wasp-lang / strong-path Goto Github PK

This is Stack project - what about cabal? What I mean is - this is a library/package - do we have to do something special to support cabal users? Do we need to test for different versions of GHC? I feel this issue is not clear because I am also not clear about what needs to be done, but I have seen other libraries do some testing with cabal and most seem to be using cabal and not stack so I am confused if we are missing on something.

I guess the question would be: since this is a library/package, how do we correctly set it up (regarding building) and how do we test it in CI to ensure all is good and it can be used by others? Is it enough to just make it a Stack project, or do we need to do something more?

What about paths that start with some amount of "../", like "../" or "../../foo" and similar? Those are valid relative paths, and right now we can't capture them with StrongPath because Path does not support them.

Investigate how hard it would be to support these!

Right now, if you want to use some XDG dir in StrongPath manner, you would do smth like this:

data UserHomeDir
data XdgDataDir

getXdgDataDir :: IO (Path System Abs (Dir XdgDataDir))
getXdgDataDir = fromJust $ parseRelDir $ System.Directory.getXdgDirectory System.Directory.XdgData ""

We could add functions like this directly into StrongPath! Let's say in StrongPath.Xdg module?
Another option, in case we don't want to bloat package with stuff like this, is to put it in a separate package: strong-path-xdg. Or strong-path-extra.

This idea came as a suggestion from @4caraml .

I opened an issue on stack repo: commercialhaskell/stack#5583

For now I disabled building project on Windows on CI (check TODOs in ci.yaml), but we should re-enable it once we figure out how to get it working in the stack/GHC9.0.1/Win combo.

Since we need to explicitly care about dependency version bounds anyway, due to strong-path being published as a library to Hackage, in which case it is important to have version bounds, I don't think we need Stack -> it actually just confuses me from time to time. I would be up for replacing it with cabal-install!

Instructions here: https://github.com/commercialhaskell/stackage/blob/master/MAINTAINERS.md#adding-a-package

https://hackage.haskell.org/package/hashable-1.4.0.1/docs/Data-Hashable.html#t:Hashable

Ideally we could just forward to the Hasheable instance of Path, but there is some extra info we keep next to the Path, that might not be viable, and we will have to do it on our own.

This is important to enable usage of StrongPath in certain situations, for example as a key in a HashMap.

Take a look at https://hackage.haskell.org/package/path-0.9.0/docs/Path-Posix.html -> stripProperPrefix and isProperPrefixOf.

We want to implement that same function here.

We did this for Wasp compiler and I am very happy with the results - Github Actions are free, we need just one config file, and they are even faster (mac machines get provisioned pretty quickly).

This is how we did it for Wasp: https://github.com/wasp-lang/wasp/blob/master/.github/workflows/ci.yaml .

The main thing I am not sure about is, how do we need to set things up to test for different versions of GHC. Do we need to test for different versions of GHC? What about cabal? I am not sure hm!

Learn more about how they do versioning -> it is called PVP and is not exactly the same as semver.

This should give readers a quick idea of who is this for and what are the benefits.

Mention this library is for those that are dealing a lot with different filepaths and want to make sure they are correctly concatenating and passing them around, that no paths are getting mixed up, be it regarding to their standard (posix/win), their relativity (rel vs abs) or their nature (file vs dir).

They have this functionality in https://github.com/commercialhaskell/path and it is great. Right now we expect users to use path package to create paths in compile time and then they pass them to strong path, however this is clumsy and it means that strong path has to publicly depend on path.

Instead, we should implement the same thing in strong-path and while we will continue using pathin our implementation, we will be able to drop it from our API. We could keep additional StrongPath.Path module for interacting with Path (from/to).

This requires knowledge of template haskell so we are somewhat stuck as we don't know enough about it yet. This will have to wait until we learn more about template haskell or until somebody helps.

They should work the same way relDirToPosix and relFileToPosix work.

Strong path needs path 0.9.
path 0.9 is not available yet in any LTS stackage, but only in nightly.
Therefore we are compiling strong-path with nightly resolver.
But, nightly uses template-haskell 2.17 while LTS uses 2.16.
So we get a problem there when trying to use it in Stack project that is using LTS.

I am not sure how to go about this - should we somehow modify the dependencies that strong-path is relying on, so it can be easily used in Stack projects? Maybe switch it to LTS and specify path 0.9 as some kind of exception?

Or is there an ok way to include strong-path as it is in Stack project that are using LTS resolver at the moment?

I feel like I don't know enough about all this together, I need to investigate this more and understand it better or have someone help out.

We have a bunch of functions operating in the MonadThrow, which means they can throw an error/exception.

The thing is, right now, most of the errors thrown are coming directly from the underlying path functions that we call, which means user needs to import path library in order to be able to handle them!

Instead, we should catch and convert those errors into our own errors, that user's can handle without knowing about and importing path library.

Currently, the types in this package do not have kind polymorphism, meaning you can only have Dir a if a :: *. But sometimes it would be convenient to allow a to have a different kind.

For example, imagine I want to define a function readREADME :: Project -> IO String which gets the README file from a couple of explicitly defined projects:

data Project = WaspcProject | WasplsProject

To do this with strong-path, I would like to be able to write:

{-# LANGUAGE DataKinds #-}
{-# LANGUAGE PolyKinds #-}

data Project = WaspcProject | WasplsProject

readREADME :: Project -> IO String
readREADME project =
  let readmeFile = case
        WaspcProject -> waspcProjectDir </> readmeFileInProjectDir
        WasplsProject -> wasplsProjectDir </> readmeFileInProjectDir
  in readFile $ toFilePath readmeFile

data READMEFile

readmeFileInProjectDir :: forall (base :: Project). Path' (Rel base) (File READMEFile)
readmeFileInProjectDir = [relfile|README.md|]

waspcProjectDir :: Path' Abs (Dir 'WaspcProject)
waspcProjectDir = [absdir|~/waspc|]

wasplsProjectDir :: Path' Abs (Dir 'WasplsProject)
wasplsProjectDir = [absdir|~/waspls|]

Currently, I have define some extra types WaspcDir and WasplsDir, and then I have to make sure to keep those types in-sync with Project in case I add more constructors. I would also have to make a typeclass to act similar to the Project kind that I could use to limit what readmeFileInProjectDir can be joined with. The goal is to reduce that little bit of boilerplate and work required to keep the constructors and types in-sync.

I think this should be possible to do by enabling -XPolyKinds in this package.

add

  schedule:
    # additionally run once per week (At 00:00 on Sunday) to maintain cache
    - cron: '0 0 * * 0'

to on: in github actions workflow.

Inspired by https://github.com/kowainik/stan/blob/main/.github/workflows/ci.yml .

I made a PR here commercialhaskell/path#167, it fixes all the issues of Path.Posix and Path.Windows not behaving consistently regardless of the underlying platform.
Once it gets merged, I should be able to remove the workarounds that are patching these problems.

Right now Stackage has LTS that uses path 0.8.*, and we need >= 0.9 due to bugs in 0.8.

So the idea is to wait for the next LTS that will have path 0.9 and then we can add strong-path to it.

strong-path is already in the nightly, where path is 0.9.

This methods exists in commercialhaskell/path -> add them to strong path also. They are useful and needed.

There are also couple more functions dealing with properPrefix, and there are some functions dealing with file extensions -> consider adding those to.

Sometimes we are referencing files on the disk that are bundled with the project, and they should exist during development and later when distributed, so it would be cool if compiler could confirm their existence on the disk.

I imagine that when we do something like [absfile|/some/path|], we could also somehow verify via TH that it exists. Maybe it would be special kind of quoter, smth like [absfile'|/some/path|], where ' indicates it will check for its existence on the disk? What about relative paths, for which we know what they are relative to, how would we do that kind of check?

I am not sure how feasible this really is, especially since location of the file can change from development to distribution, but it would be interesting to explore.

This also sounds quite costly, since during each compilation we would have to check for the files on the disk, I wonder if that is feasible.

We use Test.Tasty.Hspec + tasty-discover, but with new changes to Test.Tasty.Hspec, they seem to discourage using tasty + hspec together and suggest we use just one.

So should we switch to Tasty + Hunit then? Or just HSpec? We should research this and make a decision.

And make it so that in the future we can also easily test for couple older LTS versions.

I should document well the process of determining / updating version bounds, and how the CI needs to be updated based on that (or vice versa).

Good resources for what I need to do are here:

• https://www.reddit.com/r/haskell/comments/qhn8es/how_do_i_figure_out_dependency_version_bounds_for/
• commercialhaskell/stackage#6256

The path library offers a bunch more utility functions (https://hackage.haskell.org/package/path-0.9.2/docs/Path-Posix.html#g:4) that strong-path would benefit from having. Currently, to do things like work with extensions you have to convert to a path Path, use functions from that library, and then back to a strong-path Path.

Same as we have castDir, we should also have castFile.

If StrongPath.Path contains ../, transforming it into Path.Path is not possible because Path.Path does not support ../, and right now error happens, which makes the function unsafe. Instead we should have it operate in the MonadThrow, same as Path.Path does in such situations.

Right now, README.md is the main documentation.

However, from what I understood, packages are expected to have decent Haddock documentation.

I am not yet sure how to go about this -> should I write decent Haddock documentation and then link to it from the README.md, or should I have some kind of dummy Haddock documentation that just links to the README.md? It doesn't seem to make sense to maintain both, that will be too much duplication.

I should look in to some existing, well written packages (like path) and figure out how are they doing it.

These are unsafe versions of relDirToPosix and relFileToPosix.

I didn't document them, I should probably do that.

But I am concerned about the name -> is it clear enough that they are unsafe? Should we name them relDirToPosixUnsafe? Should we even bother with them, maybe we should kick them out?

I got following advice from Hackage admins:

I also noticed that your package does not currently have version bounds for its dependencies:

https://github.com/wasp-lang/strong-path/blob/master/package.yaml#L30-L32

Packages that are uploaded to Hackage are generally expected to follow the Package Versioning Policy and have version bounds on all dependencies indicating which versions they are known to build with. Here are some links that have more information:

http://hackage.haskell.org/upload#versioning_and_curation
https://pvp.haskell.org/

Also, here are a few things to keep in mind when uploading to Hackage that help keep the ecosystem working reliably for everyone:

1. Hackage is intended to be a permanent record. Therefore uploads cannot be changed or removed.
2. Only upload things that work, are useful to other people, and that you intend to maintain.
3. Use cabal gen-bounds to put PVP-compliant version bounds (lower AND upper) on ALL your unique dependencies so your package will still be buildable years down the road. One important thing to note is that you only need to include version bounds once. For example, if you depend on the same package in your library and your test suite, you only need to put the version bounds for that dependency in one place. This keeps the dependency bounds information DRY.
4. Package candidates CAN be changed, so use them to test things out and get everything right before you publish permanently to the main index.

Here are a few important resources where you can learn more:

The hackage home page that contains some of the main guidelines for Hackage packages:
http://hackage.haskell.org/

Information about package candidates can be found here:
http://hackage.haskell.org/packages/candidates/

There's also a handy repo for autogenerating Travis CI configs: https://github.com/haskell-CI/haskell-ci

If you have any questions, just let us know and we'll be glad to help. Happy coding!

We should look into all the points here and take care of them before uploading strong-path to Hackage!

There is a lot of duplication inside the code of the strong-path library because we have 12 different constructors we are dealing with. I am pretty sure we can remove some of that duplication using TemplateHaskell, so we could try that. Maybe there is an even more elegant way though? Worth researching.

We should have more examples with it! Maybe even special small section about it.