GithubHelp home page GithubHelp logo

srid / haskell-flake Goto Github PK

View Code? Open in Web Editor NEW
134.0 8.0 15.0 681 KB

A `flake-parts` Nix module for Haskell development

Home Page: https://community.flake.parts/haskell-flake

License: MIT License

Nix 98.65% Haskell 0.71% Just 0.54% Shell 0.10%
haskell nix flake-parts

haskell-flake's Introduction

project chat Harmeless Code of Conduct

haskell-flake - Manage Haskell projects conveniently with Nix

There are several ways to manage Haskell packages using Nix with varying degrees of integration. haskell-flake makes Haskell development, packaging and deployment with Nix flakes a lot simpler than other existing approaches. This project is set up as a modern flake-parts module to integrate easily into other Nix projects and shell development environments in a lightweight and modular way.

To see more background information, guides and best practices, visit https://community.flake.parts/haskell-flake

Caveat: haskell-flake only supports the Haskell package manager Cabal, so your project must have a top-level .cabal file (single package project) or a cabal.project file (multi-package project).

Getting started

The minimal changes to your flake.nix to introduce the haskell-flake and flake-parts modules will look similar to:

# file: flake.nix
{
  inputs = {
    ...
    flake-parts.url = "github:hercules-ci/flake-parts";
    haskell-flake.url = "github:srid/haskell-flake";
  };

  outputs = inputs:
    inputs.flake-parts.lib.mkFlake { inherit inputs; } {
      systems = [ "x86_64-linux", ... ];
      imports = [
        ...
        inputs.haskell-flake.flakeModule
      ];
      perSystem = { self', system, lib, config, pkgs, ... }: {
        haskellProjects.default = {
          # basePackages = pkgs.haskellPackages;

          # Packages to add on top of `basePackages`, e.g. from Hackage
          packages = {
            aeson.source = "1.5.0.0" # Hackage version
          };

          # my-haskell-package development shell configuration
          devShell = {
            hlsCheck.enable = false;
          };

          # What should haskell-flake add to flake outputs?
          autoWire = [ "packages" "apps" "checks" ]; # Wire all but the devShell
        };

        devShells.default = pkgs.mkShell {
          name = "my-haskell-package custom development shell";
          inputsFrom = [
            ...
            config.haskellProjects.default.outputs.devShell
          ];
          nativeBuildInputs = with pkgs; [
            # other development tools.
          ];
        };
      };
    };
}

haskell-flake scans your folder automatically for a .cabal or cabal.project file. In this example an imaginary my-haskell-package.cabal project is used.

To see in more detail how to use haskell-flake in a realistic Haskell project with several other development tools, take a look at the corresponding Haskell single-package project Nix template and this Haskell multi-package project Nix example.

Documentation

https://community.flake.parts/haskell-flake

Discussion

Zulip is the primary venue for discussion; we also have Github Discussions enabled.

haskell-flake's People

Contributors

aravindgopall avatar dixslyf avatar dontdieych avatar github-actions[bot] avatar hariamoor-professional avatar malteneuss avatar ozkutuk avatar roberth avatar shivaraj-bh avatar srid avatar yuanwang-wf avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

haskell-flake's Issues

Support `nix flake show --option allow-import-from-derivation false`

This will allow inclusion of haskell-flake based projects into the NixOS flakes search.

I believe it can be done, but we'll have to make the set of attributes less dynamic, in particular we have to predict outputs, the names of the outputs.

lib.lazyDerivation NixOS/nixpkgs@fce8b01 produces an attrset that is a valid "derivation attrset" but only needs to evaluate the argument derivation for attributes that really need it, like drvPath, outPath, etc.

Our package definitions would have to look somewhat like

fix (pkg:
  lazyDerivation {
    derivation = callCabal2nix ...;
    meta = ... hardcoded or perhaps some `readFile` on `.cabal` sources if we dare ...;
    passthru = {
      inherit (pkg)
        overrideCabal
        overrideAttrs
        # ... any useful unconditional attributes
        ;
    };
  }
)

I can add multi-output support to lazyDerivation.

HLS check fails by trying to access the network

The check works on my local machine but not on Github Actions runner or Garnix CI.

https://github.com/srid/haskell-template/runs/8060856223?check_suite_focus=true

default-hls-check> Severity: DsError
default-hls-check> Message:
default-hls-check>   Error when calling cabal exec -v0 -- ghc --print-libdir
default-hls-check>   dieVerbatim: user error (cabal: Couldn't establish HTTP connection. Possible cause: HTTP proxy
default-hls-check>   server
default-hls-check>   is down.
default-hls-check>   )

Very strange.

buildTools does not pay attention to overrides

Here if I do something like

              haskellPackages = pkgs'.haskell.packages.ghc942;
              buildTools = hp: {
                cabal-doctest = null;
                cabal-install = null;
                ghcid = null;
                haskell-language-server = null;
                hedgehog = null;
                hlint = pkgs'.haskell.packages.ghc942.hlint;
                hoogle = null;
                hoogle-with-packages = null;
                warp = null;
              };
              source-overrides = { };
              overrides = final: prev:
                with pkgs.haskell.lib;
                (horizon-plutus.packages.x86_64-linux) //
                {
                };
            };

it does not pay attention to the overrides in horizon, but uses versions of base-compat that I am not sure where they are coming from

      > base >=4.3 && <4.16
       >
       For full logs, run 'nix log /nix/store/hqacgvv9lkn11nak8lk8s8j3ndlk7rj6-base-compat-0.11.2.drv'.

Add `cabal check` to `checks`

In arion I had a separate build with

                  checked = super.haskell.lib.overrideCabal pkg (o: {
                    postConfigure = ''${o.postConfigure or ""}
                      if ! ${hsuper.cabal-install}/bin/cabal check;
                      then
                        echo 1>&2 ERROR: cabal file is invalid. Above warnings were errors.
                        exit 1
                      fi
                    '';
                  });

I think we could touch the outputs and exit 0 to speed this up.
Doing it in runCommand would be fragile.
This should probably call Setup.hs instead of cabal-install.

Running it as part of the main build isn't ideal because the check may be more strict than necessary. I think it should only run in CI.

Option to disable the development environment

Haskell developers will always remain the primary audience of haskell-flake. For users who just want to package the Haskell app without needing the full-blown development environment, the following configuration is useful:

haskelProjects.default = {
  buildTools = _: {
    cabal-install = null;
    ghcid = null;
    hlint = null;
    haskell-language-server = null;
  };
}

(These just disable the default build tools added by haskell-flake).

However, this configuration does not prevent the addition of devShell itself that is going to remain unused by these users.

From an API standpoint, how can we provide a simpler option to disable the whole development shell and friends for this target audience (non-Haskell developer users)?

Default for `packages` option should use `cabal.project` if it exists

... instead of the current */*.cabal (or *.cabal) globbing.

The algorithm would then be roughly: check for cabal.project or stack.yaml or *.cabal in the root and use that to set the packages option; if not present, require the user to set that option (or some helper option that sets it, for instance an option that specifies the location of a cabal.project file).

An error message should also be provided when the user hasn't set this option and their project has multiple-packages (no top-level .cabal) but without a cabal.project.

Originally posted by @roberth in #49 (comment)

Remove `hlintCheck`

treefmt can do this: https://github.com/srid/haskell-template/pull/83/files

And hlint does not need to be run in the devShell, which is all the more reason to decouple it from haskell-flake. haskell-flake should perhaps have a 'recommendations' section in the README, advising people to use treefmt-nix if they want goodies like auto-formatting, hlint, etc.

Facilitate predictable overlay merging

Background: #67 (comment)

In the below imports, the overlays are applied in reverse order:

            imports = [
              {
                overrides = self: super: {
                  aeson = super.aeson-backup;
                };
              }
              {
                overrides = self: super: {
                  aeson-backup = super.aeson;
                  aeson = null;
                };
              }
            ]

The last overlay is applied first, and then the first one is applied next. This is how it is as of NixOS/nixpkgs@bbf7742 - but there is no guarantee that this order will not change.

Currently when multiple overlays are in use, haskell-flake will print this warning:

trace: WARNING[haskell-flake]: Multiple haskell overlays are applied in arbitrary order.

But still, this can be surprising and annoying to deal with.

We want something like NixOS/nixpkgs#215486 - or at least something to allow the current project overrides take priority (even if the overlays of dependencies themselves, when imported, are applied in arbitrary order).

Add `haskellProjects.foo.finalPackages` to expose the final package set

We want to expose the following as an option:

finalPackages = cfg.haskellPackages.extend finalOverlay;

This will be useful here: https://github.com/srid/ema-template/pull/45/files#diff-206b9ce276ab5971a2489d75eb1b12999d4bf3843b7988cbe8d687cfde61dea0L63

Where instead of using lib.getExe pkgs.haskellPackages.ghcid (incorrect, because ghcid was overriden) I can instead accurately use lib.getExe config.haskellPackages.main.finalPackages.ghcid.

Configurable auto-wiring

Currently haskell-flake implicitly sets packages, devShells and checks. Let's call it "auto wiring".

Unless upstream implements it, we can add an haskellProjects.foo.autoWire bool option (default is true) that users can disable if they want to "wire" the flake outputs themselves, eg.:

{
  haskellProjects.foo = {
    ..
    autoWire = false;
  };
  packages.default = config.haskellProjects.foo.output.packages.mypkg;
  devShells.default = config.haskellProjects.foo.output.devShell;
  # Disabling checks temporarily
  checks = config.haskellProjects.foo.output.checks;
}

Add `haskell-flake-lib` ?

Like flake-parts-lib, should we add a haskell-flake-lib for use in both haskellProjects and haskellFlakeProjectModules?

This attrset may initially contain the following:

  • The default value for haskellFlakeProjectModules #108 (unless there is a better way to do this?)
  • composePipe of #101 (comment) (though this really is better to be inlined in most cases, as lib.mapAttrs (k: v: lib.pipe super.${k} v)).

Unable to use this flake on darwin

I've tried both duplicating the haskell-template repo and starting with a fresh repo and adding the flake specified in this repo to no avail.

Upon running nix develop, it seems to properly download all the dependencies and stuff so after creating the flake.lock I get this error

error: flake 'path:/Users/$USER/Documents/Workspace/$PROJNAME' does not provide attribute 'devShells.x86-64-darwin.default', 'devShell.x86-64-darwin', 'packages.x86-64-darwin.default' or 'defaultPackage.x86-64-darwin' Did you mean one of devShells or x86_64-darwin?

From what I can tell the default package/devShell isn't getting set on darwin but I don't have any other operating systems to check this with.

The only resource I was able to find that helped me understand this a little more was this issue on the o.g. nix repo, but that seems to be pretty specific to the 'nix flake new' command which I'm not using for this. (EDIT: the proposed solution seems to be to use flake-utils whereas this flake uses flake-parts)

Is there something I should manually be changing to let this work on macOS?

Thank you for your time and energy maintaining this package!!

Add a `check` for `haskell-language-server`

To sanity check that the project is configured correctly for haskell-language-server to work, we run the following in nix shell:

nix develop -i -c haskell-language-server

... and check that it successfully exits without any error messages.

The goal of this issue is to have haskell-flake automatically add a flake check that checks the same. API-wise, it could look like this:

haskellProjects.foo = {
  checks.haskell-language-server = true;  # True by default if buildTools.haskell-language-server is non-null
};

Add a check for `hlint`

Like #18 but for hlint (enabled by default if .hlint.yaml exists at project root and buildTools.hlint is non-null).

ghc 9.4.3 support

I've been curious about haskell-flake and flake-parts for some time, so I tried to convert an existing project that's already using nix flakes to haskell-flake, but I wasn't able to get far, unfortunately.

Here is the smallest broken flake file.

{
  description = "project";
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
    flake-parts.url = "github:hercules-ci/flake-parts";
    haskell-flake.github:srid/haskell-flake";
  };
  outputs = inputs@{ self, nixpkgs, flake-parts, haskell-flake, ... }:
    flake-parts.lib.mkFlake { inherit inputs; } {
      systems = nixpkgs.lib.systems.flakeExposed;
      imports = [
        haskell-flake.flakeModule
      ];
      perSystem = { config, pkgs, ... }: {
        haskellProjects.default = {
          haskellPackages = pkgs.haskell.packages.ghc943;
        };
      };
    };
}

I get the following error:

image

Commenting out the haskellPackages line works, but I get ghc 9.2 instead of 9.4 which this project needs.

Am I doing something wrong?

Remove `pkgFiltered`

contd. from #85 (comment)

@roberth I still don't understand why we need this transformation:

pkgFiltered = pkgs.haskell.lib.overrideSrc pkgProto {
src = filterSrc name value.root;
};

If the code is not needed for any project or has no effect, it should be removed to minimize maintenance.

I'm trying to understand your comment here

I think it's supposed to cut the src directory from any parent directory contents [..] Maybe there's something different about the situation where I used it

Is there a concrete project example that demonstrates the utility of this code? Without which the project will rebuild unnecessarily? I cannot see it in https://github.com/srid/haskell-multi-nix - so I wonder what I'm missing.

nix build . don't work

I use this example https://github.com/srid/ema/blob/master/flake.nix with GHC8107 (nixpkgs unstable and I can't build the package, with nix build . or nix run . I've got this error :
flake 'git+file:///home/manu/build/hsnix/mong-servant' does not provide attribute 'packages.x86_64-linux.default' or 'defaultPackage.x86_64-linux'. Have you got an idea ?
Thanks

Group package set related options

Detached from #52 (comment)

haskellProjects.foo = {
  devShell.enable = false;
  packageSet = {
    # The overlays are composed in the order they are shown here.
    base = pkgs.haskell.packages.ghc944;
    local = {
      mypkg.root = ./.;  # Assumes mypkg.cabal
    };
    source-overrides = { .. }; # Overrides by direct path to source
    overrides = self: super:  { .. };  # Raw Haskell overlay
  };
};

Users can then use config.haskellPackages.foo.packageSet.final to access all the packages. The idea is that we can say "start from base", then we apply local, then source-overrides and finally overrides, before creating the project specific haskell package set.

finalPackages of #68 would become packageSet.final.

Add a flake check for hpack <-> `.cabal` consistency

Based on #90 (comment)

Create a hpackCheck (analogous to the existing hlsCheck) if (and only if) the repository has both hpack and cabal files. The check will succeed only if running hpack generates the same cabal file.

This is useful in workflows where developers commit the generated cabal file to Git repo. The check will ensure that nobody has forgotten to commit package.yaml without the corresponding .cabal change.

include in `NixOS/templates`?

May I suggest including haskell-flake or haskell-template here https://github.com/NixOS/templates?
Then the project would be included in nix flake show templates, and it would enable the same thing that haskell.nix has:

nix flake init --template templates#haskell-nix --impure

Thanks, this Flake is great

Require `name` to be explicitly specified?

name is currently optional. Without specifying it, the .drv file is created like this:

❯ nix run .
warning: Git tree '/Users/srid/code/flake-to-gitlab-ci' is dirty
warning: Git tree '/Users/srid/code/flake-to-gitlab-ci' is dirty
error: a 'x86_64-linux' with features {} is required to build '/nix/store/bi5flxinmyvcyxccvciwkaw054xad5vm-cabal2nix-.drv', but I am a 'aarch64-darwin' with features {benchmark, big-parallel, nixos-test}
(use '--show-trace' to show detailed location information)

Notice that bi5flxinmyvcyxccvciwkaw054xad5vm-cabal2nix-.drv part (around the hyphen).

Once we specify name, as in:

--- a/flake.nix
+++ b/flake.nix
@@ -14,12 +14,13 @@
         haskellProjects.default = {
           haskellPackages = pkgs.haskell.packages.ghc924;
           root = self;
+          name = "flake-to-gitlab-ci";
           overrides = self: super: with pkgs.haskell.lib; {
             lens-aeson = dontCheck (self.callHackage "lens-aeson" "1.2.1" { });
             shh = doJailbreak super.shh;

it becomes:

error: a 'x86_64-linux' with features {} is required to build '/nix/store/yjl4h8wmcqr5r919y7c6bzwnqra678kf-cabal2nix-flake-to-gitlab-ci.drv', but I am a 'aarch64-darwin' with features {benchmark, big-parallel, nixos-test}
(use '--show-trace' to show detailed location information)

i.e., with the project name; viz.: yjl4h8wmcqr5r919y7c6bzwnqra678kf-cabal2nix-flake-to-gitlab-ci.drv.

Rewrite tests, for exhaustive coverage

We want to be exhaustive in what we test, specifically multiple projects with differing configurations, such as

  • those using hpack; #116
  • those using project modules #119

bash is neither a suitable nor friendly language (and runtest.sh is growing in size & complexity), so let's feel free to write them in a more suitable language (like Haskell) as long as we don't complicate things too much.

`nix run` is broken

❯ nix run github:srid/emanote
warning: Using saved setting for 'extra-substituters = https://cache.garnix.io' from ~/.local/share/nix/trusted-settings.json.
warning: Using saved setting for 'extra-trusted-public-keys = cache.garnix.io:CTFPyKSLcx5RMJKfLo5EEPUObbA78b0YQ2DTCJXqr9g=' from ~/.local/share/nix/trusted-settings.json.
error: Package emanote-0.6.7.1 does not have meta.mainProgram set, so I don't know how to find the main executable. You can set meta.mainProgram, or pass the full path to executable, e.g. program = "${pkg}/bin/foo"

A flake app to upload to Hackage

Normally I use cabal sdist followed by cabal upload --publish1 in the nix shell to upload a new release to Hackage.

  • Can this be automated in an universal fashion?
  • If so, would it be useful for this module to provide a flake app to do it all?

Then, something like nix run .#hackage-publish -- <package-name> could be used to build a sdist and publish to Hackage automatically. Though it is not clear which exact interface (CLI arguments, for instance) we want to support.

Footnotes

  1. @locallycompact pointed me to this command.

Autowire `haskellFlakeProjectModules`

Let's say that 'qux' depends on 'bar' which depends on 'foo'.

At the moment, bar's flake.nix manually wires things up like this:

      flake.haskellFlakeProjectModules = rec {
        input = { pkgs, ... }: {
          imports = [
            inputs.foo.haskellFlakeProjectModules.output
          ];
        };
        output = { pkgs, lib, ... }: withSystem pkgs.system (ctx@{ config, ... }: {
          # Pass along the dependency overrides to consuming flake.
          imports = [
            input
          ];
          # Pass local packages, as overlay, to the consuming flake.
          source-overrides =
            lib.mapAttrs (name: ks: ks.root)
              config.haskellProjects.default.packages;
        });
      }

Notice how 'bar' uses 'foo's module, thus inheriting its overlays and local packages. Thereon, it exports it all, along with its own location packages, for 'qux' to consume from (via importing its output module).

The general pattern here is to generate both input and output modules, thus enabling the user to wire them up to form a graph of Haskell packages. The overall goal here is simplify the use of haskell-flake when managing multiple Haskell packages across multiple repos (as opposed to a mono repo) that depend on each other.


The purpose of this issue is to figure out the best way to minimize this boilerplate while enabling this pattern, and implement it. Here's one approach:

  perSystem.haskellProjects.default.autoWire.outputs = true;  # Creates 'packages', 'devShells', etc.
  perSystem.haskellProjects.default.autoWire.modules = true;  # Creates 'flake.haskellFlakeProjectModules.{input,output}'

The first option would be for #62. I'm not sure if the second option should be in perSystem or flake. If it has to be in flake, we would possibly have something like flake.haskellFlake.autoWire.modules = true; - but it feels weird to spread this outside of project config. OTOH, the latter is more agreeable when we consider the fact that autowiring of modules is done only for the 'default' project.

Do not create `app` if there is none

Ema is a library:

❯ nix run github:srid/ema/multisite
error: unable to execute '/nix/store/44gpjhbn175y00jyy5z1k0jwy6q946wg-ema-0.7.2.0/bin/ema': No such file or directory

One solution: we could just add an executable = true; option for this.

CI: use same `nixpkgs` pin for template flake as test flake

Otherwise, this slows down the CI due to fetching & building two different nixpkgs.

Two possible approaches:

  • Generate test/flake.lock, copy that over to example/flake.lock, or
  • Use --override-input for nixpkgs, flake-parts, etc.

nix flake init -t $FLAKE
# Build haskell executable
nix build --override-input haskell-flake path:${FLAKE}
# Test haskell devshell (via HLS check)
nix develop --override-input haskell-flake path:${FLAKE} -c haskell-language-server

Source filtering

Changes to unrelated files cause rebuilds, especially when the package is at the root of the repo, where non-package files have no other place to go.

Some solutions:
a. parse the cabal file in Nix: no experimental features required, but somewhat hard to get right, not future proof
b. or use ca-derivations with buildFromSdist or buildFromCabalSdist (NixOS/nixpkgs#174176)

Re-evaluate the structure of `outputs`

I don't think we need this option (added in #63). Here's why:

  • finalPackages already contains outputs.packages
  • For outputs.devShells, we can expose the devshell in config.haskellPackages.foo.devShell.drv
  • Likewise, the hls check in outputs.checks can be exposed in config.haskellPackages.foo.devShell.hlsCheck.drv.

I can't think of a reason to retain outputs option in its current form.

Projects using hpack (not .cabal) cannot be built

Seems to be a regression from #57 #42

       > No cabal.project file or cabal file matching the default glob './*.cabal' was found.
       > Please create a package description file <pkgname>.cabal or a cabal.project file referencing the packages you want to build.

Turns out that the use of buildFromCabalSdist is the culprit:

let
fromSdist = self.buildFromCabalSdist or (builtins.trace "Your version of Nixpkgs does not support hs.buildFromCabalSdist yet." (pkg: pkg));
filterSrc = name: src: lib.cleanSourceWith { inherit src name; filter = path: type: true; };
in
lib.mapAttrs
(name: value:
let
# callCabal2nix does not need a filtered source. It will
# only pick out the cabal and/or hpack file.
pkgProto = self.callCabal2nix name value.root { };
pkgFiltered = pkgs.haskell.lib.overrideSrc pkgProto {
src = filterSrc name value.root;
};
in
fromSdist pkgFiltered)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.