tweag / opam-nix

Turn opam-based OCaml projects into Nix derivations

opam-nix

Turn opam-based OCaml projects into Nix derivations, automatically resolving both OCaml and system dependencies.

opam-nix can build packages

• from opam and dune-project files,
• for Linux and macOS,
• on x86_64 and aarch64 (including M1 macs),
• using a compiler from nixpkgs or from opam-repository,
• with either dynamic or static linking.

It also comes with the power of Nix, allowing you to effortlessly manage multiple projects with different dependency versions, override dependencies, cache builds, and more.

Quick start

For a quick introduction to opam-nix and a guide to get you started, read this blog post on the Tweag blog.

Templates

opam-nix comes with some templates that can help you package opam packages with Nix.

Note

All of these templates assume that you already have an OCaml project packaged with opam, and just want to package it with Nix. If you're starting from scratch, you have to set up the opam files separately.

• A simple package build, no frills: nix flake init -t github:tweag/opam-nix
• A more featured flake, building an executable and providing a shell in which you can conveniently work on it: nix flake init -t github:tweag/opam-nix#executable
• Build multiple packages from the same workspace, and have a shell in which you can work on them: nix flake init -t github:tweag/opam-nix#multi-package

Note

If you're using Git, you should git add flake.nix after initializing, as Nix operates on the git index contents.

Examples

There are also some examples which can give you some ideas of what is possible with opam-nix:

• Building a package from opam-repository
• Building a package from a custom github repository
• Building a package from a multi-package github repository with submodules
• Building a Coq package
• Building a static version of a package using compiler from nixpkgs
• Building a static version of a package using the compiler from opam
• Building a GUI package
• Building the entirety of Tezos

All examples are checks and packages, so you can do e.g. nix build github:tweag/opam-nix#opam-ed to try them out individually, or nix flake check github:tweag/opam-nix to build them all.

Building packages from opam-repository with Nix

opam-nix provides all packages from opam-repository in its legacyPackages flake output. You can use this to quickly check out some package, or get a shell with particular version without messing with opam switches.

For example:

• nix build 'github:tweag/opam-nix#utop."2.10.0"' to build a specific version of utop (available in ./result),
• nix shell github:tweag/opam-nix#camyll.latest to get a shell with the latest version of camyll.

Complete documentation

Complete documentation for opam-nix is available in the DOCUMENTATION.md file.

Questions? Problems?

You can reach out to us via the Discussions page or directly to the maintainer via email.

Related projects

• Nix: A powerful package manager that makes package management reliable and reproducible;
• opam: the OCaml package manager;
• hillingar: Tool for building MirageOS unikernels with opam-nix.

opam-nix's Issues

Unable to build `vscoq-language-server` 2.1.2 as an opam dependency using `opam-nix`

Describe the bug

Unable to have vscoq-language-server as an opam dependency in a coq project using opam-nix.

To Reproduce

Using opam-nix to get a dev shell for the following .opam project file works well:

opam-version: "2.0"
depends: [
  "ocaml"
  "coq" {>= "8.18"}
  "coq-hammer"
  "coq-aac-tactics"
  "coq-smtcoq"
]

Associated flake:

{
  inputs = {
    opam-nix.url = "github:tweag/opam-nix";
    opam-repository.url = "github:ocaml/opam-repository";
    opam-repository.flake = false;
    opam-coq-archive.url = "github:coq/opam-coq-archive";
    opam-coq-archive.flake = false;
    flake-utils.url = "github:numtide/flake-utils";
    nixpkgs.follows = "opam-nix/nixpkgs";
  };
  outputs = { self, flake-utils, opam-nix, opam-repository, opam-coq-archive, nixpkgs }@inputs:
    let package = "destination";
    in flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = nixpkgs.legacyPackages.${system};
        on = opam-nix.lib.${system};
        inherit (opam-nix.lib.${system}) queryToScope;
        scope = on.buildOpamProject'
          {
            repos = [ "${opam-repository}" "${opam-coq-archive}/released" ];
          }
          ./.
          {
            ocaml-base-compiler = "*";
          };
        overlay = final: prev:
          {
            # Your overrides go here
          };
        scope' = scope.overrideScope' overlay;
        main = scope'.${package};
      in {
        legacyPackages = scope';

        packages.default = main;

        devShells.default = pkgs.mkShell {
          inputsFrom = [ main ];
          buildInputs = [
            #additional packages from nixpkgs
            pkgs.eprover
            pkgs.cvc5
            pkgs.vampire
          ];
        };
      });
}

But it fails as soon as I add vscoq-language-server dependency (without constraint bounds):

opam-version: "2.0"
depends: [
  "ocaml"
  "coq" {>= "8.18"}
  "coq-hammer"
  "coq-aac-tactics"
  "vscoq-language-server"
]

$ nix develop
[...]
error: builder for '/nix/store/xfxxlv440lq0wwyv39131h57jz417k6r-vscoq-language-server-2.1.2.drv' failed with exit code 1;
       last 10 log lines:
       > Hint: coqc --config requires the coq-stdlib package in order to function
       > properly.
       > File "language/dune", line 4, characters 99-119:
       > 4 |  (preprocess (pps ppx_optcomp -- -cookie "ppx_optcomp.env=env ~coq:(Defined \"%{coq:version.major}.%{coq:version.minor}\")"))
       >                                                                                                        ^^^^^^^^^^^^^^^^^^^^
       > Error: Could not expand %{coq:version.minor} as running coqc --config failed.
       > /nix/store/5k4b2v1cn9rl0hhabarjdf2d8sr7znj6-coq-core-8.18.0/bin/coqc --config failed with exit code 1.
       > Hint: coqc --config requires the coq-stdlib package in order to function
       > properly.
       >
       For full logs, run 'nix log /nix/store/xfxxlv440lq0wwyv39131h57jz417k6r-vscoq-language-server-2.1.2.drv'.
error: 1 dependencies of derivation '/nix/store/8ick5xh2mzcsfp25x8wb1k035vgc9i6r-nix-shell-env.drv' failed to build

Using a globally installed opam, and using opam pin coq 8.18.0 and opam install vscoq-language-server works fine.

Already tried: setting "coq" { = "8.18.0" } instead of "coq" { >= "8.18" } doesn't help.

Expected behavior

I expect opam-nix to successfully build vscoq-language-server as it would using opam install

Environment

• OS name + version: Kubuntu 22.04 + nix
• Version of the code: opam-nix master from 21/03/2024:

{
        "lastModified": 1710414181,
        "narHash": "sha256-Cfiyf0o5ySmi2BHpNDvWEKM/dtExNcmIiTaBbRZImqs=",
        "owner": "tweag",
        "repo": "opam-nix",
        "rev": "8578c83725e4ceaf04913cf04da9bb36d85ad0bc",
        "type": "github"
}

Additional context

I just managed to get a working dev shell using the following opam file:

opam-version: "2.0"
depends: [
]

and flake:

{
  inputs = {
    opam-nix.url = "github:tweag/opam-nix";
    opam-repository.url = "github:ocaml/opam-repository";
    opam-repository.flake = false;
    opam-coq-archive.url = "github:coq/opam-coq-archive";
    opam-coq-archive.flake = false;
    flake-utils.url = "github:numtide/flake-utils";
    nixpkgs.follows = "opam-nix/nixpkgs";
  };
  outputs = { self, flake-utils, opam-nix, opam-repository, opam-coq-archive, nixpkgs }@inputs:
    let package = "destination";
    in flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = nixpkgs.legacyPackages.${system};
        on = opam-nix.lib.${system};
        # inherit (opam-nix.lib.${system}) queryToScope;
        devPackagesQuery = {
          # You can add "development" packages here. They will get added to the devShell automatically.
          coq = "8.18.0";
          coq-hammer = "*";
          coq-aac-tactics = "*";
          coq-smtcoq = "*";
          vscoq-language-server = "2.0.3+coq8.18";
        };
        query = devPackagesQuery // {
          ## You can force versions of certain packages here, e.g:
          ## - force the ocaml compiler to be taken from opam-repository:
          ocaml-base-compiler = "5.0.0";
          ## - or force the compiler to be taken from nixpkgs and be a certain version:
          # ocaml-system = "4.14.0";
          ## - or force ocamlfind to be a certain version:
        };
        scope = on.buildOpamProject' {
          repos = [ "${opam-repository}" "${opam-coq-archive}/released" ];
        } ./. query;
        overlay = final: prev: {
          # You can add overrides here
          ${package} = prev.${package}.overrideAttrs (_: {
            # Prevent the ocaml dependencies from leaking into dependent environments
            doNixSupport = false;
          });
        };
        scope' = scope.overrideScope' overlay;
        # The main package containing the executable
        main = scope'.${package};
        # Packages from devPackagesQuery
        devPackages = builtins.attrValues
          (pkgs.lib.getAttrs (builtins.attrNames devPackagesQuery) scope');
      in {
        legacyPackages = scope';

        packages.default = main;

        devShells.default = pkgs.mkShell {
          inputsFrom = [ main ];
          buildInputs = devPackages ++ [
            #additional packages from nixpkgs
            pkgs.eprover
            pkgs.cvc5
            pkgs.vampire
          ];
        };
      });
}

I don't know yet what made it work under this form.

`getUrl` doesn't provide a hash, so `--impure` is required to build.

getUrl doesn't provide a hash attribute, so when it's result is used in the builder we get the error in pure evaluation mode, 'fetchTarball' requires a 'sha256' argument.

A workaround is to build with --impure. However, this negates a big advantage of flakes, especially if this requires a downstream user with a project built using opam-nix as a flake input to rebuild their NixOS config with --impure.

Materialization can be used to get all the dependencies of a project statically, instead of resolving them with opam at every build. Do you think it would be possibly to statically get hashes of these dependencies at this point and commit them to the json file produced my materialization?

Filtering `project` without `--impure` option for `build*Project`

Is your feature request related to a problem? Please describe.
Currently, in buildDuneProject (and probably also in buildOpamProject), in the project argument with filtering ./. will require --impure. It would be nice to be able to do without it.

I am using https://github.com/numtide/nix-filter, but it would happen with lib.sources.sourceFilesBySuffices or buildins.path also.

I think it is very important to filter src. Currently, building with this flake is getting involved with flake.nix and other files that are not needed for the build ocaml project because can't filtering without --impure, so even if I change unrelated parts of those files, they get re-built, which annoys me.

Describe the solution you'd like
Build with filtering project arg, without --impure option.

Describe alternatives you've considered
Use --impure

Additional context

This problem is occurring while working on this branch. https://github.com/SnO2WMaN/SATySFi/tree/sno2wman/nix-flake for PR

example file

I've been reading the excellent example here: https://github.com/brendanzab/ocaml-flake-example (from this thread: https://discourse.nixos.org/t/ocaml-example-project-using-flakes/16892)

It would be nice to have something similar for opam-nix, I admit that I'm a bit lost with the README :)

Add the ability to override the default build command

Is your feature request related to a problem? Please describe.
Dune supports different build envs that can be used to pass custom compiler flags during compilation. Currently the installation will use the build attribute found in the opam file by default and there is no way to provide an alternative build command (to use a custom env for example).

Describe the solution you'd like
After creating a scope I would like to be able to override the build attribute for my package.

Currently the following does not work and still uses the build command from the opam file:

let
  scope = opam-nix.lib.${pkgs.system}.buildOpamProject {
    inherit pkgs;
  } "my_project" src { ocaml-system = null; };

  overlay = self: super: {
    my_project = super.my_project.overrideAttrs (oa: { build = [ "xxx" ]; });
  };
in
(scope.overrideScope' overlay).my_project

Describe alternatives you've considered
Alternatively maybe there is a way to adapt buildDuneProject to allow passing a custom build argument.

compareVersions' does not correctly handle versions containing `~`.

Describe the bug
When finding the list of input OCaml packages for a derivation, ocaml-nix attempts to compare version constraints with the compareVersions' function.

However this function does not quite match opam's behaviour for packages containing ~: opam treats ~ as smaller than ~beta, but opam-nix treats it as larger. This means that a dependency may show up in opam's solution, but will not be added as a build dependency in the actual nix derivation.

This was encountered when trying to depend on OCaml 5.0.0~alpha1. This specifies a version bound of >= 5.0.0~ && < 5.0.1~, but 5.0.0~alpha1 is treated as smaller than 5.0.0~ by opam-nix.

To Reproduce
There's an expected ordering of versions on the opam manual, which we can compare against opam-nix's behaviour:

let 
  actual =  (builtins.sort (compareVersions' "lt")
    ["0.1" "1.0-test" "1.0.1" "1.0.10" "1.0" "1.0~beta" "dev" "trunk" "~" "~~" "~beta10" "~beta2"]);
  exp = ["~~" "~" "~beta2" "~beta10" "0.1" "1.0~beta" "1.0" "1.0-test" "1.0.1" "1.0.10" "dev" "trunk"]; 
in
if actual != exp then 
  builtins.throw "Expected: ${builtins.concatStringsSep ", " exp}\nActual:   ${builtins.concatStringsSep ", " actual}"
else
  true

# error: Expected: ~~, ~, ~beta2, ~beta10, 0.1, 1.0~beta, 1.0, 1.0-test, 1.0.1, 1.0.10, dev, trunk
#        Actual:   ~beta2, ~beta10, 0.1, 1.0, 1.0-test, 1.0~beta, 1.0.1, 1.0.10, dev, trunk, ~~, ~

In the case of building OCaml, running nix build with the following crate:

flake.nix

{
  description = "A very basic flake";

  inputs.opam-nix.url = "github:tweag/opam-nix";
  inputs.opam-repository = { url = "github:ocaml/opam-repository"; flake = false; };
  inputs.ocaml-beta-repository = { url = "github:ocaml/ocaml-beta-repository"; flake = false; };

  outputs = { self, opam-nix, opam-repository, ocaml-beta-repository }:
    let 
      system = "x86_64-linux";
      inherit (opam-nix.lib.${system}) queryToScope;
      scope = queryToScope { repos = [ocaml-beta-repository opam-repository]; } { 
        ocaml = "5.0.0";
        ocaml-base-compiler = "*";
      };
    in
    {
      defaultPackage.x86_64-linux = scope.ocaml;
    };
}

fails when building the ocaml opam package with the error: ocaml: command not found, as opam-base-compiler is never added as a dependency!

Expected behavior
I'd expect the behaviour to exactly match opam.

Environment

• OS name + version:
• Version of the code: ea5924d

Additional context
I'm happy to have a look at fixing this if preferred, just wanted to raise this first to discuss possible solutions.

The most comprehensive option would of course just be to port opam's version comparison implementation to nix, but not sure if that's a little overkill? (and is licenced under the LGPL, which may not be acceptable for you).

Static Binaries

Is your feature request related to a problem? Please describe.

I need to produce a static build of an ocaml project for deployment purposes.

Describe the solution you'd like

It would be really cool if we could do this with opam-nix and musl.

Describe alternatives you've considered

I'm not very familiar with ocaml tooling but it looks like the alternative is to try to use dune a la this blog post:
https://ocamlpro.com/blog/2021_09_02_generating_static_and_portable_executables_with_ocaml/

Additional context
Add any other context or screenshots about the feature request here.

`nix flake show` on this repo doesn't work on Darwin

Describe the bug
nix flake show on this repository fails on darwin

To Reproduce

[1] rgrinberg@rgrinberg-16> nix flake show github:tweag/opam-nix                                                                                                                                 ~/github/tweag/opam-nix
github:tweag/opam-nix/ab96c52560432f5e1d36cbb92b2dd088c3bbea4a
├───aux: unknown
├───checks
│   ├───aarch64-darwin
error: builder for '/nix/store/mfzs64lknk5iv1lspdr9j59cgf544sjd-readme-check.drv' failed with exit code 1;
       last 2 log lines:
       >
       > ErroSysError~executing '/nix/store/7aqi1cxp0q594zl71c8fpkv7zjnzf6rw-bash-5.1-p12/bin/bash': No such file or directory
       For full logs, run 'nix log /nix/store/mfzs64lknk5iv1lspdr9j59cgf544sjd-readme-check.drv'.
(use '--show-trace' to show detailed location information)

Expected behavior
I expected the command to succeed

Environment

• OS name + version: Darwin
• Version of the code: master (ab96c52)

Additional context
n/a

Question: Add build flags to specific package

Hello,

First of all, apologies for opening an issue, I don't know where else to ask it.

I'm trying to put why3 v1.6.0 in a flake. The problem is that, I need to enable the IDE with the --enable-ide flag at build time (similar to configureFlags). Is there a way to do this currently?
I've tried to use the overlay mechanism to add the flag, but it didn't work since it's more designed for env variables as far as I understand.

Here's the nix file I currently have:

{
  inputs = {
    flake-utils.url = "github:numtide/flake-utils";
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-23.11";
    opam-nix.url = "github:tweag/opam-nix";
  };

  outputs = {
    self,
    nixpkgs,
    opam-nix,
    flake-utils,
  }:
    flake-utils.lib.eachDefaultSystem (system: let
      why3 = let
        inherit (opam-nix.lib.${system}) queryToScope;
        scope = queryToScope {} {
          why3 = "1.6.0";
          ocaml-base-compiler = "*";
        };
        overlay = self: super: {
          why3 = super.why3.overrideAttrs (
            oa: { }
          );
        };
      in (scope.overrideScope' overlay).why3;

      pkgs = nixpkgs.legacyPackages.${system};
    in {
      devShell = pkgs.mkShell {
        packages = [
          why3
        ];
      };
    });
}

Thank you for your time :)

ocaml5.1.1-opam-core-2.2.0 fails to build

Describe the bug
opam-core fails to build:

ocaml5.1.1-opam-core> Opam will be built WITH its default built-in solver
ocaml5.1.1-opam-core> Executables will be installed in /nix/store/lpfjkzdn9v946g3p39zjv62r2gav0v40-ocaml5.1.1-opam-core-2.2.0/bin
ocaml5.1.1-opam-core> Manual pages will be installed in /nix/store/lpfjkzdn9v946g3p39zjv62r2gav0v40-ocaml5.1.1-opam-core-2.2.0/share/man
ocaml5.1.1-opam-core> Running phase: buildPhase
ocaml5.1.1-opam-core> File "src/core/dune", line 6, characters 67-71:
ocaml5.1.1-opam-core> 6 |   (libraries   re (re_export ocamlgraph) unix sha jsonm swhid_core uutf
ocaml5.1.1-opam-core>                                                                        ^^^^
ocaml5.1.1-opam-core> Error: Library "uutf" not found.
ocaml5.1.1-opam-core> -> required by library "opam-core" in _build/default/src/core
ocaml5.1.1-opam-core> -> required by _build/default/META.opam-core
ocaml5.1.1-opam-core> -> required by _build/install/default/lib/opam-core/META
ocaml5.1.1-opam-core> -> required by _build/default/opam-core.install
ocaml5.1.1-opam-core> -> required by alias install

This is the dune file:

https://github.com/ocaml/opam/blob/a1c9c34417735687fd9310e7dc5c4c177e020441/src/core/dune

To Reproduce
Run nix build in this repo.

Expected behavior
It builds.

Environment

• NixOS 24.05
• Version of the code:

    "opam-nix": {
      "locked": {
        "lastModified": 1716292162,
        "narHash": "sha256-UOJNCbqvxABD56JZtZkv3C9ufdqrs7/Ep4AKkCHgPuo=",
        "owner": "tweag",
        "repo": "opam-nix",
        "rev": "1d3cbd6d3f247db77cb581c88c9a1d72e4acad60",
        "type": "github"
      },
  },

Full build logs:

ocaml5.1.1-opam-core> sourcing setup hook '/nix/store/fvp288mpbp1sk089mfcxkbd3rn288g8f-ocaml5.1.1-findlib-1.9.6/nix-support/setup-hook'
ocaml5.1.1-opam-core> sourcing setup hook '/nix/store/brfgny6xf2i3qfsfwnxchbgn9jxa6vzl-unzip-6.0/nix-support/setup-hook'
ocaml5.1.1-opam-core> sourcing setup hook '/nix/store/yq6n8b0mnk0qxzbs3ajsjcp8ziwqylrl-patchelf-0.15.0/nix-support/setup-hook'
ocaml5.1.1-opam-core> sourcing setup hook '/nix/store/iks1pihvbilsh5sy8qvpd638k422w9i8-update-autotools-gnu-config-scripts-hook/nix-support/setup-hook'
ocaml5.1.1-opam-core> sourcing setup hook '/nix/store/h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh'
ocaml5.1.1-opam-core> sourcing setup hook '/nix/store/m54bmrhj6fqz8nds5zcj97w9s9bckc9v-compress-man-pages.sh'
ocaml5.1.1-opam-core> sourcing setup hook '/nix/store/wgrbkkaldkrlrni33ccvm3b6vbxzb656-make-symlinks-relative.sh'
ocaml5.1.1-opam-core> sourcing setup hook '/nix/store/5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh'
ocaml5.1.1-opam-core> sourcing setup hook '/nix/store/fyaryjvghbkpfnsyw97hb3lyb37s1pd6-move-lib64.sh'
ocaml5.1.1-opam-core> sourcing setup hook '/nix/store/kd4xwxjpjxi71jkm6ka0np72if9rm3y0-move-sbin.sh'
ocaml5.1.1-opam-core> sourcing setup hook '/nix/store/pag6l61paj1dc9sv15l7bm5c17xn5kyk-move-systemd-user-units.sh'
ocaml5.1.1-opam-core> sourcing setup hook '/nix/store/jivxp510zxakaaic7qkrb7v1dd2rdbw9-multiple-outputs.sh'
ocaml5.1.1-opam-core> sourcing setup hook '/nix/store/ilaf1w22bxi6jsi45alhmvvdgy4ly3zs-patch-shebangs.sh'
ocaml5.1.1-opam-core> sourcing setup hook '/nix/store/cickvswrvann041nqxb0rxilc46svw1n-prune-libtool-files.sh'
ocaml5.1.1-opam-core> sourcing setup hook '/nix/store/xyff06pkhki3qy1ls77w10s0v79c9il0-reproducible-builds.sh'
ocaml5.1.1-opam-core> sourcing setup hook '/nix/store/ngg1cv31c8c7bcm2n8ww4g06nq7s4zhm-set-source-date-epoch-to-latest.sh'
ocaml5.1.1-opam-core> sourcing setup hook '/nix/store/gps9qrh99j7g02840wv5x78ykmz30byp-strip.sh'
ocaml5.1.1-opam-core> sourcing setup hook '/nix/store/kp2j7yn0wzwq5piy494r54dafrh83s6s-gcc-wrapper-13.3.0/nix-support/setup-hook'
ocaml5.1.1-opam-core> sourcing setup hook '/nix/store/9rz99vibv1782dxjkwvxr29s7f2f86wl-binutils-wrapper-2.42/nix-support/setup-hook'
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/rr9yfqhvlcpl0sdz9v1fxbfvbhlrllmx-ocaml-5.1.1
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/3mswiwwah2697vs00063cg3nxb5w3vds-dune-3.16.0
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/fvp288mpbp1sk089mfcxkbd3rn288g8f-ocaml5.1.1-findlib-1.9.6
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/brfgny6xf2i3qfsfwnxchbgn9jxa6vzl-unzip-6.0
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/c5fnp2k2z1wg708scr2q1hd5bk0x1ysi-ocaml5.1.1-cppo-1.6.9
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/yq6n8b0mnk0qxzbs3ajsjcp8ziwqylrl-patchelf-0.15.0
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/iks1pihvbilsh5sy8qvpd638k422w9i8-update-autotools-gnu-config-scripts-hook
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/m54bmrhj6fqz8nds5zcj97w9s9bckc9v-compress-man-pages.sh
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/wgrbkkaldkrlrni33ccvm3b6vbxzb656-make-symlinks-relative.sh
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/fyaryjvghbkpfnsyw97hb3lyb37s1pd6-move-lib64.sh
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/kd4xwxjpjxi71jkm6ka0np72if9rm3y0-move-sbin.sh
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/pag6l61paj1dc9sv15l7bm5c17xn5kyk-move-systemd-user-units.sh
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/jivxp510zxakaaic7qkrb7v1dd2rdbw9-multiple-outputs.sh
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/ilaf1w22bxi6jsi45alhmvvdgy4ly3zs-patch-shebangs.sh
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/cickvswrvann041nqxb0rxilc46svw1n-prune-libtool-files.sh
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/xyff06pkhki3qy1ls77w10s0v79c9il0-reproducible-builds.sh
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/ngg1cv31c8c7bcm2n8ww4g06nq7s4zhm-set-source-date-epoch-to-latest.sh
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/gps9qrh99j7g02840wv5x78ykmz30byp-strip.sh
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/kp2j7yn0wzwq5piy494r54dafrh83s6s-gcc-wrapper-13.3.0
ocaml5.1.1-opam-core> calling 'envBuildHostHook' function hook 'exportOcamlDestDir' /nix/store/9rz99vibv1782dxjkwvxr29s7f2f86wl-binutils-wrapper-2.42
ocaml5.1.1-opam-core> calling 'envHostTargetHook' function hook 'addOCamlPath' /nix/store/w57hgyh0xhxkvm811nrlaakbpfarbbhr-ocaml5.1.1-ocamlgraph-2.1.0
ocaml5.1.1-opam-core> calling 'envHostTargetHook' function hook 'ccWrapper_addCVars' /nix/store/w57hgyh0xhxkvm811nrlaakbpfarbbhr-ocaml5.1.1-ocamlgraph-2.1.0
ocaml5.1.1-opam-core> calling 'envHostTargetHook' function hook 'bintoolsWrapper_addLDVars' /nix/store/w57hgyh0xhxkvm811nrlaakbpfarbbhr-ocaml5.1.1-ocamlgraph-2.1.0
ocaml5.1.1-opam-core> calling 'envHostTargetHook' function hook 'addOCamlPath' /nix/store/ahggc6fmha5bfk7aw75ds7hclg8kkfrk-ocaml5.1.1-stdlib-shims-0.3.0
ocaml5.1.1-opam-core> calling 'envHostTargetHook' function hook 'ccWrapper_addCVars' /nix/store/ahggc6fmha5bfk7aw75ds7hclg8kkfrk-ocaml5.1.1-stdlib-shims-0.3.0
ocaml5.1.1-opam-core> calling 'envHostTargetHook' function hook 'bintoolsWrapper_addLDVars' /nix/store/ahggc6fmha5bfk7aw75ds7hclg8kkfrk-ocaml5.1.1-stdlib-shims-0.3.0
ocaml5.1.1-opam-core> calling 'envHostTargetHook' function hook 'addOCamlPath' /nix/store/mxc97j88z7is25v6dh1vgr3hrl63myzl-ocaml5.1.1-re-1.11.0
ocaml5.1.1-opam-core> calling 'envHostTargetHook' function hook 'ccWrapper_addCVars' /nix/store/mxc97j88z7is25v6dh1vgr3hrl63myzl-ocaml5.1.1-re-1.11.0
ocaml5.1.1-opam-core> calling 'envHostTargetHook' function hook 'bintoolsWrapper_addLDVars' /nix/store/mxc97j88z7is25v6dh1vgr3hrl63myzl-ocaml5.1.1-re-1.11.0
ocaml5.1.1-opam-core> calling 'envHostTargetHook' function hook 'addOCamlPath' /nix/store/pn5zxypfycg7p485s8z0zflh2c2l3jnc-ocaml5.1.1-seq-0.1
ocaml5.1.1-opam-core> calling 'envHostTargetHook' function hook 'ccWrapper_addCVars' /nix/store/pn5zxypfycg7p485s8z0zflh2c2l3jnc-ocaml5.1.1-seq-0.1
ocaml5.1.1-opam-core> calling 'envHostTargetHook' function hook 'bintoolsWrapper_addLDVars' /nix/store/pn5zxypfycg7p485s8z0zflh2c2l3jnc-ocaml5.1.1-seq-0.1
ocaml5.1.1-opam-core> calling 'postHook' function hook 'detectOcamlConflicts'
ocaml5.1.1-opam-core> Running phase: unpackPhase
ocaml5.1.1-opam-core> unpacking source archive /nix/store/02z7xakx0w5ay7jn1fjzkffcj2s0rcrz-opam-full-2.2.0-2.tar.gz
ocaml5.1.1-opam-core> calling 'unpackCmd' function hook '_tryUnzip' /nix/store/02z7xakx0w5ay7jn1fjzkffcj2s0rcrz-opam-full-2.2.0-2.tar.gz
ocaml5.1.1-opam-core> calling 'unpackCmd' function hook '_defaultUnpack' /nix/store/02z7xakx0w5ay7jn1fjzkffcj2s0rcrz-opam-full-2.2.0-2.tar.gz
ocaml5.1.1-opam-core> source root is opam-full-2.2.0
ocaml5.1.1-opam-core> calling 'postUnpack' function hook '_updateSourceDateEpochFromSourceRoot'
ocaml5.1.1-opam-core> setting SOURCE_DATE_EPOCH to timestamp 1720010379 of file opam-full-2.2.0/src_ext/archives/dune-3.14.2.tbz
ocaml5.1.1-opam-core> Running phase: patchPhase
ocaml5.1.1-opam-core> Running phase: updateAutotoolsGnuConfigScriptsPhase
ocaml5.1.1-opam-core> Running phase: configurePhase
ocaml5.1.1-opam-core> calling 'preConfigure' function hook '_multioutConfig'
ocaml5.1.1-opam-core> patching script interpreter paths in ./configure
ocaml5.1.1-opam-core> ./configure: interpreter directive changed from "#! /bin/sh" to "/nix/store/m101dg80ngyjdb02g6jwy80sr7kzj26g-bash-5.2p26/bin/sh"
ocaml5.1.1-opam-core> configure flags: --disable-static --prefix=/nix/store/lpfjkzdn9v946g3p39zjv62r2gav0v40-ocaml5.1.1-opam-core-2.2.0 --disable-checks
ocaml5.1.1-opam-core> checking for ocamlc... ocamlc
ocaml5.1.1-opam-core> OCaml version is 5.1.1
ocaml5.1.1-opam-core> OCaml library path is /nix/store/rr9yfqhvlcpl0sdz9v1fxbfvbhlrllmx-ocaml-5.1.1/lib/ocaml
ocaml5.1.1-opam-core> checking for ocamlopt... ocamlopt
ocaml5.1.1-opam-core> checking for ocamlc.opt... ocamlc.opt
ocaml5.1.1-opam-core> checking for ocamlopt.opt... ocamlopt.opt
ocaml5.1.1-opam-core> checking for ocaml... ocaml
ocaml5.1.1-opam-core> checking for ocamldep... ocamldep
ocaml5.1.1-opam-core> checking for ocamldep.opt... ocamldep.opt
ocaml5.1.1-opam-core> checking for ocamlmktop... ocamlmktop
ocaml5.1.1-opam-core> checking for ocamlmklib... ocamlmklib
ocaml5.1.1-opam-core> checking for ocamldoc... ocamldoc
ocaml5.1.1-opam-core> checking for ocamldoc.opt... ocamldoc.opt
ocaml5.1.1-opam-core> checking for ocamlbuild... no
ocaml5.1.1-opam-core> checking OCaml Sys.os_type... Unix
ocaml5.1.1-opam-core> checking for gawk... gawk
ocaml5.1.1-opam-core> checking for compiler type... cc
ocaml5.1.1-opam-core> checking for compiler architecture... amd64
ocaml5.1.1-opam-core> checking for compiler system... linux
ocaml5.1.1-opam-core> checking for compiler target... x86_64-unknown-linux-gnu
ocaml5.1.1-opam-core> checking whether the C compiler works... yes
ocaml5.1.1-opam-core> checking for C compiler default output file name... a.out
ocaml5.1.1-opam-core> checking for suffix of executables...
ocaml5.1.1-opam-core> checking whether we are cross compiling... no
ocaml5.1.1-opam-core> checking for suffix of object files... o
ocaml5.1.1-opam-core> checking whether the compiler supports GNU C... yes
ocaml5.1.1-opam-core> checking whether gcc accepts -g... yes
ocaml5.1.1-opam-core> checking for gcc option to enable C11 features... none needed
ocaml5.1.1-opam-core> checking for stdio.h... yes
ocaml5.1.1-opam-core> checking for stdlib.h... yes
ocaml5.1.1-opam-core> checking for string.h... yes
ocaml5.1.1-opam-core> checking for inttypes.h... yes
ocaml5.1.1-opam-core> checking for stdint.h... yes
ocaml5.1.1-opam-core> checking for strings.h... yes
ocaml5.1.1-opam-core> checking for sys/stat.h... yes
ocaml5.1.1-opam-core> checking for sys/types.h... yes
ocaml5.1.1-opam-core> checking for unistd.h... yes
ocaml5.1.1-opam-core> checking for acl/libacl.h... no
ocaml5.1.1-opam-core> checking for library containing acl_get_perm... no
ocaml5.1.1-opam-core> checking for ocamlobjinfo... ocamlobjinfo
ocaml5.1.1-opam-core> checking for ocamlfind... ocamlfind
ocaml5.1.1-opam-core> checking for curl... no
ocaml5.1.1-opam-core> checking for wget... no
ocaml5.1.1-opam-core> checking for dune... dune
ocaml5.1.1-opam-core> checking for patch... patch
ocaml5.1.1-opam-core> checking for bunzip2... bunzip2
ocaml5.1.1-opam-core> checking for linking method... shared
ocaml5.1.1-opam-core> checking for OCaml findlib package unix... found
ocaml5.1.1-opam-core> checking for OCaml findlib package re 1.9.0 or later... found 1.11.0
ocaml5.1.1-opam-core> checking for OCaml findlib package base64 3.1.0 or later... no
ocaml5.1.1-opam-core> checking for OCaml findlib package cmdliner... not found
ocaml5.1.1-opam-core> checking for OCaml findlib package ocamlgraph... found
ocaml5.1.1-opam-core> checking for OCaml findlib package cudf 0.7 or later... no
ocaml5.1.1-opam-core> checking for OCaml findlib package dose3.common 6.1 or later... no
ocaml5.1.1-opam-core> checking for OCaml findlib package dose3.algo 6.1 or later... no
ocaml5.1.1-opam-core> checking for OCaml findlib package opam-file-format 2.1.4 or later... no
ocaml5.1.1-opam-core> checking for OCaml findlib package spdx_licenses... not found
ocaml5.1.1-opam-core> checking for OCaml findlib package opam-0install-cudf 0.4 or later... no
ocaml5.1.1-opam-core> checking for OCaml findlib package jsonm... not found
ocaml5.1.1-opam-core> checking for OCaml findlib package uutf... not found
ocaml5.1.1-opam-core> checking for OCaml findlib package sha... not found
ocaml5.1.1-opam-core> checking for OCaml findlib package swhid_core... not found
ocaml5.1.1-opam-core> checking for OCaml findlib package mccs 1.1+17 or later... no
ocaml5.1.1-opam-core> checking whether the compiler supports GNU C++... yes
ocaml5.1.1-opam-core> checking whether g++ accepts -g... yes
ocaml5.1.1-opam-core> checking for g++ option to enable C++11 features... none needed
ocaml5.1.1-opam-core> checking whether g++ really is a C++ compiler... yes
ocaml5.1.1-opam-core> configure: creating ./config.status
ocaml5.1.1-opam-core> config.status: creating Makefile.config
ocaml5.1.1-opam-core> config.status: creating src/ocaml-flags-configure.sexp
ocaml5.1.1-opam-core> config.status: creating src/stubs/c-flags.sexp
ocaml5.1.1-opam-core> config.status: creating src/stubs/libacl/c-libraries.sexp
ocaml5.1.1-opam-core> Opam will be built WITH its default built-in solver
ocaml5.1.1-opam-core> Executables will be installed in /nix/store/lpfjkzdn9v946g3p39zjv62r2gav0v40-ocaml5.1.1-opam-core-2.2.0/bin
ocaml5.1.1-opam-core> Manual pages will be installed in /nix/store/lpfjkzdn9v946g3p39zjv62r2gav0v40-ocaml5.1.1-opam-core-2.2.0/share/man
ocaml5.1.1-opam-core> Running phase: buildPhase
ocaml5.1.1-opam-core> File "src/core/dune", line 6, characters 67-71:
ocaml5.1.1-opam-core> 6 |   (libraries   re (re_export ocamlgraph) unix sha jsonm swhid_core uutf
ocaml5.1.1-opam-core>                                                                        ^^^^
ocaml5.1.1-opam-core> Error: Library "uutf" not found.
ocaml5.1.1-opam-core> -> required by library "opam-core" in _build/default/src/core
ocaml5.1.1-opam-core> -> required by _build/default/META.opam-core
ocaml5.1.1-opam-core> -> required by _build/install/default/lib/opam-core/META
ocaml5.1.1-opam-core> -> required by _build/default/opam-core.install
ocaml5.1.1-opam-core> -> required by alias install

pin-depends fails to parse tags

Describe the bug
When dealing with a pin-depend URL with a tag, such as git+https://github.com/foo/bar#0.1.0, opam-nix parses the tag as a SHA1:

error: hash '0.1.0' has wrong length for hash type 'sha1'

To Reproduce
Here's my flake.nix:

{
  description = "Experimental implementation of Cartesian cubical type theory";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-22.05";
    flake-utils.url = "github:numtide/flake-utils";

    opam-repository = {
      url = "github:ocaml/opam-repository";
      flake = false;
    };
    
    opam-nix = {
      url = "github:tweag/opam-nix";
      inputs.nixpkgs.follows = "nixpkgs";
      inputs.flake-utils.follows = "flake-utils";
      inputs.opam-repository.follows = "opam-repository";
    };
  };

  outputs = { self
            , flake-utils
            , opam-nix
            , nixpkgs
            , opam-repository
            }@inputs:
    flake-utils.lib.eachDefaultSystem (system: {
      legacyPackages = let
        on = opam-nix.lib.${system};
      in on.buildDuneProject { } "cooltt" ./. {
        ocaml-base-compiler = null;
      };

      defaultPackage = self.legacyPackages.${system}."cooltt";
    });
}

and cooltt.opam:

opam-version: "2.0"
name: "cooltt"
version: "0.0"
maintainer: "Jonathan Sterling <[email protected]>"
authors: ["The RedPRL Development Team"]
homepage: "https://github.com/RedPRL/cooltt"
bug-reports: "https://github.com/RedPRL/cooltt/issues"
dev-repo: "git+https://github.com/RedPRL/cooltt.git"
synopsis: "Experimental implementation of Cartesian cubical type theory"
license: "Apache-2.0"
depends: [
  "dune" {>= "2.0"}
  "ocaml" {>= "4.10.0"}
  "ppx_deriving" {>= "4.4.1"}
  "bantorra"
  "bwd" {>= "1.2"}
  "cmdliner" {>= "1.1"}
  "containers" {>= "3.4"}
  "ezjsonm" {>= "1.2.0"}
  "menhir" {>= "20180703"}
  "uuseg" {>= "12.0.0"}
  "uutf" {>= "1.0.2"}
  "yuujinchou" {>= "2.0.0" & < "3"}
  "odoc" {with-doc}
  "kado"
]
pin-depends: [
  [ "kado.~dev" "git+https://github.com/RedPRL/kado" ]
  [ "bantorra.0.1.0" "git+https://github.com/RedPRL/bantorra#0.1.0" ]
]
build: [
  ["dune" "build" "-p" name "-j" jobs]
  ["dune" "build" "-p" name "-j" jobs "@runtest"] {with-test}
  ["dune" "build" "-p" name "-j" jobs "@doc"] {with-doc}
]

Expected behavior
It should pull from the tag.

Environment

• OS name + version: NixOS 22.05
• Version of the code: Latest as of report

Why dontPatchShebangsEarly?

This is more of a question than an issue but why does opam-nix use dontPatchShebangsEarly instead of a more common name dontPatchShebangs?

As part of our build process we run dune subst which will invoke git to obtain the current revision, setting dontPatchShebangsEarly is important to avoid the -dirty suffix in the version. I spent a few hours debugging this until realised that the builder in opam-nix does not use dontPatchShebangs.

unmet availability conditions: 'enable-ocaml-beta-repository'

I maintain the flake for this project:

https://git.sr.ht/~jonsterling/ocaml-forester

It stopped working after some packages were updated, and I haven't been able to figure out how to fix it:

resolve> [ERROR] No solution including optional dependencies for forester.3.0.0 & ocaml-system:   * Incompatible packages:
resolve>             - forester >= 3.0.0 -> eio_main >= 0.13 -> eio_linux >= 0.13 -> eio >= 0.13 -> ocaml >= 5.1.0 -> ocaml-base-compiler >= 5.1.0~
resolve>             - ocaml-system
resolve>           * Missing dependency:
resolve>             - forester >= 3.0.0 -> eio_main >= 0.13 -> eio_linux >= 0.13 -> eio >= 0.13 -> ocaml >= 5.1.0 -> ocaml-variants < 5.1.1~ -> ocaml-beta
resolve>             unmet availability conditions: 'enable-ocaml-beta-repository'

To Reproduce

nix run sourcehut:~jonsterling/ocaml-forester for the original breakage.
After cloning and updating the flake, nix build -L results in the same error

Expected behavior
It builds

Environment

• 24.05.20231222.6df37dc (Uakari)
• Version of the code: (Do you mean version of opam-nix? latest)

ocamlfind: Hunk #1 FAILED on findlib 1.9.5

Describe the bug
When trying to build ocamlfind as part of an opam-nix derivation, the following error arises:

@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/jdhi5amjwlp8ymf0c7f342ax42b6f8s4-findlib-1.9.5.tar.gz
source root is findlib-1.9.5
setting SOURCE_DATE_EPOCH to timestamp 1655063092 of file findlib-1.9.5/src/findlib/topfind_rd1.p
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
applying patch /nix/store/xcdgajkwqw7jfqyll1pqjf53610nqafs-ldconf.patch
patching file findlib.conf.in
applying patch /nix/store/xs144bvl5yy4530n1aa4dn0grg0jjj3l-install_topfind.patch
patching file src/findlib/Makefile
Hunk #1 FAILED at 123.
1 out of 1 hunk FAILED -- saving rejects to file src/findlib/Makefile.rej

I'm not entirely sure where these patches are coming from. Running nix shell nixpkgs#ocamlPackages.findlib works fine and puts ocamlfind in PATH, so presumably it has something to do with an issue here?

The issue is probably to do with the fact that I'm pulling findlib 1.9.5, not 1.9.3. The reason I have a different opam-repository as an input is because bwd isn't in the outdated version in this repo's flake.lock.

To Reproduce
Here's my flake.nix:

{
  description = "Experimental implementation of Cartesian cubical type theory";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";

    opam-repository = {
      url = "github:ocaml/opam-repository";
      flake = false;
    };
    
    opam-nix = {
      url = "github:tweag/opam-nix";
      inputs.nixpkgs.follows = "nixpkgs";
      inputs.flake-utils.follows = "flake-utils";
      inputs.opam-repository.follows = "opam-repository";
    };

    kado = {
      url = "github:RedPRL/kado";
      flake = false;
    };
    bantorra = {
      url = "github:RedPRL/bantorra/0.1.0";
      flake = false;
    };
  };

  outputs = { self, flake-utils, opam-nix, nixpkgs, kado, bantorra, opam-repository }@inputs:
    flake-utils.lib.eachDefaultSystem (system: {
      legacyPackages = let
        pkgs = nixpkgs.legacyPackages.${system};
        on = opam-nix.lib.${system};
        scope = on.buildDuneProject {
          repos = [
            (on.makeOpamRepo kado)
            (on.makeOpamRepo bantorra)
            opam-nix.inputs.opam-repository
          ];
        } "cooltt" ./. { 
          ocaml-base-compiler = null;
        };
        overlay = self: super:
          {
            # Your overrides go here
          };
      in scope.overrideScope' overlay;

      defaultPackage = self.legacyPackages.${system}."cooltt";
    });
}

and cooltt.opam:

opam-version: "2.0"
name: "cooltt"
version: "0.0"
maintainer: "Jonathan Sterling <[email protected]>"
authors: ["The RedPRL Development Team"]
homepage: "https://github.com/RedPRL/cooltt"
bug-reports: "https://github.com/RedPRL/cooltt/issues"
dev-repo: "git+https://github.com/RedPRL/cooltt.git"
synopsis: "Experimental implementation of Cartesian cubical type theory"
license: "Apache-2.0"
depends: [
  "dune" {>= "2.0"}
  "ocaml" {>= "4.10.0"}
  "ppx_deriving" {>= "4.4.1"}
  "bantorra"
  "bwd" {>= "1.2"}
  "cmdliner" {>= "1.1"}
  "containers" {>= "3.4"}
  "ezjsonm" {>= "1.2.0"}
  "menhir" {>= "20180703"}
  "uuseg" {>= "12.0.0"}
  "uutf" {>= "1.0.2"}
  "yuujinchou" {>= "2.0.0" & < "3"}
  "odoc" {with-doc}
  "kado"
]
pin-depends: [
  [ "kado.~dev" "git+https://github.com/RedPRL/kado" ]
  [ "bantorra.0.1.0" "git+https://github.com/RedPRL/bantorra#1e78633d9a2ef7104552a24585bb8bea36d4117b" ]
]
build: [
  ["dune" "build" "-p" name "-j" jobs]
  ["dune" "build" "-p" name "-j" jobs "@runtest"] {with-test}
  ["dune" "build" "-p" name "-j" jobs "@doc"] {with-doc}
]

The rest of the code for the actual project is as in https://github.com/RedPRL/cooltt.

Expected behavior
ocamlfind should build and install successfully.

Environment

• OS name + version: NixOS 22.05
• Version of the code: 23c55a8

Additional context
Add any other context about the problem here.

Fake `opam` support of the `exec` subcommand

Is your feature request related to a problem? Please describe.

I am contributing on an OPAM-based OCaml project with both Nix and non-Nix people. I setup opam-nix for my convenience, but not everyone goes through it to work on the project in question. Now, for non-Nix OPAM-based scripts, it is common to prefix commands by opam exec -- to make sure that they are run in the correct environment. Because the fake opam command does not support exec, though, those scripts do not work for me. Depending on the version of opam-nix, I get either:

$ opam exec -- echo foo
2.0.0

or

$ opam exec -- echo foo
opam-nix fake opam doesn't understand these arguments: exec -- echo foo

Describe the solution you'd like

I would like for the fake opam command to recognize the exec subcommand. Considering that the fake opam only exists when in a Nix context, then opam exec should basically be a no-op running the given command (eg. opam exec - echo foo just runs echo foo).

Describe alternatives you've considered

For now, I am just stripping all uses of opam exec from scripts before running anything. This is doable and not very difficult. Alternatively, I could provide yet another fake opam in my Nix flake, taking precedence over opam-nix's, and recognizing exec. I feel the best would still be for it to be supported in opam-nix.

`eio-posix-1.1` fails to build on macOS ventura 13.6.7

error: builder for '/nix/store/zpssmmd2cifm4xlsyksnx04vmskx0iwi-eio_posix-1.1.drv' failed with exit code 1;
       last 20 log lines:
       > Running phase: unpackPhase
       > unpacking source archive /nix/store/2giwhf2h39bchbz2h8gmyydk0xzyqivh-eio-1.1.tbz
       > source root is eio-1.1
       > setting SOURCE_DATE_EPOCH to timestamp 1716894397 of file eio-1.1/tests/trace.md
       > Running phase: patchPhase
       > Running phase: configurePhase
       > patching script interpreter paths in .
       > Running phase: buildPhase
       > File "lib_eio_posix/dune", line 9, characters 9-24:
       > 9 |   (names eio_posix_stubs))
       >              ^^^^^^^^^^^^^^^
       > (cd _build/default/lib_eio_posix && /nix/store/iyk4jyp5aqmrp6mq5mlvirjdijhrdbna-clang-wrapper-16.0.6/bin/cc -O2 -fno-strict-aliasing -fwrapv -pthread -D_FILE_OFFSET_BITS=64 -fdiagnostics-color=always -D_LARGEFILE64_SOURCE -g -I /nix/store/6sc7asvwvwqicyczhsiyhhaqgkwd81a3-ocaml-base-compiler-5.1.1/lib/ocaml -I /nix/store/6sc7asvwvwqicyczhsiyhhaqgkwd81a3-ocaml-base-compiler-5.1.1/lib/ocaml/runtime_events -I /nix/store/6sc7asvwvwqicyczhsiyhhaqgkwd81a3-ocaml-base-compiler-5.1.1/lib/ocaml/threads -I /nix/store/6sc7asvwvwqicyczhsiyhhaqgkwd81a3-ocaml-base-compiler-5.1.1/lib/ocaml/unix -I /nix/store/6z4nynxml3hm47rjlrivncpbyp75rwcc-bigstringaf-0.9.1/lib/ocaml/5.1.1/site-lib/bigstringaf -I /nix/store/77pxgjv7ndrhm11wij2pwmnw1l37cl68-iomux-0.3/lib/ocaml/5.1.1/site-lib/iomux -I /nix/store/9nhfnqlcx5wc4749whcpjjfgh9ljip07-optint-0.3.0/lib/ocaml/5.1.1/site-lib/optint -I /nix/store/f9y9cimmydclqbvh5nsym0n329nn9cmv-lwt-dllist-1.0.1/lib/ocaml/5.1.1/site-lib/lwt-dllist -I /nix/store/fbd6i566p0s915i4c4p16sa2pdnpfkk3-seq-base/lib/ocaml/5.1.1/site-lib/seq -I /nix/store/h7rl940vdrv7ghz5f7iq2mv5b7g3ypkg-eio-1.1/lib/ocaml/5.1.1/site-lib/eio -I /nix/store/h7rl940vdrv7ghz5f7iq2mv5b7g3ypkg-eio-1.1/lib/ocaml/5.1.1/site-lib/eio/core -I /nix/store/h7rl940vdrv7ghz5f7iq2mv5b7g3ypkg-eio-1.1/lib/ocaml/5.1.1/site-lib/eio/runtime_events -I /nix/store/h7rl940vdrv7ghz5f7iq2mv5b7g3ypkg-eio-1.1/lib/ocaml/5.1.1/site-lib/eio/unix -I /nix/store/h7rl940vdrv7ghz5f7iq2mv5b7g3ypkg-eio-1.1/lib/ocaml/5.1.1/site-lib/eio/utils -I /nix/store/hylnp4rnal9mfk80lgyyk24hgi7a861v-thread-table-1.0.0/lib/ocaml/5.1.1/site-lib/thread-table -I /nix/store/jh6j1kg1vk1w4pxm18ljgkfc1zjfpp5x-mtime-2.0.0/lib/ocaml/5.1.1/site-lib/mtime -I /nix/store/jh6j1kg1vk1w4pxm18ljgkfc1zjfpp5x-mtime-2.0.0/lib/ocaml/5.1.1/site-lib/mtime/clock/os -I /nix/store/l8g8g5mxgvwbn7rxcgw0l1qv26rv464a-cstruct-6.2.0/lib/ocaml/5.1.1/site-lib/cstruct -I /nix/store/mj0ri949k37ar70192vzghhdfyf3r520-psq-0.2.1/lib/ocaml/5.1.1/site-lib/psq -I /nix/store/qikgfx8jqhhnyih7k3qhka1hn3vm9scp-fmt-0.9.0/lib/ocaml/5.1.1/site-lib/fmt -I /nix/store/rwnkfah6ckvg65sizrvks7m7v8azy5jj-hmap-0.8.1/lib/ocaml/5.1.1/site-lib/hmap -I /nix/store/ynq5inhfj63d1ds9ifhdsi1297ndllzz-domain-local-await-1.0.1/lib/ocaml/5.1.1/site-lib/domain-local-await -I ../lib_eio/unix/include -o eio_posix_stubs.o -c eio_posix_stubs.c)
       > eio_posix_stubs.c:133:7: error: call to undeclared function 'preadv'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
       >   r = preadv(Int_val(v_fd), iov, n_bufs, offset);
       >       ^
       > eio_posix_stubs.c:150:7: error: call to undeclared function 'pwritev'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
       >   r = pwritev(Int_val(v_fd), iov, n_bufs, offset);
       >       ^
       > 2 errors generated.
       >
       For full logs, run 'nix-store -l /nix/store/zpssmmd2cifm4xlsyksnx04vmskx0iwi-eio_posix-1.1.drv'.
error: 1 dependencies of derivation '/nix/store/36z756dqwgjgk4nsbrlfdzxnpbnz94yg-eio_main-1.1.drv' failed to build
error: 1 dependencies of derivation '/nix/store/sfyqcshn69716iaij95zlmi92h41yfb2-forester-4.1.0.drv' failed to build

To Reproduce
Run nix build on https://git.sr.ht/~jonsterling/ocaml-forester/commit/e9bd3a5a784e8fe1a34928d85ef0188e3ea41947

Expected behavior
Build succeeds

Environment

• macOS Ventura 13.6.7
• Version of the code:

   "opam-nix": {
     "locked": {
       "lastModified": 1716292162,
       "narHash": "sha256-UOJNCbqvxABD56JZtZkv3C9ufdqrs7/Ep4AKkCHgPuo=",
       "owner": "tweag",
       "repo": "opam-nix",
       "rev": "1d3cbd6d3f247db77cb581c88c9a1d72e4acad60",
       "type": "github"
     },
...
   },

Build does succeed on at least macOS Sonoma 14.5

Version information?

Is your feature request related to a problem? Please describe.

When building a package from a git repository using opam-nix, the Build_info.V1.version () from dune-build-info is always None.
Moreover, dune subst does not seem to be properly run, because I still get the %%VERSION_NUM%% placeholder in the resulting binary (even though the .opam file specifies dune subst).

Describe the solution you'd like

Build_info.V1.version () should return the appropriate version, and the dune subst placeholders should be replaced with the appropriate values when building using opam-nix.

Note: if I am missing something and this should work, please do tell — I am fairly new to nix and opam-nix and just used the auto-generated flake.nix.

See this repo: bclement-ocp/opam-nix-dune-build-info

Building with opam:

$ opam install .
# ...
$ opam-nix-dune-build-info 
dune-build-info: ddfc70d
dune subst: ddfc70d

Buliding with opam-nix:

$ nix build
# ...
$ ./result/bin/opam-nix-dune-build-info 
dune-build-info: <no version>
dune subst: %%VERSION_NUM%%

Materialization does not work with local projects and pin-depends.

Description
Currently materialize and materializedDefsToScope do not support working with local projects and thus cannot capture pinned dependencies. It would be great if there was a way to get a fully "locked" version of the project with resolved package versions and necessary URLs to be later loaded and built.

Expected behavior
Ideally, a common workflow present in other tools as mentioned by @brendanzab, would fit this use case. Specifically:

• lock:
  • load project's opam file(s);
  • solve dependency versions (including pinned packages);
  • create a lock file with the solution containing URLs and hashes;
• build:
  • fetch sources included in the lock file;
  • build the dependencies and the project.

I realise this is potentially significantly different to what opam-nix does presently. Could you share some details on the rationale behind the current design? I'd be happy to discuss alternative approaches, if you think this doesn't make sense.

An interesting project to look at for inspiration is opam-monorepo, which generates reproducible lock-files like: https://github.com/ocamllabs/opam-monorepo/blob/main/opam-monorepo.opam.locked.

Adding dependences to dune project

Describe the bug
This might just be me being absolutely new to the OCaml scene, but I don't seem to be able to add new dependences to a dune project. For reference, I'm following the official Getting Started guide.

To Reproduce
On a machine that has never seen any Ocaml software, I did the following

mkdir ocamltest
cd ocamltest
git init
nix flake init -t github:tweag/opam-nix#multi-package
git add .
cd ..
dune init proj ocamltest
cd ocamltest
git add .
dune build

Then I edited bin/dune to include dream as a dependency.

(executable
 (public_name ocamltest)
 (name main)
 (libraries ocamltest dream))

and finally

git add bin/dune
exit
nix develop

This puts me in an environment that does not have dream available (dune build fails).

Expected behavior
By reading this intro I was expecting the flake to be able to figure out the dependencies by looking at the project.
What am I doing wrong? Did I miss something?

Environment
x86_64 GNU/Linux NixOS

Opam-Nix not providing the latest library versions from opam-repository

Describe the bug
I can see in opam-repository that bwd-2.2 and yuujinchou-5.0.2 are available:

https://github.com/ocaml/opam-repository/blob/master/packages/bwd/bwd.2.2.0/opam
https://github.com/ocaml/opam-repository/blob/master/packages/yuujinchou/yuujinchou.5.0.2/opam

However Opam-nix is giving me bwd-2.1 and yuujinchou-4.0.2.

To Reproduce

➜ nix shell github:tweag/opam-nix#bwd.latest       

➜ echo $PATH
/nix/store/gk1f1i3hba3cgbxmpg9i144qbmszw9lc-bwd-2.1.0/bin:/run/wrappers/bin:/home/solomon/.nix-profile/bin:/etc/profiles/per-user/solomon/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin:/home/solomon/.local/scripts:/home/solomon/.local/bin:/home/solomon/.cabal/bin:/home/solomon/.zsh/plugins/zsh-nix-shell

➜ nix shell github:tweag/opam-nix#yuujinchou.latest

➜ echo $PATH
/nix/store/9di530vv5hwafixg94gan84xgqimiwvx-yuujinchou-4.0.0/bin:/nix/store/gk1f1i3hba3cgbxmpg9i144qbmszw9lc-bwd-2.1.0/bin:/run/wrappers/bin:/home/solomon/.nix-profile/bin:/etc/profiles/per-user/solomon/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin:/home/solomon/.local/scripts:/home/solomon/.local/bin:/home/solomon/.cabal/bin:/home/solomon/.zsh/plugins/zsh-nix-shell

➜ nix shell github:tweag/opam-nix#yuujinchou.5.0.2
error: flake 'github:tweag/opam-nix' does not provide attribute 'packages.x86_64-linux.yuujinchou.5.0.2', 'legacyPackages.x86_64-linux.yuujinchou.5.0.2' or 'yuujinchou.5.0.2'

Expected behavior
I would expect the latest version from opam-repository to be provided.

Environment

• Nixos 23.05

Handle ocaml stublibs properly

Is your feature request related to a problem? Please describe.

Currently, opam-nix installs ocaml stublibs to $OCAMLFIND_DESTDIR/${pname} together with the rest of the libraries.
However, ocaml compiler expects them to be present in $OCAMLFIND_DESTDIR/stublibs directory.

This is the case for e.g. zarith library.

Describe the solution you'd like

The following postInstall workaround seems to work:

postInstall = ''
    mkdir -p "$OCAMLFIND_DESTDIR/stublibs"
    ln -s "$OCAMLFIND_DESTDIR"/${o.pname}/dll*.so "$OCAMLFIND_DESTDIR"/stublibs/
  '';

Ideally, opam-nix should handle this automatically. Maybe there is a way to specify a place to looks for stublibs.

Support `.locked` opam files

Support an alternative workflow where pindepends are picked up from an opam lockfile instead of using the Opam solver with opam admin list.

error: attribute 'mypackage' missing

Describe the bug
A clear and concise description of what the bug is.

I am unable to run nix build, it fails with the following:

error: attribute 'mypackage' missing

       at /nix/store/xf0glwam0jk3yym888y1c7ny4f92j8lk-source/src/opam.nix:494:23:

          493|       pinDeps =
          494|         getPinDepends repo.passthru.pkgdefs.${name}.${latestVersions.${name}};
             |                       ^
          495|     in queryToScope {

       … while evaluating the attribute 'passthru.pkgdefs.mypackage."${(latestVersions)."${name}"}"'

       at /nix/store/xf0glwam0jk3yym888y1c7ny4f92j8lk-source/src/opam.nix:259:38:

          258|       repo = linkFarm "opam-repo" ([ repo-description ] ++ opamFileLinks);
          259|     in repo // { passthru = { inherit sourceMap pkgdefs; }; };
             |                                      ^
          260|

       … while evaluating 'getPinDepends'

       at /nix/store/xf0glwam0jk3yym888y1c7ny4f92j8lk-source/src/opam.nix:475:19:

          474|
          475|   getPinDepends = pkgdef:
             |                   ^
          476|     if pkgdef ? pin-depends then

       … from call site

       at /nix/store/xf0glwam0jk3yym888y1c7ny4f92j8lk-source/src/opam.nix:494:9:

          493|       pinDeps =
          494|         getPinDepends repo.passthru.pkgdefs.${name}.${latestVersions.${name}};
             |         ^
          495|     in queryToScope {

       … while evaluating 'optionals'

       at /nix/store/q3l9x3ncgi0hppw0i65ki9jrxj8kmgac-source/lib/lists.nix:271:5:

          270|     # List to return if condition is true
          271|     elems: if cond then elems else [];
             |     ^
          272|

       … from call site

       at /nix/store/xf0glwam0jk3yym888y1c7ny4f92j8lk-source/src/opam.nix:496:27:

          495|     in queryToScope {
          496|       repos = [ repo ] ++ optionals pinDepends pinDeps ++ repos;
             |                           ^
          497|       overlays = overlays;

       … while evaluating 'joinRepos'

       at /nix/store/xf0glwam0jk3yym888y1c7ny4f92j8lk-source/src/opam.nix:372:15:

          371|
          372|   joinRepos = repos:
             |               ^
          373|     if length repos == 0 then

       … from call site

       at /nix/store/xf0glwam0jk3yym888y1c7ny4f92j8lk-source/src/opam.nix:455:18:

          454|     pipe query [
          455|       (opamList (joinRepos repos) resolveArgs)
             |                  ^
          456|       (opamListToQuery)

       … while evaluating the attribute 'buildCommand' of the derivation 'resolve'

       at /nix/store/q3l9x3ncgi0hppw0i65ki9jrxj8kmgac-source/pkgs/stdenv/generic/make-derivation.nix:302:7:

          301|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          302|       name =
             |       ^
          303|         let

       … while realising the context of a path

       at /nix/store/q3l9x3ncgi0hppw0i65ki9jrxj8kmgac-source/lib/strings.nix:1076:43:

         1075|   */
         1076|   fileContents = file: removeSuffix "\n" (readFile file);
             |                                           ^
         1077|

       … while evaluating anonymous lambda

       at /nix/store/q3l9x3ncgi0hppw0i65ki9jrxj8kmgac-source/lib/trivial.nix:357:50:

          356|   */
          357|   warnIf = cond: msg: if cond then warn msg else x: x;
             |                                                  ^
          358|

       … from call site

       at /nix/store/q3l9x3ncgi0hppw0i65ki9jrxj8kmgac-source/lib/strings.nix:654:5:

          653|     # to strings and comparing. This was surprising and confusing.
          654|     warnIf
             |     ^
          655|       (isPath suffix)

       … while evaluating 'removeSuffix'

       at /nix/store/q3l9x3ncgi0hppw0i65ki9jrxj8kmgac-source/lib/strings.nix:651:5:

          650|     # Input string
          651|     str:
             |     ^
          652|     # Before 23.05, paths would be copied to the store before converting them

       … from call site

       at /nix/store/q3l9x3ncgi0hppw0i65ki9jrxj8kmgac-source/lib/strings.nix:1076:24:

         1075|   */
         1076|   fileContents = file: removeSuffix "\n" (readFile file);
             |                        ^
         1077|

       … while evaluating 'fileContents'

       at /nix/store/q3l9x3ncgi0hppw0i65ki9jrxj8kmgac-source/lib/strings.nix:1076:18:

         1075|   */
         1076|   fileContents = file: removeSuffix "\n" (readFile file);
             |                  ^
         1077|

       … from call site

       at /nix/store/xf0glwam0jk3yym888y1c7ny4f92j8lk-source/src/opam.nix:201:18:

          200|         '';
          201|       solution = fileContents resolve-drv;
             |                  ^
          202|

       … while evaluating 'splitString'

       at /nix/store/q3l9x3ncgi0hppw0i65ki9jrxj8kmgac-source/lib/strings.nix:598:22:

          597|   */
          598|   splitString = sep: s:
             |                      ^
          599|     let

       … from call site

       at /nix/store/xf0glwam0jk3yym888y1c7ny4f92j8lk-source/src/opam.nix:203:18:

          202|
          203|       lines = s: splitString "\n" s;
             |                  ^
          204|

       … while evaluating 'lines'

       at /nix/store/xf0glwam0jk3yym888y1c7ny4f92j8lk-source/src/opam.nix:203:15:

          202|
          203|       lines = s: splitString "\n" s;
             |               ^
          204|

       … from call site

       at /nix/store/xf0glwam0jk3yym888y1c7ny4f92j8lk-source/src/opam.nix:205:8:

          204|
          205|     in lines solution;
             |        ^
          206|

       … while evaluating 'opamList'

       at /nix/store/xf0glwam0jk3yym888y1c7ny4f92j8lk-source/src/opam.nix:159:33:

          158|
          159|   opamList = repo: resolveArgs: packages:
             |                                 ^
          160|     let

       … from call site

       at /nix/store/q3l9x3ncgi0hppw0i65ki9jrxj8kmgac-source/lib/trivial.nix:62:30:

           61|   pipe = val: functions:
           62|     let reverseApply = x: f: f x;
             |                              ^
           63|     in builtins.foldl' reverseApply val functions;

       … while evaluating 'reverseApply'

       at /nix/store/q3l9x3ncgi0hppw0i65ki9jrxj8kmgac-source/lib/trivial.nix:62:27:

           61|   pipe = val: functions:
           62|     let reverseApply = x: f: f x;
             |                           ^
           63|     in builtins.foldl' reverseApply val functions;

       … from call site

       at /nix/store/q3l9x3ncgi0hppw0i65ki9jrxj8kmgac-source/lib/trivial.nix:63:8:

           62|     let reverseApply = x: f: f x;
           63|     in builtins.foldl' reverseApply val functions;
             |        ^
           64|

       … while evaluating 'pipe'

       at /nix/store/q3l9x3ncgi0hppw0i65ki9jrxj8kmgac-source/lib/trivial.nix:61:15:

           60|   */
           61|   pipe = val: functions:
             |               ^
           62|     let reverseApply = x: f: f x;

       … from call site

       at /nix/store/xf0glwam0jk3yym888y1c7ny4f92j8lk-source/src/opam.nix:454:5:

          453|     query:
          454|     pipe query [
             |     ^
          455|       (opamList (joinRepos repos) resolveArgs)

       … while evaluating 'queryToScope'

       at /nix/store/xf0glwam0jk3yym888y1c7ny4f92j8lk-source/src/opam.nix:453:5:

          452|     , overlays ? __overlays, resolveArgs ? { } }:
          453|     query:
             |     ^
          454|     pipe query [

       … from call site

       at /nix/store/xf0glwam0jk3yym888y1c7ny4f92j8lk-source/src/opam.nix:495:8:

          494|         getPinDepends repo.passthru.pkgdefs.${name}.${latestVersions.${name}};
          495|     in queryToScope {
             |        ^
          496|       repos = [ repo ] ++ optionals pinDepends pinDeps ++ repos;

       … while evaluating 'buildOpamProject'

       at /nix/store/xf0glwam0jk3yym888y1c7ny4f92j8lk-source/src/opam.nix:488:20:

          487|     , recursive ? false }@args:
          488|     name: project: query:
             |                    ^
          489|     let

       … from call site

       at /nix/store/8agzjbynw7bb9n030radf916a6xkyhxv-source/flake.nix:15:11:

           14|         scope =
           15|           on.buildOpamProject { } package ./. { ocaml-base-compiler = "*"; };
             |           ^
           16|         overlay = final: prev:

       … while evaluating the attribute 'legacyPackages'

       at /nix/store/8agzjbynw7bb9n030radf916a6xkyhxv-source/flake.nix:21:9:

           20|       in {
           21|         legacyPackages = scope.overrideScope' overlay;
             |         ^
           22|

       … while evaluating the attribute 'legacyPackages.x86_64-linux.mypackage'

       at /nix/store/pgid9c9xfcrbqx2giry0an0bi0df7s5c-source/lib.nix:40:24:

           39|                 ${key} = (attrs.${key} or { })
           40|                   // { ${system} = ret.${key}; };
             |                        ^
           41|               }

To Reproduce
Steps to reproduce the behavior.

nix flake init -t github:tweag/opam-nix

Change package name and then you get this error

Environment

• OS name + version: void linux

Additional context

Full config

{
  inputs = {
    opam-nix.url = "github:tweag/opam-nix";
    flake-utils.url = "github:numtide/flake-utils";
    nixpkgs.follows = "opam-nix/nixpkgs";
  };
  outputs = { self, flake-utils, opam-nix, nixpkgs }@inputs:
    # Don't forget to put the package name instead of `throw':
    let package = "mypackage";
    in flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = nixpkgs.legacyPackages.${system};
        on = opam-nix.lib.${system};
        scope =
          on.buildOpamProject { } package ./. { ocaml-base-compiler = "*"; };
        overlay = final: prev:
          {
            # Your overrides go here
          };
      in {
        legacyPackages = scope.overrideScope' overlay;

        packages.default = self.legacyPackages.${system}.${package};
      });
}

My opam file

# This file is generated by dune, edit dune-project instead
opam-version: "2.0"
synopsis: "A short synopsis"
description: "A longer description"
maintainer: ["Marc Coquand"]
authors: ["Marc Coquand"]
license: "AGPL-3.0-only"
tags: ["topics" "to describe" "your" "project"]
homepage: "https://github.com/username/reponame"
doc: "https://url/to/documentation"
bug-reports: "https://github.com/username/reponame/issues"
depends: [
  "ocaml"
  "dune" {>= "3.11"}
  "dream"
  "ppx_yojson_conv"
  "crunch"
  "cohttp"
  "lwt_ppx"
  "cohttp-lwt-unix"
  "containers"
  "calendar"
  "ptime"
  "mirage-crypto-rng"
  "mirage-crypto-rng-lwt"
  "ppx_inline_test"
  "ppx_deriving"
  "decoders-yojson"
  "odoc" {with-doc}
]
build: [
  ["dune" "subst"] {dev}
  [
    "dune"
    "build"
    "-p"
    name
    "-j"
    jobs
    "@install"
    "@runtest" {with-test}
    "@doc" {with-doc}
  ]
]
dev-repo: "git+https://github.com/username/reponame.git"

Missing dependency with non-trivial formula

Describe the bug

Building ocamlformat-lib 0.26.1 fails because the ocp-indent dependency is missing at build time.
The opam file from opam-repository lists the dependency as:

• "ocp-indent" {with-test = "false" & >= "1.8.0" | with-test & >= "1.8.1"}

To Reproduce

You can use the following flake:

{
  inputs = {
    flake-utils.url = "github:numtide/flake-utils";
    opam-nix.url = "github:tweag/opam-nix";
    opam-repository = {
      url = "github:ocaml/opam-repository";
      flake = false;
    };
  };

  outputs = { self, flake-utils, opam-nix, opam-repository }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        scope =
          opam-nix.lib.${system}.queryToScope { repos = [ opam-repository ]; } {
            ocamlformat-lib = "*";
            ocaml-base-compiler = "5.1.0";
          };
      in {
        legacyPackages = scope;
        defaultPackage = scope.ocamlformat-lib;
      });
}

It is possible to workaround this by vendoring opam-repository and replacing the problematic line to one with a simpler formula.

I achieved this with a folder with the following structure in my project:

opam-repo-overrides/
├── packages
│   └── ocamlformat-lib
│       └── ocamlformat-lib.0.26.1
│           └── opam
├── README
└── repo

The opam file there is a copy of opam-repository's, with the formula simplified to {>= "1.8.1"}.

Optional dependency on opam-overlays and mirage-opam-overlays

opam-nix have a lot of dependencies but two seem unnecessary: opam-overlays and mirage-opam-overlays.
They are added to the list of repositories used for solving, regardless of whether the packages being solved need these or not. Since #18

This adds overhead to the build but what worries me the most is the extra maintenance caused by these. I actively fight Nix tendency to use several version of each dependency and end up with a large flake.nix:

{
  inputs.nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
  inputs.opam-nix = {
    url = "github:tweag/opam-nix";
    inputs.nixpkgs.follows = "nixpkgs";
    inputs.flake-utils.follows = "flake-utils";
    inputs.opam-repository.follows = "opam-repository";
    inputs.opam-overlays.follows = "opam-overlays";
    inputs.mirage-opam-overlays.follows = "mirage-opam-overlays";
  };
  inputs.flake-utils = {
    url = "github:numtide/flake-utils";
  };
  inputs.opam-repository = {
    url = "github:ocaml/opam-repository";
    flake = false;
  };
  inputs.opam-overlays = {
    url = "github:dune-universe/opam-overlays";
    flake = false;
  };
  inputs.mirage-opam-overlays = {
    url = "github:dune-universe/mirage-opam-overlays";
    flake = false;
  };
}

These dependencies could be optional, at the cost of extra setup for users that need them and are not fighting against Nix.

Support `opam-nix-gen` picking up mutliple packages in the same project

Is your feature request related to a problem? Please describe.

My motivation is picking up vendored dependencies in materialization.

Describe the solution you'd like

For opam-nix-gen support picking up multiple packages in the same project.

An materializeOpamProject' could be useful for this. Something like:

  materializeOpamProject' = { repos ? [ opamRepository ], resolveArgs ? { }
    , regenCommand ? null, pinDepends ? true, recursive ? false }:
    name: project: query:
    let
      repo = makeOpamRepo' recursive project;
      latestVersions = mapAttrs (_: last) (listRepo repo);

      pinDeps = concatLists (attrValues (mapAttrs
        (name: version: getPinDepends repo.passthru.pkgdefs.${name}.${version})
        latestVersions));
    in materialize {
      repos = [ repo ] ++ optionals pinDepends pinDeps ++ repos;
      resolveArgs = { dev = true; } // resolveArgs;
      inherit regenCommand;
    } (llatestVersions // query);

Describe alternatives you've considered

A workaround is to just ignore these vendored dependencies and rely on dune to pick them up when building.

However if the vendored dependencies are modified to depend on an additional package these won't be picked up and will have to manually be added to the query.

Additional context

I came about this limitation while working on https://github.com/RyanGibb/aeon, which has dependencies vendored: https://github.com/RyanGibb/aeon/tree/e28f84281886394e81ba07076de21b3725aa7eec/duniverse.

Old versions of Coq alongside some Coq packages fail to build

Describe the bug

Old versions of Coq alongside some Coq packages listed in an opam file fail to install, the fix introduced in 5bfdc95 only works for newer versions of Coq. The fix doesn't work for older versions of Coq, see #48 and the resulting 34d74ca which disables it for Coq < 8.17.

To Reproduce

An opam file including, for example:

"coq" {>= "8.10" & < "8.12.0" }
"coq-ott" {>= "0.29"}

fails to build when following the standard instructions in the README of opam-nix with the error:

error: builder for '/nix/store/s5f9ss5b3i47fj85ik1k3ppv5nvv6fx4-coq-ott-dev.drv' failed with exit code 2;
       last 10 log lines:
       > install: cannot change permissions of '/nix/store/d75jswiprkwrmwm21xzv04rkjdxpwwmf-coq-8.11.2/lib/ocaml/4.11.2/site-lib/coq//user-contrib/Ott/': No such file or directory
       > install: cannot change permissions of '/nix/store/d75jswiprkwrmwm21xzv04rkjdxpwwmf-coq-8.11.2/lib/ocaml/4.11.2/site-lib/coq//user-contrib/Ott/': No such file or directory
       > install: cannot change permissions of '/nix/store/d75jswiprkwrmwm21xzv04rkjdxpwwmf-coq-8.11.2/lib/ocaml/4.11.2/site-lib/coq//user-contrib/Ott/': No such file or directory
       > install: cannot change permissions of '/nix/store/d75jswiprkwrmwm21xzv04rkjdxpwwmf-coq-8.11.2/lib/ocaml/4.11.2/site-lib/coq//user-contrib/Ott/': No such file or directory
       > install: cannot change permissions of '/nix/store/d75jswiprkwrmwm21xzv04rkjdxpwwmf-coq-8.11.2/lib/ocaml/4.11.2/site-lib/coq//user-contrib/Ott/': No such file or directory
       > install: cannot change permissions of '/nix/store/d75jswiprkwrmwm21xzv04rkjdxpwwmf-coq-8.11.2/lib/ocaml/4.11.2/site-lib/coq//user-contrib/Ott/': No such file or directory
       > make[1]: *** [Makefile.coq:490: install] Error 1
       > make[1]: Leaving directory '/build/source/coq'
       > make: *** [Makefile:5: install] Error 2
       > make: Leaving directory '/build/source/coq'
       For full logs, run 'nix log /nix/store/s5f9ss5b3i47fj85ik1k3ppv5nvv6fx4-coq-ott-dev.drv'.
error: 1 dependencies of derivation '/nix/store/ny29rycz6mqgskfmn8iv4sc57spmpv7x-coq-mi-cho-coq-dev-env.drv' failed to build

(Using nix develop --impure to trigger a build of Coq).

Expected behavior

Such an opam file would succeed at building Coq with coq-ott, following the instructions of the README.

Environment

OS name + version: NixOS
Version of the code: https://github.com/tweag/opam-nix/tree/34d74ca9d64339c453cbac3f85255644361937e2

Additional context

Removing the version constraints resulting in the following opam file, builds successfully. Giving: Coq v8.17.1

  "coq"
  "coq-ott"

I am trying to build the Coq specification of Michelson, Mi-Cho-Coq, with this .opam file.
This fails with:

error: builder for '/nix/store/cggr7bq1jzransryclk1k4k6mng60bhs-coq-error-handlers-1.2.0.drv' failed with exit code 2;
       last 10 log lines:
       > patching script interpreter paths in .
       > ./configure.sh: interpreter directive changed from "#!/usr/bin/env sh" to "/nix/store/zlf0f88vj30sc7567b80l52d19pbdmy2-bash-5.2-p15/bin/sh"
       > building
       > COQDEP VFILES
       > COQC src/All.v
       > installing
       > install: cannot change permissions of '/nix/store/d75jswiprkwrmwm21xzv04rkjdxpwwmf-coq-8.11.2/lib/ocaml/4.11.2/site-lib/coq//user-contrib/ErrorHandlers/': No such file or directory
       > install: cannot change permissions of '/nix/store/d75jswiprkwrmwm21xzv04rkjdxpwwmf-coq-8.11.2/lib/ocaml/4.11.2/site-lib/coq//user-contrib/ErrorHandlers/': No such file or directory
       > install: cannot change permissions of '/nix/store/d75jswiprkwrmwm21xzv04rkjdxpwwmf-coq-8.11.2/lib/ocaml/4.11.2/site-lib/coq//user-contrib/ErrorHandlers/': No such file or directory
       > make: *** [Makefile:490: install] Error 1
       For full logs, run 'nix log /nix/store/cggr7bq1jzransryclk1k4k6mng60bhs-coq-error-handlers-1.2.0.drv'.
error: 1 dependencies of derivation '/nix/store/7jx8n5nvn5pphwkd9xi9q9c7dwgwdmiw-coq-list-string-2.1.2.drv' failed to build
error (ignored): error: cannot unlink '/tmp/nix-build-coq-menhirlib-dev.drv-21/source/test/static/good': Directory not empty
error: 1 dependencies of derivation '/nix/store/s6f2rpr2df4lanmayilpin3npq7cfvdw-coq-mi-cho-coq-dev-env.drv' failed to build

I also tried the .opam file of the dev branch of Mi-Cho-Coq, which fails with a different error:

error: builder for '/nix/store/ldzavngayh84hm4a3z2bmib1vxd5f15r-coq-error-handlers-1.2.0.drv' failed with exit code 1;
       last 10 log lines:
       > unpacking sources
       > unpacking source archive /nix/store/qvd45x7m9a2hvnhm0fm8r7nyh4q8yl0y-1.2.0.tar.gz
       > source root is coq-error-handlers-1.2.0
       > setting SOURCE_DATE_EPOCH to timestamp 1437603510 of file coq-error-handlers-1.2.0/src/All.v
       > patching sources
       > configuring
       > patching script interpreter paths in .
       > ./configure.sh: interpreter directive changed from "#!/usr/bin/env sh" to "/nix/store/zlf0f88vj30sc7567b80l52d19pbdmy2-bash-5.2-p15/bin/sh"
       > building
       > Error: cannot find /nix/store/jcyy5nd5q4l576mm6m1z3i2kca6454za-coq-8.16.1/lib/ocaml/4.14.1/site-lib/coq/../coq-core/tools/CoqMakefile.in
       For full logs, run 'nix log /nix/store/ldzavngayh84hm4a3z2bmib1vxd5f15r-coq-error-handlers-1.2.0.drv'.
error: 1 dependencies of derivation '/nix/store/i1xx683iarcvdk14yskhaq7la1cxrs1j-coq-list-string-2.1.2.drv' failed to build
error: 1 dependencies of derivation '/nix/store/xbwp8ihlzwpgllcdib5dvdi7rp7r06k9-coq-mi-cho-coq-dev-env.drv' failed to build

The dev branch's .opam file builds successfully via opam.

Naive failure from flake init templates?

Describe the bug
Hi Folks.
Hoping its just me, but running the flake init command, changing the package string, I next get:

error: attribute '<package-name>' missing

       at /nix/store/b39yb2zy2ikm70yicc9i029xknhh6wjj-source/flake.nix:31:24:

           30|           # You can add overrides here
           31|           ${package} = prev.${package}.overrideAttrs (_: {

To Reproduce
nix flake init -t github:tweag/opam-nix#executable

nix develop

Environment

• OS name + version: nixos-unstable w nix 2.11.1
• Version of the code: unsure.

Fails to find repositories specified in `pin-depends`

Describe the bug
Opam-nix seems not to look for pin-depends.

I've been trying to experiment with yocaml. It seems not to have been published to opam yet, and people seem to be using pin-depends as a way to use it in their projects?

I searched on the opam-nix repo and it seems like pin-depends is not mentioned, so maybe it is not yet supported? I understand that it might be tricky to do this while maintaining reproducibility however? So I'm not sure what that would look like.

To Reproduce

flake.nix

{
  description = "A very basic flake";

  inputs = {
    # Utilities for writing flakes
    flake-utils.url = github:numtide/flake-utils;
    # Precisely filter files copied to the nix store
    nix-filter.url = "github:numtide/nix-filter";
    # Generate derivations from Dune and OCaml files
    opam-nix.url = github:tweag/opam-nix;
  };

  outputs = { self, nixpkgs, flake-utils, nix-filter, opam-nix }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        # Legacy packages that have not been converted to flakes
        legacyPackages = nixpkgs.legacyPackages.${system};
        # OCaml packages available on nixpkgs
        ocamlPackages = legacyPackages.ocamlPackages;

        # OCaml source files
        ocaml-src = nix-filter.lib.filter {
          root = ./.;
          include = [
            "dune-project"
            "garden.opam.template"
            (nix-filter.lib.inDirectory "bin")
          ];
        };

        # Nix source files
        nix-src = nix-filter.lib.filter {
          root = ./.;
          include = [
            (nix-filter.lib.matchExt "nix")
          ];
        };
      in
      {
        # Executed by `nix build .#<name>`
        packages.garden =
          (opam-nix.lib.${system}.buildDuneProject { } "garden" ocaml-src { }).garden;

        # Executed by `nix build`
        defaultPackage = self.packages.${system}.garden;

        # Executed by `nix run .#<name>`
        apps.garden = {
          type = "app";
          program = "${self.packages.${system}.garden}/bin/garden";
        };

        # Executed by `nix run`
        defaultApp = self.apps.${system}.garden;

        # Used by `nix develop`
        devShell = legacyPackages.mkShell {
          nativeBuildInputs = [
            ocamlPackages.dune_2
            ocamlPackages.ocaml
            legacyPackages.opam

            # Editor support
            # pkgs.ocamlformat # FIXME: fails to build `uunf` on my M1 mac :(
            ocamlPackages.merlin
            ocamlPackages.ocaml-lsp
          ];
          inputsFrom = [
            self.defaultPackage.${system}
          ];
        };
      });
}

dune-project

(lang dune 2.9)

(generate_opam_files true)

(package
 (name garden)
 (synopsis "Static page generator for my digital garden")
 (depends
  (preface (>= "0.1.0"))
  (yocaml :pinned)
  (yocaml_yaml :pinned)
  (yocaml_markdown :pinned)
  (yocaml_unix :pinned)))

bin/dune

(executable
 (name garden)
 (public_name garden)
 (promote (until-clean))
 (libraries preface yocaml yocaml_yaml yocaml_markdown yocaml_unix))

bin/garden.ml

let () =
  ()

garden.opam.template

pin-depends: [
  ["yocaml.dev" "git://github.com/xhtmlboi/yocaml.git"]
  ["yocaml_unix.dev" "git://github.com/xhtmlboi/yocaml.git"]
  ["yocaml_yaml.dev" "git://github.com/xhtmlboi/yocaml.git"]
  ["yocaml_markdown.dev" "git://github.com/xhtmlboi/yocaml.git"]
]

After running nix build I get:

nix log ...-garden-local.drv

@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/l2y5prikdgks7i4a5awc7vkxdxzxi2h9-source-copy
source root is source-copy
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
@nix { "action": "setPhase", "phase": "updateAutotoolsGnuConfigScriptsPhase" }
updateAutotoolsGnuConfigScriptsPhase
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
patching script interpreter paths in .
@nix { "action": "setPhase", "phase": "buildPhase" }
building
File "bin/dune", line 5, characters 55-66:
5 |  (libraries preface yocaml yocaml_yaml yocaml_markdown yocaml_unix))
                                                           ^^^^^^^^^^^
Error: Library "yocaml_unix" not found.
Hint: try:
  dune external-lib-deps --missing --no-config --root . --ignore-promoted-rules --default-target @install --always-show-command-line --promote-install-files false --promote-install-files --release --only-packages garden -p garden --profile release -j 10 @install

The generated opam file is:

/nix/store/...-source-copy/garden.opam

# This file is generated by dune, edit dune-project instead
opam-version: "2.0"
depends: [
  "dune" {>= "2.9"}
  "preface" {>= "0.1.0"}
  "yocaml" {pinned}
  "yocaml_yaml" {pinned}
  "yocaml_markdown" {pinned}
  "yocaml_unix" {pinned}
  "odoc" {with-doc}
]
build: [
  ["dune" "subst"] {dev}
  [
    "dune"
    "build"
    "-p"
    name
    "-j"
    jobs
    "--promote-install-files=false"
    "@install"
    "@runtest" {with-test}
    "@doc" {with-doc}
  ]
  ["dune" "install" "-p" name "--create-install-files" name]
]
pin-depends: [
  ["yocaml.dev" "git://github.com/xhtmlboi/yocaml.git"]
  ["yocaml_unix.dev" "git://github.com/xhtmlboi/yocaml.git"]
  ["yocaml_yaml.dev" "git://github.com/xhtmlboi/yocaml.git"]
  ["yocaml_markdown.dev" "git://github.com/xhtmlboi/yocaml.git"]
]

Expected behavior

The build should find the dependencies

Environment

• OS name + version: aarch64-darwin, macOS 12.1

• Version of the code:

  Inputs:
  ├───flake-utils: github:numtide/flake-utils/74f7e4319258e287b0f9cb95426c9853b282730b
  ├───nix-filter: github:numtide/nix-filter/bd03eb5242e707bc270f7a7e4f835b55713247ac
  ├───nixpkgs: github:NixOS/nixpkgs/568e0bc498ee51fdd88e1e94089de05f2fdbd18b
  └───opam-nix: github:tweag/opam-nix/03ff01c4d228814e03566ef36667599848408738
      ├───flake-compat: github:edolstra/flake-compat/12c64ca55c1014cdc1b16ed5a804aa8576601ff2
      ├───flake-utils: github:numtide/flake-utils/74f7e4319258e287b0f9cb95426c9853b282730b
      ├───nixpkgs: github:nixos/nixpkgs/eac07edbd20ed4908b98790ba299250b5527ecdf
      ├───opam-repository: github:ocaml/opam-repository/c7b45b3cbcfbe22a7b3b8af1e5bb9998f3aa0e8a
      └───opam2json: github:tweag/opam2json/7652f94ac18ec94bc1ae697169c7f7f8b809270a
          └───nixpkgs: github:NixOS/nixpkgs/5c37ad87222cfc1ec36d6cd1364514a9efc2f7f2
  

Additional context
Add any other context about the problem here.

Document adding dependencies to dune projects

Is your feature request related to a problem? Please describe.

It's not obvious how to add dependencies to a dune project. There's a particular ocaml tool that I'm able to build with opam-nix, but only by overriding the build phase to use dune build --release, and by manually adding packages to a depends in the dune-project. Since I don't maintain this tool, I'd like to be able to add the dependencies from within nix.

Describe the solution you'd like

I'd like documentation of how to add a package dependency from within the nix file in such a way that the effect is the same as adding the dependency from within the dune-project file.

Describe alternatives you've considered

I've tried some probably silly stuff: applying a patch to the dune-project during the patching phase, using passthru to override depends from builder.nix, and adding the dependencies to the query argument in buildDuneProject.

Additional context
Here's a reference flake:

  inputs = {
    opam-nix.url = "github:tweag/opam-nix";
    flake-utils.url = "github:numtide/flake-utils";
    nixpkgs.follows = "opam-nix/nixpkgs";
    sailRepo = {
      url = "github:rems-project/sail";
      flake = false;
    };
  };
  outputs = { self, flake-utils, opam-nix, sailRepo, nixpkgs }@inputs:
    flake-utils.lib.eachDefaultSystem (system:
      let
        on = opam-nix.lib.${system};
        scope = on.buildDuneProject { } "sail" sailRepo { 
          ocaml-base-compiler = "*"; 
        };
        overlay = final: prev:
          {
            sail = prev.sail.overrideAttrs (_: {
              buildPhase = "dune build --release"; 
              postFixup = "mv $out/share/sail/lib $out/share/lib";
            });
          };
      in {
        legacyPackages = scope.overrideScope' overlay;

        packages.default = self.legacyPackages.${system}.sail;
      });
}

Using this, nix build fails because of a missing base64 and omd dependency. If you apply the following patch to a clone of the sail repo

diff --git a/dune-project b/dune-project
index 23ab4217..76f403f1 100644
--- a/dune-project
+++ b/dune-project
@@ -135,4 +135,6 @@ http://www.cl.cam.ac.uk/~pes20/sail/.
     (sail_latex_backend (and (= :version) :post))
     (sail_doc_backend (and (= :version) :post))
     (sail_output (and (= :version) :post))
+    (base64 (>= 3.1.0))
+    (omd (and (>= 1.3.1) (< 1.4.0)))
     (linenoise (>= 1.1.0))))

and build in that context (moving the flake above to within the clone, and using ./. rather than sailRepo), everything builds correctly.

ppx_expect isn't compiled correctly

Describe the bug

ppx_expect isn't being compiled correctly.

To Reproduce

To reproduce, we must clone the dune repository first.

$ git clone https://github.com/ocaml/dune.git
$ cd dune
$ nix develop
$ make dev
$ ./dune.exe runtest test/expect-tests/
File "_none_", line 1:
Error: Files /nix/store/ff70rp932zf00qfw6bsdld6v53z5pl68-ppx_expect-v0.15.0/lib/ocaml/4.14.0/site-lib/ppx_expect/matcher/expect_test_matcher.cmxa
       and /nix/store/sl3k9vw69knlgnbjgsqlbpykihhkf2ld-re-1.10.4/lib/ocaml/4.14.0/site-lib/re/re.cmxa
       make inconsistent assumptions over interface Re__

Expected behavior

The tests should compile and link run.

Environment

• OS name + version: MacOS Ventura
• Version of the code: 7430da9ade302bad75f7e4cfacd9eca12783cca

Example Request: Building a dependency from source and adding it to the buildInputs.

This flake currently fails to build:

{
  inputs = {
    opam-nix.url = "github:tweag/opam-nix";
    flake-utils.url = "github:numtide/flake-utils";
    nixpkgs.follows = "opam-nix/nixpkgs";
    forester.url = "sourcehut:~jonsterling/ocaml-forester";
    forester.flake = false;
  };
  outputs = { self, flake-utils, forester, opam-nix, nixpkgs }@inputs:
    let
      package = "forester";
    in
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = nixpkgs.legacyPackages.${system};
        on = opam-nix.lib.${system};
        devPackagesQuery = {
          # You can add "development" packages here. They will get added to the devShell automatically.
          ocaml-lsp-server = "*";
          ocamlformat = "*";
        };
        query = devPackagesQuery // {
          ## You can force versions of certain packages here, e.g:
          ## - force the ocaml compiler to be taken from opam-repository:
          ocaml-base-compiler = "*";
          ## - or force the compiler to be taken from nixpkgs and be a certain version:
          # ocaml-system = "4.14.0";
          ## - or force ocamlfind to be a certain version:
          # ocamlfind = "1.9.2";
        };
        scope = on.buildOpamProject' { } forester query;
        overlay = final: prev: {
          # You can add overrides here
          ${package} = prev.${package}.overrideAttrs (_: {
            doNixSupport = false;
          });
        };
        scope' = scope.overrideScope' overlay;
        main = scope'.${package};
        devPackages = builtins.attrValues
          (pkgs.lib.getAttrs (builtins.attrNames devPackagesQuery) scope');
      in
      {
        legacyPackages = scope';
        packages.default = main;
      });
}

resolve> [ERROR] No solution including optional dependencies for forester.2.5.1 & ocaml-base-compiler & ocaml-lsp-server & ocamlformat:   * Missing dependency:
resolve>             - asai
resolve>             unknown package

The package is not contained in opam-repo but should be installable with opam: https://ocaml.org/p/asai/0.2.0
The source of the package is here: https://github.com/RedPRL/asai

Is it possible to build asai from the repo and add it to the build inputs for forester?

Thanks!

Building project with `foreign_stubs` C dependency: header file not found.

I am trying to build ocaml-fswatch which depends on fswatch.

{
  inputs = {
    opam-nix.url = "github:tweag/opam-nix";
    ocaml-fswatch.url = "github:kentookura/ocaml-fswatch";
    ocaml-fswatch.flake = false;
    flake-utils.url = "github:numtide/flake-utils";
    nixpkgs.follows = "opam-nix/nixpkgs";
  };
  outputs = { self, ocaml-fswatch, flake-utils, opam-nix, nixpkgs }@inputs:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = nixpkgs.legacyPackages.${system}.extend overlay;
        on = opam-nix.lib.${system};
        localPackagesQuery = builtins.mapAttrs (_: pkgs.lib.last)
          (on.listRepo (on.makeOpamRepo ./.));
        devPackagesQuery = {
          ocaml-lsp-server = "*";
          ocamlformat = "*";
          #fswatch = "*";
          #libfswatch = "*";
        };

        query = devPackagesQuery // { ocaml-base-compiler = "*"; };
        libfswatch = pkgs.stdenv.mkDerivation ({
          name = "libfswatch";
          src = pkgs.fetchgit {
            url = "https://github.com/emcrisostomo/fswatch";
            sha256 = "c5Thf/uEHw8aByemG/GxhPH++vz9EqE6pAu40OEvBOM=";
            fetchSubmodules = true;
          };
          nativeBuildInputs = with pkgs; [ clang cmake ];
          buildPhase = "make";
          installPhase = ''
            make install
          '';
        });
        overlay = self: super: {
          fswatch =
            super.fswatch.overrideAttrs (_: { buildInputs = [ libfswatch ]; });
        };
        scope = on.buildDuneProject { } "fswatch" ocaml-fswatch query;
        scope' = scope.overrideScope' overlay;
        main = scope'.fswatch;
        devPackages = builtins.attrValues
          (pkgs.lib.getAttrs (builtins.attrNames devPackagesQuery) scope');
        packages =
          pkgs.lib.getAttrs (builtins.attrNames localPackagesQuery) scope';

      in {
        packages = packages // {
          libfswatch = libfswatch;
          default = main;
        };
        devShells.default = pkgs.mkShell {
          inputsFrom = builtins.attrValues packages;
          buildInputs = devPackages;
        };
      });
}

This fails:

...
fswatch-dev> File "fswatch/src/dune", line 9, characters 9-13:
fswatch-dev> 9 |   (names stub)))
fswatch-dev>              ^^^^
fswatch-dev> (cd _build/default/fswatch/src && /nix/store/nlgyw2fv0cm8rkz8qm1jyw78vyif1bl9-gcc-wrapper-12.2.0/bin/gcc -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -D_FILE_OFFSET_BITS=64 -fdiagnostics-color=always -g -I /nix/store/gksszmxms1n9vhvz6fcx4ls7kaz5nm6n-ocaml-base-compiler-5.1.0_rc3/lib/ocaml -I /nix/store/gksszmxms1n9vhvz6fcx4ls7kaz5nm6n-ocaml-base-compiler-5.1.0_rc3/lib/ocaml/threads -I /nix/store/gksszmxms1n9vhvz6fcx4ls7kaz5nm6n-ocaml-base-compiler-5.1.0_rc3/lib/ocaml/unix -o stub.o -c stub.c)
fswatch-dev> stub.c:11:10: fatal error: libfswatch.h: No such file or directory
fswatch-dev>    11 | #include <libfswatch.h>
fswatch-dev>       |          ^~~~~~~~~~~~~~
fswatch-dev> compilation terminated.
error: builder for '/nix/store/4f8jr0dips13v5vxma9bascqsggjm4xp-fswatch-dev.drv' failed with exit code 1;
       last 10 log lines:
       > building
       > File "fswatch/src/dune", line 9, characters 9-13:
       > 9 |   (names stub)))
       >              ^^^^
       > (cd _build/default/fswatch/src && /nix/store/nlgyw2fv0cm8rkz8qm1jyw78vyif1bl9-gcc-wrapper-12.2.0/bin/gcc -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -D_FILE_OFFSET_BITS=64 -fdiagnostics-color=always -g -I /nix/store/gksszmxms1n9vhvz6fcx4ls7kaz5nm6n-ocaml-base-compiler-5.1.0_rc3/lib/ocaml -I /nix/store/gksszmxms1n9vhvz6fcx4ls7kaz5nm6n-ocaml-base-compiler-5.1.0_rc3/lib/ocaml/threads -I /nix/store/gksszmxms1n9vhvz6fcx4ls7kaz5nm6n-ocaml-base-compiler-5.1.0_rc3/lib/ocaml/unix -o stub.o -c stub.c)
       > stub.c:11:10: fatal error: libfswatch.h: No such file or directory
       >    11 | #include <libfswatch.h>
       >       |          ^~~~~~~~~~~~~~
       > compilation terminated.
       >

So the problem lies with the following command:

/nix/store/...-gcc-wrapper-12.2.0/bin/gcc -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -D_FILE_OFFSET_BITS=64 -fdiagnostics-color=always -g -I /nix/store/...-ocaml-base-compiler-5.1.0_rc3/lib/ocaml -I /nix/store/...-ocaml-base-compiler-5.1.0_rc3/lib/ocaml/threads -I /nix/store/...-ocaml-base-compiler-5.1.0_rc3/lib/ocaml/unix -o stub.o -c stub.c

Am I passing the buildInputs correctly to opam-nix?

Failed to build the dune template on macOS: `symbols not found`

Hi,

I configured a freshly initialized dune project (by dune init proj) with opam-nix. I can build it successfully in an arm NixOS VM, but I failed building it under an apple silicon mac (I tried both architectures, x86_64 and aarch64). I've uploaded the dune project and flake here: https://github.com/mistzzt/minimal-opam-nix-example

Also, below are the relevant error messages. I have no idea what to try next -- any suggestion would be appreciated.

$ nix build
error: builder for '/nix/store/99jv7xb5skyg4m1vh7lh369517dgp63g-opam-2.1.5.drv' failed with exit code 2;
       last 10 log lines:
       >       ml2c_property(Virtual_packages*, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, CUDFProperty*, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, CUDFProperty*>>>*, long) in libmccs_stubs.a(mccs_stubs.o)
       >       ml2c_package(Virtual_packages*, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, CUDFProperty*, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, CUDFProperty*>>>*, int&, long) in libmccs_stubs.a(mccs_stubs.o)
       >       ml2c_propertydef(Virtual_packages*, long) in libmccs_stubs.a(mccs_stubs.o)
       >       ml2c_propertydeflist(Virtual_packages*, long) in libmccs_stubs.a(mccs_stubs.o)
       >       ...
       > ld: symbol(s) not found for architecture arm64
       > clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
       > File "caml_startup", line 1:
       > Error: Error during linking (exit code 1)
       > make: *** [Makefile:135: build-opam] Error 1
       For full logs, run 'nix log /nix/store/99jv7xb5skyg4m1vh7lh369517dgp63g-opam-2.1.5.drv'.
error:
       … while evaluating the attribute 'packages.aarch64-darwin.test2'

         at /nix/store/fcnzp53fbzryz8igkx787vp1zfp0nfac-source/flake.nix:30:7:

           29|     in {
           30|       test2 =
             |       ^
           31|         (buildDuneProject {} "test2" ./. {

       … while calling the 'foldl'' builtin

         at /nix/store/dqf6f40n3dxhx1cxbdza1lalncpyy8cl-source/lib/trivial.nix:63:8:

           62|     let reverseApply = x: f: f x;
           63|     in builtins.foldl' reverseApply val functions;
             |        ^
           64|

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: 1 dependencies of derivation '/nix/store/vgmj1z4dl2n3wwq5hvvrs9ssx5a9vxxf-resolve.drv' failed to build

$ nix log /nix/store/99jv7xb5skyg4m1vh7lh369517dgp63g-opam-2.1.5.drv
...
  "___gxx_personality_v0", referenced from:
      ml2c_vpkg(Virtual_packages*, long) in libmccs_stubs.a(mccs_stubs.o)
      Virtual_packages::get(char const*) in libmccs_stubs.a(mccs_stubs.o)
      ml2c_vpkglist(Virtual_packages*, long) in libmccs_stubs.a(mccs_stubs.o)
      ml2c_property(Virtual_packages*, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, CUDFProperty*, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, >
      ml2c_package(Virtual_packages*, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, CUDFProperty*, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, s>
      ml2c_propertydef(Virtual_packages*, long) in libmccs_stubs.a(mccs_stubs.o)
      ml2c_propertydeflist(Virtual_packages*, long) in libmccs_stubs.a(mccs_stubs.o)
      ...
ld: symbol(s) not found for architecture arm64
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
File "caml_startup", line 1:
Error: Error during linking (exit code 1)

$ nix --version
nix (Nix) 2.18.1

$ uname -a
Darwin symmetry 23.3.0 Darwin Kernel Version 23.3.0: Wed Dec 20 21:30:44 PST 2023; root:xnu-10002.81.5~7/RELEASE_ARM64_T6000 arm64 arm Darwin

Thanks!

Example Request: Multi Package Project

Lots of OCaml projects contain more than one opam package per project. But as far as I can tell, all the examples cover projects with only a single opam package. It would be good to include an example of this common project type as well.

Multi-package template fails to install opam package `bogue`

Describe the bug
The bogue package fails to build with the following error:

error: builder for '/nix/store/7mwqar1ajgybsl9wdkjx763ibavg0kzf-conf-sdl2-image-1.drv' failed with exit code 1;
       last 7 log lines:
       > unpacking sources
       > unpacking source archive /nix/store/4d0ix5djms3n2jnjdc58l916cwack1rp-empty-directory
       > source root is empty-directory
       > patching sources
       > configuring
       > patching script interpreter paths in .
       > building
       For full logs, run 'nix log /nix/store/7mwqar1ajgybsl9wdkjx763ibavg0kzf-conf-sdl2-image-1.drv'.
error: 1 dependencies of derivation '/nix/store/z9hgyzcgv53y8cav7dqanwpsp9x58aca-tsdl-image-0.5.drv' failed to build
error: 1 dependencies of derivation '/nix/store/h5fw4ps8ws5hgvrm9pyjp1dgldfymcph-bogue-20221112.drv' failed to build

To Reproduce
The simplest way to reproduce is nix shell github:tweag/opam-nix#bogue.latest.

Personally, I am trying to build a devShell using the below flake, with direnv use flake.

Click to show flake

{
  description = "A Nix-flake-based OCaml development environment";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    opam-nix.url = "github:tweag/opam-nix";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = {
    self,
    opam-nix,
    nixpkgs,
    flake-utils,
  } @ inputs:
    flake-utils.lib.eachDefaultSystem (system: let
      pkgs = nixpkgs.legacyPackages.${system};
      on = opam-nix.lib.${system};
      localPackagesQuery =
        builtins.mapAttrs (_: pkgs.lib.last)
        (on.listRepo (on.makeOpamRepo ./.));
      devPackagesQuery = {
        "bogue" = "*";
        "core" = "*";
        "utop" = "*";
      };
      query =
        devPackagesQuery
        // {};
      scope = on.buildOpamProject' {} ./. query;
      overlay = final: prev: {};
      scope' = scope.overrideScope' overlay;
      devPackages =
        builtins.attrValues
        (pkgs.lib.getAttrs (builtins.attrNames devPackagesQuery) scope');
      packages =
        pkgs.lib.getAttrs (builtins.attrNames localPackagesQuery) scope';
    in {
      legacyPackages = scope';
      inherit packages;

      devShells.default = pkgs.mkShell {
        inputsFrom = builtins.attrValues packages;
        buildInputs = devPackages ++ (with pkgs; [SDL_image]);
      };
    });
}

Expected behavior
A correctly produced dev shell with the bogue package.

Environment

• OS name + version: nixos 23.11 (unstable)
• Version of the code: e83bd1d

Additional context
Opam Repository Link for conf-sdl2-image
Source code for tsdl-image
Source code for bogue

Installing an old version of Coq via opam-nix fails, related to coq-stdlib

Describe the bug
Old versions of Coq listed in an opam file fail to install, likely due to: 5bfdc95 ocaml.nix - line 133 and that coq-stdlib is only for newer versions of Coq.

To Reproduce
An opam file with: "coq" {>= "8.10" & < "8.11.0" } fails to build when following the standard instructions in the README of opam-nix with the error: error: attribute 'coq-stdlib' missing.

It seems that coq-of-ocaml is possibly meant as an alternative to writing coq. Using this still does not work, giving the error No agreement on the version of coq-of-ocaml.

(Using nix develop to trigger a build of Coq).

Expected behavior
An opam file with "coq" {>= "8.10" & < "8.11.0" } would succeed at building Coq following the instructions of the README.

Environment

• OS name + version: NixOS
• Version of the code: 5bfdc95

Additional context
Reverting 5bfdc95 fixes the issue.

pin materialization `opam-repository`

I want to use the same flake inputs (specifically opam-repository) for materialization so ended up creating a derivation that calls materializeOpamProject: https://github.com/RyanGibb/aeon/blob/f4ecf576a7f7cd8b882c221339bd071ddfa711d0/flake.nix#L44-L46

Is there a better way to do this? If not, is this worth documenting for others?

Fail to compile `raylib-ocaml`

Describe the bug
I want to use the raylib-ocaml library (github and opam) in my dune project. Trying to build it fails for conf-libxcursor.

To Reproduce

1. nix-shell -p dune opam
2. opam exec -- dune init proj hello
3. cd hello
4. nix flake init -t github:tweag/opam-nix
5. edit the flake.nix
   1. change buildOpamProject to buildDuneProject
   2. add raylib = "*"; to scope
   3. add ${package} = prev.${package}.overrideAttrs (oa: { buildInputs = oa.buildInputs ++ [ final.raylib ]; }); to overlay
6. nix build

Expected behavior
Successful build.

Environment

• OS name + version: NixOS 2.18.1

Additional context
Add any other context about the problem here.
The full flake is:

{
    inputs = {
        opam-nix.url = "github:tweag/opam-nix";
        flake-utils.url = "github:numtide/flake-utils";
        nixpkgs.follows = "opam-nix/nixpkgs";
    };

    outputs = { self, flake-utils, opam-nix, nixpkgs }@inputs:
    let 
        package = "hello";
    in flake-utils.lib.eachDefaultSystem (system:
        let
            pkgs = nixpkgs.legacyPackages.${system};
            on = opam-nix.lib.${system};
            scope = on.buildDuneProject { } package ./. { 
                ocaml-base-compiler = "*"; 
                raylib = "*";
            };
            overlay = final: prev: {
                ${package} = prev.${package}.overrideAttrs (oa: {
                    buildInputs = oa.buildInputs ++ [ final.raylib ];
                    buildPhase = "dune build --release"; 
                });
            };
        in {
            legacyPackages = scope.overrideScope' overlay;
            packages.default = self.legacyPackages.${system}.${package};
        }
    );
}

Full output from build:

❯ nix build
error: builder for '/nix/store/j5j86cs6ckhrh5fs4hbrslf4814rhpvr-conf-libxcursor-1.drv' failed with exit code 1;
       last 7 log lines:
       > unpacking sources
       > unpacking source archive /nix/store/4d0ix5djms3n2jnjdc58l916cwack1rp-empty-directory
       > source root is empty-directory
       > patching sources
       > configuring
       > patching script interpreter paths in .
       > building
       For full logs, run 'nix log /nix/store/j5j86cs6ckhrh5fs4hbrslf4814rhpvr-conf-libxcursor-1.drv'.
error: 1 dependencies of derivation '/nix/store/if85wf0mgkf7plxd54mw4vgzdkh6jgrx-raylib-1.1.0.drv' failed to build
error (ignored): error: cannot unlink '/tmp/nix-build-ocaml-base-compiler-5.1.1_rc1.drv-6/ocaml-5.1.1-rc1': Directory not empty
error: 1 dependencies of derivation '/nix/store/alklyc02s7jz2gadg4bhvgwd8crhz48i-hello-dev.drv' failed to build

fake-opam is overriding actual opam

Describe the bug

opam-nix provides a fake opam binary. This binary overrides the actual opam binary which is necessary for dune's test suite.

To Reproduce

Consider the base commit in this PR[https://github.com/ocaml/dune/pull/6180] and run

$ nix develop --command opam
  2.0.0

argon2 leads to infinite recursion

Describe the bug
argon2 fails to install and results in an infinite recursion error.

To Reproduce
nix shell github:tweag/opam-nix#argon2.latest

Expected behavior
The argon2 derivation should work properly.

Environment

• OS name + version: NixOS 23.11.20230929.f5892dd (Tapir)
• Version of the code: latest

ocamlfind: Hunk #1 FAILED on findlib 1.9.6

This is essentially #11 happening again with 1.9.5. The context is the same, I need a more up-to-date version of opam-repository because some of my dependencies are more recent than the locked opam-repository in the opam-nix repo.

Cannot build Coq projects that depend on the Coq package index

Describe the bug
I am not able to build the Coq package Verdi (or other Coq projects that depend on the Coq package index) using buildOpamProject, even though Verdi is an opam project.

To Reproduce

1. Clone my fork of Verdi that adds a flake.nix and checkout the branch opam-nix-issue
2. Run nix build --show-trace
3. Be met with the following trace

error:
       … while evaluating the attribute 'buildInputs' of the derivation 'coq-verdi-dev'

       at /nix/store/q3l9x3ncgi0hppw0i65ki9jrxj8kmgac-source/pkgs/stdenv/generic/make-derivation.nix:302:7:

          301|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          302|       name =
             |       ^
          303|         let

       … while evaluating the attribute 'src' of the derivation 'coq-inf-seq-ext-dev'

       at /nix/store/q3l9x3ncgi0hppw0i65ki9jrxj8kmgac-source/pkgs/stdenv/generic/make-derivation.nix:302:7:

          301|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          302|       name =
             |       ^
          303|         let

       … while evaluating the attribute 'src'

       at /nix/store/hd3481707v5r6prhf52y8gyk9x048d00-source/src/evaluator/default.nix:444:17:

          443|         pkgdef.src or pkgs.pkgsBuildBuild.emptyDirectory;
          444|     in { inherit archive src; };
             |                 ^
          445|

       error: in pure evaluation mode, 'fetchTarball' requires a 'sha256' argument

4. Just for fun, run nix build --impure --show-trace to see if that fixes anything
5. Be met with the following trace

error:
       … while evaluating the attribute 'buildInputs' of the derivation 'coq-verdi-dev'

       at /nix/store/q3l9x3ncgi0hppw0i65ki9jrxj8kmgac-source/pkgs/stdenv/generic/make-derivation.nix:302:7:

          301|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          302|       name =
             |       ^
          303|         let

       … while evaluating the attribute 'src' of the derivation 'coq-inf-seq-ext-dev'

       at /nix/store/q3l9x3ncgi0hppw0i65ki9jrxj8kmgac-source/pkgs/stdenv/generic/make-derivation.nix:302:7:

          301|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          302|       name =
             |       ^
          303|         let

       … while evaluating the attribute 'src'

       at /nix/store/hd3481707v5r6prhf52y8gyk9x048d00-source/src/evaluator/default.nix:444:17:

          443|         pkgdef.src or pkgs.pkgsBuildBuild.emptyDirectory;
          444|     in { inherit archive src; };
             |                 ^
          445|

       error: unable to download 'git+https://github.com/DistributedComponents/InfSeqExt.git#master': Unsupported protocol (1)

The error appears to stem from this line. Building other Coq projects with similar dependencies also leads to the same sort of "Unsupported protocol" error, so I do not believe this is specific to Coq's distributed components library, nor to Verdi. But I don't understand opam-nix well enough to know what is happening here.

Expected behavior
Because Verdi is an opam project, the first nix build should succeed and build Verdi for me.

Environment

• NixOS following the 23.05 release branch

Additional context
The flake.nix contains a line like

          on.buildOpamProject
            { repos = [ opam-repository
                        ("${opam-coq-archive}" + "/extra-dev")
                      ];
            }

The string interpolation is because opam-coq-archive contains multiple opam repos (unfortunately), so I have to specify the extra-dev repo in particular.

bug: failed to build package cf

Describe the bug
The cf is a OCaml bindings to macOS CoreFoundation
that is depends on ctypes.

When I try to build a package that depends on cf, I get a lot of errors like:

`nix log /nix/store/sc20b74lvbin9qns41fxggw5fmwwbrmn-cf-0.5.0.drv`

@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/b1rbpaikwf7mpcgqihpway4gpdfyi393-cf-lwt-0.5.0.tbz
source root is cf-lwt-0.5.0
setting SOURCE_DATE_EPOCH to timestamp 1648823425 of file cf-lwt-0.5.0/lwt/dune
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
patching script interpreter paths in .
@nix { "action": "setPhase", "phase": "buildPhase" }
building
Done: 31% (18/58, 40 left) (jobs: 0)^M                                    ^MFile "lib_gen/dune", line 27, characters 0-185:
27 | (rule
28 |  (targets detect.exe)
29 |  (enabled_if
30 |   (= %{system} macosx))
31 |  (deps
32 |   detect.c
33 |   (package ctypes))
34 |  (action
35 |   (run %{cc} -I %{ocaml_where} -I %{lib:ctypes:} -o %{targets} %{deps})))
Error: File unavailable:
/nix/store/xb81cf7qgnyvsyvp9nfj15dmqchc24gk-ctypes-0.22.0/doc/ctypes/CHANGES.md
Done: 31% (18/58, 40 left) (jobs: 0)^M                                    ^MFile "lib_gen/dune", line 27, characters 0-185:
27 | (rule
28 |  (targets detect.exe)
29 |  (enabled_if
30 |   (= %{system} macosx))
31 |  (deps
32 |   detect.c
33 |   (package ctypes))
34 |  (action
35 |   (run %{cc} -I %{ocaml_where} -I %{lib:ctypes:} -o %{targets} %{deps})))
Error: File unavailable:
/nix/store/xb81cf7qgnyvsyvp9nfj15dmqchc24gk-ctypes-0.22.0/doc/ctypes/LICENSE

... repeated n times similar error ...

Done: 31% (18/58, 40 left) (jobs: 0)^M                                    ^MFile "lib_gen/dune", line 27, characters 0-185:
27 | (rule
28 |  (targets detect.exe)
29 |  (enabled_if
30 |   (= %{system} macosx))
31 |  (deps
32 |   detect.c
33 |   (package ctypes))
34 |  (action
35 |   (run %{cc} -I %{ocaml_where} -I %{lib:ctypes:} -o %{targets} %{deps})))

Error: File unavailable:
/nix/store/xb81cf7qgnyvsyvp9nfj15dmqchc24gk-ctypes-0.22.0/lib/stublibs/dllctypes_stubs.so
Done: 31% (18/58, 40 left) (jobs: 0)^M                                    ^MDone: 33% (35/105, 70 left, 221 failed) (jobs: 5)^M              >
/nix/store/j645h2bcqjww615dfkka4v4q1nyyy5m1-cctools-port-973.0.1/libexec/as/x86_64/as: this system assembler is deprecated. Please migrate to>
Done: 33% (35/105, 70 left, 221 failed) (jobs: 5)^M                                                 ^MDone: 45% (51/113, 62 left, 221 failed)>
/nix/store/j645h2bcqjww615dfkka4v4q1nyyy5m1-cctools-port-973.0.1/libexec/as/x86_64/as: this system assembler is deprecated. Please migrate to>
Done: 45% (51/113, 62 left, 221 failed) (jobs: 3)^M                                                 ^M(cd _build/default && /nix/store/b7pfan>
/nix/store/j645h2bcqjww615dfkka4v4q1nyyy5m1-cctools-port-973.0.1/libexec/as/x86_64/as: this system assembler is deprecated. Please migrate to>
Done: 45% (51/113, 62 left, 221 failed) (jobs: 3)^M                                                 ^M(cd _build/default && /nix/store/b7pfan>
/nix/store/j645h2bcqjww615dfkka4v4q1nyyy5m1-cctools-port-973.0.1/libexec/as/x86_64/as: this system assembler is deprecated. Please migrate to>
Done: 45% (51/113, 62 left, 221 failed) (jobs: 3)^M                                                 ^MDone: 49% (56/113, 57 left, 221 failed)>
clang-11: warning: argument unused during compilation: '-fno-strict-overflow' [-Wunused-command-line-argument]
Done: 49% (56/113, 57 left, 221 failed) (jobs: 2)^M                                                 ^MDone: 50% (57/113, 56 left, 221 failed)>
/nix/store/j645h2bcqjww615dfkka4v4q1nyyy5m1-cctools-port-973.0.1/libexec/as/x86_64/as: this system assembler is deprecated. Please migrate to>
Done: 50% (57/113, 56 left, 221 failed) (jobs: 1)^M

The path /nix/store/xb81cf7qgnyvsyvp9nfj15dmqchc24gk-ctypes-0.22.0/doc/ctypes/CHANGES.md does not exist,
I think the correct path is:

/nix/store/xb81cf7qgnyvsyvp9nfj15dmqchc24gk-ctypes-0.22.0
├── lib
│  └── ocaml
├── nix-support
│  ├── is-opam-nix-package
│  ├── propagated-build-inputs
│  ├── propagated-native-build-inputs
│  └── setup-hook
└── share
   ├── doc
   └── man

And the content of nix-support/setup-hook is:

opam__ctypes__bin="${opam__ctypes__bin-/nix/store/xb81cf7qgnyvsyvp9nfj15dmqchc24gk-ctypes-0.22.0/bin}"
opam__ctypes__build="${opam__ctypes__build-null}"
opam__ctypes__build_id="${opam__ctypes__build_id-null}"
opam__ctypes__dev="${opam__ctypes__dev-false}"
opam__ctypes__doc="${opam__ctypes__doc-/nix/store/xb81cf7qgnyvsyvp9nfj15dmqchc24gk-ctypes-0.22.0/share/doc}"
opam__ctypes__enable="${opam__ctypes__enable-enable}"
opam__ctypes__etc="${opam__ctypes__etc-/nix/store/xb81cf7qgnyvsyvp9nfj15dmqchc24gk-ctypes-0.22.0/etc}"
opam__ctypes__hash="${opam__ctypes__hash-null}"
opam__ctypes__installed="${opam__ctypes__installed-true}"
opam__ctypes__lib="${opam__ctypes__lib-/nix/store/xb81cf7qgnyvsyvp9nfj15dmqchc24gk-ctypes-0.22.0/lib/ocaml/5.1.1/site-lib}"
opam__ctypes__man="${opam__ctypes__man-/nix/store/xb81cf7qgnyvsyvp9nfj15dmqchc24gk-ctypes-0.22.0/share/man}"
opam__ctypes__name="${opam__ctypes__name-ctypes}"
opam__ctypes__opamfile="${opam__ctypes__opamfile-null}"
opam__ctypes__pinned="${opam__ctypes__pinned-false}"
opam__ctypes__prefix="${opam__ctypes__prefix-/nix/store/xb81cf7qgnyvsyvp9nfj15dmqchc24gk-ctypes-0.22.0}"
opam__ctypes__sbin="${opam__ctypes__sbin-/nix/store/xb81cf7qgnyvsyvp9nfj15dmqchc24gk-ctypes-0.22.0/bin}"
opam__ctypes__share="${opam__ctypes__share-/nix/store/xb81cf7qgnyvsyvp9nfj15dmqchc24gk-ctypes-0.22.0/share}"
opam__ctypes__version="${opam__ctypes__version-0.22.0}"
export OCAMLPATH=${OCAMLPATH-}${OCAMLPATH:+:}/nix/store/xb81cf7qgnyvsyvp9nfj15dmqchc24gk-ctypes-0.22.0/lib/ocaml/5.1.1/site-lib
export CAML_LD_LIBRARY_PATH=${CAML_LD_LIBRARY_PATH-}${CAML_LD_LIBRARY_PATH:+:}/nix/store/xb81cf7qgnyvsyvp9nfj15dmqchc24gk-ctypes-0.22.0/lib/ocaml/5.1.1/site-lib/stublibs

To Reproduce

nix build 'github:tweag/opam-nix#cf."0.5.0"'

OR

I made a minimal example that depends on cf: with_cf.

nix build

or only cf:

nix build .#cf

Expected behavior
build successfully

Environment

• OS name + version: macOS Sonoma 14.2.1 23C71 x86_64 (Darwin 23.2.0)
• Version of the code: github:tweag/opam-nix/464863fba44c7ecc50bd1a2967274482a2c33daf

"locked": {
  "lastModified": 1712645768,
  "narHash": "sha256-9dUh8nElGtC74Q4gIDV6DM0FKgF1oXh0PUkCxdbp+sg=",
  "owner": "tweag",
  "repo": "opam-nix",
  "rev": "464863fba44c7ecc50bd1a2967274482a2c33daf",
  "type": "github"
},

Additional context
Add any other context about the problem here.

Most recent stable version is not selected when using `"*"` in queries

Describe the bug
opam-nix selects ocaml 5.0.0~rc1 over ocaml 5.0.0 when using ocaml-base-compiler = "*"

To Reproduce

• Clone https://github.com/brendanzab/language-garden.git and checkout brendanzab/language-garden@97f9f2d
• Check the ocaml version:

  $ nix develop --command ocaml --version
  The OCaml toplevel, version 5.0.0

• Apply the following diff:

  diff --git a/flake.nix b/flake.nix
  index 012f4aa..2134b1d 100644
  --- a/flake.nix
  +++ b/flake.nix
  @@ -72,7 +72,7 @@
         # when building the opam project with `opam-nix`.
         query = devPackagesQuery // {
           # Force the ocaml compiler to be taken from opam-repository:
  -        ocaml-base-compiler = "5.0.0"; # Fix for opam-nix choosing 5.0.0~rc1 over 5.0.0
  +        ocaml-base-compiler = "*";
         };
  
         # Package-specific derivation overrides.

• Check the ocaml version:

  $ nix develop --command ocaml --version
  The OCaml toplevel, version 5.0.0~rc1

Expected behavior
opam-nix would select the most recent stable version when "*" is specified. Alternatively it would select the most recent unstable version (there are more recent versions of ocaml-base-compiler now).

Environment

• OS name + version: macOS 13.4.1
• Version of the code: e83bd1d

Unable to verify version bounds for a pin-depend

Describe the bug
When placing version bounds in depends as well as having a pin-depend for the same dependency, opam-nix (or its call to OPAM) fails to verify that the package is of the correct version (despite running OPAM manually succeeding):

error: builder for '/nix/store/9rlbv11lhf3ghpxdyclr1f2x2n0mxy47-resolve.drv' failed with exit code 20;
       last 5 log lines:
       > [ERROR] No solution including optional dependencies for cooltt.0.0 & ocaml-base-compiler:   * Missing dependency:
       >             - bantorra < 0.2
       >             no matching version
       >
       >

To Reproduce
flake.nix is exactly the same as in #12.

cooltt.opam (different):

opam-version: "2.0"
name: "cooltt"
version: "0.0"
maintainer: "Jonathan Sterling <[email protected]>"
authors: ["The RedPRL Development Team"]
homepage: "https://github.com/RedPRL/cooltt"
bug-reports: "https://github.com/RedPRL/cooltt/issues"
dev-repo: "git+https://github.com/RedPRL/cooltt.git"
synopsis: "Experimental implementation of Cartesian cubical type theory"
license: "Apache-2.0"
depends: [
  "dune" {>= "2.0"}
  "ocaml" {>= "4.10.0"}
  "ppx_deriving" {>= "4.4.1"}
  "bantorra" {>= "0.1" & < "0.2"}
  "bwd" {>= "1.2"}
  "cmdliner" {>= "1.1"}
  "containers" {>= "3.4"}
  "ezjsonm" {>= "1.2.0"}
  "menhir" {>= "20180703"}
  "uuseg" {>= "12.0.0"}
  "uutf" {>= "1.0.2"}
  "yuujinchou" {>= "2.0.0" & < "3"}
  "odoc" {with-doc}
  "kado"
]
pin-depends: [
  [ "kado.~dev" "git+https://github.com/RedPRL/kado" ]
  [ "bantorra.0.1.0" "git+https://github.com/RedPRL/bantorra#1e78633d9a2ef7104552a24585bb8bea36d4117b" ]
]
build: [
  ["dune" "build" "-p" name "-j" jobs]
  ["dune" "build" "-p" name "-j" jobs "@runtest"] {with-test}
  ["dune" "build" "-p" name "-j" jobs "@doc"] {with-doc}
]

Expected behavior
It should find the package and build normally.

Environment

• OS name + version: NixOS 22.05
• Version of the code: Latest as of writing

Use dune 3 with buildDuneProject

Currently, buildDuneProject insists on dune 2. Dune 3 is the latest version and should be used by default.

Unable to statically build package that depends on `dllunix`

Describe the bug
I'm trying to statically build a project that has a transitive dependency on dllunix.so and am getting the following error:

error: builder for '/nix/store/x4jq8kzc7m7awl74pxyib11sn92fy1lc-jsonrpc-static-x86_64-unknown-linux-musl-1.15.1-5.0.drv' failed with exit code 2;
       last 10 log lines:
       > building
       > File "_none_", line 1:
       > Alert ocaml_deprecated_auto_include:
       > OCaml's lib directory layout changed in 5.0. The unix subdirectory has been
       > automatically added to the search path, but you should add -I +unix to the
       > command-line to silence this alert (e.g. by adding unix to the list of
       > libraries in your dune file, or adding use_unix to your _tags file for
       > ocamlbuild, or using -package unix for ocamlfind).
       > Cannot load required shared library dllunix.
       > Reason: dllunix.so: dynamic loading not supported on this platform.
       For full logs, run 'nix log /nix/store/x4jq8kzc7m7awl74pxyib11sn92fy1lc-jsonrpc-static-x86_64-unknown-linux-musl-1.15.1-5.0.drv'.
error: 1 dependencies of derivation '/nix/store/ym9m7ipv68y04vmw6a9008290y9i4js6-lsp-static-x86_64-unknown-linux-musl-1.15.1-5.0.drv' failed to build
error (ignored): error: cannot unlink '/tmp/nix-build-menhirLib-static-x86_64-unknown-linux-musl-20230415.drv-2': Directory not empty
error: 1 dependencies of derivation '/nix/store/7gv25hzwjg801axwzpm5n6l728x0qxar-asai-static-x86_64-unknown-linux-musl-0.1.0_dev.drv' failed to build
error: 1 dependencies of derivation '/nix/store/18d0sz41m84cliq8w84waj5116rgg32x-polytt-static-x86_64-unknown-linux-musl-dev-env.drv' failed to build

To Reproduce
My project is not public but here is my flake.nix:

{
  inputs = {
    opam-nix.url = "github:tweag/opam-nix";
    flake-utils.url = "github:numtide/flake-utils";
    nixpkgs.follows = "opam-nix/nixpkgs";
  };
  outputs = { self, flake-utils, opam-nix, nixpkgs }@inputs:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = nixpkgs.legacyPackages.${system};
        on = opam-nix.lib.${system};

        localPackagesQuery = builtins.mapAttrs (_: pkgs.lib.last)
          (on.listRepo (on.makeOpamRepo ./.));

        query = {
          ocaml-base-compiler = "*";
        };

        scope = on.buildDuneProject { pkgs = pkgs.pkgsStatic; } "polytt" ./. query;

        overlay = self: super: {
          # Prevent unnecessary dependencies on the resulting derivation
          polytt = super.polytt.overrideAttrs (_: {
            removeOcamlReferences = true;
            postFixup = "rm -rf $out/nix-support";
          });
        };

        scope' = scope.overrideScope' overlay;

        # Packages in this workspace
        packages =
          pkgs.lib.getAttrs (builtins.attrNames localPackagesQuery) scope';
      in
      {
        legacyPackages = scope';

        packages = packages // { default = packages.polytt; };
      });
}

This should fail on any package that depends on, for example, the ocaml lsp package.

Expected behavior
I would expect a successful static build.

Environment

• Nixos 23.05
• Version of the code: ocaml 5.0

Additional context
Add any other context about the problem here.

Pin-depends do not work for packages outside of opam repository.

Describe the bug
It seems that the pin-depends feature implemented in opam-nix is currently only implemented as an override for packages that are already part of the official opam repository. The same functionality in opam allows to pin packages that are not part of the opam repository and I think similar treatment should be implemented in opam-nix.

To Reproduce

1. Create an opam file with a pin-depends section containing a package that is not part of the opam repository.
2. Use buildOpamProject or buildDuneProject with pinDepends = true.
3. Build the project and confirm the error (where the pinned package is called options in my case).

error: builder for '/nix/store/hd1mg3ylzq7j7v6psc867dlnvrjdh260-resolve.drv' failed with exit code 20;
       > [ERROR] No solution including optional dependencies for fmt & hello.dev:   * Missing dependency:
       >             - options
       >             unknown package

See the files below that can reproduce the described issue.

flake.nix

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/21.11";
    flake-utils.url = "github:numtide/flake-utils";
    nix-filter.url = "github:numtide/nix-filter";
    opam-nix.url = "path:./opam-nix";
  };
  outputs = { self, nixpkgs, flake-utils, nix-filter, opam-nix }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs { inherit system; };
        src = nix-filter.lib.filter {
          root = ./.;
          include = [
            "dune-project"
            "hello.opam"
            (nix-filter.lib.inDirectory "hello")
          ];
        };
        scope = opam-nix.lib.${system}.buildOpamProject {
          inherit pkgs;
          pinDepends = true;
        } "hello" src { fmt = null; };

      in rec {
        defaultPackage = scope.hello;
        defaultApp = flake-utils.lib.mkApp { drv = defaultPackage; };
        devShell = pkgs.mkShell {
          nativeBuildInputs = defaultPackage.nativeBuildInputs;
        };
      });
}

hello.opam

opam-version: "2.0"
build: [ ["dune" "build" "-p" name] ]
depends: [
  "ocaml" {= "4.13.1"}
  "dune" {build}
  "fmt"
  "options"
]
pin-depends: [
   ["options.dev" "git+https://github.com/odis-labs/options.git#master"]
]

Expected behavior
The expected behavior is for any kind of opam project outside of opam repository to be included in the resolution and build process.

Environment

• OS name + version: macOS 12.0.1 arm64
• Version of the code: 272fdbc (latest commit)

Additional context
From my cursory look at the code in opam.nix the issue seems to be that:

• in buildOpamProject the pinDepsOverlay is added to overlays for queryToScope;
• in queryToScope the overlays (containing pin deps) are applied after the resolution happens.

I think the potential solution could be expanding all pin-depends into separate repositories with makeOpamRepo and adding the pins to repos, instead of overlays.

opam package coq-of-ocaml not working

Describe the bug
Building an opam-nix derivation with coq-of-ocaml doesn't work.

To Reproduce
Add coq-of-ocaml to the dev deps of an opam nix build.

Expected behavior
For the package to work.

Additional context

error: builder for '/nix/store/xn35ysz8r9kwvyd51yir61vdy0mhpsw2-coq-of-ocaml-2.5.3+4.14.drv' failed with exit code 1;
       last 3 log lines:
       > unpacking sources
       > unpacking source archive /nix/store/47888vypivqxqgjkhk3kfz3fhrah6yw3-coq-of-ocaml-full.2.5.3+4.14.tar.gz
       > unpacker produced multiple directories
       For full logs, run 'nix log /nix/store/xn35ysz8r9kwvyd51yir61vdy0mhpsw2-coq-of-ocaml-2.5.3+4.14.drv'.
error: 1 dependencies of derivation '/nix/store/zlkvpv32nb6gi7n0vb2vs71cnim5hspg-nix-shell-env.drv' failed to build