alexxnica / nixcrates

reads rust-lang/crates.io-index and outputs nix expressions into fractalide/nix-crates-index

nixcrates

An early stage drop-in replacement for cargo which uses crates.io index.

The very awesome guys over at https://nixcloud.io implemented this for fractalide. Please do consider contracting them for your hardcore nix related jobs!

Usage

Iterative updates (you'll mostly need this)

1. initial setup of environment, thereafter not needed:

        $ cd $dev # your development directory
        $ mkdir nixcrates && cd nixcrates
        $ git clone https://github.com/rust-lang/crates.io-index
        $ git clone https://github.com/fractalide/nix-crates-index.git
        $ fork https://github.com/fractalide/nixcrates.git
        $ git clone https://github.com/your_account/nixcrates.git
        $ cd nixcrates
        $ nix-env -f default.nix -iA nixcrates

2. update the crates.io-index repository:

        $ cd crates.io-index
        $ git pull
        $ git rev-parse master
        $ fc336466a7ede6f14ce427deb5a64c5d02d14be0

3. use nixcrates to update it

        $ cd nix-crates-index/
        $ rm -r */
        $ nixcrates ../crates.io-index/ ./
        $ git add *
        # just use the rev from the crates.io-index below        
        $ git commit -m 'crate.io rev @ fc3364, <explain what feature you want or why you're bumping it>'

4. send a pull request!

5. for added security add your crates to the allTargets attribute nixcrates/default.nix to check that they built properly

    nix-build -A allTargets
   

Build examples

Currently we use rustc nightly from most recent nipxkgs master!

Example targets:

git clone https://github.com/fractalide/nixcrates.git
cd nixcrates
nix-build default.nix -A nixcrates
nix-build default.nix -A allTargets
nix-build default.nix -A allCrates.rustache
nix-build default.nix -A allCrates.serde

Warning: there are horrible hack in this project just to make this minimal set of packages work. we were under a lot of time-pressure and low in funding. on judgement day, hopefully god didn't see it!

How it works

Parse the JSON files of github.com/rust-lang/crates.io-index repository and translating them into nix files. Considering dependencies we are currently only supporting versioning via ~ and ^ as well as single versions. If anything else is used we switch to the newest version. In case of ^ and ~ we are using the highest allowed version.

What do versions look like when created?

{package_name} points on the newest version. all__{package_name} contains a set of all versions.

There are also several shortcuts to address the newest subversion. For every missing semver version number simply the newest is used. For example all__{package_name}.{package_name}_0_1 points the the version of {package_name} with 0.1.X where X is the highest number found.

What failes and why?

Missing dependencies

For some reason sometimes the crates.io json file is not listing all the dependencies. If this occurs the missing dependency has to be added by hand. However editing an autogenerated file is a pain. Therefore this can be done in the nix-crates-index/default.nix file. At the bottom of the file you can append additional dependencies. For this see the rusqlite crate example where pkg-config was added.

Resolving dependencies

Currently semver is resolved by choosing the highest allowed version from rust-lang/cartes.io-index repository. This works for targets called allTargets. However here is a constructed case that would fail:

For example your project has 2 dependencies {dep_a} and {dep_b}. Lets assume:

• {dep_a} depends on {dep_c} with 0.1.1 <= version <= 0.2.0
• {dep_b} depends on {dep_c} with 0.0.1 <= version <= 0.1.2

Now our repo would compile dep_a with dep_c_0_2_0 and dep_b with dep_c_0_1_2. This is a problem as soon as {dep_a} as well as {dep_b} are exposing types from {dep_c} in their interfaces.

Not supported stuff... yet please do help out with this

Bundled C-code

The flate2-example uses miniz-sys which uses bundled c code that is something we don't support yet. To see the error:

nix-build -A allCrates.miniz-sys

Override x_crate with a local x_crate, this helps develop locally

Xml-rs != xml, crate name and extern names don't match, a mapping file needs to be passed in.

What about build.rs files?

Tests

Crates come with tests but we didn't have time to implement even though we would have loved to do that. Building crates with tests requires dev-dependencies to be around. We currently just remove them because they caused a lot cyclic dependencies which Nix (and even cargo) can't handle.

Dependencies

Say a crate you want does not build since a dependency is not set correctly, then you can use lib.recursiveUpdate in nix-crates-index/default.nix to change/extend them. This holds also true for pkgs from nixpkgs!