tiiuae / sbomnix

A suite of utilities to help with software supply chain challenges on nix targets

sbomnix

This repository is home to various command line tools and Python libraries that aim to help with software supply chain challenges:

• sbomnix is a utility that generates SBOMs given Nix derivation or out path.
• nixgraph helps query and visualize dependency graphs for Nix derivation or out path.
• nixmeta summarizes nixpkgs meta-attributes from the given nixpkgs version.
• vulnxscan is a vulnerability scanner demonstrating the usage of SBOMs in running vulnerability scans.
• repology_cli and repology_cve are command line clients to repology.org.
• nix_outdated is a utility that finds outdated nix dependencies for given out path, listing the outdated packages in priority order based on how many other packages depend on the given outdated package.
• provenance is a command line tool to generate SLSA v1.0 compliant provenance attestation files in json format for any nix flake or derivation.

For an example of how to use the tooling provided in this repository to automate daily vulnerability scans for a nix flake project, see: ghafscan.

The CycloneDX and SPDX SBOMs for each release of sbomnix tooling is available in the release assets.

All the tools in this repository originate from Ghaf Framework.

Table of Contents

• Getting Started
  • Running as Nix Flake
  • Running from Nix Development Shell
• Buildtime vs Runtime Dependencies
  • Buildtime Dependencies
  • Runtime Dependencies
• Usage Examples
  • Generate SBOM Based on Derivation File or Out-path
  • Generate SBOM Including Meta Information
  • Generate SBOM Including Buildtime Dependencies
  • Generate SBOM Based on Result Symlink
  • Generate SBOM Based on Flake Reference
  • Visualize Package Dependencies
• Contribute
• License
• Acknowledgements

Getting Started

sbomnix requires common Nix tools like nix and nix-store. These tools are expected to be in $PATH.

Running as Nix Flake

sbomnix can be run as a Nix flake from the tiiuae/sbomnix repository:

# '--' signifies the end of argument list for `nix`.
# '--help' is the first argument to `sbomnix`
$ nix run github:tiiuae/sbomnix#sbomnix -- --help

or from a local repository:

$ git clone https://github.com/tiiuae/sbomnix
$ cd sbomnix
$ nix run .#sbomnix -- --help

See the full list of supported flake targets by running nix flake show.

Running from Nix Development Shell

If you have nix flakes enabled, start a development shell:

$ git clone https://github.com/tiiuae/sbomnix
$ cd sbomnix
$ nix develop

You can also use nix-shell to enter the development shell:

$ git clone https://github.com/tiiuae/sbomnix
$ cd sbomnix
$ nix-shell

Keep in mind this doesn't add the various entrypoint binaries to your PATH directly. They are produced during the setuptools build.

While you're in the devshell, you can run various command line tools via the entrypoint files directly:

# sbomnix:
$ src/sbomnix/main.py --help

# nixgraph:
$ src/nixgraph/main.py --help

# nixmeta:
$ src/nixmeta/main.py --help

# vulnxscan:
$ src/vulnxscan/vulnxscan_cli.py --help

# repology_cli:
$ src/repology/repology_cli.py --help

# repology_cve:
$ src/repology/repology_cve.py --help

# nix_outdated:
$ src/nixupdate/nix_outdated.py --help

# provenance:
$ src/provenance/main.py --help

Buildtime vs Runtime Dependencies

Buildtime Dependencies

Closure of a nix store path is a list of all the dependent store paths, recursively, referenced by the target store path. For a package, the closure of it's derivation lists all the buildtime dependencies. As an example, for a simple C program, the buildtime dependencies include packages to bootstrap gcc, stdenv, glibc, bash, etc. on the target architecture. Even a simple hello-world C program typically includes over 150 packages in its list of buildtime dependencies. It's important to note that generating buildtime dependencies in Nix does not require building the target.

For reference, following is a link to graph from an example hello-world C program that includes the first two layers of buildtime dependencies: direct dependencies and the first level of transitive dependencies: C hello-world buildtime, depth=2.

Runtime Dependencies

Runtime dependencies are a subset of buildtime dependencies. Nix automatically determines the runtime dependencies by scanning the generated output paths (i.e. build output) for the buildtime dependencies' store paths. This means nix needs to build the target output first, before runtime dependencies can be determined. For reference, below is a complete runtime dependency graph of an example hello-world C program:

By default, where applicable, the tools in this repository assume runtime dependencies. This means, for instance, that unless specified otherwise, sbomnix will output an SBOM including runtime-only dependencies, nixgraph outputs runtime dependency graph, and vulnxscan and nix_outdated scan runtime dependencies. Since Nix needs to build the target output to determine the runtime dependencies, all the tools in this repository will also build (force-realise) the target output as part of each tool's invocation when determining the runtime dependencies. All the mentioned tools in this repository also support working with buildtime dependencies instead of runtime dependencies with the help of --buildtime command line argument. As mentioned earlier, generating buildtime dependencies in Nix does not require building the target. Similarly, when --buildtime is specified, the tools in this repository do not need to be build the given target.

Usage Examples

The usage examples work for both the built package, as well as inside the devshell.

Keep in mind inside the devshell, calls to sbomnix need to be replaced with src/sbomnix/main.py (and similar for other entrypoints).

In the below examples, we use Nix package wget as an example target. To print wget out-path on your local system, try:

$ nix eval -f '<nixpkgs>' 'wget.outPath'
"/nix/store/8nbv1drmvh588pwiwsxa47iprzlgwx6j-wget-1.21.3"

Generate SBOM Based on Derivation File or Out-path

By default sbomnix scans the given target and generates an SBOM including the runtime dependencies. Notice: determining the target runtime dependencies in Nix requires building the target.

# Target can be specified with flakeref too, e.g.:
# sbomnix .
# sbomnix github:tiiuae/sbomnix
# sbomnix nixpkgs#wget
# Ref: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake.html#flake-references
$ sbomnix /nix/store/8nbv1drmvh588pwiwsxa47iprzlgwx6j-wget-1.21.3
...
INFO     Wrote: sbom.cdx.json
INFO     Wrote: sbom.spdx.json
INFO     Wrote: sbom.csv

Main outputs are the SBOM json files sbom.cdx.json and sbom.spdx.json in CycloneDX and SPDX formats.

Generate SBOM Including Buildtime Dependencies

By default sbomnix scans the given target for runtime dependencies. You can tell sbomnix to determine the buildtime dependencies using the --buildtime argument. Below example generates SBOM including buildtime dependencies. Notice: as opposed to runtime dependencies, determining the buildtime dependencies does not require building the target.

$ sbomnix /nix/store/8nbv1drmvh588pwiwsxa47iprzlgwx6j-wget-1.21.3 --buildtime

Generate SBOM Based on Result Symlink

sbomnix can be used with output paths too (e.g. anything which produces a result symlink):

$ sbomnix /path/to/result 

Generate SBOM Based on Flake Reference

sbomnix also supports scanning flake references:

$ sbomnix github:NixOS/nixpkgs?ref=nixos-unstable#wget --buildtime

Visualize Package Dependencies

sbomnix finds the package dependencies using nixgraph. Moreover, nixgraph can also be used as a stand-alone tool for visualizing package dependencies. Below, we show an example of visualizing package wget runtime dependencies:

$ nixgraph /nix/store/8nbv1drmvh588pwiwsxa47iprzlgwx6j-wget-1.21.3 --depth=2

Which outputs the dependency graph as an image (with maxdepth 2):

For more examples on querying and visualizing the package dependencies, see: nixgraph.

Contribute

Any pull requests, questions and error reports are welcome. To start development, we recommend using Nix flakes development shell:

$ git clone https://github.com/tiiuae/sbomnix
$ cd sbomnix/
# Optionally, install git hooks to check the git commit message
$ ./githooks/install-git-hooks.sh
$ nix develop

Run make help to see the list of supported make targets. Prior to sending any pull requests, make sure at least the make pre-push runs without failures.

To deactivate the Nix devshell, run exit in your shell. To see other Nix flake targets, run nix flake show.

License

This project is licensed under the Apache-2.0 license - see the Apache-2.0.txt file for details.

Acknowledgements

sbomnix uses Nix store derivation scanner (nix.py and derivation.py) originally from vulnix.

sbomnix's Issues

how to `nix build` vulnxscan?

nix build .#sbomnix works fine

but nix build .#vulnxscan gives

does not provide attribute 'packages.x86_64-linux.vulnxscan', 'legacyPackages.x86_64-linux.vulnxscan' or 'vulnxscan'

I see in the flake.nix that vulnxscan is an app. And it is clear to me than an app is meant to be nix run-d.

Except that if I run it this way, no gc root is created, and after reboot, all the derivations are gone, and I have to go make coffee while the run command fetches everything again. In my testing, only actually building the app/package will cause a gc root to be created, thereby caching the derivation.

Sorry if this is a stupid question, nix has been just head banging from the get go for me, I've only started using it over the weekend.

allow reading runtime dependencies without requiring the package availability in local nix store

Currently, to get the runtime dependencies for a given target, sbomnix requires the target to be available in local nix store.
Note: for buildtime-only dependencies, only the derivation file is required.

The task in this item is to change sbomnix so that runtime dependencies can also be found without requiring the package be installed locally.

Example (nmap is not locally installed):

Buildtime:

sbomnix /nix/store/n54bb3k7p568x2i6abvkavmzc2nwgihh-nmap-7.93.drv --type=buildtime

WARNING  Command line argument '--meta' missing: SBOM will not include license information (see '--help' for more details)
INFO     Loading buildtime dependencies referenced by '/nix/store/n54bb3k7p568x2i6abvkavmzc2nwgihh-nmap-7.93.drv'
INFO     Wrote: sbom.cdx.json
INFO     Wrote: sbom.spdx.json
INFO     Wrote: sbom.csv

Runtime:

sbomnix /nix/store/n54bb3k7p568x2i6abvkavmzc2nwgihh-nmap-7.93.drv --type=runtime
WARNING  Command line argument '--meta' missing: SBOM will not include license information (see '--help' for more details)
INFO     Loading runtime dependencies referenced by '/nix/store/n54bb3k7p568x2i6abvkavmzc2nwgihh-nmap-7.93.drv'
Traceback (most recent call last):
  File "/home/hrosten/projects/sbomnix/venv/bin/sbomnix", line 33, in <module>
    sys.exit(load_entry_point('sbomnix', 'console_scripts', 'sbomnix')())
  File "/nix/store/g7mndp0nh7jy7xc9gxv1jjdl6jxac7hi-python3.10-sbomnix-1.4.5/lib/python3.10/site-packages/sbomnix/main.py", line 87, in main
    sbomdb = SbomDb(target_path, runtime, buildtime, args.meta)
  File "/nix/store/g7mndp0nh7jy7xc9gxv1jjdl6jxac7hi-python3.10-sbomnix-1.4.5/lib/python3.10/site-packages/sbomnix/sbomdb.py", line 48, in __init__
    self._init_dependencies(nix_path)
  File "/nix/store/g7mndp0nh7jy7xc9gxv1jjdl6jxac7hi-python3.10-sbomnix-1.4.5/lib/python3.10/site-packages/sbomnix/sbomdb.py", line 62, in _init_dependencies
    runtime_dependencies = NixDependencies(nix_path, buildtime=False)
  File "/nix/store/g7mndp0nh7jy7xc9gxv1jjdl6jxac7hi-python3.10-sbomnix-1.4.5/lib/python3.10/site-packages/nixgraph/graph.py", line 255, in __init__
    self._parse_runtime_dependencies(nix_path)
  File "/nix/store/g7mndp0nh7jy7xc9gxv1jjdl6jxac7hi-python3.10-sbomnix-1.4.5/lib/python3.10/site-packages/nixgraph/graph.py", line 261, in _parse_runtime_dependencies
    nix_out = exec_cmd(
  File "/nix/store/g7mndp0nh7jy7xc9gxv1jjdl6jxac7hi-python3.10-sbomnix-1.4.5/lib/python3.10/site-packages/sbomnix/utils.py", line 114, in exec_cmd
    raise error
  File "/nix/store/g7mndp0nh7jy7xc9gxv1jjdl6jxac7hi-python3.10-sbomnix-1.4.5/lib/python3.10/site-packages/sbomnix/utils.py", line 104, in exec_cmd
    ret = subprocess.run(cmd, capture_output=True, encoding="utf-8", check=True)
  File "/nix/store/syz2y6j53y5hpzbs7l0965zwxshi8iyl-python3-3.10.10/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['nix', '--extra-experimental-features', 'nix-command', 'path-info', '/nix/store/n54bb3k7p568x2i6abvkavmzc2nwgihh-nmap-7.93.drv']' returned non-zero exit status 1.

sbomnix: support aarch64-darwin

PR #58 added support for multiple systems, but @nikitawootten mentioned dropping support for aarch64-darwin due to failing tests.

This issue is a reminder to analyze the issue and, if possible, add sbomnix support for aarch64-darwin.

rewrite mapping nix packages to cpe identifiers

NVD plans to retire legacy data feeds on 09/2023: https://nvd.nist.gov/products/cpe

Currently, sbomnix uses NVD "CPE Dictionary" in mapping the nix pakcages to CPE identifiers, see: https://github.com/tiiuae/sbomnix/blob/main/scripts/cpedict/update-cpedict.sh and https://github.com/tiiuae/sbomnix/blob/main/sbomnix/cpe.py.

We need to rethink how to properly do this in sbomnix to make it more accurate and so that it does not rely on the to-be-retired NVD data feed.

All suggestions or ideas how to improve the CPE mapping are welcome.

nixgraph: allow loading state

Change nixgraph (cli) so that it can load an earlier state (sbom?), to allow visualizing and querying the build target graph ‘postmortem’ without having to re-produce the earlier build environment locally.

vulnxscan fails if the Nix output path is a JSON file

First of all, thanks for you project!

The nix2container project produces container image specification as JSON file. When running vulnxscan on a nix2container output path, it fails such as:

$ nix run github:tiiuae/sbomnix#vulnxscan -- ./result
CRITICAL Specified target is not a nix artifact: 'result'

$ file ./result
./result: symbolic link to /nix/store/7s4y6dcmfc6frqv38j8y6g7ifmazh5hx-image-bash.json

It seems to be because vulnxscan only consider non JSON file as Nix artifact:

sbomnix/scripts/vulnxscan/vulnxscan.py

Line 338 in f763f71

if _is_json(target_path_abs):

Instead of testing the file type, maybe you should run a Nix command on this store path to let Nix deciding if it is a Nix artefact or not. (nix-store -q <FILE> for instance).

Note this is related to this nix2container issue.

vulnxscan: add severity information

Add severity information to vulnxscan results for all scanners/databases that include the severity information.

improve the way sbomnix reads pacakge metadata

sbomnix reads the nix package meta info from the json file given via the command-line --meta option.

This needs to be re-done properly, ultimately aiming to get rid of the need for --meta command line argument completely.

Add option to scan nix profile?

I.e. it can take ~/.nix-profile/manifest.json as input

Currently you get:

% sbomnix ~/.nix-profile              
INFO     Evaluating '/Users/arian/.nix-profile'
INFO     Try force-realising store-path '/Users/arian/.nix-profile'
INFO     Loading runtime dependencies referenced by '/nix/store/pwcgic86vfhhdkpbh03cn7pv7a58vdqh-profile'
CRITICAL No deriver found for: '/nix/store/pwcgic86vfhhdkpbh03cn7pv7a58vdqh-profile

add tips&tricks page

add tips&tricks markdown page that would include some interesting non-obvious use-cases of sbomnix such as this:
https://discourse.nixos.org/t/how-to-get-package-maintainers-of-all-installed-packages/27307/12

and this, that creates a list of fetched urls given a result symlink:

# Run sbomnix; include both buildtime and runtime dependencies: 
$ nix run github:tiiuae/sbomnix#sbomnix -- ./result --type=both

# Query the sbom for fetched urls:
$ nix-shell -p csvkit --run \
  'csvsql --verbose --query "select urls from sbom where urls != '\'''\'' " sbom.csv' \
  | tr " " "\n" | grep -v urls | sort | uniq \
  > fetch_urls.txt

Example or documentation on how to use it via an API

Hello,

I'm currently investigating how I could potentially embed SBOMs in OCI containers made with Nix.
I would like to use this project which seems quite mature, but I wish I could use it through a Nix API, it would make sense to use it through it when building the container.

I have opened an issue at https://discourse.nixos.org/t/generate-sbom-from-oci-container-made-with-nix/39430, and I'm currently able to do what I want, but I wish I could also evaluate the use of this project too.

Thanks for shedding some lights on this!

scanning can take a long time, is there any ability to keep / cache the results for n days?

Running the vulnxscan command twice in a row shows that there is some caching going on, because the second run returns almost immediately.

Rebooting causes the same vulnxscan command to run for a long time again.

I there a way to have finer grained control over this caching behaviour? Say to invalidate the cache after n days.

Any advice (or clarification of how caching is done) is appreciated.

only produce `output_path` for the actually-used output

when creating the runtime sbom for the Gnome ISO with sbomnix $(nix-build nixos/release-combined.nix -A nixos.iso_gnome.x86_64-linux) on nixpkgs 0e74ca98a74bc7270d28838369593635a5db3260, the CycloneDX JSON output contains:

{
      "type": "application",
      "bom-ref": "/nix/store/vkx2srall0p9q68hm73zpwq9cmv8pmjg-qemu-host-cpu-only-8.2.1.drv",
      "name": "qemu-host-cpu-only",
      "version": "8.2.1",
      "purl": "pkg:nix/[email protected]",
      "cpe": "cpe:2.3:a:qemu-host-cpu-only:qemu-host-cpu-only:8.2.1:*:*:*:*:*:*:*",
      "description": "A generic and open source machine emulator and virtualizer",
      "licenses": [
        {
          "license": {
            "id": "GPL-2.0-or-later"
          }
        }
      ],
      "properties": [
        {
          "name": "nix:output_path",
          "value": "/nix/store/38nyj07s5k4l7dhnpjap5clqdgx3b38k-qemu-host-cpu-only-8.2.1-ga"
        },
        {
          "name": "nix:output_path",
          "value": "/nix/store/c7cw8hgpb1wks5f91ijkm1xv9nmp2zk8-qemu-host-cpu-only-8.2.1"
        },
        {
          "name": "nix:drv_path",
          "value": "/nix/store/vkx2srall0p9q68hm73zpwq9cmv8pmjg-qemu-host-cpu-only-8.2.1.drv"
        },
        {
          "name": "homepage",
          "value": "https://www.qemu.org/"
        }
      ]
    },

However, looking at the output of nix-store -q --tree $(nix-build nixos/release-combined.nix -A nixos.iso_gnome.x86_64-linux), I would expect only the /nix/store/38nyj07s5k4l7dhnpjap5clqdgx3b38k-qemu-host-cpu-only-8.2.1-ga output path and not /nix/store/c7cw8hgpb1wks5f91ijkm1xv9nmp2zk8-qemu-host-cpu-only-8.2.1

sbomnix: add option to include only direct dependencies

Maybe a --depth argument as seen in nixgraph

TypeError: sequence item 0: expected str instance, list found

When running sbomnix, I get the following error:

Traceback (most recent call last):
  File "/nix/store/mzck8rim82y9vx4xm18dp312gn88vni2-python3.10-sbomnix-1.4.6/bin/.sbomnix-wrapped", line 9, in <module>
    sys.exit(main())
  File "/nix/store/mzck8rim82y9vx4xm18dp312gn88vni2-python3.10-sbomnix-1.4.6/lib/python3.10/site-packages/sbomnix/main.py", line 100, in main
    sbomdb = SbomDb(target_path, runtime, buildtime, args.meta, args.depth)
  File "/nix/store/mzck8rim82y9vx4xm18dp312gn88vni2-python3.10-sbomnix-1.4.6/lib/python3.10/site-packages/sbomnix/sbomdb.py", line 55, in __init__
    self._init_sbomdb()
  File "/nix/store/mzck8rim82y9vx4xm18dp312gn88vni2-python3.10-sbomnix-1.4.6/lib/python3.10/site-packages/sbomnix/sbomdb.py", line 105, in _init_sbomdb
    self._sbomdb_join_meta(self.meta_path)
  File "/nix/store/mzck8rim82y9vx4xm18dp312gn88vni2-python3.10-sbomnix-1.4.6/lib/python3.10/site-packages/sbomnix/sbomdb.py", line 116, in _sbomdb_join_meta
    df_meta = _parse_json_metadata(meta_path)
  File "/nix/store/mzck8rim82y9vx4xm18dp312gn88vni2-python3.10-sbomnix-1.4.6/lib/python3.10/site-packages/sbomnix/sbomdb.py", line 471, in _parse_json_metadata
    setcol("meta_maintainers_email", []).append(";".join(emails))
TypeError: sequence item 0: expected str instance, list found

The json metadata was created using the following command:

nix-env -qa --meta --json '.*' > ~/nix-meta.json

In case it's important, I invoked sbomnix using the following command:

nix run github:tiiuae/sbomnix#sbomnix -- $(readlink result) --type=both --meta=nix-meta.json

Packaging latest version in nixpkgs: "error: illegal path references in fixed-output derivation"

Hi,

I tried updating sbomnix (1.4.5 -> 1.6.0) in nixpkgs but hit this weird issue:

$ nix-build -A sbomnix
these 2 derivations will be built:
  /nix/store/g9zk3n7zhj4bgb22llns0y2kh2d89nr1-source.drv
  /nix/store/0y11vlrxgs9zlaxfv6dm6dkbclc0ml6h-sbomnix-1.6.0.drv
building '/nix/store/g9zk3n7zhj4bgb22llns0y2kh2d89nr1-source.drv'...

trying https://github.com/tiiuae/sbomnix/archive/refs/tags/v1.6.0.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  163k    0  163k    0     0   329k      0 --:--:-- --:--:-- --:--:--  329k
unpacking source archive /build/v1.6.0.tar.gz
error: illegal path references in fixed-output derivation '/nix/store/g9zk3n7zhj4bgb22llns0y2kh2d89nr1-source.drv'
error: 1 dependencies of derivation '/nix/store/0y11vlrxgs9zlaxfv6dm6dkbclc0ml6h-sbomnix-1.6.0.drv' failed to build

Not sure if this is an issue with Nix, nixpkgs or sbomnix, but thought I should start documenting the issue somewhere.

sbomnix: go and rust dependencies

Dependencies from rust and go projects are missing because they fetch their dependencies directly, so the dependencies are not included in the dependency trees produced by nix-store --query --graph, which is what sbomnix uses internally.

For rust, this should be fixed with: NixOS/nixpkgs#217084, but the problem still persists for all other ecosystems that fetch their own dependencies.

All suggestions for how to improve the dependency lookup for sbomnix are welcome.