GithubHelp home page GithubHelp logo

nix-community / trustix Goto Github PK

View Code? Open in Web Editor NEW
248.0 7.0 21.0 3.95 MB

Trustix: Distributed trust and reproducibility tracking for binary caches [maintainer=@adisbladis]

Home Page: https://nix-community.github.io/trustix/

Makefile 0.18% Go 71.18% Nix 9.59% Shell 0.43% TeX 0.04% Procfile 0.38% JavaScript 0.24% HTML 0.15% TypeScript 17.76% CSS 0.05%

trustix's Introduction

Trustix - A new model for Nix binary substitutions

Trustix is a tool that compares build outputs for a given build input across a group of independent providers to establish trust in software binaries.

Overview

We often use pre-built software binaries and trust that they correspond to the program we want. But nothing assures that these binaries were really built from the program's sources and reasonable built instructions. Common, costly supply chain attacks exploit this to distribute malicious software, which is one reason why most software is delivered through centralized, highly secured providers. Trustix, a tool developed via an NGI0 PET grant, establishes trust in binaries in a different, decentralized manner. This increases security, and paves the way for an internet where small providers can deliver safe code, ultimately with a safer and larger offer for the user.

Trustix is developed for the Nix ecosystem.

How does this translate to Nix?

In the Nix ecosystem, pre-built binaries are distributed through so-called binary substituters. Similar to other centralized caching systems, they are a single point of failure in the chain of trust when delivering a package to a user. This is problematic for several reasons:

First, if anyone manages to compromise the NixOS Hydra build machines and its keys, they could upload backdoored builds to users. In the Nix ecosystem, a compromised key is even more dangerous because https://cache.nixos.org can't use a rolling key because of the way it is set up. This means that a compromised key would realistically mean that all packages in the cache are compromised. They would have to be rebuilt or garbage collected which is very costly.

Second, the NixOS Hydra hardware, on which the binaries are built, may also be compromised and not considered trustworthy by more security conscious users.

For some more background see the original project announcement.

Trustix design

Trustix aims to solve this problem via distributed trust & trust agility. Essentially it compares build outputs across a group of independent builders that log and exchange hashes of build input/output pairs. This is achieved through the following methodology:

  • Each builder is associated with a public-private key pair
  • In a post-build hook the output hash (NAR hash) of the build is uploaded to a ledger (a signed append-only log of build results).

This allows a user to trust binary substitutions based on an M-of-N vote among the participating builders.

Here is an example: Let's say we have 4 builders configured: Alice, Bob, Chuck & Dan. We have configured Trustix to require a 3/4 majority for a build to be trusted. Alice, Bob, Dan and Chuck all claim to have built the hello derivation. All builders participate in the Trustix network and communicate precisely what they have built with a hash that describes the build inputs of hello, and what have obtained as output with another hash. For the same input, the first 3 builders have arrived at the same output hash but Chuck has obtained something different.

This information can now be used by a Trustix user to:

  • track build reproducibility across a large number of builders.
  • trust only builds that have been confirmed by a majority of selected builders.
  • automatically identify and exclude misbehaving builders such as Chuck in above's example.

Related Projects

Documentation

Documentation is built as a part of CI and published on Github Pages.

Further resources

A collection of external resources to dive deeper into Trustix and its ideas.

Developing

For notes on development see HACKING.md

About the project

The developmentent of Trustix has been sponsored by Tweag I/O and funded by the NLNet foundation and the European Commission’s Next Generation Internet programme through the NGI Zero PET (privacy and trust enhancing technologies) fund.

NGI0 logo NLNet banner Tweag logo

trustix's People

Contributors

06kellyjac avatar adisbladis avatar bbigras avatar cgeorgii avatar chkno avatar davidak avatar flokli avatar hxr404 avatar jurraca avatar mkaito avatar mmesch avatar peterbourgon avatar profpatsch avatar raboof avatar radvendii avatar renovate[bot] avatar zimbatm avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

trustix's Issues

error: A definition for option `services.trustix.deciders.nix' is not of type `submodule'.

Describe the bug

[root@gaming:~]# nixos-rebuild switch
building Nix...
building the system configuration...
error: A definition for option `services.trustix.deciders.nix' is not of type `submodule'. Definition values:
       - In `/etc/nixos/configuration.nix':
           [
             {
               engine = "percentage";
               percentage = {
                 minimum = 66;
           ...
(use '--show-trace' to show detailed location information)

To Reproduce

https://tweag.github.io/trustix/binarycache.html#trustix---binary-cache-setup

Expected behavior

no errors when following documentation

Environment
OS name + version: NixOS 22.05pre375061.c777cdf5c56
Version of the code: 7ed5270

Additional context
Add any other context about the problem here.

remote "https://demo.trustix.dev" fails

The documentation says i should configure this demo remote: https://nix-community.github.io/trustix/howto-nix/subscribing.html#configuring

But it does not work!

...
Dec 03 11:23:30 gaming trustix[8856]: time="2022-12-03T11:23:30+01:00" level=error msg="Error while getting logs from 'https://demo.trustix.dev': unavailable: x509: certificate is valid for r13y.trustix.dev, not demo.trustix.dev"
Dec 03 11:23:30 gaming trustix[8856]: panic: runtime error: invalid memory address or nil pointer dereference
Dec 03 11:23:30 gaming trustix[8856]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0xa6f5d7]
Dec 03 11:23:30 gaming trustix[8856]: goroutine 23 [running]:
Dec 03 11:23:30 gaming trustix[8856]: github.com/nix-community/trustix/packages/trustix/internal/pool.(*ClientPool).Dial.func1()
Dec 03 11:23:30 gaming trustix[8856]:         github.com/nix-community/trustix/packages/trustix/internal/pool/pool.go:225 +0x157
Dec 03 11:23:30 gaming trustix[8856]: created by github.com/nix-community/trustix/packages/trustix/internal/pool.(*ClientPool).Dial
Dec 03 11:23:30 gaming trustix[8856]:         github.com/nix-community/trustix/packages/trustix/internal/pool/pool.go:218 +0xf8
Dec 03 11:23:30 gaming systemd[1]: trustix.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Dec 03 11:23:30 gaming systemd[1]: trustix.service: Failed with result 'exit-code'.

The other domain does also not work:

...
Dec 03 11:30:11 gaming trustix[12976]: time="2022-12-03T11:30:11+01:00" level=error msg="Error while getting logs from 'https://r13y.trustix.dev': unknown: 405 Method Not Allowed"
Dec 03 11:30:11 gaming trustix[12976]: panic: runtime error: invalid memory address or nil pointer dereference
Dec 03 11:30:11 gaming trustix[12976]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0xa6f5d7]
Dec 03 11:30:11 gaming trustix[12976]: goroutine 23 [running]:
Dec 03 11:30:11 gaming trustix[12976]: github.com/nix-community/trustix/packages/trustix/internal/pool.(*ClientPool).Dial.func1()
Dec 03 11:30:11 gaming trustix[12976]:         github.com/nix-community/trustix/packages/trustix/internal/pool/pool.go:225 +0x157
Dec 03 11:30:11 gaming trustix[12976]: created by github.com/nix-community/trustix/packages/trustix/internal/pool.(*ClientPool).Dial
Dec 03 11:30:11 gaming trustix[12976]:         github.com/nix-community/trustix/packages/trustix/internal/pool/pool.go:218 +0xf8
Dec 03 11:30:11 gaming systemd[1]: trustix.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Dec 03 11:30:11 gaming systemd[1]: trustix.service: Failed with result 'exit-code'.

Please setup a working demo and update the documentation accordingly.

Automatically generate keys when missing

Manually generating keys causes a lot of issues (#81), especially when the documentation is not correct!

To make it more straight forward, the keys should get automatically generated (as suggested in #28 (comment)), so the user does not have to do anything.

I could add this in the systemd service, but i think it would be better to add it to the software directly, so it does also work when used not on nixos. I know several programs that does that like Caddy and BOINC.

Conflicting systemd socket definitions between trustix and trustix-nix-cache services

Describe the bug

When following the directions in https://tweag.github.io/trustix/binarycache.html , should both services.trustix.enable=true and services.trustix-nix-cache.enable=true be set?

If I set them both, the NixOS configuration evaluation fails, saying:

error: The option `systemd.sockets.trustix.description' has conflicting definition values:
  - In `/nix/store/.../packages/trustix/nixos': "Socket for the Trustix daemon"
  - In `/nix/store/.../packages/trustix-nix/nixos/binarycache.nix': "Socket for the Trustix Nix binary cache daemon"

If I set only services.trustix-nix-cache.enable=true, the service does not start:

systemd[1]: trustix.socket: Socket service trustix.service not loaded, refusing.
systemd[1]: Failed to listen on Socket for the Trustix Nix binary cache daemon.
systemd[1]: Dependency failed for Trustix Nix binary cache daemon.
systemd[1]: trustix-nix-cache.service: Job trustix-nix-cache.service/start failed with result 'dependency'.

To Reproduce

$ git clone https://github.com/chkno/trustix-integration-tests
$ cd trustix-integration-tests
$ git checkout issue24
$ nix --experimental-features 'nix-command flakes' --print-build-logs build .#checks.i686-linux.one-publisher
...
error: The option `systemd.sockets.trustix.description' has conflicting definition values:
...
$ git checkout HEAD^
$ nix --experimental-features 'nix-command flakes' --print-build-logs build .#checks.i686-linux.one-publisher
...
vm-test-run-one-publisher> clint # [  256.598503] systemd[1]: trustix.socket: Socket service trustix.service not loaded, refusing.
vm-test-run-one-publisher> clint # [  256.605378] systemd[1]: Failed to listen on Socket for the Trustix Nix binary cache daemon.
vm-test-run-one-publisher> clint # [  256.612300] systemd[1]: Dependency failed for Trustix Nix binary cache daemon.
vm-test-run-one-publisher> clint # [  256.620804] systemd[1]: trustix-nix-cache.service: Job trustix-nix-cache.service/start failed with result 'dependency'.
...
vm-test-run-one-publisher> Exception: command `grep hello /var/log/local-builds` unexpectedly succeeded

Expected behavior
There is a concrete-enough example somewhere, either in documentation or test cases, to get a Trustix instance running.

Environment

error: The option `services.trustix-nix-build-hook.logID' is used but not defined.

Describe the bug

[root@gaming:~]# time nixos-rebuild switch
building Nix...
building the system configuration...
error: The option `services.trustix-nix-build-hook.logID' is used but not defined.

Related to #19.

To Reproduce

https://tweag.github.io/trustix/publishing.html

Expected behavior

The config needs to be like this:

  # Push local builds via the post-build hook
  services.trustix-nix-build-hook = {
    enable = true;
    logID = "";
  };

But i don't know what to put in there.

Environment

  • OS name + version:
  • Version of the code:

Additional context

https://github.com/tweag/trustix/blob/fe726118f9f6ecd9739554ac16f32b499ad7a981/packages/trustix-nix/nixos/post-build-hook.nix#L28

Should it generate an ID here?

https://github.com/tweag/trustix/blob/f4e0241b837b9a090af2e59953f3fdef350a3a16/packages/trustix/internal/protocols/protocols.go#L49

How to use

Is your feature request related to a problem? Please describe.
I'm very interested in using this, but i don't know how. There is no documentation.

Describe the solution you'd like
Add a Usage section to the README. Describe how to install the tool. Describe how to reproduce the hello package build. Maybe a more complicated example.

Describe alternatives you've considered
everyone asks on IRC or finds some outdated instructions with google and everyone is frustrated with this project

Additional context
Documentation is essential for the success of a project. When no one know how to use it, no one can/will use it.

Add NLNet and NGI Zero Pet logos and information to the README

Describe the desired solution

Add logos as found here to the README, as well as short mention:

The developmentent of Trustix is carried out by Tweag I/O and funded by the NLNet foundation and the European Commission’s Next Generation Internet programme through the NGI Zero PET (privacy and trust enhancing technologies) fund.

Additional context
NGI0_tag
nlnet-banner

@MMesch Could you double check the wording above?

The format for the trustix service is different to docs and requires I set values for all deciders

Describe the bug
A clear and concise description of what the bug is.

I can only successfully build with this config

  services.trustix = {
    enable = true;
    deciders.nix = {
      engine = "percentage";
      percentage.minimum = 66;
      javascript.minimum = "blah";
      logid.minimum = "blah";
    };
  };

If I don't set values for javascript.minimum or logid.minimum:

  services.trustix = {
    enable = true;
    deciders.nix = {
      engine = "percentage";
      percentage.minimum = 66;
      # javascript.minimum = "bob";
      # logid.minimum = "bob";
    };
  };

I get this error error: The option services.trustix.deciders.nix.javascript' is used but not defined.`

And using the example from the docs here https://tweag.github.io/trustix/binarycache.html#configuring:

error: syntax error, unexpected '{'

       at /nix/store/apg3j6wchilb4vzk1hcv0agkp3pllihj-source/nixos/hosts/whatever/configuration.nix:90:9:

           89|       {
           90|         {
             |         ^
           91|           type = "percentage";

To Reproduce
Steps to reproduce the behavior.

Clone the repo, use the nixos module, try the configs above

Expected behavior
A clear and concise description of what you expected to happen.

Environment

  • OS name + version:
  • Version of the code:

Additional context
Add any other context about the problem here.

Fails to start after update

I updated my system to NixOS 22.05.4270.b68a6a27adb and trustix from 32ee78a to c852d3c.

After the nixos-rebuild switch, trustix-nix-cache.service and trustix.service do not start.

warning: the following units failed: trustix-nix-cache.service, trustix.service

Here, for some reason, it can't read the private key anymore.

Dec 03 06:58:03 gaming systemd[1]: Stopping Trustix daemon...
Dec 03 06:58:23 gaming trustix[1106]: time="2022-12-03T06:58:23+01:00" level=error msg="Could not update STH" error="rpc error: code = Unavailable desc = timed out waiting for server handshake" logI>
Dec 03 06:58:23 gaming systemd[1]: trustix.service: Deactivated successfully.
Dec 03 06:58:23 gaming systemd[1]: Stopped Trustix daemon.
Dec 03 06:58:23 gaming systemd[1]: trustix.service: Consumed 1d 47min 58.636s CPU time, no IP traffic.
Dec 03 06:58:25 gaming systemd[1]: Started Trustix daemon.
Dec 03 06:58:25 gaming trustix[2956960]: time="2022-12-03T06:58:25+01:00" level=info msg="Creating state directory" directory=.
Dec 03 06:58:25 gaming trustix[2956960]: time="2022-12-03T06:58:25+01:00" level=info msg="Creating signer" name=davidak type=ed25519
Dec 03 06:58:25 gaming trustix[2956960]: Error: open /var/trustix/keys/private: permission denied
Dec 03 06:58:25 gaming trustix[2956960]: Usage:
Dec 03 06:58:25 gaming trustix[2956960]:   trustix daemon [flags]
Dec 03 06:58:25 gaming trustix[2956960]: Flags:
Dec 03 06:58:25 gaming trustix[2956960]:       --config string    Path to config.toml/json
Dec 03 06:58:25 gaming trustix[2956960]:   -h, --help             help for daemon
Dec 03 06:58:25 gaming trustix[2956960]:       --interval float   Log poll interval in seconds (default 1800)
Dec 03 06:58:25 gaming trustix[2956960]:       --listen strings   Listen to address
Dec 03 06:58:25 gaming trustix[2956960]:       --state string     State directory (default ".local/share/trustix")
Dec 03 06:58:25 gaming trustix[2956960]: Global Flags:
Dec 03 06:58:25 gaming trustix[2956960]:       --address string   Connect to address (default "unix:///tmp/trustix.sock")
Dec 03 06:58:25 gaming trustix[2956960]:       --log-id string    Log ID
Dec 03 06:58:25 gaming trustix[2956960]:       --timeout int      Timeout in seconds (default 20)
Dec 03 06:58:25 gaming trustix[2956960]: open /var/trustix/keys/private: permission denied
Dec 03 06:58:25 gaming systemd[1]: trustix.service: Main process exited, code=exited, status=1/FAILURE
Dec 03 06:58:25 gaming systemd[1]: trustix.service: Failed with result 'exit-code'.

Maybe this crashes, because the socket does not exist:

Dec 03 06:58:25 gaming systemd[1]: trustix-nix-cache.service: Failed with result 'exit-code'.
Dec 03 06:58:26 gaming systemd[1]: Started Trustix Nix binary cache daemon.
Dec 03 06:58:26 gaming trustix-nix[2957284]: time="2022-12-03T06:58:26+01:00" level=debug msg="Creating client for remote" address="unix:///run/trustix-daemon.socket"
Dec 03 06:58:26 gaming trustix-nix[2957284]: panic: runtime error: invalid memory address or nil pointer dereference
Dec 03 06:58:26 gaming trustix-nix[2957284]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x7b55be]
Dec 03 06:58:26 gaming trustix-nix[2957284]: goroutine 1 [running]:
Dec 03 06:58:26 gaming trustix-nix[2957284]: github.com/bufbuild/connect-go.newClientConfig({0xc000140100?, 0xc000140100?}, {0xc000197ab0, 0x1, 0x90d240?})
Dec 03 06:58:26 gaming trustix-nix[2957284]:         github.com/bufbuild/connect-go/client.go:198 +0x1de
Dec 03 06:58:26 gaming trustix-nix[2957284]: github.com/bufbuild/connect-go.NewClient[...]({0xad43a0, 0xc000126570?}, {0xc000140100, 0x3f}, {0xc000197ab0, 0x1, 0x1})
Dec 03 06:58:26 gaming trustix-nix[2957284]:         github.com/bufbuild/connect-go/client.go:41 +0xa9
Dec 03 06:58:26 gaming trustix-nix[2957284]: github.com/nix-community/trustix/packages/trustix-proto/api/apiconnect.NewLogAPIClient({0xad43a0, 0xc000126570}, {0x7fff73150b72?, 0x7?}, {0xc000197ab0, >
Dec 03 06:58:26 gaming trustix-nix[2957284]:         github.com/nix-community/trustix/packages/trustix-proto/api/apiconnect/api.connect.go:140 +0xcd
Dec 03 06:58:26 gaming trustix-nix[2957284]: github.com/nix-community/trustix/packages/trustix/client.newLogAPIConnectClient(...)
Dec 03 06:58:26 gaming trustix-nix[2957284]:         github.com/nix-community/trustix/packages/trustix/client/connect_logapi.go:23
Dec 03 06:58:26 gaming trustix-nix[2957284]: github.com/nix-community/trustix/packages/trustix/client.CreateClient({0x7fff73150b72, 0x21}, {0xc000197ab0, 0x1, 0x1})
Dec 03 06:58:26 gaming trustix-nix[2957284]:         github.com/nix-community/trustix/packages/trustix/client/conn.go:35 +0x1d5
Dec 03 06:58:26 gaming trustix-nix[2957284]: github.com/nix-community/trustix/packages/trustix-nix/cmd.glob..func1(0xe4d5a0?, {0x9f4c1f?, 0x6?, 0x6?})
Dec 03 06:58:26 gaming trustix-nix[2957284]:         github.com/nix-community/trustix/packages/trustix-nix/cmd/binary-cache-proxy.go:155 +0x205
Dec 03 06:58:26 gaming trustix-nix[2957284]: github.com/spf13/cobra.(*Command).execute(0xe4d5a0, {0xc00010e120, 0x6, 0x6})
Dec 03 06:58:26 gaming trustix-nix[2957284]:         github.com/spf13/cobra/command.go:872 +0x694
Dec 03 06:58:26 gaming trustix-nix[2957284]: github.com/spf13/cobra.(*Command).ExecuteC(0xe4daa0)
Dec 03 06:58:26 gaming trustix-nix[2957284]:         github.com/spf13/cobra/command.go:990 +0x3bd
Dec 03 06:58:26 gaming trustix-nix[2957284]: github.com/spf13/cobra.(*Command).Execute(...)
Dec 03 06:58:26 gaming trustix-nix[2957284]:         github.com/spf13/cobra/command.go:918
Dec 03 06:58:26 gaming trustix-nix[2957284]: github.com/nix-community/trustix/packages/trustix-nix/cmd.Execute()
Dec 03 06:58:26 gaming trustix-nix[2957284]:         github.com/nix-community/trustix/packages/trustix-nix/cmd/root.go:57 +0x45
Dec 03 06:58:26 gaming trustix-nix[2957284]: main.main()
Dec 03 06:58:26 gaming trustix-nix[2957284]:         github.com/nix-community/trustix/packages/trustix-nix/main.go:11 +0x17
Dec 03 06:58:26 gaming systemd[1]: trustix-nix-cache.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Dec 03 06:58:26 gaming systemd[1]: trustix-nix-cache.service: Failed with result 'exit-code'.

I debug it further...

Log spammed with msg="Updated STH"

Describe the bug

Now i have the trustix daemon running. It puts out this log line 247 times a second!

trustix[373644]: time="2022-05-08T01:55:12+02:00" level=info msg="Updated STH" logID=***** oldTreeSize=0 treeSize=0

This results in a 6% CPU usage of the process doing nothing on a 5 GHz i9 processor.

Please stop spamming my logs!

To Reproduce

run trustix daemon

Expected behavior
a program should not spam the logs with the same message over and over again multiple times a second!!!!1

Environment

  • OS name + version: NixOS 22.05pre375061.c777cdf5c56
  • Version of the code: 7ed5270

Additional context
Add any other context about the problem here.

error: anonymous function at .../trustix/default.nix:1:1 called with unexpected argument 'lib'

Describe the bug

I followed the manual to this page and rebuild my system with the provided config, but it fails.

https://tweag.github.io/trustix/publishing.html

[root@gaming:~]# nixos-rebuild switch
error: anonymous function at /home/davidak/code/trustix/default.nix:1:1 called with unexpected argument 'lib'

       at /nix/var/nix/profiles/per-user/root/channels/nixos/lib/modules.nix:500:8:

          499|       # works.
          500|     in f (args // extraArgs)
             |        ^
          501|   else
(use '--show-trace' to show detailed location information)
building Nix...
error: anonymous function at /home/davidak/code/trustix/default.nix:1:1 called with unexpected argument 'lib'

       at /nix/var/nix/profiles/per-user/root/channels/nixos/lib/modules.nix:500:8:

          499|       # works.
          500|     in f (args // extraArgs)
             |        ^
          501|   else
(use '--show-trace' to show detailed location information)
building the system configuration...
error: anonymous function at /home/davidak/code/trustix/default.nix:1:1 called with unexpected argument 'lib'

       at /nix/var/nix/profiles/per-user/root/channels/nixos/lib/modules.nix:500:8:

          499|       # works.
          500|     in f (args // extraArgs)
             |        ^
          501|   else
(use '--show-trace' to show detailed location information)

To Reproduce

{ config, pkgs, lib, ... }:

{
  imports =
    [
      /etc/nixos/hardware-configuration.nix
      ...
      /home/davidak/code/trustix
    ];

...
}

Expected behavior
No errors when following instructions from manual.

Environment

  • OS name + version: NixOS 22.05pre372961.e10da1c7f54
  • Version of the code: 7ed5270

Additional context

My configuration.nix starts with:

{ config, pkgs, lib, ... }:

Removing lib from there does not solve the issue.

log-id not passed to post-build-hook

When I configure a publisher using the module, it automatically registers the post-build-hook, calling trustix-nix without a log-id parameter:

https://github.com/tweag/trustix/blob/9bf6f32ab9b28c49fdc12c6e7a847a2b6dc1aa00/packages/trustix-nix/nixos/post-build-hook.nix#L32

However, this parameter is mandatory:

https://github.com/tweag/trustix/blob/9bf6f32ab9b28c49fdc12c6e7a847a2b6dc1aa00/packages/trustix-nix/cmd/post-build-hook.go#L33-L35

It seems the LogID can be derived from the key type and public key, which are also present in the publisher config. Perhaps post-build-hook.nix should by default get those two fields from the publisher config, and pass them to post-build-hook.go so it can derive the LogID from them? Or am I misunderstanding how this is meant to hang together?

NixOS Reproducibility Community Dashboard

There seem to be work started for something like that in trustix-nix-reprod. It would be great to have a dashboard and documented workflow how one can contribute to it.

Multiple people from https://matrix.to/#/#reproducible-builds:nixos.org are running the r13y tests themselves. It would be great to spread the work better and get a more robust result if different people run the tests on different systems.

Can you finish this project, so NixOS can host the dashboard? Maybe the foundation can sponsor the development.

Log spammed with "Could not update STH"

When i configure a subscriber, that can not be found, the log is spammed with this line 443 times a second!

Dec 03 11:48:57 gaming trustix[28249]: time="2022-12-03T11:48:57+01:00" level=error msg="Could not update STH" error="Couldn't find client for tag: e0f263745e4e3ab07ab5275b00b44f594e0b6d2bd35892a8ebd10a7f86322eb7" logID=e0f263745e4e3ab07ab5275b00b44f594e0b6d2bd35892a8ebd10a7f86322eb7

    subscribers = [
      # local
      {
        protocol = "nix";
        publicKey = {
          type = "ed25519";
          key = "APAu/WvYTnCQSIg+5OzWKC/D+JzWFJZmvNsPPBbfhcQ=";
        };
      }
      # demo.trustix.dev
#      {
#        protocol = "nix";
#        publicKey = {
#          type = "ed25519";
#          key = "2uy8gNIOYEewTiV7iB7cUxBGpXxQtdlFepFoRvJTCJo=";
#        };
#      }
    ];

(the commented out part)

Similar to #34

mv: cannot move 'trustix-priv' to '/var/trustix/keys/trustix-priv': No such file or directory

Describe the bug

In the manual, i'm instructed to run this command:

mv trustix-priv /var/trustix/keys/trustix-priv

but it fails:

[davidak@gaming:~/code/trustix]$ mv trustix-priv /var/trustix/keys/trustix-priv
mv: cannot move 'trustix-priv' to '/var/trustix/keys/trustix-priv': No such file or directory

To Reproduce
https://tweag.github.io/trustix/publishing.html

Expected behavior
Following the manual should not lead to errors.

Environment

  • OS name + version: NixOS 22.05pre372961.e10da1c7f54
  • Version of the code: 7ed5270

Additional context

These are the steps i have done:

sudo mkdir -p /var/trustix/keys
sudo mv trustix-priv /var/trustix/keys/
sudo mv trustix-pub /var/trustix/keys/
sudo chown root:root /var/trustix/keys/*
sudo chmod 600 /var/trustix/keys/*
ll /var/trustix/keys/
total 8
-rw------- 1 root root 88 May  7 22:05 trustix-priv
-rw------- 1 root root 44 May  7 22:05 trustix-pub

I have yet to see if it works with these permissions.

Update: It does not.

× trustix.service - Trustix daemon
     Loaded: loaded (/etc/systemd/system/trustix.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Sat 2022-05-07 23:58:20 CEST; 2s ago
TriggeredBy: ● trustix.socket
    Process: 347890 ExecStart=/nix/store/sql7z5xkbmydnqinyp3brpp3m76vj13p-trustix-dev/bin/trustix daemon --state . --config /nix/store/gwhp4kd41mgzkh7015864ibwxcql0ls0-trustix-config.json (code=exited, status=1/FAILURE)
   Main PID: 347890 (code=exited, status=1/FAILURE)
         IP: 0B in, 0B out
        CPU: 11ms

May 07 23:58:20 gaming trustix[347890]:       --interval float   Log poll interval in seconds (default 1800)
May 07 23:58:20 gaming trustix[347890]:       --listen strings   Listen to address
May 07 23:58:20 gaming trustix[347890]:       --state string     State directory (default ".local/share/trustix")
May 07 23:58:20 gaming trustix[347890]: Global Flags:
May 07 23:58:20 gaming trustix[347890]:       --address string   Connect to address (default "unix:///tmp/trustix.sock")
May 07 23:58:20 gaming trustix[347890]:       --log-id string    Log ID
May 07 23:58:20 gaming trustix[347890]:       --timeout int      Timeout in seconds (default 20)
May 07 23:58:20 gaming trustix[347890]: open /var/trustix/keys/priv: permission denied
May 07 23:58:20 gaming systemd[1]: trustix.service: Main process exited, code=exited, status=1/FAILURE
May 07 23:58:20 gaming systemd[1]: trustix.service: Failed with result 'exit-code'.
warning: error(s) occurred while switching to the new configuration

I can't configure the user trustix for the files, since the user does not exist before i reconfigure the system with this, which lead to a broken system. This is not thought through.

Error while getting logs from 'grpc+https://trustix.example.com'

Describe the bug

The manual says i should set that remote.

Obviously it does not exist.

This causes the Trustix daemon to crash and a bad first impression for the user.

To Reproduce

https://tweag.github.io/trustix/subscribing.html

Expected behavior

The documentation should have instructions that let the user test the software. Following the documentation should not let the software crash!

It should enable the user to come to a useful state.

Environment

  • OS name + version:
  • Version of the code:

Additional context

Log:

May 08 00:55:53 gaming systemd[1]: Started Trustix daemon.
May 08 00:55:53 gaming trustix[362048]: time="2022-05-08T00:55:53+02:00" level=info msg="Creating state directory" directory=.
May 08 00:55:53 gaming trustix[362048]: time="2022-05-08T00:55:53+02:00" level=info msg="Creating signer" name=davidak type=ed25519
May 08 00:55:53 gaming trustix[362048]: time="2022-05-08T00:55:53+02:00" level=debug msg="Dialing remote" address="grpc+https://trustix.example.com"
May 08 00:55:53 gaming trustix[362048]: time="2022-05-08T00:55:53+02:00" level=info msg="Adding log subscriber" id=***** pubkey="*****"
May 08 00:55:53 gaming trustix[362048]: time="2022-05-08T00:55:53+02:00" level=info msg="Adding log" id=***** pubkey="*****"
May 08 00:55:53 gaming trustix[362048]: time="2022-05-08T00:55:53+02:00" level=error msg="Could not update STH" error="Couldn't find client for tag: *****" logID=***>
May 08 00:55:53 gaming trustix[362048]: time="2022-05-08T00:55:53+02:00" level=debug msg="Creating gRPC servers"
May 08 00:55:53 gaming trustix[362048]: time="2022-05-08T00:55:53+02:00" level=info msg="Using socket activated listener" address=/run/trustix-daemon.socket
May 08 00:55:53 gaming trustix[362048]: time="2022-05-08T00:55:53+02:00" level=debug msg="Checking peer credential for socket"
May 08 00:55:53 gaming trustix[362048]: time="2022-05-08T00:55:53+02:00" level=info msg="Updated STH" logID=***** oldTreeSize=0 treeSize=0
May 08 00:55:53 gaming trustix[362048]: time="2022-05-08T00:55:53+02:00" level=info msg="Updated STH" logID=***** oldTreeSize=0 treeSize=0
May 08 00:55:53 gaming trustix[362048]: time="2022-05-08T00:55:53+02:00" level=info msg="Updated STH" logID=***** oldTreeSize=0 treeSize=0
May 08 00:55:53 gaming trustix[362048]: time="2022-05-08T00:55:53+02:00" level=error msg="Error while getting logs from 'grpc+https://trustix.example.com': rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while di>
May 08 00:55:53 gaming trustix[362048]: panic: runtime error: invalid memory address or nil pointer dereference
May 08 00:55:53 gaming trustix[362048]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0xa996aa]
May 08 00:55:53 gaming trustix[362048]: goroutine 25 [running]:
May 08 00:55:53 gaming trustix[362048]: github.com/tweag/trustix/packages/trustix/internal/pool.(*ClientPool).Dial.func1()
May 08 00:55:53 gaming trustix[362048]:         /build/source/internal/pool/pool.go:228 +0x14a
May 08 00:55:53 gaming trustix[362048]: created by github.com/tweag/trustix/packages/trustix/internal/pool.(*ClientPool).Dial
May 08 00:55:53 gaming trustix[362048]:         /build/source/internal/pool/pool.go:221 +0xe8
May 08 00:55:53 gaming systemd[1]: trustix.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
May 08 00:55:53 gaming systemd[1]: trustix.service: Failed with result 'exit-code'.

failed to get current user: user: unknown userid 62583

Running the trustix daemon from the nixos module, it is running as a DynamicUser systemd module:

https://github.com/tweag/trustix/blob/9bf6f32ab9b28c49fdc12c6e7a847a2b6dc1aa00/packages/trustix/nixos/default.nix#L240-L246

If I understand correctly from http://0pointer.net/blog/dynamic-users-with-systemd.html, systemd should an actual user for this dynamic user, not just a PID. However, I only see a pid:

62583 1341 0.0 0.0 1312748 16264 ? Ssl 14:46 0:00 /nix/store/k17mjpk7hfbmvx9r2abh11rw24x97dlz-trustix-dev/bin/trustix daemon --state . --config /nix/store/yqpsksmayry4qmp6sicimzz6ck0ivxa8-trustix-config.toml

Then, when I try to publish a build (say, nix-build '<nixpkgs>' -A hello --check), it fails on not being able to find the user:

post-build-hook: time="2021-06-21T15:11:59+02:00" level=debug msg="Submitting mapping" storePath=/nix/store/vf8fb2avrxnnhc6n70dkqdlk25f61rwm-hello-2.10
post-build-hook: time="2021-06-21T15:11:59+02:00" level=debug msg="Dialing remote" address="unix:///run/trustix-daemon.socket"
post-build-hook: time="2021-06-21T15:11:59+02:00" level=fatal msg="could not submit: rpc error: code = Unknown desc = failed to get current user: user: unknown userid 62583"

The root cause might be in systemd not creating the user, but perhaps other trustix users have run into the same problem? I'm on nixos-unstable at 33d42ad7cf2769ce6364ed4e52afa8e9d1439d58

trustix-nix-reprod: No version that satisfies the requirement diffoscope<172,>=171

Describe the bug

[davidak@gaming:~/code/trustix]$ nix-build ./. -A packages.trustix-nix-reprod
this derivation will be built:
  /nix/store/scanjik9qs1r40l5gs9wpravj77szwv7-python3.8-trustix-nix-reprod-0.1.0.drv
these 50 paths will be fetched (15.63 MiB download, 70.96 MiB unpacked):
  /nix/store/01y2yl6gw2lsxx3yah2kssxcs9fwzki4-squashfs-4.4
  /nix/store/07zqfpr47mg8rynxh19yi2zhcyfn9w45-imlib2-1.7.1
  /nix/store/271pdmg1vh31fns381m8jvw7k1cpy8dx-python3.8-libarchive-c-2.9
  /nix/store/2mmqlgqqzmahqwxjrndksn7p4ly9xdxw-python3.8-defusedxml-0.7.0
  /nix/store/49xip1fqvxsqs4jljf7crl5scz1nbd1c-python3.8-chardet-3.0.4
  /nix/store/4cddvmfw30d43afx81xnk4s0ffc363zs-python3.8-requests-toolbelt-0.9.1
  /nix/store/4l0m5abhbaky2fbajirdkdfshhpslw37-python3.8-argcomplete-1.12.2
  /nix/store/5hz6ggb9nnly1zglp29yj5510nm3vj76-python3.8-editorconfig-0.12.2
  /nix/store/60n64ml3fl88kidsn23izk09yz96cm5g-libpng-1.2.59
  /nix/store/70nblr4d6rbcxz59lq04zkbf187bk49g-python3.8-python-magic-0.4.22
  /nix/store/7c3s3lraq18055nrj7r0jajzn2v2xj19-python3.8-dicttoxml-1.7.4
  /nix/store/7gafa9azsajafb39jag9w9ppz0h6s40g-acl-2.2.53-bin
  /nix/store/7s09cnjlmi9nzvs2xsf0i213pb8b3czb-lzo-2.10
  /nix/store/83kgzg9jrrx1b6q4kzmf6iczirgwy0d3-python3.8-jsondiff-1.2.0
  /nix/store/8pxgjk70b5hv6gq82zs20qbbb9qbdmcl-sng-1.1.0
  /nix/store/8xhm968gk4vr4dwq1dg3gxigkc4gjmnw-e2fsprogs-1.45.6-bin
  /nix/store/95vyfn0yc7f9h4k0rczgq6rn28qd4c7m-diffoscope-166
  /nix/store/97ky3m2fsda0sjp6aswxgw64gp0wf0nm-python3.8-pyxattr-0.7.2
  /nix/store/9dpvvsqhsz8mcic2w57w8xc8j6xpib0b-pgpdump-0.33
  /nix/store/9izlcbb6mnwm4nbxn7hpv7ap5q2md4a4-woff2-1.0.2-lib
  /nix/store/9ydwcnshdrqnw200i19gz8iqy3hcpcrj-python3.8-more-itertools-8.6.0
  /nix/store/ap9ydmw7kqryh079f6hh4yx3bjb16y3p-zip-3.0
  /nix/store/bkgh1d8mx9msc5p54v08qx6m3d9mc7jw-jsbeautifier-1.13.5
  /nix/store/bl8dkmvjkvwmiclcbfqyx38ic9kbwfzp-libcaca-0.99.beta19
  /nix/store/bnyz816gwamgflzz3pymn5yyhm54cy76-dtc-1.6.0
  /nix/store/dviqx9x1hyagmphvx468k9ify9w16v29-python3.8-zipp-3.4.0
  /nix/store/f30mrvjcb84pzxw2lkmkcgi7v8qwmgwp-python3.8-tlsh-4.5.0
  /nix/store/g0casx8sxb8ldh1947nxzqpx856xnnqi-libarchive-3.5.1
  /nix/store/gzi1i4dnpfvvvbqjccvx85ih5w836ynw-python3.8-wcwidth-0.2.5
  /nix/store/h1zkkd9hi3rqrnkksqy93gy8k69xhln0-python3.8-PyPDF2-1.26.0
  /nix/store/k4xdivwv9jmmp0zk2h1hzd3m9bgj97q6-netpbm-10.92.0
  /nix/store/kkgdii4a197v8bvpi6sbnf9zmyv5457w-libid3tag-0.15.1b
  /nix/store/pyjb0qk4s6g7xygphn9mvqwcraxzrvl1-python3.8-ptyprocess-0.6.0
  /nix/store/rffbd45zb07nbv2x6dc47n4yvqkpgw35-libungif-4.1.4
  /nix/store/rh8lvf0jklz59b8sjbm01slv54rf6syy-python3.8-progressbar33-2.4
  /nix/store/rz7saw2wyclxz3inn3paljhns3vviwy6-colordiff-1.0.19
  /nix/store/sg8c2w86gd0a8g4qprl398qfgb5dn5yi-xxd-1003.1-2008
  /nix/store/vhxcssfr7b866ippb3l1d0cap8xfydrq-vim-8.2.1522
  /nix/store/w7rjgzgcr7ncifr7n3w742a5zip00n2m-fontforge-20201107
  /nix/store/wdxn89sgq02ayqfj8cwxxixvdgcfdz3n-python3.8-python-debian-0.1.39
  /nix/store/x3bz3fpg96qa4a1fjc9wjwadmqb4d6gp-python3.8-requests-2.25.1
  /nix/store/xc8nbxr692drvbxgarjlp4az80x8x8sc-python3.8-pexpect-4.8.0
  /nix/store/xrx963rmp92g41k6iidrsp1idi7bxsmn-e2fsprogs-1.45.6
  /nix/store/xz4riy7vck8g8yh4083xsc0jdij27rff-lz4-1.9.3-bin
  /nix/store/y97dvplpwy4k214ndyya9xzrqhsf9rwx-python3.8-importlib-metadata-1.7.0
  /nix/store/yf3wjxc895q03q8ckrkzslqjj2g8sgb9-python3.8-prettytable-2.0.0
  /nix/store/yqzqn3h84h3nybwvaj6siw1clw1vpv6l-nix-2.3.10-dev
  /nix/store/yrl946k33gqc91vvrqc4ffkykjd8c92j-libuninameslist-20200413
  /nix/store/z4yr5l36zaikgh8svgdi13kybpy2a5by-libcaca-0.99.beta19-bin
  /nix/store/z8dfc8mhkzk6vpzymlhwd2pp4q2ffikz-db-5.3.28-bin
copying path '/nix/store/yqzqn3h84h3nybwvaj6siw1clw1vpv6l-nix-2.3.10-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/7gafa9azsajafb39jag9w9ppz0h6s40g-acl-2.2.53-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/rz7saw2wyclxz3inn3paljhns3vviwy6-colordiff-1.0.19' from 'https://cache.nixos.org'...
copying path '/nix/store/z8dfc8mhkzk6vpzymlhwd2pp4q2ffikz-db-5.3.28-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/bnyz816gwamgflzz3pymn5yyhm54cy76-dtc-1.6.0' from 'https://cache.nixos.org'...
copying path '/nix/store/xrx963rmp92g41k6iidrsp1idi7bxsmn-e2fsprogs-1.45.6' from 'https://cache.nixos.org'...
copying path '/nix/store/g0casx8sxb8ldh1947nxzqpx856xnnqi-libarchive-3.5.1' from 'https://cache.nixos.org'...
copying path '/nix/store/xz4riy7vck8g8yh4083xsc0jdij27rff-lz4-1.9.3-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/9dpvvsqhsz8mcic2w57w8xc8j6xpib0b-pgpdump-0.33' from 'https://cache.nixos.org'...
copying path '/nix/store/h1zkkd9hi3rqrnkksqy93gy8k69xhln0-python3.8-PyPDF2-1.26.0' from 'https://cache.nixos.org'...
copying path '/nix/store/49xip1fqvxsqs4jljf7crl5scz1nbd1c-python3.8-chardet-3.0.4' from 'https://cache.nixos.org'...
copying path '/nix/store/2mmqlgqqzmahqwxjrndksn7p4ly9xdxw-python3.8-defusedxml-0.7.0' from 'https://cache.nixos.org'...
copying path '/nix/store/7c3s3lraq18055nrj7r0jajzn2v2xj19-python3.8-dicttoxml-1.7.4' from 'https://cache.nixos.org'...
copying path '/nix/store/5hz6ggb9nnly1zglp29yj5510nm3vj76-python3.8-editorconfig-0.12.2' from 'https://cache.nixos.org'...
copying path '/nix/store/83kgzg9jrrx1b6q4kzmf6iczirgwy0d3-python3.8-jsondiff-1.2.0' from 'https://cache.nixos.org'...
copying path '/nix/store/271pdmg1vh31fns381m8jvw7k1cpy8dx-python3.8-libarchive-c-2.9' from 'https://cache.nixos.org'...
copying path '/nix/store/bl8dkmvjkvwmiclcbfqyx38ic9kbwfzp-libcaca-0.99.beta19' from 'https://cache.nixos.org'...
copying path '/nix/store/kkgdii4a197v8bvpi6sbnf9zmyv5457w-libid3tag-0.15.1b' from 'https://cache.nixos.org'...
copying path '/nix/store/60n64ml3fl88kidsn23izk09yz96cm5g-libpng-1.2.59' from 'https://cache.nixos.org'...
copying path '/nix/store/rffbd45zb07nbv2x6dc47n4yvqkpgw35-libungif-4.1.4' from 'https://cache.nixos.org'...
copying path '/nix/store/yrl946k33gqc91vvrqc4ffkykjd8c92j-libuninameslist-20200413' from 'https://cache.nixos.org'...
copying path '/nix/store/7s09cnjlmi9nzvs2xsf0i213pb8b3czb-lzo-2.10' from 'https://cache.nixos.org'...
copying path '/nix/store/k4xdivwv9jmmp0zk2h1hzd3m9bgj97q6-netpbm-10.92.0' from 'https://cache.nixos.org'...
copying path '/nix/store/bkgh1d8mx9msc5p54v08qx6m3d9mc7jw-jsbeautifier-1.13.5' from 'https://cache.nixos.org'...
copying path '/nix/store/9ydwcnshdrqnw200i19gz8iqy3hcpcrj-python3.8-more-itertools-8.6.0' from 'https://cache.nixos.org'...
copying path '/nix/store/rh8lvf0jklz59b8sjbm01slv54rf6syy-python3.8-progressbar33-2.4' from 'https://cache.nixos.org'...
copying path '/nix/store/pyjb0qk4s6g7xygphn9mvqwcraxzrvl1-python3.8-ptyprocess-0.6.0' from 'https://cache.nixos.org'...
copying path '/nix/store/wdxn89sgq02ayqfj8cwxxixvdgcfdz3n-python3.8-python-debian-0.1.39' from 'https://cache.nixos.org'...
copying path '/nix/store/70nblr4d6rbcxz59lq04zkbf187bk49g-python3.8-python-magic-0.4.22' from 'https://cache.nixos.org'...
copying path '/nix/store/97ky3m2fsda0sjp6aswxgw64gp0wf0nm-python3.8-pyxattr-0.7.2' from 'https://cache.nixos.org'...
copying path '/nix/store/x3bz3fpg96qa4a1fjc9wjwadmqb4d6gp-python3.8-requests-2.25.1' from 'https://cache.nixos.org'...
copying path '/nix/store/f30mrvjcb84pzxw2lkmkcgi7v8qwmgwp-python3.8-tlsh-4.5.0' from 'https://cache.nixos.org'...
copying path '/nix/store/gzi1i4dnpfvvvbqjccvx85ih5w836ynw-python3.8-wcwidth-0.2.5' from 'https://cache.nixos.org'...
copying path '/nix/store/07zqfpr47mg8rynxh19yi2zhcyfn9w45-imlib2-1.7.1' from 'https://cache.nixos.org'...
copying path '/nix/store/4cddvmfw30d43afx81xnk4s0ffc363zs-python3.8-requests-toolbelt-0.9.1' from 'https://cache.nixos.org'...
copying path '/nix/store/dviqx9x1hyagmphvx468k9ify9w16v29-python3.8-zipp-3.4.0' from 'https://cache.nixos.org'...
copying path '/nix/store/vhxcssfr7b866ippb3l1d0cap8xfydrq-vim-8.2.1522' from 'https://cache.nixos.org'...
copying path '/nix/store/9izlcbb6mnwm4nbxn7hpv7ap5q2md4a4-woff2-1.0.2-lib' from 'https://cache.nixos.org'...
copying path '/nix/store/01y2yl6gw2lsxx3yah2kssxcs9fwzki4-squashfs-4.4' from 'https://cache.nixos.org'...
copying path '/nix/store/ap9ydmw7kqryh079f6hh4yx3bjb16y3p-zip-3.0' from 'https://cache.nixos.org'...
copying path '/nix/store/xc8nbxr692drvbxgarjlp4az80x8x8sc-python3.8-pexpect-4.8.0' from 'https://cache.nixos.org'...
copying path '/nix/store/8pxgjk70b5hv6gq82zs20qbbb9qbdmcl-sng-1.1.0' from 'https://cache.nixos.org'...
copying path '/nix/store/y97dvplpwy4k214ndyya9xzrqhsf9rwx-python3.8-importlib-metadata-1.7.0' from 'https://cache.nixos.org'...
copying path '/nix/store/yf3wjxc895q03q8ckrkzslqjj2g8sgb9-python3.8-prettytable-2.0.0' from 'https://cache.nixos.org'...
copying path '/nix/store/8xhm968gk4vr4dwq1dg3gxigkc4gjmnw-e2fsprogs-1.45.6-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/z4yr5l36zaikgh8svgdi13kybpy2a5by-libcaca-0.99.beta19-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/4l0m5abhbaky2fbajirdkdfshhpslw37-python3.8-argcomplete-1.12.2' from 'https://cache.nixos.org'...
copying path '/nix/store/w7rjgzgcr7ncifr7n3w742a5zip00n2m-fontforge-20201107' from 'https://cache.nixos.org'...
copying path '/nix/store/sg8c2w86gd0a8g4qprl398qfgb5dn5yi-xxd-1003.1-2008' from 'https://cache.nixos.org'...
copying path '/nix/store/95vyfn0yc7f9h4k0rczgq6rn28qd4c7m-diffoscope-166' from 'https://cache.nixos.org'...
building '/nix/store/scanjik9qs1r40l5gs9wpravj77szwv7-python3.8-trustix-nix-reprod-0.1.0.drv'...
Sourcing python-remove-tests-dir-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing pip-build-hook
Using pipBuildPhase
Using pipShellHook
Sourcing pip-install-hook
Using pipInstallPhase
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing python-namespaces-hook
unpacking sources
unpacking source archive /nix/store/lry24jsz28rawy90as1j471mn10xvhyp-source
source root is source
setting SOURCE_DATE_EPOCH to timestamp 315619200 of file source/trustix_nix_reprod/templates/search.jinja2
patching sources
configuring
no configure script, doing nothing
building
Executing pipBuildPhase
Creating a wheel...
WARNING: The directory '/homeless-shelter/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Processing /build/source
    Preparing wheel metadata ... done
Building wheels for collected packages: trustix-nix-reprod
  Building wheel for trustix-nix-reprod (PEP 517) ... done
  Created wheel for trustix-nix-reprod: filename=trustix_nix_reprod-0.1.0-py3-none-any.whl size=71325 sha256=39a3691a2f8678ac9c37a017fb312545398ad77bc731150b02334bf9a6186a79
  Stored in directory: /build/pip-ephem-wheel-cache-rumekrmr/wheels/e1/fb/ff/3c7bf8327fe7b38e0848fbd084e404c8a61b776977c550714b
Successfully built trustix-nix-reprod
Finished creating a wheel...
Finished executing pipBuildPhase
installing
Executing pipInstallPhase
/build/source/dist /build/source
Processing ./trustix_nix_reprod-0.1.0-py3-none-any.whl
Requirement already satisfied: uvicorn[standard]<0.14.0,>=0.13.4 in /nix/store/yqilzd4hn2vs8s4i56k6p13jmpd6r6yi-python3.8-uvicorn-0.13.4/lib/python3.8/site-packages (from trustix-nix-reprod==0.1.0) (0.13.4)
Requirement already satisfied: aiohttp[speedups]<4.0.0,>=3.7.4 in /nix/store/7nmaaiklmz14fh4yxgilxdp7xfqd693j-python3.8-aiohttp-3.7.4.post0/lib/python3.8/site-packages (from trustix-nix-reprod==0.1.0) (3.7.4.post0)
Requirement already satisfied: trustix-python in /nix/store/5vsl4w7rimf12i1dvf1a6547nxzzwajj-python3.8-trustix-python-0.1.0/lib/python3.8/site-packages (from trustix-nix-reprod==0.1.0) (0.1.0)
Requirement already satisfied: aiofiles<0.7.0,>=0.6.0 in /nix/store/yprd4cjj4gq3y5h2ck26a6w1g5z6ryf2-python3.8-aiofiles-0.6.0/lib/python3.8/site-packages (from trustix-nix-reprod==0.1.0) (0.6.0)
Requirement already satisfied: requests<3.0.0,>=2.25.1 in /nix/store/azmvgbdprg3blf9ngba1j3147aissl2c-python3.8-requests-2.25.1/lib/python3.8/site-packages (from trustix-nix-reprod==0.1.0) (2.25.1)
Requirement already satisfied: ijson<4.0.0,>=3.1.3 in /nix/store/bvcs4rr6i90fcf8c0d2qiwb4igw08pmh-python3.8-ijson-3.1.4/lib/python3.8/site-packages (from trustix-nix-reprod==0.1.0) (3.1.4)
Requirement already satisfied: aioredis<2.0.0,>=1.3.1 in /nix/store/i67zm0rx7ihjpsl02fa7v19qrc0ssrix-python3.8-aioredis-1.3.1/lib/python3.8/site-packages (from trustix-nix-reprod==0.1.0) (1.3.1)
Requirement already satisfied: fastapi[standard]<0.64.0,>=0.63.0 in /nix/store/2sgqk6rr60b5ahmvbzybkm41c2b20394-python3.8-fastapi-0.63.0/lib/python3.8/site-packages (from trustix-nix-reprod==0.1.0) (0.63.0)
  WARNING: fastapi 0.63.0 does not provide the extra 'standard'
Requirement already satisfied: orjson<4.0.0,>=3.5.1 in /nix/store/46fr3mx3gwz8jiwa8b9c6mc2jbmrcba6-python3.8-orjson-3.5.1/lib/python3.8/site-packages (from trustix-nix-reprod==0.1.0) (3.5.1)
ERROR: Could not find a version that satisfies the requirement diffoscope<172,>=171 (from trustix-nix-reprod==0.1.0) (from versions: none)
ERROR: No matching distribution found for diffoscope<172,>=171 (from trustix-nix-reprod==0.1.0)
error: builder for '/nix/store/scanjik9qs1r40l5gs9wpravj77szwv7-python3.8-trustix-nix-reprod-0.1.0.drv' failed with exit code 1;
       last 10 log lines:
       > Requirement already satisfied: trustix-python in /nix/store/5vsl4w7rimf12i1dvf1a6547nxzzwajj-python3.8-trustix-python-0.1.0/lib/python3.8/site-packages (from trustix-nix-reprod==0.1.0) (0.1.0)
       > Requirement already satisfied: aiofiles<0.7.0,>=0.6.0 in /nix/store/yprd4cjj4gq3y5h2ck26a6w1g5z6ryf2-python3.8-aiofiles-0.6.0/lib/python3.8/site-packages (from trustix-nix-reprod==0.1.0) (0.6.0)
       > Requirement already satisfied: requests<3.0.0,>=2.25.1 in /nix/store/azmvgbdprg3blf9ngba1j3147aissl2c-python3.8-requests-2.25.1/lib/python3.8/site-packages (from trustix-nix-reprod==0.1.0) (2.25.1)
       > Requirement already satisfied: ijson<4.0.0,>=3.1.3 in /nix/store/bvcs4rr6i90fcf8c0d2qiwb4igw08pmh-python3.8-ijson-3.1.4/lib/python3.8/site-packages (from trustix-nix-reprod==0.1.0) (3.1.4)
       > Requirement already satisfied: aioredis<2.0.0,>=1.3.1 in /nix/store/i67zm0rx7ihjpsl02fa7v19qrc0ssrix-python3.8-aioredis-1.3.1/lib/python3.8/site-packages (from trustix-nix-reprod==0.1.0) (1.3.1)
       > Requirement already satisfied: fastapi[standard]<0.64.0,>=0.63.0 in /nix/store/2sgqk6rr60b5ahmvbzybkm41c2b20394-python3.8-fastapi-0.63.0/lib/python3.8/site-packages (from trustix-nix-reprod==0.1.0) (0.63.0)
       >   WARNING: fastapi 0.63.0 does not provide the extra 'standard'
       > Requirement already satisfied: orjson<4.0.0,>=3.5.1 in /nix/store/46fr3mx3gwz8jiwa8b9c6mc2jbmrcba6-python3.8-orjson-3.5.1/lib/python3.8/site-packages (from trustix-nix-reprod==0.1.0) (3.5.1)
       > ERROR: Could not find a version that satisfies the requirement diffoscope<172,>=171 (from trustix-nix-reprod==0.1.0) (from versions: none)
       > ERROR: No matching distribution found for diffoscope<172,>=171 (from trustix-nix-reprod==0.1.0)
       For full logs, run 'nix log /nix/store/scanjik9qs1r40l5gs9wpravj77szwv7-python3.8-trustix-nix-reprod-0.1.0.drv'.

To Reproduce

[davidak@gaming:~/code/trustix]$ nix-build ./. -A packages.trustix-nix-reprod

Expected behavior
No errors

Environment
OS name + version: NixOS 22.05pre375061.c777cdf5c56
Version of the code: 7ed5270

Additional context
Add any other context about the problem here.

Building `trustix` on `aarch64-darwin`

Is your feature request related to a problem? Please describe.
There are internals tied to Unix syscalls:

internal/rpc/auth/auth.go:22:16: undefined: syscall.Ucred
internal/rpc/auth/peercred.go:54:23: undefined: syscall.GetsockoptUcred
internal/rpc/auth/peercred.go:55:47: undefined: syscall.SO_PEERCRED

These symbols are not present on darwin, but there are similes. This discussion is relevant. There is an abstraction library linked at the end that might help make trustix portable.

Additional context
First, the package seems required to generate keys, although I suspect this should be doable manually. While you may intend to run Trustix on a NixOS server, you might be writing the configuration on a Mac, and may want to generate your keys locally.

Second, it is likely desirable (and probably out of scope for now) to run Trustix itself on a Mac.

Switch SMT library from lazyledger org to celestiaorg

I don't have the mental bandwidth to figure out the differences between the older and the newer version right now. #61

Probably something that still needs to be done at some point so this issue can track it, either for later or someone else can pick it up.

Clarify license

Is your feature request related to a problem? Please describe.

As a user and contributor, i want to know which license the code has.

Describe the solution you'd like

Add a "License" section to the README that describes which components are licensed under which license.

It would be also great if you follow the REUSE standard: https://reuse.software/

You added license information to some files, but it is not machine readable, so the reuse tool can not detect it.

[davidak@gaming:~/code/trustix]$ nix run nixpkgs#reuse lint
...

# SUMMARY

* Bad licenses:
* Deprecated licenses:
* Licenses without file extension:
* Missing licenses:
* Unused licenses:
* Used licenses:
* Read errors: 0
* Files with copyright information: 114 / 308
* Files with license information: 0 / 308

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

If you have multiple licenses (as it seems), place the license text in a LICENSES/ directory. The reuse tool can do that for you!

Additional context

I see you use different licenses:

7c09014
7b35b0e
39a88df
https://github.com/tweag/trustix/blob/a434c7143943a7c8d963e1fb372f3b3d3164dfe2/packages/trustix-python/pyproject.toml
...

Crash when contacting trustix-daemon

[root@gaming:~]# nix-build -E '(import <nixpkgs> {}).writeText "hello" "Hello World!"' --no-out-link --check
checking outputs of '/nix/store/fmvm1z6l9k78s6xcgbj8qkrhkbiz5ar4-hello.drv'...
running post-build-hook '/nix/store/7nzwfm3plmajjyvjvvry6vl2gyz3kvhd-trustix-hook'...
post-build-hook: time="2022-12-03T13:20:09+01:00" level=debug msg="Submitting mapping" storePath=/nix/store/cswi7m0frxljix1l6ixsjdryg5z75i3y-hello
post-build-hook: time="2022-12-03T13:20:09+01:00" level=debug msg="Creating client for remote" address="unix:///run/trustix-daemon.socket"
post-build-hook: panic: runtime error: invalid memory address or nil pointer dereference
post-build-hook: [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x7b55be]
post-build-hook: 
post-build-hook: goroutine 1 [running]:
post-build-hook: github.com/bufbuild/connect-go.newClientConfig({0xc0002b4200?, 0xc0002b4200?}, {0xc0001a9d68, 0x1, 0x90d240?})
post-build-hook:        github.com/bufbuild/connect-go/client.go:198 +0x1de
post-build-hook: github.com/bufbuild/connect-go.NewClient[...]({0xad43a0, 0xc00029aa20?}, {0xc0002b4200, 0x3f}, {0xc0001a9d68, 0x1, 0x1})
post-build-hook:        github.com/bufbuild/connect-go/client.go:41 +0xa9
post-build-hook: github.com/nix-community/trustix/packages/trustix-proto/api/apiconnect.NewLogAPIClient({0xad43a0, 0xc00029aa20}, {0x7ffd10f965b1?, 0x7?}, {0xc0001a9d68, 0x1, 0x1})
post-build-hook:        github.com/nix-community/trustix/packages/trustix-proto/api/apiconnect/api.connect.go:140 +0xcd
post-build-hook: github.com/nix-community/trustix/packages/trustix/client.newLogAPIConnectClient(...)
post-build-hook:        github.com/nix-community/trustix/packages/trustix/client/connect_logapi.go:23
post-build-hook: github.com/nix-community/trustix/packages/trustix/client.CreateClient({0x7ffd10f965b1, 0x21}, {0xc0001a9d68, 0x1, 0x1})
post-build-hook:        github.com/nix-community/trustix/packages/trustix/client/conn.go:35 +0x1d5
post-build-hook: github.com/nix-community/trustix/packages/trustix-nix/cmd.glob..func2(0xe4d820?, {0x9f4c1f?, 0x4?, 0x4?})
post-build-hook:        github.com/nix-community/trustix/packages/trustix-nix/cmd/post-build-hook.go:87 +0x35d
post-build-hook: github.com/spf13/cobra.(*Command).execute(0xe4d820, {0xc0002c20c0, 0x4, 0x4})
post-build-hook:        github.com/spf13/cobra/command.go:872 +0x694
post-build-hook: github.com/spf13/cobra.(*Command).ExecuteC(0xe4daa0)
post-build-hook:        github.com/spf13/cobra/command.go:990 +0x3bd
post-build-hook: github.com/spf13/cobra.(*Command).Execute(...)
post-build-hook:        github.com/spf13/cobra/command.go:918
post-build-hook: github.com/nix-community/trustix/packages/trustix-nix/cmd.Execute()
post-build-hook:        github.com/nix-community/trustix/packages/trustix-nix/cmd/root.go:57 +0x45
post-build-hook: main.main()
post-build-hook:        github.com/nix-community/trustix/packages/trustix-nix/main.go:11 +0x17
error: program '/nix/store/7nzwfm3plmajjyvjvvry6vl2gyz3kvhd-trustix-hook' failed with exit code 2

This was also reported by @raboof in chat: https://matrix.to/#/!tCEqPTBHfrsSDeIuFP:trustix.dev/$3iWGOoZwyGMluozypjdK7jZMsCwkUvj9ud3alb-80bI?via=blad.is&via=matrix.org&via=nixos.dev

This issue is reproducible since e688127, but might exist before and is hidden by other issues. Ref: #81 (comment)

Tested on NixOS 22.05.4270.b68a6a27adb.

Add a way to pin the root hash for a log id

Is your feature request related to a problem? Please describe.
If a key from a log is compromised you will want to have a way to say "this is the last published thing that I trust from this log".

Describe the solution you'd like
The correct solution is probably to have a config file option to pin the last known good hash.

Describe alternatives you've considered
No alternatives considered in earnest.
All key revocation mechanisms comes with significant cost.

Crash instead of error message

Trustix crash a lot, in any situation where it can't work. Wrong configuration, unreachable socket, wrong file permission -> CRASH!

Please create software that does not crash and show a useful error message instead.

Some examples:

Here trustix crash because of your certificate:

...
Dec 03 11:23:30 gaming trustix[8856]: time="2022-12-03T11:23:30+01:00" level=error msg="Error while getting logs from 'https://demo.trustix.dev': unavailable: x509: certificate is valid for r13y.trustix.dev, not demo.trustix.dev"
Dec 03 11:23:30 gaming trustix[8856]: panic: runtime error: invalid memory address or nil pointer dereference
Dec 03 11:23:30 gaming trustix[8856]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0xa6f5d7]
Dec 03 11:23:30 gaming trustix[8856]: goroutine 23 [running]:
Dec 03 11:23:30 gaming trustix[8856]: github.com/nix-community/trustix/packages/trustix/internal/pool.(*ClientPool).Dial.func1()
Dec 03 11:23:30 gaming trustix[8856]:         github.com/nix-community/trustix/packages/trustix/internal/pool/pool.go:225 +0x157
Dec 03 11:23:30 gaming trustix[8856]: created by github.com/nix-community/trustix/packages/trustix/internal/pool.(*ClientPool).Dial
Dec 03 11:23:30 gaming trustix[8856]:         github.com/nix-community/trustix/packages/trustix/internal/pool/pool.go:218 +0xf8
Dec 03 11:23:30 gaming systemd[1]: trustix.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Dec 03 11:23:30 gaming systemd[1]: trustix.service: Failed with result 'exit-code'.

I got the domain from https://nix-community.github.io/trustix/howto-nix/subscribing.html#configuring.

It does also not work with http:

Dec 03 11:46:03 gaming trustix[27556]: time="2022-12-03T11:46:03+01:00" level=error msg="Error while getting logs from 'http://demo.trustix.dev': unimplemented: 404 Not Found"
Dec 03 11:46:03 gaming trustix[27556]: panic: runtime error: invalid memory address or nil pointer dereference
Dec 03 11:46:03 gaming trustix[27556]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0xa6f5d7]
Dec 03 11:46:03 gaming trustix[27556]: goroutine 13 [running]:
Dec 03 11:46:03 gaming trustix[27556]: github.com/nix-community/trustix/packages/trustix/internal/pool.(*ClientPool).Dial.func1()
Dec 03 11:46:03 gaming trustix[27556]:         github.com/nix-community/trustix/packages/trustix/internal/pool/pool.go:225 +0x157
Dec 03 11:46:03 gaming trustix[27556]: created by github.com/nix-community/trustix/packages/trustix/internal/pool.(*ClientPool).Dial
Dec 03 11:46:03 gaming trustix[27556]:         github.com/nix-community/trustix/packages/trustix/internal/pool/pool.go:218 +0xf8
Dec 03 11:46:03 gaming systemd[1]: trustix.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Dec 03 11:46:03 gaming systemd[1]: trustix.service: Failed with result 'exit-code'.

But the other domain does also not work:

...
Dec 03 11:30:11 gaming trustix[12976]: time="2022-12-03T11:30:11+01:00" level=error msg="Error while getting logs from 'https://r13y.trustix.dev': unknown: 405 Method Not Allowed"
Dec 03 11:30:11 gaming trustix[12976]: panic: runtime error: invalid memory address or nil pointer dereference
Dec 03 11:30:11 gaming trustix[12976]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0xa6f5d7]
Dec 03 11:30:11 gaming trustix[12976]: goroutine 23 [running]:
Dec 03 11:30:11 gaming trustix[12976]: github.com/nix-community/trustix/packages/trustix/internal/pool.(*ClientPool).Dial.func1()
Dec 03 11:30:11 gaming trustix[12976]:         github.com/nix-community/trustix/packages/trustix/internal/pool/pool.go:225 +0x157
Dec 03 11:30:11 gaming trustix[12976]: created by github.com/nix-community/trustix/packages/trustix/internal/pool.(*ClientPool).Dial
Dec 03 11:30:11 gaming trustix[12976]:         github.com/nix-community/trustix/packages/trustix/internal/pool/pool.go:218 +0xf8
Dec 03 11:30:11 gaming systemd[1]: trustix.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Dec 03 11:30:11 gaming systemd[1]: trustix.service: Failed with result 'exit-code'.

When i configure "unix:///run/trustix-daemon.socket" as remote, trustix also crashes, because it does not exist before it is started. I guess it also does not make sense.

Dec 03 11:38:40 gaming trustix[23369]: time="2022-12-03T11:38:40+01:00" level=error msg="Error while getting logs from 'unix:///run/trustix-daemon.socket': unavailable: dial unix /run/trustix-daemon.sock: connect: no such file or directory"
Dec 03 11:38:40 gaming trustix[23369]: panic: runtime error: invalid memory address or nil pointer dereference
Dec 03 11:38:40 gaming trustix[23369]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0xa6f5d7]
Dec 03 11:38:40 gaming trustix[23369]: goroutine 7 [running]:
Dec 03 11:38:40 gaming trustix[23369]: github.com/nix-community/trustix/packages/trustix/internal/pool.(*ClientPool).Dial.func1()
Dec 03 11:38:40 gaming trustix[23369]:         github.com/nix-community/trustix/packages/trustix/internal/pool/pool.go:225 +0x157
Dec 03 11:38:40 gaming trustix[23369]: created by github.com/nix-community/trustix/packages/trustix/internal/pool.(*ClientPool).Dial
Dec 03 11:38:40 gaming trustix[23369]:         github.com/nix-community/trustix/packages/trustix/internal/pool/pool.go:218 +0xf8
Dec 03 11:38:40 gaming systemd[1]: trustix.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Dec 03 11:38:40 gaming systemd[1]: trustix.service: Failed with result 'exit-code'.

signer = "snakeoil";

Describe the bug

I guess a user should add their nickname there.

To Reproduce

https://tweag.github.io/trustix/publishing.html

Expected behavior

The manual should tell a user every step they have to do.

Environment

  • OS name + version:
  • Version of the code:

Additional context

And what do you imply with the example name "snakeoil"? That the whole project is snakeoil and does not contribute to security at all? Is that the impression a new user should have?

Submit already built hashes

Is your feature request related to a problem? Please describe.
This tool is not ready yet but i already build a lot of packages. It would be great if i can still calculate and submit the hashes of all packages that i build myself (not from hydra).

Describe the solution you'd like
Implement and document a workflow that allows me to submit hashes of self-built packages in my local nix store.

Describe alternatives you've considered
Not having the hashes of packages...

Additional context

A proof-of-work blockchain for "binary transparency" and "data transparency"

I was thinking of an international proof-of-work blockchain for "binary transparency" and "data transparency". (data/content-transparency can be used in decentralized web projects like ZeroNet)
In proof-of-work blockchain, can remain immune to split-view-attack because client can independently verify the difficulty of the blockchain so it can assure it is seeing an international powerful blockchain.
Who pays for Miners?
While writing to this blockchain can be free they can also be a donation wallet used for rewarding miners. Those you care not to become victim of split-view attack can donate to this wallet.

Notes:
1.I'm not expert
2.it can be in combination of your multi-signature approach

Link to blog article in readme

Is your feature request related to a problem? Please describe.
There are more information in the blog post than in the documentation (readme). In best case it would not be this way, since the information could be outdated and should be updated with the code (unlike a blog article),

Describe the solution you'd like
but for now it would help to add the link to the blog article to the readme.

https://www.tweag.io/blog/2020-12-16-trustix-announcement/

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Reward becoming a builder

Trustix clients should pay small amount in every connection (through a privacycoins). becoming a builder should be rewarded by these money. Without incentive there is no decentralization

Document real world use

How would someone actually use this?

Who is this for? Only for people with higher security requirements that don't trust hydra builds, or should it be enabled on every Nix install at some point?

How would it be configured on a mobile device (notebook, no static network, behind firewall), workstation (pc, static ip, behind firewall) or server (VPS, static ip, public ip)?

What if i use all 3 kinds of devices? Should i collect logs only on one? Or should i have different names and key-pairs for all of them?

What to do if one builder get compromised? Do i have to revoke my vote that i have build the software, or does the system naturally handle the case that one hash is different than the others? (i think yes)

What is a sane policy for a nixos user (and contributor)? As i understand e.g. 2/3 majority requires at least 3 different builds, so i have to subscribe to 3 logs (how do remotes relate to that?) that have build the package i want.

To what log should i subscribe? Do i have to trust it? (that would defeat the whole purpose and i could just add their binaryCaches)
Or can i just subscribe to random logs? What if N$4 actually advertises millions of logs, i subscribe to them and they succeed in a 51% attack?

A use case might be unfree packages that are always build locally, like Steam. So i would subscribe to the logs of other steam users and when 3 people have build it before, i can just download it from them. But providing such packages for download might be a copyright violation. So maybe not share such packages? Maybe we as foss community could get a reference court decision that when it's clear that i get the same file if i build it myself or download, it's legally fine to download it and it's considered as if i built it myself.

Could someone guess my passwort in a config file and pull that file from me to verify it? Should we add and share configurations at all? (the problem is Nix internally don't know what a package, test or configuration is)

I'm often building packages changed in a nixpkgs PR to review it. The submitter has build it too as well as ofborg. How can we verify that we have the same hash (to see the package is build reproducible)? How can we do that on a big scale for all nixpkgs contributors?

How can we check all published logs if a package have different hashes and is therefore not reproducible? I guess trustix-nix-reprod is intended for that. But there is no documentation yet.

What happens when a package is not reproducible? The hashes will never match and it will not be a majority, so it will be always build locally.

Can i configure this to get packages that where build by hydra from a closer cache that i don't have to trust? So i ask hydra for the hash and ask random close stores (local network at company or university, random from nearby ip range measured by geoip and ping)?

How to configure it to always fall back to hydra?

How would a use case look like where i don't trust hydra builds?

Imagine this is enabled on NixOS by default. What are the default subscribers? Can i as a regular user add my own logs to that list?

Do remotes need to be a public domain that resolves to a public IP? How can that work for notebooks behind nat + firewall?

When i destroy a system (like temporary container/vm), can i transfer the logs to another system to preserve them?

What else is possible when this would have 5000 users?

not everyone has the same level of trust in different build servers, or the same security requirements. By defining consensus and having the vote client-side, we end up with a model that is much more flexible and can be tuned to your use cases and threat model.

can we have multiple sane defaults for security levels, like

i trust no one = always build locally
i only trust myself = use other logs owned by myself
i trust the community = list of community logs maintained in nixpkgs. optionally only verified nixpkgs contributors or committers
i trust nixos foundation = use cache.nixos.org as truth, but also use other caches like from community when hashes match
i trust everyone = use all logs available (are they discovered peer-to-peer?)

If the default voting mechanism is majority-rules, then an attacker would need to gain control of 51% of all configured nix Trustix logs, including access to the log’s private key, an attack which is extremely unlikely. Keep in mind that even if this should happen, high-profile targets would likely have configured their own voting procedure, so they would be unaffected.

how does a voting procedure look like that is unaffected from 51% attack? 2/3 majority? 9/10 majority?
how much do i have to trust the logs i subscribe? would you add my log if you where a high-profile target? i might be kind of trustworthy since i'm around for 5 years, but they could pay me good and i will add their malware... but they would have to do that with many community members and it's unlikely that no one talks about it. so just subscribe to a diverse set of logs from different community members?

Can you add a community registry where people can add their logs, so we can test with as many logs as possible?
It should have some metadata like maintainer (from nixpkgs. used as name, e-mail, ...).
Have a nixos option to enable them.

Add more channels and build targets to https://r13y.trustix.dev/

nixos-unstable (for users and developers)
nixos.iso_minimal.x86_64-linux
nixos-gnome-x86_64-linux.iso
every package of nixpkgs

nixos-stable: 22.05 (for users, for release announcement)
nixos.iso_minimal.x86_64-linux
nixos-gnome-x86_64-linux.iso
every package of nixpkgs

maybe also ARM, RISCV, darwin, ..... don't know if there is any reproducabillity efforts

so we are prepared to people start using it and already have a design for multiple trackings

panic: open /var/trustix/keys/cache-private-key.pem: permission denied

Describe the bug

Again, following the documentation lead to program crash!

[root@gaming:~]# journalctl -u trustix-nix-cache.service
May 08 02:56:35 gaming systemd[1]: Started Trustix Nix binary cache daemon.
May 08 02:56:35 gaming trustix-nix[385966]: panic: open /var/trustix/keys/cache-private-key.pem: permission denied
May 08 02:56:35 gaming trustix-nix[385966]: goroutine 1 [running]:
May 08 02:56:35 gaming trustix-nix[385966]: github.com/tweag/trustix/packages/trustix-nix/cmd.readKey({0x7ffe937e9bbc, 0xe5b460})
May 08 02:56:35 gaming trustix-nix[385966]:         /build/source/cmd/binary-cache-proxy.go:70 +0x26d
May 08 02:56:35 gaming trustix-nix[385966]: github.com/tweag/trustix/packages/trustix-nix/cmd.glob..func1(0xe4ef40, {0x9c58a8, 0x2, 0x2})
May 08 02:56:35 gaming trustix-nix[385966]:         /build/source/cmd/binary-cache-proxy.go:101 +0x85
May 08 02:56:35 gaming trustix-nix[385966]: github.com/spf13/cobra.(*Command).execute(0xe4ef40, {0xc0000807e0, 0x2, 0x2})
May 08 02:56:35 gaming trustix-nix[385966]:         /build/source/vendor/github.com/spf13/cobra/command.go:850 +0x60e
May 08 02:56:35 gaming trustix-nix[385966]: github.com/spf13/cobra.(*Command).ExecuteC(0xe4f480)
May 08 02:56:35 gaming trustix-nix[385966]:         /build/source/vendor/github.com/spf13/cobra/command.go:958 +0x3ad
May 08 02:56:35 gaming trustix-nix[385966]: github.com/spf13/cobra.(*Command).Execute(...)
May 08 02:56:35 gaming trustix-nix[385966]:         /build/source/vendor/github.com/spf13/cobra/command.go:895
May 08 02:56:35 gaming trustix-nix[385966]: github.com/tweag/trustix/packages/trustix-nix/cmd.Execute()
May 08 02:56:35 gaming trustix-nix[385966]:         /build/source/cmd/root.go:61 +0x45
May 08 02:56:35 gaming trustix-nix[385966]: main.main()
May 08 02:56:35 gaming trustix-nix[385966]:         /build/source/main.go:14 +0x17
May 08 02:56:35 gaming systemd[1]: trustix-nix-cache.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
May 08 02:56:35 gaming systemd[1]: trustix-nix-cache.service: Failed with result 'exit-code'.

To Reproduce

https://tweag.github.io/trustix/binarycache.html#trustix---binary-cache-setup

Expected behavior

  1. programs should never crash; rather provide meaningful error messages
  2. following the documentation should lead to working example

Environment

  • OS name + version:
  • Version of the code:

Additional context

Same as in #28. Auto-generate on first start if file does not exist.

Rebuild command on https://r13y.trustix.dev/

Let's consider i want to contribute to e.g. https://r13y.trustix.dev/drv?storePath=%2Fnix%2Fstore%2F6741gxgxx598589zkw0m4w324h4ymv3l-nixos-minimal-22.11pre0.-x86_64-linux.iso.drv.

There should be a command on the site that i can copy in my terminal (with trustix setup) and it builds the complete derivation, to add my opinion on what the correct result is.

And a second command to only build and submit those packages that are missing. The goal is to figure out if a package is reproducible. So 2 builds of every package, while 1 already comes from Hydra when it's an official channel.

When the 2 builds have different results, it might help debugging to have a third? Maybe apply this rule just for unstable and others, not stable? I guess we don't try to fix such issues on stable.

But do what the r13y community wants and needs.

Connecting to log server via proxy (and not the substituters)

Is it possible that Trustix downloads packages directly but connects to the transparency log server via proxy? It makes "split view attack" harder. Also better if you force even ordinary users to use Tor to connect to log server (by making the log server's address .onion)

(There is a monetized version of Tor, also available)

Document debugging

For example:

  • How to view the database?

It is a Bolt embedded key-value store using the bbolt implementation.

We have the CLI Browser for BoltDB files boltbrowser in nixpkgs. The database file does must not be locked to be able to open it, even in read-only mode.

So stop the services first:

systemctl stop trustix.socket

Now you can have a look at the database:

$ sudo nix run nixpkgs#boltbrowser -- -ro /var/lib/trustix/trustix.db

Screenshot from 2022-08-03 13-21-13

Screenshot from 2022-08-03 13-22-00

When you are ready, start the services again:

systemctl start trustix.service
systemctl start trustix-nix-cache.service
  • How to view a list of all log entries?

Use the database browser i guess.

  • How to view a single log entry?

/nix/store/sql7z5xkbmydnqinyp3brpp3m76vj13p-trustix-dev/bin/trustix query --key string

but where to get string

Doing it like in this test works:

https://github.com/tweag/trustix/blob/e89875c2a40588c7badeb52ec31ea23fa2e0c7b6/packages/trustix/tests/default.nix#L24-L40

I still don't know how i get from "/nix/store/hkgpl034l6c5zgzhks2dyp7p41z6qyc4-hello-2.12" to the key.

Why does it return a different value than the one i submitted? Isn't the purpose of a database to store values and not alter them?

  • How to test decide?
[davidak@gaming:~]$ sudo /nix/store/sql7z5xkbmydnqinyp3brpp3m76vj13p-trustix-dev/bin/trustix decide --protocol '5138a791-8d00-4182-96bc-f1f2688cdde2' --key '6574' --address unix:///run/trustix-daemon.socket --log-id *****
[sudo] password for davidak: 
DEBU[0000] Dialing remote                                address="unix:///run/trustix-daemon.socket"
DEBU[0000] Requesting output mappings for                key=6574
FATA[0000] could not decide: rpc error: code = Unknown desc = No decider configured for protocol '5138a791-8d00-4182-96bc-f1f2688cdde2'

Roadmap for small-scale community testing

When we reach a level of beta quality (currently i consider it alpha, but broken), we can do small-scale community testing with as many people as possible, to see how well it works and what benefits it brings. That can grow organically as more people get interested in it.

A roadmap can list the steps that need to be done to achieve this goal.

I currently don't understand how this software would work in practice, so i need #41 for better understanding and thinking about it.

what is meant by "a ledger"

Is your feature request related to a problem? Please describe.
The readme says:

In a post-build hook the output hash (NAR hash) of the build is uploaded to a ledger

it is not clear to me what is meant by "a ledger"

Describe the solution you'd like
document which ledger is used and how. can i view all hashes for a package there?

the article said it's not centralized, so is it peer-to-peer? what happens when my machine is offline? are my submitted hashes still accessible?

Describe alternatives you've considered
not documenting it and calling it magic

Additional context
please document more

Complete LUA -> JS transition

Describe the bug

There is still LUA in the code:

[davidak@gaming:~/code/trustix]$ grep lua -R .
...
./packages/trustix-nix/gomod2nix.toml:["github.com/Shopify/go-lua"]
./packages/trustix-nix/gomod2nix.toml:  ["github.com/Shopify/go-lua".fetch]
./packages/trustix-nix/gomod2nix.toml:    url = "https://github.com/Shopify/go-lua"

Related:

To Reproduce
Steps to reproduce the behavior.

Expected behavior
A clear and concise description of what you expected to happen.

Environment

  • OS name + version:
  • Version of the code:

Additional context
Add any other context about the problem here.

More logging

Is your feature request related to a problem? Please describe.

When the post-build hook sends hashes to the log, i only see this line in trustix log:

May 09 07:44:11 gaming trustix[893843]: time="2022-05-09T07:44:11+02:00" level=debug msg="Checking peer credential for socket"

Describe the solution you'd like

Also log "added key x"

Describe alternatives you've considered

Additional context
you probably also missed a lot of actions that can be logged, even if just in debug mode

e.g. when my result is submitted to https://r13y.trustix.dev/ i want to see a link to the results derivation, to see if there are other results and if they match mine

Future of build-transparency.org

At the start of the project, there was this page created: https://build-transparency.org/

I think it was never updated and it is not really useful.

If you want to have a landing page, you should get a domain with trustix in it and create a new, more end-user focused page.

build-transparency.org can probably just get deleted.

No decider configured for protocol '5138a791-8d00-4182-96bc-f1f2688cdde2' (nix)

Describe the bug

Submit key and value:

[davidak@gaming:~]$ sudo /nix/store/sql7z5xkbmydnqinyp3brpp3m76vj13p-trustix-dev/bin/trustix --address unix:///run/trustix-daemon.socket --log-id ***** submit --key "bc63f28a4e8dda15107f687e6c3a8848492e89e3bc7726a56a0f1ee68dd9350d" --value "28899cec2bd12feeabb5d82a3b1eafd23221798ac30a20f449144015746e2321"
DEBU[0000] Dialing remote                                address="unix:///run/trustix-daemon.socket"
DEBU[0000] Submitting mapping                            key=bc63f28a4e8dda15107f687e6c3a8848492e89e3bc7726a56a0f1ee68dd9350d value=28899cec2bd12feeabb5d82a3b1eafd23221798ac30a20f449144015746e2321
OK

Flush

[davidak@gaming:~]$ sudo /nix/store/sql7z5xkbmydnqinyp3brpp3m76vj13p-trustix-dev/bin/trustix --address unix:///run/trustix-daemon.socket --log-id ***** flush
DEBU[0000] Dialing remote                                address="unix:///run/trustix-daemon.socket"

Query key:

[davidak@gaming:~]$ sudo /nix/store/sql7z5xkbmydnqinyp3brpp3m76vj13p-trustix-dev/bin/trustix --address unix:///run/trustix-daemon.socket --log-id ccbfc3d79bc7c05ebb0655fb62e694ab3a69ae55ed16a86faede15f2b8b1e190 query --key "bc63f28a4e8dda15107f687e6c3a8848492e89e3bc7726a56a0f1ee68dd9350d"
DEBU[0000] Dialing remote                                address="unix:///run/trustix-daemon.socket"
DEBU[0000] Requesting log head                          
DEBU[0000] Requesting output mapping for                 key=bc63f28a4e8dda15107f687e6c3a8848492e89e3bc7726a56a0f1ee68dd9350d
Output digest: 5768f7201db3dccf3ec8c5ec2be5108c411396ad8c1351d89294f515456cdc23

Try decide:

[davidak@gaming:~]$ sudo /nix/store/sql7z5xkbmydnqinyp3brpp3m76vj13p-trustix-dev/bin/trustix --address unix:///run/trustix-daemon.socket decide --protocol nix --key bc63f28a4e8dda15107f687e6c3a8848492e89e3bc7726a56a0f1ee68dd9350d
DEBU[0000] Dialing remote                                address="unix:///run/trustix-daemon.socket"
DEBU[0000] Requesting output mappings for                key=bc63f28a4e8dda15107f687e6c3a8848492e89e3bc7726a56a0f1ee68dd9350d
FATA[0000] could not decide: rpc error: code = Unknown desc = No decider configured for protocol '5138a791-8d00-4182-96bc-f1f2688cdde2'

journalctl -u trustix.service -f

May 09 00:49:41 gaming trustix[652959]: time="2022-05-09T00:49:41+02:00" level=debug msg="Checking peer credential for socket"
May 09 00:49:41 gaming trustix[652959]: time="2022-05-09T00:49:41+02:00" level=info msg="Received Decide request" key=bc63f28a4e8dda15107f687e6c3a8848492e89e3bc7726a56a0f1ee68dd9350d

To Reproduce

Needs #38.

  services.trustix = {
    enable = true;

    signers.davidak = {
      type = "ed25519";
      ed25519 = {
        private-key-path = "/var/trustix/keys/private";
      };
    };

    publishers = [
      {
        signer = "davidak";
        protocol = "nix";
        publicKey = {
          type = "ed25519";
          key = "*****";
        };
      }
    ];

    subscribers = [
      {
        protocol = "nix";
        publicKey = {
          type = "ed25519";
          key = "*****";
        };
      }
    ];

    deciders.nix = {
      engine = "percentage";
      #percentage.minimum = 66;
      percentage.minimum = 0;
    };

    # A remote can expose many logs and they are not neccesarily created by the remote in question
    remotes = [
      #"grpc+http://127.0.0.1"
      #"https://cache.nixos.org"
      "unix:///run/trustix-daemon.socket"
    ];

  };

  # Push local builds via the post-build hook
  services.trustix-nix-build-hook = {
    enable = true;
    logID = "*****";
  };

  # Enable the local binary cache server
  services.trustix-nix-cache = {
    enable = true;
    private-key = "/var/trustix/keys/cache-private-key.pem";
    listen = "0.0.0.0";
    port = 9001;
    openFirewall = true;
  };

  # Configure Nix to use it
  nix = {
    binaryCaches = [
      "http://127.0.0.1:9001"
    ];
    binaryCachePublicKeys = [
      "binarycache.example.com:*****"
    ];
  };

Expected behavior

decide should work according to the configuration.

i had write the config differently than the documentation says to avoid errors. this is the only way i found works without errors

Environment

  • OS name + version: NixOS 22.05pre375061.c777cdf5c56
  • Version of the code: 7ed5270

Additional context
Add any other context about the problem here.

Please provide a `flake.nix`

Is your feature request related to a problem? Please describe.

The current installation instructions assume that a local clone of the repository can be made and easily accessed through an absolute path.

This is not possible in flakes, also it is barely reproducible to keep a user maintained clone around.

Describe the solution you'd like

Provide a flake.nix file that declares the important outputs and makes them available to consuming flakes.

Describe alternatives you've considered

Using "trustix" as a flake = false input.

panic: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial unix /tmp/trustix.sock: connect: no such file or directory"

Describe the bug

trustix-nix-cache expects the trustix socket at unix:///tmp/trustix.sock, but it is at /run/trustix-daemon.socket by default.

To Reproduce

https://tweag.github.io/trustix/binarycache.html

  # Enable the local binary cache server
  services.trustix-nix-cache = {
    enable = true;
    private-key = "/var/trustix/keys/cache-private-key.pem";
    port = 9001;
  };

Expected behavior

  1. programs should never crash; rather provide meaningful error messages
  2. following the documentation should lead to working example

Environment

  • OS name + version: NixOS 22.05pre375061.c777cdf5c56
  • Version of the code: 7ed5270

Additional context

/tmp/trustix.sock is the default from trustix-nix

there is an option for trustix-rpc in

https://github.com/tweag/trustix/blob/58208de059dbabef6347e560db325ac98acc36d8/packages/trustix-nix/nixos/binarycache.nix#L32-L36

but it is not used

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.