snyk / broker

A broker system between a public service and a private service

---

snyk/broker

Snyk Broker proxies access between snyk.io and your Git repositories, such as GitHub Enterprise, GitHub.com and Bitbucket Server. Snyk Broker can also be used to enable a secure connection with your on-premise Jira deployment.

The Broker server and client establish an applicative tunnel, proxying requests from snyk.io to the Git (fetching manifest files from monitored repositories), and vice versa (webhooks posted by the Git).

The Broker client runs within the user's internal network, keeping sensitive data such as Git tokens within the network perimeter. The applicative tunnel scans and adds only relevant requests to an approved list, narrowing down the access permissions to the bare minimum required for Snyk to actively monitor a repository.

More details here

Usage

Please refer to our extensive documentation.

broker's Issues

Filtering incoming Client connections

I think I've discovered a vulnerability with the incoming Client connection.

When Client with BROKER_ID 123 is connected to the server and 2nd (evil) Client connects with the same BROKER_ID, it overtakes original Client's position and all Server calls that should go to the original Client goes only to the evil one. Could cause both data leaks and mocked responses to mess with Server.

There are a few ways to mitigate this, although they are not perfect:

• IP filtering: not sure if we can have a stable IP/IP range from those internal networks, but if so, it sounds reasonable
• per-client Server-Client certificate: Client will specify what cert it expects from Server and Server serves different per-client certificate - could be bypassed with a dedicated client ignoring errors IMHO
• OAuth 1: I know, I know… but it would also come with a MAC, key rotation etc.

What prevents it now is that, Nginx can't distinguish incoming connections before handshake because path looks like this path="/primus/?_primuscb=LU9fg6t&EIO=3&transport=polling&t=LU9fg6v&b64=1" and BROKER_ID is in payload so it might have to be resolved on Server - check that BROKER_ID is allowed in ENV and its ENV value matches IP/IP range (in socket.on('identify'))

tl;dr: I'd propose a Server would only accept Client connections if they are named in the ENV and their IP/IP range matches. So Server would also have a MY-CLIENT-ID=<IP> in ENV for each allowed Client.

Drop JSON configs in favor of YAML

My reason would be that JSON is more machine readable and we want these configs to be easier to audit. Compare this part of config:

{
  "public":
  [
      {
        "//": "used for pushing up webhooks from github",
        "method": "POST",
        "path": "/webhook/github",
        "valid": [
          {
            "//": "accept all pull request state changes (these don't have files in them)",
            "path": "pull_request.state",
            "value": "open"
          },
          {
            "path": "commits.*.added.*",
            "value": "package.json"
          },
          {
            "path": "commits.*.modified.*",
            "value": "package.json"
          },
          {
            "path": "commits.*.added.*",
            "value": ".snyk"
          },
          {
            "path": "commits.*.modified.*",
            "value": ".snyk"
          }
        ]
      }
  ]
}

public:
# used for pushing up webhooks from github
- method: POST
  path: "/webhook/github"
  valid:
  # accept all pull request state changes (these don't have files in them)
  - path: pull_request.state
    value: open
  - path: commits.*.added.*
    value: package.json
  - path: commits.*.modified.*
    value: package.json
  - path: commits.*.added.*
    value: ".snyk"
  - path: commits.*.modified.*
    value: ".snyk"

Thoughts?

<SPAM>

    > Hi, Are other container engines besides Docker supported for running the broker container ? For example: Podman - which is currently included and supported in EL8 and newer and Ubuntu 22.10 and newer. The Community Edition of Docker is a 3rd party package usually not supported by any Linux distro vendors. Thanks !

Originally posted by @Jimimaku in #455 (comment)

Additional (auth) headers? And/or features in general

This is more of a feature/project mgmt question than an issue so skipping the issue template.

There are currently two different auth header options we can add on the client side to the outgoing connection, basic and token. We would have the need to use something like bearer or even raw (without any prepended word) as well in our environments.

Is this project open to accept pull requests to add those kind of additions into the package or is this functionality very Snyk specific and you'd recommend to fork the repo instead?

This would be something that we (RoadieHQ) would be looking forward to contribute in.

Audit log

We need to provide a sensible logging for audit - as in who accessed what resource when and with what outcome. Debug is useful for development, but currently, it on one side exposes tokens etc. and doesn't record IP of request origin etc.

We have some internal logging format, but it's far from perfect in current state. Do you have more experience with this? The best resource I found so far was this: https://www.owasp.org/index.php/Logging_Cheat_Sheet I can prepare some proposal/PR in the spirit of OWASP guide.

From a technical standpoint, standard output to console should be sufficient afaik - implemented with https://github.com/winstonjs/winston for example.

Documentation: CPU and memory requirements for the docker

Please add CPU and memory requirements for the docker to documentation.

4.129.0 fails to start and core dumps

• node -v:

node -v
v16.15.1

• npm -v:

npm -v
8.11.0

• Command run:

We updated from snyk broker from 4.122.0 to 4.129.0 and now the broker fails to start with the following logs:

node[1]: ../src/node_platform.cc:61:std::unique_ptr<long unsigned int> node::WorkerThreadsTaskRunner::DelayedTaskScheduler::Start(): Assertion `(0) == (uv_thread_create(t.get(), start_thread, this))' failed.
 1: 0xb09c10 node::Abort() [node]
 2: 0xb09c8e  [node]
 3: 0xb794ce  [node]
 4: 0xb79596 node::NodePlatform::NodePlatform(int, v8::TracingController*) [node]
 5: 0xacd0e9 node::InitializeOncePerProcess(int, char**, node::InitializationSettingsFlags, node::ProcessFlags::Flags) [node]
 6: 0xacd399 node::Start(int, char**) [node]
 7: 0x7f4d5d387d90  [/lib/x86_64-linux-gnu/libc.so.6]
 8: 0x7f4d5d387e40 __libc_start_main [/lib/x86_64-linux-gnu/libc.so.6]
 9: 0xa4067c  [node]

Adding debug logging doesn't produce any more logs. Host that its running on has plenty of resources and its not a ulimit issue (nodejs/node#43064)

Expected behaviour

Broker should start.

Actual behaviour

Broker dies.

Steps to reproduce

---

If applicable, please append the --debug flag on your command and include the output here **ensuring to remove any sensitive/personal details or tokens.

How to handle self-signed certs on Internal services that Client interacts with

The best option for these cases seems to have an envvar with a CA certificate. I'd discourage anyone from using it with NODE_TLS_REJECT_UNAUTHORIZED - and let's see if we even get to such situation. However, if this happens, maybe we can improve it with:

• certificate pinning (Client-Server) - not sure if this can be enforced with NODE_TLS_REJECT_UNAUTHORIZED active
• Additional encryption/TLS? MAC signing?

I'd remove the NODE_TLS_REJECT_UNAUTHORIZED note from Readme as it is too dangerous in my opinion.

README.md cites the wrong port

In the README, we provide commands in this format:

docker run --restart=always \
           -p 8000:8000 \
...

However, the mapping of the 8000:8000 port doesn't work since we default to 7341. I recommend instead we specify

-p 7341:7341
as a default, or even
-p 8000:7341
if we're interested in using port 8000. In the present form, 8000:8000 goes nowhere and the misalignment to the actual port may confuse our end users.

Other container engine than Docker

Hi,
Are other container engines besides Docker supported for running the broker container ?
For example: Podman - which is currently included and supported in EL8 and newer and Ubuntu 22.10 and newer.
The Community Edition of Docker is a 3rd party package usually not supported by any Linux distro vendors.
Thanks !

Snyk Broker not starting: Illegal operation on a directory

I set up the current snyk-broker version via GCPs Cloud Run via Kubernetes. Unfortunately, the container does not spawn correctly. From the log I can see the default config is being created (filters appear and the server listens on 8080), but an error prevents the container from spinning up completely:

Error: EISDIR: illegal operation on a directory, read
    at main (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/lib/webserver.js:37:25)
    at Object.module.exports [as client] (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/lib/client/index.js:21:52)
    at Object.main (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/lib/index.js:34:23)
    at module.exports (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/cli/exec.js:8:42)
    at Object.<anonymous> (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/cli/index.js:22:5) {

My config:

Any idea what directory is being accessed illegally here?

Confidential data (hashed broker token) stored in path

file - https://github.com/snyk/broker/blob/master/lib/client/dispatcher/client/api.ts
line - 24

Storing confidential information such as broker token in path is generally a bad idea.
This information should be moved to the POST body to let the TLS do the work.

Helm chart?

Is there a plan to create an official helm chart for simple Kubernetes deployment?

:tada: This PR is included in version 4.86.0 :tada:

    :tada: This PR is included in version 4.86.0 :tada:

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

Originally posted by @snyksec in #261 (comment)

Tough-Cookie vulnerability. Possible approaches?

We received a notification that there is a vulnerable version of tough-cookie coming from one of the dependencies of this package.

The CVE is in here:

• https://nvd.nist.gov/vuln/detail/CVE-2023-26136
• https://security.snyk.io/vuln/SNYK-JS-TOUGHCOOKIE-5672873

There was an attempt to remove this dep within this PR: https://github.com/snyk/broker/pull/570/files but that never made to the final image since the dependency is needed.

What would be the best approaches to mitigate this?

missing contributing.md file

Suggest we should have a CONTRIBUTING.md file in this repository.

Require Linux/ARM64 docker image for snyk/broker

Hi Team,

I have been working on building snyk/broker docker image for both amd64 and arm64 platform. Successfully built and tested the docker image for both the platforms.

Do you have any plans for releasing ARM64 images?

It will be very helpful if an ARM64 image is available.

Prometheus metrics support?

How can we monitor the broker? We are planning to host it on Kuberentes, so prometheus metrics support (disabled by default, with paramter that enable them) will be great. This express middleware looks promising.

Misleading error message

When installing the Snyk-broker on a Openshift-platform with limitations on file access due to security context a misleading error message is given. This will, at least it did for us, possibly result in time waste for the "sys-admins". We spent many ours trying to figure out what the No Filters found. A Broker requires filters to run. Shutting down.-message meant, as there is no reference to any filters in the documentation or in the Helm-chart. We finally had a look at the source code and was able to backtrack the message

Expected behaviour

If the application is not able to load the files in defaultFilters a complementary and precise error message should be given. In our case the process did not have access to load the files (access denied)

Actual behaviour

The error message No Filters found. A Broker requires filters to run. Shutting down. is given.

broker/lib/index.ts

Line 51 in f001b36

`No Filters found. A Broker requires filters to run. Shutting down.`,

Steps to reproduce

Make the files in defaultFilters inaccessible for the process while the application starts

Implement /readinesscheck combining both healthchecks for Kubernetes deployments

For deployments on Kubernetes it would be great to have a single health check endpoint the combines the results from the already existing healthchecks /healthcheck and /systemcheck for deployments in Kubernetes, because Kubernetes only allows the configuration of a single readiness or liveness check.

The behavior should be a single AND of the status of the already existing checks to clearly report to the operator that there is an issue with connecting to either network endpoint.

Ability to disable variable substitution

It would be fantastic if it was possible to disable the body and header substitution feature of the broker client/server. Ideally this could be done at the filter level with a new option like "disableVarsSubstitution" and or "disableBodyVarsSubstitution/disableHeaderVarsSubstitution". Another option is to have the ability to disable var substitution globally on a client. e.g. broker --disableVarsSubstitution or something like that.

SSL routines:ssl3_get_record:wrong version number

Hello,

docker image : broker:4.157.1-bitbucket-server

We have installed the snyk broker on our Openshift instance (We do not have a token yet). Outbound connection to snyk is performed through our enterprise proxy (the url is whitelisted as expected)

But the broker fails because of SSL routines:ssl3_get_record:wrong version number

{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":30,"brokerClientId":"7190f16f-8413-49fc-83e6-3d92880cbeab","msg":"generated broker client id","time":"2023-07-13T12:54:14.970Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":30,"enabled":false,"msg":"checking for HA mode","time":"2023-07-13T12:54:14.970Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":30,"enabled":true,"msg":"verifying if preflight checks are enabled","time":"2023-07-13T12:54:14.970Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":30,"enabled":false,"msg":"checking for HA mode","time":"2023-07-13T12:54:14.971Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":40,"retryCount":3,"errorMessage":"write EPROTO 140403255646144:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:332:\n","url":"https://broker.snyk.io/healthcheck","msg":"retrying request x 3 ","time":"2023-07-13T12:54:16.238Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":40,"attempt":0,"operation":"http check broker-server-status","timeout":100,"msg":"waiting for 100ms before next try","time":"2023-07-13T12:54:17.250Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":40,"retryCount":3,"errorMessage":"write EPROTO 140403255646144:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:332:\n","url":"https://broker.snyk.io/healthcheck","msg":"retrying request x 3 ","time":"2023-07-13T12:54:18.372Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":40,"attempt":1,"operation":"http check broker-server-status","timeout":200,"msg":"waiting for 200ms before next try","time":"2023-07-13T12:54:19.387Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":40,"retryCount":3,"errorMessage":"write EPROTO 140403255646144:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:332:\n","url":"https://broker.snyk.io/healthcheck","msg":"retrying request x 3 ","time":"2023-07-13T12:54:20.576Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":40,"attempt":2,"operation":"http check broker-server-status","timeout":400,"msg":"waiting for 400ms before next try","time":"2023-07-13T12:54:21.565Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":40,"retryCount":3,"errorMessage":"write EPROTO 140403255646144:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:332:\n","url":"https://broker.snyk.io/healthcheck","msg":"retrying request x 3 ","time":"2023-07-13T12:54:23.042Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":50,"error":{"name":"Error","message":"Error executing check with checkId broker-server-status","stack":"Error: Error executing check with checkId broker-server-status\n at HttpCheckService.run (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/lib/client/checks/http/http-check-service.js:33:19)\n at processTicksAndRejections (node:internal/process/task_queues:96:5)\n at async retry (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/lib/client/retry/exponential-backoff.js:7:16)\n at async executePreflightChecks (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/lib/client/checks/index.js:28:29)\n at async Object.module.exports [as client] (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/lib/client/index.js:33:37)\n at async Object.main (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/lib/index.js:40:12)\n at async module.exports (/home/node/.npm-global/lib/n...
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":30,"url":"https://broker.snyk.io","serverId":"","msg":"broker client is connecting to broker server","time":"2023-07-13T12:54:24.074Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":30,"rulesCount":110,"msg":"loading new rules","time":"2023-07-13T12:54:24.075Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":30,"port":"8000","msg":"local server is listening","time":"2023-07-13T12:54:24.146Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":30,"rulesCount":1,"msg":"loading new rules","time":"2023-07-13T12:54:24.147Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":50,"type":"TransportError","description":404,"msg":"Failed to connect to broker server","time":"2023-07-13T12:54:24.528Z","v":0}

Running the docker image outside of Openshift leads to the same error.
The proxy configuration is OK, because running this from the node cli works as expected from the container itself

const request = require('request');
request('https://broker.snyk.io/healthcheck', function (error, response, body) {
...   console.error('error:', error);
...   console.log('statusCode:', response && response.statusCode);
...   console.log('body:', body); 
... });

Response

> error: null
statusCode: 200
body: {"ok":true,"version":"4.155.0"}

Thanks