stakater / gitwebhookproxy

A proxy to let webhooks reach running services behind a firewall – [✩Star] if you're using it!

GitWebhookProxy

A proxy to let webhooks to reach a Jenkins instance running behind a firewall

PROBLEM

Jenkins is awesome and matchless tool for both CI & CD; but unfortunately its a gold mine if left in wild with wide open access; so, we always want to put it behind a firewall. But when we put it behind firewall then webhooks don't work anymore and no one wants the pull based polling but rather prefer the build to start as soon as there is a commit!

SOLUTION

This little proxy makes webhooks start working again!

Supported Providers

Currently we support the following git providers out of the box:

• Github
• Gitlab

Configuration

GitWebhookProxy can be configured by providing the following arguments either via command line or via environment variables:

Parameter	Description	Default	Example
listenAddress	Address on which the proxy listens.	:8080	127.0.0.1:80
upstreamURL	URL to which the proxy requests will be forwarded (required)		https://someci-instance-url.com/webhook/
secret	Secret of the Webhook API. If not set validation is not made.		iamasecret
provider	Git Provider which generates the Webhook	github	github or gitlab
allowedPaths	Comma-Separated String List of allowed paths on the proxy		/project or github-webhook/,project/
ignoredUsers	Comma-Separated String List of users to ignore while proxying Webhook request		someuser
allowedUsers	Comma-Separated String List of users to allow while proxying Webhook request		someuser

DEPLOYING TO KUBERNETES

The GitWebhookProxy can be deployed with vanilla manifests or Helm Charts.

Vanilla Manifests

For Vanilla manifests, you can either first clone the respository or download the deployments/kubernetes/gitwebhookproxy.yaml file only.

Configuring

Below mentioned attributes in gitwebhookproxy.yaml have been hard coded to run in our cluster. Please make sure to update values of these according to your own configuration.

1. Change below mentioned attribute's values in Ingress in gitwebhookproxy.yaml

 rules:
  - host: gitwebhookproxy.example.com

  tls:
  - hosts:
    - gitwebhookproxy.example.com

2. Change below mentioned attribute's values in Secret in gitwebhookproxy.yaml

data:
  secret: example

3. Change below mentioned attribute's values in ConfigMap in gitwebhookproxy.yaml

data:
  provider: github
  upstreamURL: https://jenkins.example.com
  allowedPaths: /github-webhook,/project
  ignoredUsers: stakater-user

Deploying

Then you can deploy GitwebhookProxy by running the following kubectl commands:

kubectl apply -f gitwebhookproxy.yaml -n <namespace>

Note: Make sure to update the port in deployment.yaml as well as service.yaml if you change the default listenAddress port.

Helm Charts

Alternatively if you have configured helm on your cluster, you can add gitwebhookproxy to helm from our public chart repository and deploy it via helm using below mentioned commands

1. Add the chart repo:

   i. helm repo add stakater https://stakater.github.io/stakater-charts/

   ii. helm repo update

2. Set configuration as discussed in the Configuring section

   i. helm fetch --untar stakater/gitwebhookproxy

   ii. Open and edit gitwebhookproxy/values.yaml in a text editor and update the values mentioned in Configuring section.

3. Install the chart

   • helm install stakater/gitwebhookproxy -f gitwebhookproxy/values.yaml -n gitwebhookproxy

Running outside Kubernetes

Run with Docker

To run the docker container outside of Kubernetes, you can pass the configuration as the Container Entrypoint arguments. The docker image is available on docker hub. Example below:

docker run stakater/gitwebhookproxy:v0.2.63 -listen :8080 -upstreamURL google.com -provider github -secret "test"

Run with Docker compose

For docker compose, the syntax is a bit different

jenkinswebhookproxy: 
    image: 'stakater/gitwebhookproxy:latest'
    command: ["-listen", ":8080", "-secret", "test", "-upstreamURL", "jenkins.example.com, "-allowedPaths", "/github-webhook,/ghprbhook"]
    restart: on-failure

Troubleshooting

405 Method Not Allowed with Jenkins & github plugin

If you get the following error when setting up webhooks for your jobs in Jenkins, make sure you have the trailing / in the webhook configured in Jenkins.

Error Redirecting '/github-webhook' to upstream', Upstream Redirect Status: 405 Method Not Allowed

Help

Got a question? File a GitHub issue, or send us an email.

Talk to us on Slack

Join and talk to us on the #tools-gwp channel for discussing about GitWebhookProxy

Contributing

Bug Reports & Feature Requests

Please use the issue tracker to report any bugs or file feature requests.

Developing

PRs are welcome. In general, we follow the "fork-and-pull" Git workflow.

1. Fork the repo on GitHub
2. Clone the project to your own machine
3. Commit changes to your own branch
4. Push your work back up to your fork
5. Submit a Pull request so that we can review your changes

NOTE: Be sure to merge the latest from "upstream" before making a pull request!

Changelog

View our closed Pull Requests.

License

Apache2 © Stakater

About

GitWebhookProxy is maintained by Stakater. Like it? Please let us know at [email protected]

See our other projects or contact us in case of professional services and queries on [email protected]

gitwebhookproxy's Issues

Other event types

I couldn't find in the documentation that this only works with Push and Pull events, that does seem to be the case. Any chance this could would with other event types?
We have a project using release, which the Jenkins Generic Webhook trigger can use, but GitWebhookProxy throws an error on that event type, since it seems to assume the GitHub trigger.

Add docs on how to Make GWP work

Add docs on how to Make GWP work i.e. the part after GWP is running, how to create a webhook on github and test it with GWP, add docs with screenshots

Proxy strips secret from webhook

I noticed that my secret is no longer valid after the webhook traverses the proxy. Would it be possible to preserve the secret so i can also validate with my listener running on internal kube ?

Can't handle team as a reviewer

When I assign a team as a reviewer in PR, GWP won't work properly.

Feature request: prometheus metrics

Would be nice if the service had some metrics.

Specify in ReadMe how to use jumbo manifest for deployment

Lets update README so, that user can deploy using the jumbo manifest; also clearly mention in the README that user must change these hardcoded values:

  rules:
  - host: gitwebhookproxy.tools.stackator.com

  tls:
  - hosts:
    - gitwebhookproxy.tools.stackator.com

upstreamURL: https://jenkins.tools.stackator.com

Create single manifest to deploy

Create a single manifest through the pipeline for deploying GWP easily through 1 yaml file.

Implement `Ignored Committers` functionality for Gitlab

We have implemented functionality to provide list of ignored committers for GitHub, it does not forward push event if the committer is in the list of ignored committers.

Similar implementation for Gitlab is also required.

Docker running but requests not passing

Hi guys,

I'm running the tool in Docker but I don't receive the requests into Jenkins, this is my docker command
docker run -p 80:8080 stakater/gitwebhookproxy:v0.2.70 -listen :80 -upstreamURL :8080 -provider gitlab -allowedPaths /project -secret supersecret
Note that I'm using -p to map the ports, otherwise the docker ps command shows no port open for this image

then I get this output from docker:
2019/11/09 16:55:39 Stakater Git WebHook Proxy started with provider 'gitlab' 2019/11/09 16:55:39 Listening at: :80

After that I sent a [POST] to :80/project/ that I assume should be logged into the stdout but nothing occurs

If I hit directly :8080/project/ I can run the build successfully

I'm running the docker image in Ubuntu 16.04.6 LTS
already tried v0.2.63 but same behaviour is observed

so my questions are: should I expect any http request logged in stdout? should I keep using -p argument?

Thanks in advance!

allowedusers logic should allow empty list

It seems that the proxy requires to whitelist all users:

GitWebhookProxy/pkg/proxy/proxy.go

Line 73 in 035d2b4

if len(p.allowedUsers) > 0 {

GitWebhookProxy/pkg/proxy/proxy.go

Line 145 in 035d2b4

if p.isIgnoredUser(committer) && (!p.isAllowedUser(committer)) {

Would it make more sense to the allow-check if arraysize > 0, else accept it? It seems a bit cumbersome to have to list each and every user.

Github payload unmarshaling failed for Push event

Using image v0.2.79.

When making a push on Github, the delivery has the green checkmark and "looks" successful, but in the background, the proxy is getting this log:
Github payload unmarshaling failed for Push event: invalid character 'p' looking for beginning of value

Content type is set to application/x-www-form-urlencoded
If I try json instead, it's returning log: Error Validating Hook: <nil>

Any idea on what could be causing this?

Reverse proxy for webhook for Jenkins and Bitbucket cloud

we are having Jenkins which is sitting behind firewall( private subnets) where out of the world cannot access we would like to implemnet reverse proxy webhooks from Bitbucket cloud but seems to be not found any documentation over here.

It would be nice if anyone can confirm if it also supports Bitbucket cloud too

[Feature Request] Standalone Docker Image

Hello, nice tool.

It would be great if you had a Docker image ready-to-go with your software. I currently run regular Docker not Kubernetes so it would be nice to have something to work with.

Possibility to whitelist webhooks ip addresses

In the case of GitHub, would it be possible to add an option to query GitHub api for IP ranges used by webhooks and only allow access from them? It seems like the most secure option.

Don't know what's the situation with GitLab, but maybe they have similar API.

Misc chart cleanups

1. should have a securityContext to run as non-priv. user + readonlyrootfs
2. ignoredUsers/allowedUsers are expected to be found in the configmap, but these are not set if values are ""

Upstream Redirect Status: 413 Request Entity Too Large

In cases where the commit is too big the error Upstream Redirect Status: 413 Request Entity Too Large throws in the GitWebhookProxy, is it possible to increase the payload size that the proxy can handle?

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Update alpine Docker tag to v3.20
• Update golang Docker tag to v1.22.6
• Update module github.com/jarcoal/httpmock to v1.3.1
• Update stakater/base-alpine Docker tag to v3.11
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

dockerfile

build/package/Dockerfile

• stakater/base-alpine 3.7

build/package/Dockerfile.build

• golang 1.13.1-alpine

build/package/Dockerfile.run

• alpine 3.9

gomod

go.mod

• go 1.13
• github.com/jarcoal/httpmock v1.0.4
• github.com/julienschmidt/httprouter v1.3.0
• github.com/namsral/flag v1.7.4-pre

---

• Check this box to trigger a request for Renovate to run again on this repository

Ingress parsing error when installing with helm

Followed the directions exactly how it instructs on the readme. Only changed the host, upstreamURL and allowedPaths as instructed.

Error: release gitwebhookproxy failed: Ingress in version "v1beta1" cannot be handled as a Ingress: v1beta1.Ingress.ObjectMeta: v1.ObjectMeta.Annotations: ReadString: expects " or n, but found t, error found in #10 byte of ...|edirect":true,"kuber|..., bigger context ...|ons":{"ingress.kubernetes.io/force-ssl-redirect":true,"kubernetes.io/ingress.class":"ingress-externa|...

New release

Last week you merged some fixes and improvements to the project. I was wondering if there's a release policy so I can have some expectations about when this new changes will be released and when the new artifacts will be pushed.

Thanks!

Create ingress by default

Add ingress to the charts and manifests, make it optional but default.
Also make the expose label optional

Can't handle if Github PR is labelled

2019/10/01 20:52:19 Received event type: pull_request 2019/10/01 20:52:19 Github payload unmarshaling failed for Pull Request event: json: cannot unmarshal object into Go struct field .labels of type string 2019/10/01 20:52:19 Incoming request from user: 2019/10/01 20:52:19 Ignoring request for user:

How can we whitelist the GitHub to the firewall so that ,the GitHub webbook trigger the pipeline running in the Jenkins?

Does not work with Jenkins Github Plugin

When attempting to override the Web hook url for the Github plugin a 400 response is returned. The gitwebhookproxy logs show that this is because of missing headers. The Github plugin is trying to validate the endpoint by sending a request with a test header: https://github.com/jenkinsci/github-plugin/blob/master/src/main/java/org/jenkinsci/plugins/github/config/GitHubPluginConfig.java#L214-L241

Mock request for tests

I used httpbin.org for a few test cases, The requests should be mocked instead of relying on an external service

unmarshaling failed for Pull Request event: invalid character 'p' looking for beginning of value

I'm running stakater/gitwebhookproxy:v0.2.75 with GWP_SECRET exposed as an environment variable and with this command passed into docker:

        "-allowedPaths", "/events",
        "-upstreamURL", "http://MY_UPSTREAM_HOSTNAME:8080",

When I trigger a webhook at GitHub, the hook isn't proxied as expected, with this error shown in the logs:

Github payload unmarshaling failed for Push event: invalid character 'p' looking for beginning of value

What is going wrong here?