GithubHelp home page GithubHelp logo

ghostunnel / ghostunnel Goto Github PK

View Code? Open in Web Editor NEW
1.8K 50.0 260.0 20.39 MB

A simple SSL/TLS proxy with mutual authentication for securing non-TLS services.

License: Apache License 2.0

Go 66.00% Python 33.03% Makefile 0.75% Dockerfile 0.16% Open Policy Agent 0.06%
tls tunnel ssl proxy stunnel go security crypto pkcs11 hsm

ghostunnel's Introduction

Ghostunnel

license release docker test coverage report

๐Ÿ‘ป

Ghostunnel is a simple TLS proxy with mutual authentication support for securing non-TLS backend applications.

Ghostunnel supports two modes, client mode and server mode. Ghostunnel in server mode runs in front of a backend server and accepts TLS-secured connections, which are then proxied to the (insecure) backend. A backend can be a TCP domain/port or a UNIX domain socket. Ghostunnel in client mode accepts (insecure) connections through a TCP or UNIX domain socket and proxies them to a TLS-secured service. In other words, ghostunnel is a replacement for stunnel.

Supported platforms: Ghostunnel is developed primarily for Linux and Darwin (macOS), although it should run on any UNIX system that exposes SO_REUSEPORT, including FreeBSD, OpenBSD and NetBSD. Ghostunnel also supports running on Windows, though with a reduced feature set.

See ghostunnel --help, ghostunnel server --help and ghostunnel client --help.

Features

Access control: Ghostunnel enforces mutual authentication by requiring a valid client certificate for all connections. Policies can enforce checks on the peer certificate in a connection, either via simple flags or declarative policies using Open Policy Agent. This is useful for restricting access to services that don't have native access control.

Certificate hotswapping: Ghostunnel can reload certificates at runtime without dropping existing connections. Certificates can be loaded from disk, the SPIFFE Workload API, or a PKCS#11 module. This allows short-lived certificates to be used with Ghostunnel as you can pick up new certificates transparently.

ACME Support: In server mode, Ghostunnel can optionally obtain and automatically renew a public TLS certificate via the ACME protocol, such as through Let's Encrypt. Note that this requires a valid FQDN accessible on the public internet for verification.

Monitoring and metrics: Ghostunnel has a built-in status feature that can be used to collect metrics and monitor a running instance. Metrics can be fed into Graphite or Prometheus to see number of open connections, rate of new connections, connection lifetimes, timeouts, and other info.

Emphasis on security: We have put some thought into making Ghostunnel secure by default and prevent accidental misconfiguration. For example, we always negotiate TLS v1.2 (or greater) and only use safe cipher suites. Ghostunnel also supports PKCS#11 which makes it possible to use Hardware Security Modules (HSMs) to protect private keys.

Getting Started

To get started and play around with the implementation, you will need to generate some test certificates. If you want to bootstrap a full PKI, one good way to get started is to use a package like square/certstrap. If you only need some test certificates for playing around with the tunnel, you can find some pre-generated ones in the test-keys directory (alongside instructions on how to generate new ones with OpenSSL).

Install

Ghostunnel is available through GitHub releases and through Docker Hub.

# Compile for local architecture
make ghostunnel

Note that ghostunnel requires Go 1.19 or later to build, and CGO is required for PKCS#11 support.

Develop

Ghostunnel has an extensive suite of integration tests. Our integration test suite requires Python 3.5 (or later) and gocovmerge to run. We use Go modules for managing vendored dependencies.

To run tests:

# Option 1: run unit & integration tests locally
make test

# Option 2: run unit & integration tests in a Docker container
# This also runs PKCS#11 integration tests using SoftHSM in the container
GO_VERSION=1.22 make docker-test

# Open coverage information in browser
go tool cover -html coverage/all.profile

For more information on how to contribute, please see the CONTRIBUTING file.

Usage

By default, ghostunnel runs in the foreground and logs to stderr. You can set --syslog to log to syslog instead of stderr. If you want to run ghostunnel in the background, we recommend using a service manager such as systemd or runit, or use a wrapper such as daemonize or dumb-init.

Certificates

Ghostunnel accepts certificates in multiple different file formats.

The --keystore flag can take a PKCS#12 keystore or a combined PEM file with the certificate chain and private key as input (format is auto-detected). The --cert / --key flags can be used to load a certificate chain and key from separate PEM files (instead of a combined one).

Ghostunnel also supports loading identities from the macOS keychain or the SPIFFE Workload API and having private keys backed by PKCS#11 modules, see the "Advanced Features" section below for more information.

Server mode

This is an example for how to launch ghostunnel in server mode, listening for incoming TLS connections on localhost:8443 and forwarding them to localhost:8080. Note that while we use TCP sockets on localhost in this example, both the listen and target flags can also accept paths to UNIX domain sockets as their argument.

To set allowed clients, you must specify at least one of --allow-all, --allow-cn, --allow-ou, --allow-dns or --allow-uri. All checks are made against the certificate of the client. Multiple flags are treated as a logical disjunction (OR), meaning clients can connect as long as any of the flags matches (see ACCESS-FLAGS for more information). In this example, we assume that the CN of the client cert we want to accept connections from is client.

Start a backend server:

nc -l localhost 8080

Start a ghostunnel in server mode to proxy connections:

ghostunnel server \
    --listen localhost:8443 \
    --target localhost:8080 \
    --keystore test-keys/server-keystore.p12 \
    --cacert test-keys/cacert.pem \
    --allow-cn client

Verify that clients can connect with their client certificate:

openssl s_client \
    -connect localhost:8443 \
    -cert test-keys/client-combined.pem \
    -key test-keys/client-combined.pem \
    -CAfile test-keys/cacert.pem

Now we have a TLS proxy running for our backend service. We terminate TLS in ghostunnel and forward the connections to the insecure backend.

Client mode

This is an example for how to launch ghostunnel in client mode, listening on localhost:8080 and proxying requests to a TLS server on localhost:8443.

Start a backend TLS server:

openssl s_server \
    -accept 8443 \
    -cert test-keys/server-combined.pem \
    -key test-keys/server-combined.pem \
    -CAfile test-keys/cacert.pem

Start a ghostunnel with a client certificate to forward connections:

ghostunnel client \
    --listen localhost:8080 \
    --target localhost:8443 \
    --keystore test-keys/client-combined.pem \
    --cacert test-keys/cacert.pem

Verify that we can connect to 8080:

nc -v localhost 8080

Now we have a TLS proxy running for our client. We take the insecure local connection, wrap them in TLS, and forward them to the secure backend.

Full tunnel (client plus server)

We can combine the above two examples to get a full tunnel. Note that you can start the ghostunnels in either order.

Start netcat on port 8001:

nc -l localhost 8001

Start the ghostunnel server:

ghostunnel server \
    --listen localhost:8002 \
    --target localhost:8001 \
    --keystore test-keys/server-combined.pem \
    --cacert test-keys/cacert.pem \
    --allow-cn client

Start the ghostunnel client:

ghostunnel client \
    --listen localhost:8003 \
    --target localhost:8002 \
    --keystore test-keys/client-keystore.p12 \
    --cacert test-keys/cacert.pem

Verify that we can connect to 8003:

nc -v localhost 8003

Now we have a full tunnel running. We take insecure client connections, forward them to the server side of the tunnel via TLS, and finally terminate and proxy the connection to the insecure backend.

Advanced Features

Access Control Flags

Ghostunnel supports different types of access control flags in both client and server modes. All checks are made against the certificate of the client or server. Multiple flags are treated as a logical disjunction (OR), meaning clients can connect as long as any of the flags matches. Ghostunnel is compatible with SPIFFE X.509 SVIDs.

Ghostunnel also has experimental support for Open Policy Agent (OPA) policies. Policies can be reloaded at runtime much like certificates.

See ACCESS-FLAGS for details.

Logging Options

You can silence specific types of log messages using the --quiet=... flag, such as --quiet=conns or --quiet=handshake-errs. You can pass this flag repeatedly if you want to silence multiple different kinds of log messages.

Supported values are:

  • all: silences all log messages
  • conns: silences log messages about new and closed connections.
  • conn-errs: silences log messages about connection errors encountered (post handshake).
  • handshake-errs: silences log messages about failed handshakes.

In particular we recommend setting --quiet=handshake-errs if you are running TCP health checks in Kubernetes on the listening port, and you want to avoid seeing error messages from aborted connections on each health check.

Certificate Hotswapping

To trigger a reload, simply send SIGHUP to the process or set a time-based reloading interval with the --timed-reload flag. This will cause ghostunnel to reload the certificate and private key from the files on disk. Once successful, the reloaded certificate will be used for new connections going forward.

Additionally, ghostunnel uses SO_REUSEPORT to bind the listening socket on platforms where it is supported (Linux, Apple macOS, FreeBSD, NetBSD, OpenBSD and DragonflyBSD). This means a new ghostunnel can be started on the same host/port before the old one is terminated, to minimize dropped connections (or avoid them entirely depending on how the OS implements the SO_REUSEPORT feature).

Note that if you are using an HSM/PKCS#11 module, only the certificate will be reloaded. It is assumed that the private key in the HSM remains the same. This means the updated/reissued certificate much match the private key that was loaded from the HSM previously, everything else works the same.

ACME Support

To have Ghostunnel automatically obtain and renew a public TLS certificate via ACME, use the --auto-acme-cert= flag (e.g. - --auto-acme-cert=myservice.example.com). You must also specify an email address so you will get notices from the CA about potentially important certificate lifecycle events. Specify the email address with the --auto-acme-email= flag. To use this feature, you must also specify the --auto-acme-agree-to-tos flag to indicate your explicit agreement with the CA's Terms of Service.

Ghostunnel defaults to using Let's Encrypt, but you can specify a different ACME CA URL using the --auto-acme-ca= flag. If you wish to test Ghostunnel's ACME features against a non-production ACME CA, use the --auto-acme-testca= flag. If --auto-acme-testca is specified, all ACME interaction will be with the specified test CA URL and the --auto-acme-ca= flag will be ignored.

ACME is only supported in server mode. Additionally, Ghostunnel must either be listening to a public interface on tcp/443, or somehow have a public tcp/443 listening interface forwarded to it (e.g. - systemd socket, iptables, etc.). Public DNS records must exist for a valid public DNS FQDN that resolves to the public listening interface IP.

Metrics & Profiling

Ghostunnel has a notion of "status port", a TCP port (or UNIX socket) that can be used to expose status and metrics information over HTTPS. The status port feature can be controlled via the --status flag. Profiling endpoints on the status port can be enabled with --enable-pprof.

See METRICS for details.

HSM/PKCS#11 support

Ghostunnel has support for loading private keys from PKCS#11 modules, which should work with any hardware security module that exposes a PKCS#11 interface.

See HSM-PKCS11 for details.

SPIFFE Workload API

Ghostunnel has support for maintaining up-to-date, frequently rotated identities and trusted CA certificates from the SPIFFE Workload API.

See SPIFFE-WORKLOAD-API for details.

Socket Activation (experimental)

Ghostunnel supports socket activation via both systemd (on Linux) and launchd (on macOS). Socket activation is support for the --listen and --status flags, and can be used by passing an address of the form systemd:<name> or launchd:<name>, where <name> should be the name of the socket as defined in your systemd/launchd configuration.

See SOCKET-ACTIVATION for examples.

PROXY Protocol (experimental)

Ghostunnel in server mode supports signalling of transport connection information to the backend using the PROXY protocol (v2), just pass the --proxy-protocol flag on startup. Note that the backend must also support the PROXY protocol and must be configured to use it when setting this option.

MacOS Keychain Support (experimental)

Ghostunnel supports loading certificates from the macOS keychain. This is useful if you have identities stored in your local keychain that you want to use with ghostunnel, e.g. if you want your private key(s) to be backed by the SEP on newer Touch ID MacBooks. Certificates from the keychain can be loaded by selecting them based on the serial number, Common Name (CN) of the subject, or Common Name (CN) of the issuer.

For example, to load an identity based on subject name login keychain:

ghostunnel client \
    --keychain-identity <common-name-or-serial> \
    --listen unix:/path/to/unix/socket \
    --target example.com:443 \
    --cacert test-keys/cacert.pem

Or, if you'd like to load an identity by filtering on issuer name:

ghostunnel client \
    --keychain-issuer <issuer-common-name> \
    --listen unix:/path/to/unix/socket \
    --target example.com:443 \
    --cacert test-keys/cacert.pem

Both commands above launch a ghostunnel instance that uses the certificate and private key for the selected keychain identity to proxy plaintext connections from a given UNIX socket to example.com:443. Note that combining both the identity and issuer flags in one command will cause ghostunnel to select certificates where both attributes match (matching with AND on both subject name/issuer).

ghostunnel's People

Contributors

007 avatar alokmenghrajani avatar amartinezfayo avatar azdagron avatar cablespaghetti avatar csstaub avatar dependabot-preview[bot] avatar dependabot[bot] avatar donbowman avatar elbandi avatar ewdurbin avatar falco20019 avatar gintrator avatar jsoref avatar levigross avatar marcosdy avatar mblaschke avatar mbyczkowski avatar mccurdyc avatar mcpherrinm avatar mweissbacher avatar phamann avatar romain-geissler-1a avatar ryankoski avatar sandersch avatar sgorbaty avatar shawnps avatar stouset avatar testwill avatar wuvist avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ghostunnel's Issues

Problem while building ghostunnel

Hi, I am trying to install ghostunnel on Windows 7. Can you please help me resolve this error? I am new to golang and doesnt understand what exactly it means?

C:\ghostunnel>go install github.com/square/ghostunnel
github.com/square/ghostunnel/vendor/github.com/kavu/go_reuseport
src\github.com\square\ghostunnel\vendor\github.com\kavu\go_reuseport\reuseport.go:25: undefined: maxListenerBacklog
src\github.com\square\ghostunnel\vendor\github.com\kavu\go_reuseport\reuseport.go:68: cannot assign syscall.Handle to fd (type int) in multiple assignment
src\github.com\square\ghostunnel\vendor\github.com\kavu\go_reuseport\reuseport.go:74: cannot use fd (type int) as type syscall.Handle in argument to syscall.Close
src\github.com\square\ghostunnel\vendor\github.com\kavu\go_reuseport\reuseport.go:78: cannot use fd (type int) as type syscall.Handle in argument to syscall.SetsockoptInt
src\github.com\square\ghostunnel\vendor\github.com\kavu\go_reuseport\reuseport.go:82: undefined: reusePort
src\github.com\square\ghostunnel\vendor\github.com\kavu\go_reuseport\reuseport.go:82: cannot use fd (type int) as type syscall.Handle in argument to syscall.SetsockoptInt
src\github.com\square\ghostunnel\vendor\github.com\kavu\go_reuseport\reuseport.go:86: cannot use fd (type int) as type syscall.Handle in argument to syscall.Bind
src\github.com\square\ghostunnel\vendor\github.com\kavu\go_reuseport\reuseport.go:91: cannot use fd (type int) as type syscall.Handle in argument to syscall.Listen

Cheers!
Rohit Goyal

Move to prometheus metrics

Currently you have a custom JSON metrics format that is exposed on /_metrics. Is there any plan or support for moving to prometheus.io exposition format? This would make it simple to migrate to OpenMetrics which is going to supported by all monitoring vendors.

If there is agreement to move to prometheus metrics format, I'll go ahead and make the changes.

Tunnel with no client certificate?

Doing:
ghostunnel client --listen 0.0.0.0:27017 --target [redacted]:27017 --status 0.0.0.0:8080
Prints:
ghostunnel: error: required flag --keystore not provided, try --help

I understand from this that a valid cert is mandatory for client mode, but I don't want to auth client. Just want to use ghostunnel as a TLS tunnel as in stunnel, but with the added metrics and status features.
Is it possible? Am I doing anything wrong?
TIA.

CA bundle isn't updated by signal

I am trying to integrate Spire and ghostunnel.
ghostunnel is run in the following configuration:

ghostunnel server --listen 127.0.0.1:9998 --target localhost:9987 --key svid_key.pem --cert svid.pem --cacert svid_bundle.pem --allow-uri-san spiffe://example.org/host1/workload

svid_key.pem, svid.pem and svid_bundle.pem were retrieved via Spire Agent Workload API.

After svid rotation is complete we send SIGUSR1 signal. the Ghostunnel updates the key and certificate (log output: received user defined signal 1, reloading certificates; reloading complete), but does not update the CA bundle. When the CA bundle expires, the tunnel stops accepting new connections with error: tls: failed to verify client's certificate: x509: certificate signed by unknown authority

Is it possible to preserve request originator IP/Hostname ?

we have enabled Redis Master/Slave replication over TLS through Ghosttunel full tunnel, but now master has no knowledge of real slaves connected to it through ghosttunnel (as master think ghosttunnel server listening address as the real redis slave ).

Another much bigger problem with above is sentinel fail-over, due to above incorrect real redis slave ip.

So I wonder if there is a way to preserve the request originator IP, in this case the actual redis slave ip while proxying.

Thanks!

Document build procedure for releases

Love the project! I plan to become a fairly regular user (and hopefully contributor) and would like to be able to create my own internal releases for changes that have yet to be merged in upstream.

Ad hoc creation of releases will probably be fine (my golang chops are young), but will almost certainly lead to a pitfall at some point.

Automating the builds and any necessary cross-compilation and having that be part of the source repository itself would be hugely beneficial.

For some prior art, https://github.com/hashicorp/envconsul/blob/b44605bdf87c63f40be197303545daacd746d780/Makefile#L55 is a really slick example I recently ran across.

TLS decryption error with PKCS11

Hello there,
I'm trying to use ghosttunnel to enable access to a server which is protected with TLS client authentication. Ghosttunnel works successfully when using software certificates. When using hardware tokens with PKCS11 I get the following error message:

error on dial: remote error: tls: error decrypting message

Watching the traffic with wireshark I can see that the TLS buildup is successful until my client sends client certificate and key exchange data, also OpenSC as a pkcs11 module is not having any issues, but then the server responds with an fatal decryption alert.

My commandline:

ghostunnel-v1.5.0-rc.2-windows-amd64-with-pkcs11.exe client --listen=localhos
t:8001 --target=xxx.de:443 --cert=cert_ecc.pem --pkcs11-module="C:\Program File
s\OpenSC Project\OpenSC\pkcs11\opensc-pkcs11.dll" --pkcs11-token-label=mylabel --cacert=ca.crt
[7116] 2019/10/01 11:57:59.745400 starting ghostunnel in client mode
[7116] 2019/10/01 11:58:16.703370 using target address partner.bdr.de:443
[7116] 2019/10/01 11:58:16.704370 listening for connections on localhost:8001
[7116] 2019/10/01 11:58:38.790633 error on dial: remote error: tls: error decrypting message

Note that my card works without giving pkcs11-pin directly, so thats not an issue.
Do you have any advice?

Make official Docker image build from source

Docker Hub allows you to create Automated Builds from source: https://docs.docker.com/docker-hub/builds/
It also allows to create different image tags from git tags & branches.
By making the image build via an AB, you give the resulting image verifiability and auditability. Also, the build is fully automatic. You can have the latest image tag build from HEAD and individual image tags from git's release tags.
Some people avoid non-verifiable (manually uploaded) images due to security & traceability reasons.

Docker search command clearly displays AB:

$ docker search ghostunnel
NAME                   DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
squareup/ghostunnel    A simple SSL/TLS proxy with mutual authent...   1                    
crewjam/ghostunnel                                                     0                    
pataquets/ghostunnel   ghostunnel                                      0                    [OK]
shkrid/ghostunnel                                                      0                    
apiaryio/redis-tls     Built from https://github.com/square/ghost...   0                    

error: unable to build TLS config

I got error message on Windows 10.

ghostunnel-v1.2.1-windows-amd64-with-pkcs11.exe client --listen localhost:8080 --target example.com:443 --keystore client-keystore.p12
[119128] 2018/09/05 16:15:34.196134 starting ghostunnel in client mode
error: unable to build TLS config: crypto/x509: system root pool is not available on Windows

It works well on Ubuntu.

Missing/Incorrect Host header causes servers with SNI to not work

When using ghostunnel in client mode, with the listener on my localhost, The GET requests which have an incorrect host header does not qualify for a correct GET request for the server and thus responding with an unknown server.

Ghostunnel Run:

$ ./ghostunnel client --listen localhost:9999 --target www.reddit.com:443 --disable-authentication
[33791] 2018/07/21 03:20:01.247935 starting ghostunnel in client mode
[33791] 2018/07/21 03:20:03.564880 using target address www.reddit.com:443
[33791] 2018/07/21 03:20:03.718785 listening for connections on localhost:9999
[33791] 2018/07/21 03:20:19.564512 opening pipe: tcp:127.0.0.1:54017 <-> tcp:151.101.9.140:443
[33791] 2018/07/21 03:20:19.658300 closed pipe: tcp:127.0.0.1:54017 <-> tcp:151.101.9.140:443
[33791] 2018/07/21 03:20:19.658310 error: readfrom tcp 127.0.0.1:9999->127.0.0.1:54017: read tcp 192.168.1.5:54019->151.101.9.140:443: use of closed network connection
[33791] 2018/07/21 03:20:35.707498 opening pipe: tcp:127.0.0.1:54021 <-> tcp:151.101.9.140:443
[33791] 2018/07/21 03:20:36.779824 error: readfrom tcp 127.0.0.1:9999->127.0.0.1:54021: read tcp 192.168.1.5:54022->151.101.9.140:443: use of closed network connection
[33791] 2018/07/21 03:20:36.779875 closed pipe: tcp:127.0.0.1:54021 <-> tcp:151.101.9.140:443

Curl request without host header

$ curl  -I http://localhost:9999
HTTP/1.1 500 Domain Not Found
Server: Varnish
Retry-After: 0
content-type: text/html
Cache-Control: private, no-cache
connection: keep-alive
X-Served-By: cache-sin18020-SIN
Content-Length: 239
Accept-Ranges: bytes
Date: Fri, 20 Jul 2018 21:50:19 GMT
Via: 1.1 varnish
Connection: keep-alive

Curl request with correct host header

$ curl  -I http://localhost:9999 -H "Host: www.reddit.com"
HTTP/1.1 429 Too Many Requests
Content-Type: text/html; charset=UTF-8
Content-Length: 1064
x-ua-compatible: IE=edge
retry-after: 4
x-frame-options: SAMEORIGIN
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
set-cookie: loid=00000000001ticaj40.2.1532123436599.Z0FBQUFBQmJVbGtzRFpkRjdjaUJRN2hrTFdQMUg5cGxETEtLWmRULV9DU04ydFI4Q1c2c21TQ0x2WFpKbUNBMnNMQWMzSy13TmtCd1VvM3h2NjQ5VFBhR010b0lWV0VyN3hfSkZ5NXFjWnlXaTVtR29YVGtDZWRSZ0pBRU1iYkkxZlhhRjdEUG5jMlA; Domain=reddit.com; Max-Age=63071999; Path=/; expires=Sun, 19-Jul-2020 21:50:36 GMT; secure
set-cookie: session_tracker=xW4srEaStXvtU1gSKj.0.1532123436595.Z0FBQUFBQmJVbGtzRnpJS1FPZVR2NXFxcmN5eEhSS1hsWmY4Yzk5cEIxTWhKX1FKOVRsRE9TNXNhTXcwOGtSWm4xWWRiTHF3dFdYeFdtRzZfME1DR0F5MDd2NzZWS1pXV2lJSWR6RHZrdjM2SkhYUC1MYUQyZ1N4akZHN1N0VmFSQ2dxUGhpTjl5VUE; Domain=reddit.com; Max-Age=7199; Path=/; expires=Fri, 20-Jul-2018 23:50:36 GMT; secure
cache-control: max-age=0, must-revalidate
X-Moose: majestic
Accept-Ranges: bytes
Date: Fri, 20 Jul 2018 21:50:36 GMT
Via: 1.1 varnish
Connection: keep-alive
X-Served-By: cache-sin18027-SIN
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1532123436.723003,VS0,VE1003
Vary: accept-encoding
Set-Cookie: edgebucket=NXyfd0AZHpO55jRLOx; Domain=reddit.com; Max-Age=63071999; Path=/;  secure
Server: snooserv

btw: even reddit.com sends 301 redirects to www.reddit.com

Support /_status checks

For example, passing in a flag such as --status=ADDR should start an HTTP server that listens on ADDR and serves requests to /_status with health information and other info. This would allow deployment systems like https://github.com/square/p2 to check for health on deployments.

Support for sslv3

Folks,

I have been working on a POC that is using ghostunnel in server mode, I have it in front a plain service port 2003 and ghostunnel is serving port 3003, however I have an issue where there is an error message in ghostunnel:

[18445] 2019/09/04 11:00:33.552639 error on TLS handshake from 10.X.X.X:44518: tls: client didn't provide a certificate

In the other application logs I found:

[2019-09-04 11:00:28] (ERR) failed to connect ssl stream: error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate

What I would like to confirm is that if it might be possible the second application is indeed trying to connect with sslv3 and if this is supported by ghostunnel? if not how could I enable it? I have reasons not to worry about using sslv3.

Please advice.

CRL file support

Any plans for revoked certificates (CRL) file support for server?

SIGSEV when GetClientCertificate is getting used in client mode

Command line: ./ghostunnel client --listen=127.0.0.1:1234 --target=xxxx.redis.cache.windows.net:6380 --disable-authentication --unsafe-listen

output (build locally):

[83366] 2019/07/17 20:59:01.889015 starting ghostunnel in client mode
[83366] 2019/07/17 20:59:02.174250 using target address xxxx.redis.cache.windows.net:6380
[83366] 2019/07/17 20:59:02.174539 listening for connections on 127.0.0.1:1234
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x42a3287]

goroutine 61 [running]:
crypto/tls.(*clientHandshakeState).doFullHandshake(0xc0004e3de8, 0xc0000d8f60, 0x55)
	/usr/local/Cellar/go/1.12.6/libexec/src/crypto/tls/handshake_client.go:546 +0x11f7
crypto/tls.(*clientHandshakeState).handshake(0xc0004e3de8, 0xc00010f440, 0x0)
	/usr/local/Cellar/go/1.12.6/libexec/src/crypto/tls/handshake_client.go:399 +0x3fb
crypto/tls.(*Conn).clientHandshake(0xc0001cc000, 0x0, 0x0)
	/usr/local/Cellar/go/1.12.6/libexec/src/crypto/tls/handshake_client.go:208 +0x2cd
crypto/tls.(*Conn).Handshake(0xc0001cc000, 0x0, 0x0)
	/usr/local/Cellar/go/1.12.6/libexec/src/crypto/tls/conn.go:1343 +0xef
github.com/square/ghostunnel/certloader.dialWithDialer.func2(0xc0000c06c0, 0xc0001cc000)
	/Users/mblaschke/Projects/ghostunnel/certloader/dialer.go:71 +0x2b
created by github.com/square/ghostunnel/certloader.dialWithDialer
	/Users/mblaschke/Projects/ghostunnel/certloader/dialer.go:70 +0x197

not working:

func (c *certTLSConfig) GetClientConfig() *tls.Config {
	config := c.base.Clone()
	config.GetClientCertificate = c.cert.GetClientCertificate
	config.RootCAs = c.cert.GetTrustStore()
	return config
}

working:

func (c *certTLSConfig) GetClientConfig() *tls.Config {
	config := c.base.Clone()
	//config.GetClientCertificate = c.cert.GetClientCertificate
	config.RootCAs = c.cert.GetTrustStore()
	return config
}

Still trying to search a solution but not sure if i can make a PR for this issue

Skipping server certificate verification

Hi!
Thank you so much for sharing ghostunnel, it's truly awesome!
I'm using it for dev purposes and it would be very useful if it had an option to skip certificate verification.

Cheers

Ever consider SOCKS support?

This is awesome for automated systems, or systems which are relatively static, but for those which are neither (a laptop, for instance), SOCKS support for client mode would be pretty cool. Is this something you might see as being included in the future?

Configuration should be able to come from a file, environment variable, or command line argument

In #249 / #250 we added support for environment variables for some configurables.

However, it might be nice to have a consistent and generalized way to configure Ghostunnel:

  1. Defaults from a config file (yaml or json, maybe)
  2. Override with Environment Variables
  3. Override with Command line flags

It might be easier to do this if we move from github.com/alecthomas/kingpin to github.com/alecthomas/kong, as it has some support for the "defaults in a config file".

Feature: Reload ACL flags upon receiving a signal

๐Ÿ‘‹ great job here!
I'm interested in being able to reload ACLs without losing ongoing connections.
Given that this issue is implemented, after receiving a sighup, ghostunnel would reload the ACLs so for example, one can add/remove an organization without the need to change the command line or interrupt the current connections.

  1. Would you accept a PR for this feature?
  2. Any implementation guidelines?

Thanks

Build fails without libltdl-dev installed

david.amick@machine:~$ go get github.com/square/ghostunnel
# github.com/square/ghostunnel/vendor/github.com/miekg/pkcs11
go/src/github.com/square/ghostunnel/vendor/github.com/miekg/pkcs11/pkcs11.go:28:10: fatal error: ltdl.h: No such file or directory
 #include <ltdl.h>
          ^~~~~~~~
compilation terminated.
david.amick@machine:~$

It works after sudo apt-get install libltdl-dev.

Error happened on redis disconnection

[35] 2019/04/09 18:29:49.102821 opening pipe: tcp:172.17.0.1:32846 <-> tcp:192.168.1.109:6379
[35] 2019/04/09 18:29:49.123198 error: read tcp 172.17.0.3:47390->192.168.1.109:6379: use of closed network connection // at this place redis client disconnects
[35] 2019/04/09 18:29:49.126705 closed pipe: tcp:172.17.0.1:32846 <-> tcp:192.168.1.109:6379

Ghostunnel not responding correctly to 301 redirects

I am using the client mode --listening to localhost:8000 and --targeting to some server for eg. quora.com:443.
When I send a GET request to my localhost:8000, the target host sends a HTTP 301 Moved permanently response code, to which the browser responds by making a GET connection to the Location: https://www.quora.com header of the response and because of this, the browser moves away from the tunnel and actually makes the connection.

$ ghostunnel client --listen localhost:9999 --target quora.com:443 --disable-authentication
[33476] 2018/07/21 03:11:27.322518 starting ghostunnel in client mode
[33476] 2018/07/21 03:11:29.628343 using target address quora.com:443
[33476] 2018/07/21 03:11:29.782684 listening for connections on localhost:9999
[33476] 2018/07/21 03:12:09.181264 opening pipe: tcp:127.0.0.1:53910 <-> tcp:34.235.56.176:443
[33476] 2018/07/21 03:12:09.492033 error: readfrom tcp 127.0.0.1:9999->127.0.0.1:53910: read tcp 192.168.1.5:53911->34.235.56.176:443: use of closed network connection
[33476] 2018/07/21 03:12:09.492048 closed pipe: tcp:127.0.0.1:53910 <-> tcp:34.235.56.176:443
$ curl  -I http://localhost:9999
HTTP/1.1 301 Moved Permanently
Content-Length: 178
Content-Type: text/html
Date: Fri, 20 Jul 2018 21:42:09 GMT
Location: https://www.quora.com/
Server: nginx
Connection: keep-alive

Missing release sha256 sums

Hi all, I noticed this project does not provide the sha256 sums of the released packages. Is that intentional?

I see other projects such as i.e. prometheus/node_exporter provide them as part of each release.

Thanks in advance.

Ghostunnel Performance Benchmark

Hello There,
This is an informational/clarification question rather than an issue! I have been using Ghostunnel as a TLS proxy for Redis backend and noticed that its adding 2X latency ( direct GET takes about 0.25 millisecond but through TLS proxy it takes about 0.5millisecond with the steady load of 2K request per second ) , so i wonder if you have ever done any benchmarking on additional TLS overhead that TLS proxy adds and willing to share?

Thanks in advance
Satheesha Gowda

HTTP performance best practice

Hi,

this is more of a general advice question. We are using ghostunnel to secure http requests and our database connections, for the http requests a new TCP connection is made each time causing high cpu usage inside the ghostunnel container. Is there any advice or options in ghostunnel we can use to keep a tcp connection open between the client and server processes?

When proxying the http requests we get a lot of the following errors in the log line which may be relivant:


[1] 2019/01/14 14:45:55.644754 closed pipe: tcp:172.24.3.1:48610 <-> tcp:172.28.19.175:3005
[1] 2019/01/14 14:45:55.650713 opening pipe: tcp:10.101.0.7:48614 <-> tcp:172.28.19.175:3005
[1] 2019/01/14 14:45:55.654806 error: readfrom tcp 172.24.3.51:34632->172.28.19.175:3005: read tcp 172.24.3.51:443->172.24.3.1:55284: use of closed network connection

Again thanks for this tool, it is proving very useful.

Support PEM files for certificate/key

Ghostunnel currently only supports reading PKCS12 files for the certificate and private key. We should add support for plain PEM files as well. Due to how the reloading works, it's beneficial if everything is in one file so we'll likely want to have a PEM file with the certificate and key in the same file. Could use certigo's lib package to read files which will give us free support for JCEKS/PKCS8 as well.

Allow using PKCS#11 module in client mode (for TLS client authentication)

The documentation has details about PKCS#11 support in server mode. However, I'd like to use it in client mode, so that ghostunnel can get connect to a server that requests TLS client authentication whereby key/certificate are stored in a PKCS#11 token.

I'm using ghostunnel as follows:

./ghostunnel-v1.2.1-linux-amd64-with-pkcs11 client \
    --listen=localhost:8080 \
    --target=$SERVER_REQUESTING_CLIENT_CERT:443 --disable-authentication \
    --pkcs11-module=$MODULE_WITH_CLIENT_CERT_AND_KEY

I'm receiving the following error:

[2181] 2018/07/05 19:20:26.863982 remote error: tls: handshake failure

Disable authentication not working as desired

I have discovered an issue in which --disable-authentication isn't mutually exclusive to other flags, or rather that it doesn't fully disable auth if a server receives mixed requests with, and without, client certs.

I have an insecure application running in Google Kubernetes Engine which I am fronting with ghostunnel to terminate TLS in the same pod. We are also using a Google layer 7 HTTP(s) load balancer via an Ingress controller to expose the pod to the internet.

Ideally we would want mutual TLS between the load balancer and ghostunnel, however a limitation of Google's load balancers is they don't currently support client certs. Therefore, we have had to disable authentication using the --disable-authentication flag.

Sadly, I've observed that the health checking requests from the load balancers will sometime present a client cert and sometimes not. Such as (IP's redacted):

[1] 2019/04/18 11:45:57.001862 closed pipe: tcp::49012 <-> tcp:[::1]:8080
[1] 2019/04/18 11:46:04.381232 opening pipe: tcp::51379 <-> tcp:[::1]:8080
[1] 2019/04/18 11:46:06.079706 opening pipe: tcp::52077 <-> tcp:[::1]:8080
[1] 2019/04/18 11:46:06.752197 opening pipe: tcp::61236 <-> tcp:[::1]:8080
[1] 2019/04/18 11:46:41.782057 error on TLS handshake from :62194: unauthorized: invalid principal, or principal not allowed
[1] 2019/04/18 11:46:41.912059 error on TLS handshake from :39372: unauthorized: invalid principal, or principal not allowed
[1] 2019/04/18 11:46:43.794047 error on TLS handshake from :41194: unauthorized: invalid principal, or principal not allowed
[1] 2019/04/18 11:46:45.614435 error on TLS handshake from :43302: unauthorized: invalid principal, or principal not allowed

When they do, ghostunnel returns the error unauthorized: invalid principal, or principal not allowed from auth.go, even though authentication should be disabled.
https://github.com/square/ghostunnel/blob/95a12657b2039ce10e4a1ff2c3fea6d0335dc2e1/auth/auth.go#L68-L70

This is because even when the flag is set you still configure the VerifyPeerCertificate function in the server TLS config, which throws this error.
https://github.com/square/ghostunnel/blob/95a12657b2039ce10e4a1ff2c3fea6d0335dc2e1/main.go#L445-L448

I have be able to fix the issue in a fork, by either not setting VerifyPeerCertificate if the disable-authentication flag is set, or allowing allow-all and disable to be set together which you currently guard against: https://github.com/square/ghostunnel/blob/95a12657b2039ce10e4a1ff2c3fea6d0335dc2e1/main.go#L253-L255

Please let me know if you have any further ideas and if you'd like me to submit a PR for either or the above solutions.

error on dial: x509: certificate signed by unknown authority

Hello folks,
We need this framework to wrap a graphite POC that we are doing, works fine however the tunnel is failing with this error:

Showed where the ghostunnel client is running
error on dial: x509: certificate signed by unknown authority

Showed where the ghostunnel server is running
error on TLS handshake from 10.209.114.98:39980: remote error: tls: bad certificate

I get the issue is likely related to the certificate, I have tried self signed and a .cer generated by a known authority, but I am missing something, if you guys could put me in the right direction, I would appreciate it.

Support separated certificate/key files in client mode

As per documentation at https://github.com/square/ghostunnel#certificates, ghostunnel supports loading certificate and key from separated files.

I do not know if I read it incorrectly, but the impression was that this feature was independent of the working mode (server/client). Instead, it only works in server mode, while client mode requires a bundled PEM file via the --keystore param.

It looks like #208 introduced support only for server mode, since clientValidateFlags does not support the cert/key params.
IMHO the same rationale for supporting the feature server side should be valid for client side.

If I read the code correctly, it suffices to extend the clientValidateFlags function to include the same combinations supported for server mode, and then the buildCertificate would take care of the actual loading.

ghostunnel multi-connect.

Add a new ghostunnel command to read client/servers from a file, and allow multiples of each.
This will allow easier connections for services which need multiple tunnels from the client (eg, when using sharded redis).

It also enables some other usages, like bidirectional tunnels, or meshes, which may prove useful in redis replication setups.

stunnel supports this, so this will unblock moving some remaining stunnels to ghostunnel

_status returns 503 when in server mode

This may be a bad config on my behalf, _status works fine in client mode, but in server I get 503 Service Unavailable.

my docker file uses the following command server --listen 0.0.0.0:27018 --target localhost:27017 --keystore /var/keys/outputCert.p12 --cacert /var/keys/CA.crt --allow-all --status localhost:6060

curl --fail --cacert /var/keys/CA.crt https://localhost:6060/_status

I've got around the problem by just using /_metrics for my health checks which works just fine.

tls: unknown certificate authority

I started a tunnel with the following settings:

ghostunnel server --listen 10..X.X.X:3003 --target 10.X.X.X:2003  --disable-authentication --keystore /etc/ssl/certs/serverPOC.pem --unsafe-target

However I am seeing this error in the server logs:
error on TLS handshake from 10.X.X.X:44326: remote error: tls: unknown certificate authority

Am I missing something obvious here, I thought that disable-authentication was to skip validation of certs,?

Also is this expected?

ghostunnel server --listen 10.X.X.X:3003 --target 10.X.X.X:2003  --disable-authentication  --unsafe-target
[28564] 2019/09/04 23:00:20.692038 starting ghostunnel in server mode
[28564] 2019/09/04 23:00:20.761008 error: at least one of --keystore, --cert/--key or --keychain-identity (if supported) flags is required

Thanks in advance.

binary for X86

Hello sir there is no binary for X86 linux can you add them please

Accept Keystore password from file or environment variable

Hello There,
Our Infosec team is not happy about passing keystore password as a CLI argument --storepass=PASS as its visible to everybody on executing command ps, so I wonder if --storepass can accept a password in a file --storepass=PATH or environment variable--storepass=$PATH .

  --keystore=PATH               Path to keystore (combined PEM with cert/key, or PKCS12 keystore).
  --cert=PATH                   Path to certificate (PEM with certificate chain).
  --key=PATH                    Path to certificate private key (PEM with private key).
  --storepass=PASS              Password for keystore (if using PKCS keystore, optional).

tls: handshake failure because of too strict CurvePreferences

The CurvePreferences for tls.Config is currently to strict which doesn't allow a TLS connection to some cloud based services.

Please allow custom setting or go back to the default value please.

As these are cloud services (Azure) we cannot change the tls server settings there so we have to live with that.
If i remove that setting the connection is working pretty fine.

Also some dovecot (IMAP/POP3) servers are affected.

slice bounds out of range on `--status localhost:6060`

In the 1.3.0 docker image --status localhost:6060 results in a panic.

The docker command I run is:

server --listen 0.0.0.0:27018 --target localhost:27017 --keystore /var/keys/outputCert.p12 --cacert /var/keys/CA.crt --allow-all --status localhost:6060

 panic: runtime error: slice bounds out of range

goroutine 1 [running]:
github.com/square/ghostunnel/vendor/github.com/kavu/go_reuseport.getTCPSockaddr(0x9b5d5f, 0x3, 0x7ffe70ffd17c, 0xe, 0x9b5d5f, 0x3, 0x7ffe70ffd17c, 0xe, 0xa4af80)
	/go/src/github.com/square/ghostunnel/vendor/github.com/kavu/go_reuseport/tcp.go:43 +0x364
github.com/square/ghostunnel/vendor/github.com/kavu/go_reuseport.getSockaddr(0x9b5d5f, 0x3, 0x7ffe70ffd17c, 0xe, 0xc0001f9a10, 0x8911e3, 0x9b5d5f, 0x3, 0x7ffe70ffd17c)
	/go/src/github.com/square/ghostunnel/vendor/github.com/kavu/go_reuseport/reuseport.go:30 +0x72
github.com/square/ghostunnel/vendor/github.com/kavu/go_reuseport.NewReusablePortListener(0x9b5d5f, 0x3, 0x7ffe70ffd17c, 0xe, 0x7ffe70ffd17c, 0xe, 0x7ffe70ffd17c, 0x9)
	/go/src/github.com/square/ghostunnel/vendor/github.com/kavu/go_reuseport/tcp.go:99 +0x67
main.(*Context).serveStatus(0xc000068100, 0xc00000c800, 0x2540be400)
	/go/src/github.com/square/ghostunnel/main.go:547 +0x228
main.serverListen(0xc000068100, 0x9de010, 0xc000068100)
	/go/src/github.com/square/ghostunnel/main.go:440 +0x703
main.run(0xc0000a0010, 0xd, 0xd, 0xc0001a8a50, 0xc000076058)
	/go/src/github.com/square/ghostunnel/main.go:352 +0x1348
main.main()
	/go/src/github.com/square/ghostunnel/main.go:272 +0x66

Also thank you for your work on this tool.

Using Ghostunnel to assist authorization

We'd like to use Ghostunnel not just to authenticate incoming TLS requests, but then pass some information to the HTTP service about the requesting party, so we can use role based authorization inside our service. I may take a crack at adding some custom HTTP headers with TLS certificate details (like the CN, O, OU fields), but admittedly I'd be slower.

Is this a feature that makes sense to have in the (near) future?

Support client mode

Support a client mode instead of just a server mode, i.e. a mode where TLS is spoken on the outgoing socket instead of the incoming socket.

Support metrics

Support collecting metrics via rcrowley/go-metrics. Should support metrics on connection count (new connection counter) at a minimum.

--syslog doesn't seem to work

I used the --syslog option and still the output went to stderr and nothing in /var/log/syslog.
Ubuntu 17.04 built with go1.9.1

Add support for the SPIFFE workload api

@mcpherrinm mentioned to me that SPIFFE workload api support in ghostunnel would be a welcome contribution. As a user of SPIFFE, native support for the workload api would be great as it would negate the need for me to run a shim (or maintain a fork, which I have been admittedly bad about).

I have been reviewing the codebase, and have a couple questions/comments that I wanted to raise before getting too far down the implementation path. They are:

  • CA certificates are provided directly by the workload api. Ghostunnel isn't set up well for dynamic CA certificates. Non-trivial changes would be required, including changing the TLS config hook from GetCertificate to GetConfigForClient. Is this something you're willing to support? Workload API support in this project would be only partial without it.
    • Another option is to support both methods - if CACert is provided directly, use it, otherwise use the roots provided by the workload api.
  • The workload api pushes updates rather than requiring software to poll for changes. Ghostunnel OTOH has a well established reload mechanism that is fired by a ticker. I can think of a couple options:
    • Don't use the push mechanism at all and instead open-and-close a connection to the endpoint whenever Reload() is called
    • Short circuit Reload() to do nothing (perhaps it can report an error condition if one is currently present)
    • ... something else?

One more unrelated-but-kind-of-related question... Any particular reason you're using unsafe.Pointer in the certloader back ends?

cannot validate certificate because it doesn't contain any IP SANs

Hi,
I use self-signed certificate to set up TLS tunnel between a server and a client.
However, the tunnel doesn't work because this error:
cannot validate certificate for xx.xx.xx.xx (server ip) because it doesn't contain any IP SANs
The error persists even if I pass "--verify-cn server-cn" flag to the client.

Reload certificates automatically

Rather than having to send a signal, it would be nice if certificates would be reloaded automatically. Either through periodic reloads (e.g. every two hours) or via inotify/fswatch.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.