tailscale / caddy-tailscale Goto Github PK

View Code? Open in Web Editor NEW

377.0 377.0 40.0 215 KB

A highly experimental exploration of integrating Tailscale and Caddy.

License: Apache License 2.0

Go 100.00%

caddy tailscale tsnet

caddy-tailscale's Introduction

Tailscale

https://tailscale.com

Private WireGuard® networks made easy

Overview

This repository contains the majority of Tailscale's open source code. Notably, it includes the tailscaled daemon and the tailscale CLI tool. The tailscaled daemon runs on Linux, Windows, macOS, and to varying degrees on FreeBSD and OpenBSD. The Tailscale iOS and Android apps use this repo's code, but this repo doesn't contain the mobile GUI code.

Other Tailscale repos of note:

the Android app is at https://github.com/tailscale/tailscale-android
the Synology package is at https://github.com/tailscale/tailscale-synology
the QNAP package is at https://github.com/tailscale/tailscale-qpkg
the Chocolatey packaging is at https://github.com/tailscale/tailscale-chocolatey

For background on which parts of Tailscale are open source and why, see https://tailscale.com/opensource/.

Using

We serve packages for a variety of distros and platforms at https://pkgs.tailscale.com.

Other clients

The macOS, iOS, and Windows clients use the code in this repository but additionally include small GUI wrappers. The GUI wrappers on non-open source platforms are themselves not open source.

Building

We always require the latest Go release, currently Go 1.23. (While we build releases with our Go fork, its use is not required.)

go install tailscale.com/cmd/tailscale{,d}

If you're packaging Tailscale for distribution, use build_dist.sh instead, to burn commit IDs and version info into the binaries:

./build_dist.sh tailscale.com/cmd/tailscale
./build_dist.sh tailscale.com/cmd/tailscaled

If your distro has conventions that preclude the use of build_dist.sh, please do the equivalent of what it does in your distro's way, so that bug reports contain useful version information.

Bugs

Please file any issues about this code or the hosted service on the issue tracker.

Contributing

PRs welcome! But please file bugs. Commit messages should reference bugs.

We require Developer Certificate of Origin Signed-off-by lines in commits.

See git log for our commit message style. It's basically the same as Go's style.

About Us

Tailscale is primarily developed by the people at https://github.com/orgs/tailscale/people. For other contributors, see:

Legal

WireGuard is a registered trademark of Jason A. Donenfeld.

caddy-tailscale's People

Contributors

Stargazers

Watchers

caddy-tailscale's Issues

README example fails to build against matching Caddy version

It appears that trying to compile this module into Caddy using the matching Caddy docker image is not working:

FROM caddy:2.7.6-builder AS builder
ENV XCADDY_SETCAP 0
RUN xcaddy build v2.7.6 --with github.com/tailscale/caddy-tailscale

FROM caddy:2.7.6
COPY --from=builder /usr/bin/caddy /usr/bin/caddy
COPY Caddyfile /etc/caddy/Caddyfile

Results in the following build error:

go: github.com/tailscale/[email protected] requires go >= 1.22.0 (running go 1.21.9; GOTOOLCHAIN=local)

Relates to:

It appears that this commit 0f105e89fbe2222c690b94b5b0b2a8150fa2540f introduces the Go toolchain bump. I was able to get past this by pinning to a specific commit prior to the Go toolchain bump:

FROM caddy:2.7.6-builder AS builder
ENV XCADDY_SETCAP 0
RUN xcaddy build v2.7.6 --with github.com/tailscale/caddy-tailscale@e041e1578ce91ef7d278dcec7e62e5af8362d79e

FROM caddy:2.7.6
COPY --from=builder /usr/bin/caddy /usr/bin/caddy
COPY Caddyfile /etc/caddy/Caddyfile

How to host a site with a public domain using Tailscale funnel?

@willnorris I've been looking at your code for your personal blog and it seems to me you're doing something like this:

Using your own https certificate as opposed to using the Caddy automated https like ts.net domains would use.
Using Tailscale Funnel to route traffic from your public domain to your Tailscale node.

I'm just making some assumptions here based on going through your code base which has been very helpful by the way so thank you for making that public :) I've done some experimenting of my own but have not quite been able to get this to work. I can see your Caddy config and that is making sense to me. It shows that auto_https is turned off.

My questions are:

How does this look on the DNS of your domain?
How does this look in Tailscale admin?
Any other details that might be helpful that are not obvious from looking at Caddyfile and Dockerfile?

Funnel Support

It would be cool if listeners could bind to Tailscale Funnel.

Seems to be as "simple" as calling ListenFunnel here and then plumbing some kind of option down from the config.

caddy-tailscale/module.go

Line 67 in 07491c5

ln, err := s.Listen(network, ":"+port)

I wonder how this would best be expressed from the bind directive? Maybe just add another network like the TLS case to allow for bind tailscale+funnel/myhost?

I'm down to open a PR for the change. Just wanted to check if this seemed like a useful idea. Thanks!

Proposal: Configuration block directives for loading tailscale configuration

I've seen a few comments in some PRs or issues about switching to a configuration block for loading tailscale configuration. Since it's come up multiple times I figured a GitHub issue would help track potential approaches. I had some thoughts and will include them here but we can always drop it if you've already a potential approach

Global Configuration

Anything that needs to be global across all tailscale servers would end up in a global configuration block. This lets the plugin also implement setup and teardown functionality (but only when the global options are specified). Anything that sets up a bare listener without specifying any tailscale block would not get the plugin teardown called on it.

Some examples could be:

whether servers should be ephemeral
External storage for tailscale server state
a global auth key

Per server/listener configuration

The bigger question that I might have is where to store per listener configuration (such as auth keys) particularly when http servers declare conflicting configuration. My thought is to include individual server configuration as individual blocks inside the global configuration.

The final configuration would look something like this:

tailscale {
    authkey = "TS_AUTHKEY_global"
    state {
      kubestore "secret_name"
    }
    a { # this matches tailscale/a
      authkey = "TS_AUTHKEY"
      ephemeral = true
    }
}

:80 {
  bind tailscale/a
}

:80 {
  bind tailscale/b  #tailscale/b inherits global configuration
}

Please let me know your thoughts

Failed to connect to local Tailscale daemon - /var/run/tailscale/tailscaled.sock: connect: no such file or directory

I feel like I'm missing something obvious here. I know when I manually install the tailscale client (as opposed to using the caddy-tailscale module) that /var/run/tailscale/tailscaled.sock is the correct path. I see when I ssh into the server that the /var/run/tailscale directory does not exist. But the Caddy logs seem to show the server connecting to the tailscale client. Is there a step I'm missing that I should be doing to set this up correctly?

The Tailscale logs when Caddy boots up:

{"level":"info","ts":1718132344.4524896,"logger":"tailscale","msg":"tsnet running state path /data/tailscale/my-node/tailscaled.state"}
{"level":"info","ts":1718132344.453898,"logger":"tailscale","msg":"tsnet starting with hostname \"my-node\", varRoot \"/data/tailscale/my-node\""}
{"level":"info","ts":1718132345.4562378,"logger":"tailscale","msg":"LocalBackend state is NeedsLogin; running StartLoginInteractive..."}
{"level":"info","ts":1718132350.4583023,"logger":"tailscale","msg":"AuthLoop: state is Running; done"}

foo.bar.com has A and AAAA records with private IP values from the Tailscale device my-node.
The error after making a request in browser to http://foo.bar.com:

{
  "level": "error",
  "ts": 1718132857.5938432,
  "logger": "http.handlers.authentication",
  "msg": "auth provider returned error",
  "provider": "tailscale",
  "error": "Failed to connect to local Tailscale daemon for /localapi/v0/whois; not running? Error: dial unix /var/run/tailscale/tailscaled.sock: connect: no such file or directory"
}

Request which shows up in logs right after the error:

{
  "level": "info",
  "ts": 1718150968.8068163,
  "logger": "http.log.access.tailscale",
  "msg": "handled request",
  "request": {
    "remote_ip": "100.106.231.73",
    "remote_port": "60590",
    "client_ip": "100.106.231.73",
    "proto": "HTTP/1.1",
    "method": "GET",
    "host": "foo.bar.com",
    "uri": "/",
    "headers": {
      "Accept": [
        "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"
      ],
      "Accept-Encoding": [
        "gzip, deflate"
      ],
      "Accept-Language": [
        "en-US,en;q=0.9"
      ],
      "Connection": [
        "keep-alive"
      ],
      "Dnt": [
        "1"
      ],
      "Upgrade-Insecure-Requests": [
        "1"
      ],
      "User-Agent": [
        "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36"
      ]
    }
  },
  "bytes_read": 0,
  "user_id": "",
  "duration": 0.001536612,
  "size": 0,
  "status": 401,
  "resp_headers": {
    "Server": [
      "Caddy"
    ]
  }
}

Dockerfile:

RUN /app/caddy/xcaddy build v2.8.4 \
    --with github.com/tailscale/caddy-tailscale \
    --with github.com/caddy-dns/route53 \
    --output /usr/bin/caddy

RUN chmod +x /usr/bin/caddy

CMD ["caddy", "run", "--config", "/config/caddy/caddy.json"]

Caddyfile:

{
  debug
  auto_https off
  tailscale {
    ephemeral
    auth_key "{env.TS_AUTHKEY}"
    state_dir "/data/tailscale"
  }
}

foo.bar.com:80 {
  bind tailscale/my-node
  tailscale_auth

  templates
  respond `Hello, {placeholder "http.auth.user.id"}}`
}

Caddy json config:

{
  "logging": {
    "logs": {
      "default": {
        "level": "DEBUG"
      }
    }
  },
  "apps": {
    "http": {
      "servers": {
        "srv0": {
          "listen": [
            "tailscale/my-node:80"
          ],
          "routes": [
            {
              "match": [
                {
                  "host": [
                    "foo.bar.com"
                  ]
                }
              ],
              "handle": [
                {
                  "handler": "subroute",
                  "routes": [
                    {
                      "handle": [
                        {
                          "handler": "authentication",
                          "providers": {
                            "tailscale": {}
                          }
                        },
                        {
                          "handler": "templates"
                        },
                        {
                          "body": "Hello, {placeholder \"http.auth.user.id\"}}",
                          "handler": "static_response"
                        }
                      ]
                    }
                  ]
                }
              ],
              "terminal": true
            }
          ],
          "automatic_https": {
            "disable": true
          }
        }
      }
    },
    "tailscale": {
      "auth_key": "{env.TS_AUTHKEY}",
      "ephemeral": true,
      "state_dir": "/data/tailscale"
    }
  }
}

Unable to build from master; unknown Caddy v2 revision

Current master references an unknown revision of Caddy v2

caddy-tailscale/go.mod

Line 6 in 57178f8

github.com/caddyserver/caddy/v2 v2.5.3-0.20220831215348-67098e5cbd68

2023/03/03 07:53:06 [INFO] exec (timeout=0s): /usr/bin/go get -d -v github.com/tailscale/caddy-tailscale github.com/caddyserver/caddy/v2 
go: github.com/caddyserver/caddy/[email protected]: invalid version: unknown revision 67098e5cbd68
2023/03/03 07:53:06 [FATAL] exit status 1

$ go list -m -u github.com/caddyserver/caddy/v2\
go: github.com/caddyserver/caddy/[email protected]: invalid version: unknown revision 67098e5cbd68

[Feature Request] Make the caddy-tailscale to go through quic/http3/udp

I've used the naiveproxy for many years (https://github.com/klzgrad/forwardproxy/tree/naive),
It uses
caddy web server(naive forwardproxy plugin to padding and multiplexing traffic) + chromium's network stack client
to camouflage traffic with strong censorship resistence,
and it can use http3/quic/udp to provide the best bandwidth and to penetrate the NAT.
(klzgrad/naiveproxy#175 (comment))

I'm thinking could the caddy-tailscale support http3 ?
So that the caddy-tailscale network can realize strong censorship resistence.

Caddyfile fails to load when a hostname is specified

My goal is to host a web service behind tailscale, and leverage tailscale's ability to know which user is connecting as an auth mechanism.

I want my service, running as tailscale device my-node, to be accessible over a custom domain name foo.bar.com (placeholder). Once the node is registered with tailscale, I want to set the DNS A record for foo.bar.com to point at my-node's tailscale IP.

I have the following Caddyfile:

{
    order tailscale_auth after basicauth
}

foo.bar.com:443 {
    bind tailscale/my-node
    tailscale_auth

    respond "Success!"
}

I generated a TS_AUTHKEY and set it in caddy's environment before running caddy run --config /etc/config/Caddyfile. I get the following error:

Error: loading initial config: loading new config: http app module: start: listen udp 100.97.63.133:443: bind: cannot assign requested address

I tried changing bind tailscale/my-node to bind tailscale/ and got:

Error: loading initial config: loading new config: http app module: start: listening on tailscale/:443: missing port in address

The error goes away when I drop foo.bar.com from the route directive.

FR: Add OAuth key support

I was trying for several days to get this to work, until I turned on Tailscale debug logs, and found.

"level":"debug","ts":1715360455.7697527,"logger":"tailscale","msg":"Received error: key cannot be used for node auth: {KeyCapabilityBits(CONTROL_API_SCOPE_DEVICES|OAUTH_CLIENT) [tag:caddy-test]}"}

Switching to a personal token works correctly.
The same OAuth token can be used for standard Tailscale client auth. Please add OAuth token support to this plugin.

Here are the Caddy Tailscale build info.

ep	github.com/tailscale/caddy-tailscale	v0.0.0-20240507224936-af99185a32ce	h1:hcYvYL4T6kU8JvDKzN5AkfQkxnppa7Zn5XWncEQTwRE=
dep	github.com/tailscale/golang-x-crypto	v0.0.0-20240108194725-7ce1f622c780	h1:U0J2CUrrTcc2wmr9tSLYEo+USfwNikRRsmxVLD4eZ7E=
dep	github.com/tailscale/goupnp	v1.0.1-0.20210804011211-c64d0f06ea05	h1:4chzWmimtJPxRs2O36yuGRW3f9SYV+bMTTvMBI0EKio=
dep	github.com/tailscale/hujson	v0.0.0-20221223112325-20486734a56a	h1:SJy1Pu0eH1C29XwJucQo73FrleVK6t4kYz4NVhp34Yw=
dep	github.com/tailscale/netlink	v1.1.1-0.20211101221916-cabfb018fe85	h1:zrsUcqrG2uQSPhaUPjUQwozcRdDdSxxqhNgNZ3drZFk=
dep	github.com/tailscale/peercred	v0.0.0-20240214030740-b535050b2aa4	h1:Gz0rz40FvFVLTBk/K8UNAenb36EbDSnh+q7Z9ldcC8w=
dep	github.com/tailscale/tscert	v0.0.0-20230806124524-28a91b69a046	h1:8rUlviSVOEe7TMk7W0gIPrW8MqEzYfZHpsNWSf8s2vg=
dep	github.com/tailscale/web-client-prebuilt	v0.0.0-20240226180453-5db17b287bf1	h1:tdUdyPqJ0C97SJfjB9tW6EylTtreyee9C44de+UBG0g=
dep	github.com/tailscale/wireguard-go	v0.0.0-20231121184858-cc193a0b3272	h1:zwsem4CaamMdC3tFoTpzrsUSMDPV0K6rhnQdF7kXekQ=
dep	tailscale.com	v1.62.0	h1:iI1fPDNXXETMbVEatos7xSR6Bv6aCuonD7B1X3glnPE=

Reload fails: "tsnet: listener already open for tailscale, :80"

The usual systemd unit for caddy includes an ExecReload= option:

ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile --force

When this is run on a caddy service that already connected to Tailscale, we get this error: loading config: loading new config: http app module: start: listening on tailscale/nitter:80: tsnet: listener already open for tailscale, :80

Dec 26 21:10:05 hydrangea systemd[1]: Reloading Caddy.
Dec 26 21:10:05 hydrangea caddy[2407]: {"level":"info","ts":1672089005.6320755,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":""}
Dec 26 21:10:05 hydrangea caddy[2031]: {"level":"info","ts":1672089005.6348317,"logger":"admin.api","msg":"received request","method":"POST","host":"localhost:2019","uri":"/load","remote_ip":"127.0.0.1","remote_port":"40046","headers":{"Accept-Encoding":["gzip"],"Content-Length":["203"],"Content-Type":["application/json"],"Origin":["http://localhost:2019"],"User-Agent":["Go-http-client/1.1"]}}
Dec 26 21:10:05 hydrangea caddy[2031]: {"level":"info","ts":1672089005.6360798,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
Dec 26 21:10:05 hydrangea caddy[2031]: {"level":"warn","ts":1672089005.6363716,"logger":"http","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"srv0","http_port":80}
Dec 26 21:10:05 hydrangea caddy[2031]: {"level":"info","ts":1672089005.636495,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000354a10"}
Dec 26 21:10:05 hydrangea caddy[2031]: {"level":"info","ts":1672089005.63655,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc000354a10"}
Dec 26 21:10:05 hydrangea caddy[2031]: {"level":"error","ts":1672089005.6366794,"logger":"admin.api","msg":"request error","error":"loading config: loading new config: http app module: start: listening on tailscale/nitter:80: tsnet: listener already open for tailscale, :80","status_code":400}
Dec 26 21:10:05 hydrangea caddy[2407]: Error: sending configuration to instance: caddy responded with error: HTTP 400: {"error":"loading config: loading new config: http app module: start: listening on tailscale/nitter:80: tsnet: listener already open for tailscale, :80"}
Dec 26 21:10:05 hydrangea caddy[2031]: {"level":"info","ts":1672089005.6377842,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
Dec 26 21:10:05 hydrangea systemd[1]: caddy.service: Control process exited, code=exited, status=1/FAILURE
Dec 26 21:11:35 hydrangea systemd[1]: caddy.service: Reload operation timed out. Killing reload process.

Restarting the service works fine, as expected.

Full Caddyfile (most of this is boilerplate from my usual template):

{
        admin localhost:2019
        email [email protected]
}

(global) {
        encode zstd gzip
        handle_errors {
                respond "{http.error.status_code} {http.error.status_text}"
        }
        header cache-control "public, max-age=0, must-revalidate"
        log {
                output file /var/log/caddy/access.log
        }
}

:80 {
        bind tailscale/nitter
        respond / "butts"
}

New node every start up when specifying hostname

Every time the server spins up, it creates a new node instead of using the existing device.

Caddyfile

:80 {
        bind tailscale/file

        root * /files/demo

        file_server
}

How to avoid the auto-generated numbered suffix in the node name (i.e. `mynode-1`)

Hi. I'm using this plugin in CI pipeline, where it creates some nodes in the tailnet for per-PR preview environments. Sometimes, while iterating on the project, the underlying VPS is destroyed and immediately recreated. This results in undesirable behavior where node names are automatically suffixed with an integer:

Tailnet (before):
  - myvps       (online)
  - mycaddynode (online)

Tailnet (after):
  - myvps         (offline)
  - myvps-1       (online)
  - mycaddynode   (offline)
  - mycaddynode-1 (online)

This breaks the project's naming scheme. To fix it, I log on to the tailscale admin UI, remove mycaddynode from the tailnet, then rename mycaddynode-1 to mycaddynode. The fix is simple, but can get tedious at times as it first requires identifying that this event has occured in the first place.

Therefore, I'd like to automate the fix. After searching around for a bit, I found that tailscaled's --state=mem: flag essentially works around this auto-suffix naming issue.

After trying it, the --state=mem: does indeed fix the issue, but only for "regular" tailscale nodes and not for nodes that are generated by this caddy-tailscale plugin:

Tailnet (before):
  - myvps       (online)
  - mycaddynode (online)

Tailnet (after):
  - myvps (online)
  - mycaddynode   (offline)
  - mycaddynode-1 (online)

Is there a way to get the same behavior for the "caddy nodes" too?

Workarounds

For now, I think my best bet would be to write a script that leverages that tailscale's Rest API to remove the stale offline nodes and rename the replacement node names (POST /device/{deviceId}/name) (i.e. mycaddynode-1 -> mycaddynode).

tailscale_auth seems not to work with tailscale+tls

This is my Caddyfile:

{
	order tailscale_auth after basicauth
	auto_https off
}

:80 {
	bind tailscale/dms-foo
	tailscale_auth
	reverse_proxy webserver-foo:8000

:443 {
	bind tailscale+tls/dms-foo
	tailscale_auth
	reverse_proxy webserver-foo:8000
}

curling the HTTP version returns 200, while the HTTPS version returns 401:

$ curl https://dms-foo.my-tailnet.ts.net -I
HTTP/1.1 401 Unauthorized
Server: Caddy
Date: Sat, 30 Dec 2023 21:53:52 GMT

$ curl http://dms-foo.my-tailnet.ts.net -I
HTTP/1.1 200 OK
...
Server: Caddy
...

tailscale-proxy fails with TLS

When you use tailscale-proxy with HTTPS/TLS, it tries to start QUIC and then fails:

{"level":"warn","ts":1703777283.5945904,"logger":"admin","msg":"admin endpoint disabled"}
{"level":"info","ts":1703777283.620156,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"proxy","https_port":443}
{"level":"info","ts":1703777283.620855,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"proxy"}
{"level":"info","ts":1703777283.620443,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc00021e000"}
{"level":"warn","ts":1703777283.6335611,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/data/caddy","instance":"7f43f41d-9b67-4aae-a7dc-563d96f88d7b","try_again":1703863683.63346,"try_again_in":86399.99999947}
{"level":"info","ts":1703777283.635115,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1703777283.676397,"logger":"http","msg":"enabling HTTP/3 listener","addr":"franklin:443"}
{"level":"info","ts":1703777283.6845386,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc00021e000"}
Error: loading new config: http app module: start: starting HTTP/3 QUIC listener: listen udp 100.84.250.36:443: bind: cannot assign requested address

This is from inside a container:

ARG CADDY_VERSION=latest

FROM docker.io/library/caddy:builder AS build
ENV XCADDY_SKIP_CLEANUP=1
RUN xcaddy build \
    --with github.com/tailscale/caddy-tailscale \
    $CADDY_VERSION

FROM docker.io/library/caddy
COPY --from=build /usr/bin/caddy /usr/bin/caddy

Using the command ['/usr/bin/caddy', 'tailscale-proxy', '--from', 'tailscale/myhost:443', '--to', 'backend:7071']

Failing to build with go 1.22.1 and caddy 2.7.3

It looks like a bunch of concurrency primitives are missing?

/home/offby1/go/pkg/mod/gvisor.dev/[email protected]/pkg/sync/gate_unsafe.go:144:2: undefined: gopark

Full-ish logs:

$ go version
go version go1.22.1 linux/amd64
$ xcaddy build v2.7.3 --with github.com/tailscale/caddy-tailscale
... a bunch of go get
2024/03/23 21:04:54 [INFO] exec (timeout=0s): /usr/bin/go mod tidy -e
go: downloading github.com/onsi/gomega v1.27.8
go: downloading github.com/benbjohnson/clock v1.3.0
go: downloading github.com/google/go-tpm-tools v0.3.12
go: downloading github.com/aws/aws-sdk-go v1.44.307
go: downloading google.golang.org/api v0.132.0
go: downloading cloud.google.com/go/kms v1.15.0
go: downloading cloud.google.com/go v0.110.4
go: downloading cloud.google.com/go/iam v1.1.0
go: downloading golang.org/x/oauth2 v0.10.0
go: downloading github.com/googleapis/enterprise-certificate-proxy v0.2.5
go: downloading github.com/google/s2a-go v0.1.4
go: downloading cloud.google.com/go/compute v1.20.1
2024/03/23 21:04:59 [INFO] exec (timeout=0s): /usr/bin/go build -o /home/offby1/tmp/caddy -ldflags -w -s -trimpath
# gvisor.dev/gvisor/pkg/sync
/home/offby1/go/pkg/mod/gvisor.dev/[email protected]/pkg/sync/gate_unsafe.go:114:3: undefined: goready
/home/offby1/go/pkg/mod/gvisor.dev/[email protected]/pkg/sync/gate_unsafe.go:144:2: undefined: gopark
/home/offby1/go/pkg/mod/gvisor.dev/[email protected]/pkg/sync/rwmutex_unsafe.go:76:3: undefined: semacquire
/home/offby1/go/pkg/mod/gvisor.dev/[email protected]/pkg/sync/rwmutex_unsafe.go:102:4: undefined: semrelease
/home/offby1/go/pkg/mod/gvisor.dev/[email protected]/pkg/sync/rwmutex_unsafe.go:152:3: undefined: semacquire
/home/offby1/go/pkg/mod/gvisor.dev/[email protected]/pkg/sync/rwmutex_unsafe.go:179:3: undefined: semrelease
/home/offby1/go/pkg/mod/gvisor.dev/[email protected]/pkg/sync/rwmutex_unsafe.go:207:3: undefined: semrelease
/home/offby1/go/pkg/mod/gvisor.dev/[email protected]/pkg/sync/seqcount.go:87:6: undefined: canSpin
/home/offby1/go/pkg/mod/gvisor.dev/[email protected]/pkg/sync/seqcount.go:89:4: undefined: doSpin
2024/03/23 21:05:12 [INFO] Cleaning up temporary folder: /home/offby1/.tmp/buildenv_2024-03-23-2104.2200051075
2024/03/23 21:05:12 [FATAL] exit status 1

Support for auto_tls

I was wondering what it would take to get the automatic TLS integration working properly.

I don't know too much about Caddy, but in the absence of any other mechanism for cross-server shared state, I was thinking about stuffing the tailscale Server object in a global map under the tailscale package, keyed by tsnet.Server.CertDomains(), and then calling s.LocalClient.CertPair().

It isn't the prettiest design, but I'm welcome to other suggestions if you know of a better way to share state between the cert manager and the listener.

If you're OK with that design, I could probably whip something up over the next few days.

What IP and port from RemoteAddr is the WhoIs function checking against on Tailscale side?

For this line: https://github.com/tailscale/caddy-tailscale/blob/main/module.go#L185

What is the RemoteAddr that's being passed checking against on the Tailscale side of things? If it's the Tailscale IPv4 then what is the port supposed to be in this case? If it's an endpoint in the Endpoints section, then where is the correct port supposed to come from?

I think I'm just not understanding what's supposed to be on the other side of the Caddy proxy. In my case I'm trying to put a public domain on the other side of this, so I have an Endpoint IP (client IP) and no port (80 or 443 rather but not a port from the client). Is there any way to make this work with just an Endpoint/client IP?

module.go
info, err := client.WhoIs(r.Context(), r.RemoteAddr)

localclient.go Tailscale 1.36.2

// WhoIs returns the owner of the remoteAddr, which must be an IP or IP:port.
func (lc *LocalClient) WhoIs(ctx context.Context, remoteAddr string) (*apitype.WhoIsResponse, error) {
	body, err := lc.get200(ctx, "/localapi/v0/whois?addr="+url.QueryEscape(remoteAddr))
	if err != nil {
		return nil, err
	}
	return decodeJSON[*apitype.WhoIsResponse](body)
}

The comment above implies it might work with only an IP but without the port we get this error 400 Bad Request: invalid 'addr' parameter. With a port it will do the authorization check.

Ready to use Dockerfile

Question:

As i understand it a could use this extension to generate a docker image which can be used as a base for a reverse proxy.
So starting the image it could create a device in tailscale and setup up https certs. Then the caddy plugin could forward all traffic to an remote host.

How would i accomplish that or did i misunderstand something?

Add funnel support

Allow the use of funnel to be specified per-binding in the Caddyfile.

Only the first visited https domain resolves correctly.

https://foo.tailf123a.ts.net {
	bind tailscale/foo
	reverse_proxy localhost:3000
}

https://bar.tailf123a.ts.net {
	bind tailscale/bar
	reverse_proxy localhost:5000
}

Two domains are declared in the previous caddyfile. After starting caddy, only the first domain I visit will resolve correctly. The second one will never resolve. For example:

Start caddy.
Go to https://foo.tailf123a.ts.net
Caddy does its tls thing and I'm able to connect.
Now go to https://bar.tailf123a.ts.net.
Caddy this time tries to do a http-01 challenge. This always fails. We are unable to connect https://bar.tailf123a.ts.net.

If we restart caddy, and visit https://bar.tailf123a.ts.net first, then it works. But then we can never conenct to https://foo.tailf123a.ts.net because that will do a http-01 challenge which fails.

If we declare 3 tailscale https domains in the caddyfile, the first one we visit will resolve correctly and the other 2 always fail.

Cannot build: can't find reason for requirement on github.com/google/pprof

I wanted to give this a try because it looks extremely Cool, but I'm having trouble building it, it seems that the Go package manager is getting Confused:

go: downloading github.com/jackc/pgproto3/v2 v2.3.1
go: downloading github.com/go-logfmt/logfmt v0.5.1
go: downloading github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d
go: downloading github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96
go: downloading github.com/golang/snappy v0.0.4
go: downloading github.com/golang/glog v1.0.0
go: downloading github.com/jackc/chunkreader/v2 v2.0.1
go: downloading github.com/jackc/pgpassfile v1.0.0
go: downloading github.com/jackc/pgservicefile v0.0.0-20200714003250-2b9c44734f2b
go: downloading github.com/shurcooL/sanitized_anchor_name v1.0.0
panic: internal error: can't find reason for requirement on github.com/google/[email protected]

goroutine 1 [running]:
cmd/go/internal/modget.(*resolver).updateBuildList.func1({{0xc000b26618, 0x17}, {0xc0000306f0, 0x22}})
        /usr/lib/go/src/cmd/go/internal/modget/get.go:1760 +0xd4
cmd/go/internal/modget.(*resolver).updateBuildList(0xc000156000, {0xb3bd58, 0xc00002c0e0}, {0x0, 0x0, 0x0})
        /usr/lib/go/src/cmd/go/internal/modget/get.go:1765 +0x54c
cmd/go/internal/modget.(*resolver).applyUpgrades(0xc000156000, {0xb3bd58, 0xc00002c0e0}, {0x0?, 0x0, 0xc0000a5d70?})
        /usr/lib/go/src/cmd/go/internal/modget/get.go:1312 +0x465
cmd/go/internal/modget.runGet({0xb3bd58, 0xc00002c0e0}, 0xc000028618?, {0xc000024220, 0x2, 0x2})
        /usr/lib/go/src/cmd/go/internal/modget/get.go:351 +0x458
main.invoke(0xe51280, {0xc0000241f0, 0x5, 0x5})
        /usr/lib/go/src/cmd/go/main.go:225 +0x3d9
main.main()
        /usr/lib/go/src/cmd/go/main.go:179 +0x7ce
2023/03/05 14:38:04 [FATAL] exit status 2

https stops working after updating tailscale to 1.68.0 (acme: WaitOrder: OrderError status invalid)

NOTE: As I am writing this bug report, I realize that an even newer tailscale version was released (v1.68.1). While I can't try v1.68.1 at the moment to see if it fixes the issue, I'll leave a link to the release notes in case someone is able to eyeball it and determine whether that bugfix is related to this here issue. If it is related, I'll give it a try when I get a chance.

I updated my linux machine. The update included a bump to tailscale from version 1.66.4 to 1.68.0.

After the update, I can no longer access node domains using https. I can still access them with http.

# does not work anymore
curl https://foo.tail123456.ts.net

# still works
curl http://foo.tail123456.ts.net

This is an example of the emitted caddy logs when running curl https://foo.tail123456.ts.net:

{"level":"debug","ts":1718822986.680432,"logger":"tailscale","msg":"cert(\"foo.tail12345.ts.net\"): already had ACME account."}
{"level":"debug","ts":1718822987.5187576,"logger":"tailscale","msg":"cert(\"foo.tail12345.ts.net\"): starting SetDNS call..."}
{"level":"debug","ts":1718822991.303825,"logger":"tailscale","msg":"netcheck: [v1] report: udp=true v6=false v6os=true mapvarydest=true portmap= v4a=98.765.432.198:12345 derp=00 derpdist=4v4:105ms,18v4:111ms,23v4:84ms"}
{"level":"debug","ts":1718822995.895996,"logger":"tailscale","msg":"wg: [v2] [emid9] - Receiving keepalive packet"}
{"level":"debug","ts":1718822998.1051943,"logger":"tailscale","msg":"cert(\"foo.tail12345.ts.net\"): did SetDNS"}
{"level":"debug","ts":1718822998.6333194,"logger":"tailscale","msg":"cert(\"foo.tail12345.ts.net\"): acme: WaitOrder: OrderError status \"invalid\""}
{"level":"debug","ts":1718822998.6334443,"logger":"tailscale","msg":"cert(\"foo.tail12345.ts.net\"): getCertPEM: acme: order  status: invalid"}
{"level":"debug","ts":1718822998.6339269,"logger":"http.stdlib","msg":"http: TLS handshake error from 100.12.34.56:12345: 500 Internal Server Error: acme: order  status: invalid"}

I have since rolled back the update and all is working fine at the moment.

I'm using the latest caddy-tailscale: 4e8fde6

Add reverse_proxy transport

We should implement a reverse_proxy transport so that caddy can serve as a simple proxy to expose a tailscale host onto the public internet.

I would hope to enable a config sort of like:

mypublichost.example.com {
    reverse_proxy http://node.tails-scales.ts.net {
        transport tailscale
    }
}

This would require a tsnet Server that is used for making the outbound connection. Currently, we setup that server only on a bind directive for listening, so we'd have to really think through how that should work.

Related public discussion: https://twitter.com/simonw/status/1579897624785670144 (of course, this is also doable with a separate tailscaled, but it'd be cool to support it in the caddy module)

tailscale_auth does not set Tailnet in the request header

My PoC at https://gist.github.com/tmaier/0a79b054b4bc69bb47320432017d1367 shows, that tailscale_auth does not set Tailnet as expected.

remove `tailscale+tls` network listener

We fixed TLS support in #53 so that the tailscale+tls listener is no longer needed. I left it in place for now, so as not to break folks that are currently using it, but we should probably remove it before a v1 release. I'd prefer not to leave it in, since it requires disabling caddy's auto_https support, which is not great.

Question: How to correctly configure reverse_proxy for a TLS upstream

Hello!

Thank you for this incredible project.

Consider following snippet:

:80 {
   reverse_proxy https://proxmoxvox.ts.net:8006 {
       transport tailscale
   }
}

Error message:

Error: adapting config using caddyfile: parsing caddyfile tokens for 'reverse_proxy': upstreams are configured for HTTPS but transport module does not support TLS: *tscaddy.TailscaleCaddyTransport, at Caddyfile:4

I understand that this limitation related to https://github.com/tailscale/caddy-tailscale#https-support but what would be the right syntax?

Thanks!

Trouble building with xcaddy - both 2.5.3 and 2.6.4

I'm running into issues trying to build with this plugin. I'm hoping to use caddy latest which as of writing this issue is 2.6.4.

First I tried xcaddy build --with github.com/tailscale/caddy-tailscale and got the error panic: internal error: can't find reason for requirement on github.com/google/pprof.

That led me to this issue which clarifies that this is an upstream Golang issue: caddyserver/caddy#5301

I tried the tip from this comment caddyserver/caddy#5301 (comment) from mholt to try and build using v2.6.4 as well as hopefully solve the above problem.

xcaddy build \
  --with github.com/tailscale/caddy-tailscale \
  --with github.com/caddyserver/caddy/v2=github.com/caddyserver/caddy/[email protected]

That leads to a new error message go: github.com/caddyserver/caddy/[email protected]: invalid version: unknown revision 2.6.4 which is similar to #9. Although I think in my case the cause is described by this stack overflow comment since 2.6.4 probably didn't exist when this plugin was built.

At this point I'm going to fork and try to build this plugin using 2.6.4 following these instructions: caddyserver/caddy#5301 (comment)

If I can't get 2.6.4 to build I'll try using the version listed in the dependencies 2.5.3. I'll update here with my findings.

Any help is much appreciated.

Tailscale host is not cleaned up

Caddy Version: v2.6.4
caddy-tailscale version: latest I think?

I'm running into the same issue that's described here but I can confirm I'm using the same authkey. Based on some experimentation, if the tsnet listener get's closed then an ephemeral node should be cleaned up from the tailscale control plane.

The caddy-tailscale plugin has a Destruct method but I don't think it's getting called because it's not implemented as a caddy.CleanerUpper?

Correction, it's exactly what's happening in #7. I mistook the node key for the authkey. I'm running this in Nomad with an ephemeral key so new deployments get a new node key. My experimentation still says that closing the Listener would logout the node from tailscale and allow the next deployment to use the same name.

So far I haven't been able to ensure that the CleanerUpper executes. If y'all know more about Caddy or can correct my idea please let me know!

Add trusted proxies support

The range of proxy servers from which requests should be trusted (trusted_proxies) can be extended with plugins to maintain a dynamic list of IP ranges. It would be nice if we could use Tailscale tags for this.

eg something like this

trusted_proxies tailscale tag:servers

which would trust requests from machines with the servers tag.