lightninglabs / aperture Goto Github PK

First of all, thanks for Aperture. We have been playing around with it and its great.

Seems like the Authorization Header checker is looking for the LSAT prefix, while the protocol specification says the value should be L402.

This piece of code seems to be the cause:

var (
	authRegex  = regexp.MustCompile("LSAT (.*?):([a-f0-9]{64})")
	authFormat = "LSAT %s:%s"
)

Today we require etcd, which might be a bit more exotic to some devs. We should add support for sqlite+postgres via sqlc as we've done in some other projects.

We only need to store the onion private key, and also the macaroon root keys. With taproot assets, we have an implementation of the macaroon storage, so we can use that: https://github.com/lightninglabs/taproot-assets/blob/main/tapdb/macaroons.go

The BOS scores endpoint referenced here seems to be down and has been returning a 404 since at least yesterday. FYI!

$ curl -k -v https://test.swap.lightning.today:11010/availability/v1/btc.json
*   Trying 35.160.63.96...
* TCP_NODELAY set
* Connected to test.swap.lightning.today (35.160.63.96) port 11010 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=test.swap.lightning.today
*  start date: Jul 26 00:00:00 2020 GMT
*  expire date: Aug 26 12:00:00 2021 GMT
*  issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
*  SSL certificate verify ok.
> GET /availability/v1/btc.json HTTP/1.1
> Host: test.swap.lightning.today:11010
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Content-Type: text/plain; charset=utf-8
< X-Content-Type-Options: nosniff
< Date: Wed, 16 Jun 2021 20:31:23 GMT
< Content-Length: 19
<
404 page not found
* Connection #0 to host test.swap.lightning.today left intact
* Closing connection 0

A necessary component is an operation HTTP/2+HTTP+gRPC reverse proxy. The proxy should be able to transparently proxy any sort of requests to those 3 target protocols. The proxy should have a static configuration of its target backend(s). The proxy should be able to handle TLS termination through the target end point (may involve self-signed certs).

In preparation for the greater system, the package should accept an abstract credentials/authentication interface. A draft version of this interface may look something like the following:

type Authenticator interface {
    Accept(http.Header) bool

    FreshChallengeHeader(r *http.Request) (http.Header, error)
}

Notice that such an interface allows the precise authentication protocol to be fully abstracted from the proxy itself and live entirely in a distinct package.

In lightninglabs/loop/pull/445, I bumped the following dependencies by mistake:

google.golang.org/grpc v1.38.0 -> google.golang.org/grpc v1.42.0
google.golang.org/protobuf v1.26.0 -> google.golang.org/protobuf v1.27.1

This caused all paywalled aperture apis to fail with rpc error: code = Unknown desc = cannot initiate swap: unexpected HTTP status code received from server: 402 (Payment Required), despite the interceptor being active (confirmed with ad-hoc logging).

This is not an issue at present, since we have no pressing need to upgrade.
Should we decide to upgrade lnd/loop/pool/lit, we will need to remember to upgrade aperture as well.

As explained here https://docs.lightning.engineering/lightning-network-tools/aperture/mailbox#docs-internal-guid-6f5d734c-7fff-7276-2045-8790bdb8ac96 , it is not possible to bind to the external IP used in many VPS setups. This is because it's behind NAT and the external IP doesn't terminate on the VM.

My question is, is it possible to use the hashmail feature of aperture with this networking setup? Part of the solution is to set listenaddr: "0.0.0.0:443" to bind to an externally accessible interface, but when I do this I get the following error when trying to connect via lnc:

[DBG] PRXY: No backend service matched request [lightning.brodie.rocks:443/hashmailrpc.HashMail/RecvStream].

I'm guessing the problem is no service is defined for that FQDN because it is bound to 0.0.0.0 ? I looked through how to configure services but it all seems geared towards LSAT configuration, where in this case I'm only interested in setting up a hashmail proxy.

aperture.yaml used:
listenaddr: "0.0.0.0:443" debuglevel: "trace" autocert: false servername: lightning.brodie.rocks authenticator: disable: true hashmail: enabled: true messagerate: 1ms messageburstallowance: 99999999 prometheus: enabled: false

Thanks for any tips/suggestions.

Similar to #4, we'll also need awareness of an LSAT within nautilus itself. This will require mirror server-side unary and streaming interceptors which can live in this repo as they'll only be used within the nautilus system for now.

Not sure if the golang update is the problem or something similar. But our race detection tests are currently broken in the master branch:

WARNING: DATA RACE

Read at 0x00c0004c1960 by goroutine 54:
  net/http.(*http2requestBody).Close()
      /home/travis/.gimme/versions/go1.16.13.linux.amd64/src/net/http/h2_bundle.go:5982 +0x91
  net/http/httputil.(*ReverseProxy).ServeHTTP()
      /home/travis/.gimme/versions/go1.16.13.linux.amd64/src/net/http/httputil/reverseproxy.go:358 +0x1327
  github.com/lightninglabs/aperture/proxy.(*Proxy).ServeHTTP()
      /home/travis/gopath/src/github.com/lightninglabs/aperture/proxy/proxy.go:240 +0x46f
  github.com/lightninglabs/aperture/proxy.(*Proxy).ServeHTTP-fm()
      /home/travis/gopath/src/github.com/lightninglabs/aperture/proxy/proxy.go:99 +0x68
  net/http.HandlerFunc.ServeHTTP()
      /home/travis/.gimme/versions/go1.16.13.linux.amd64/src/net/http/server.go:2050 +0x51
  net/http.serverHandler.ServeHTTP()
      /home/travis/.gimme/versions/go1.16.13.linux.amd64/src/net/http/server.go:2868 +0xca
  net/http.initALPNRequest.ServeHTTP()
      /home/travis/.gimme/versions/go1.16.13.linux.amd64/src/net/http/server.go:3440 +0x104
  net/http.(*initALPNRequest).ServeHTTP()
      <autogenerated>:1 +0xa6
  net/http.Handler.ServeHTTP-fm()
      /home/travis/.gimme/versions/go1.16.13.linux.amd64/src/net/http/server.go:87 +0x6d
  net/http.(*http2serverConn).runHandler()
      /home/travis/.gimme/versions/go1.16.13.linux.amd64/src/net/http/h2_bundle.go:5840 +0xc5

Previous write at 0x00c0004c1960 by goroutine 79:
  net/http.(*http2requestBody).Close()
      /home/travis/.gimme/versions/go1.16.13.linux.amd64/src/net/http/h2_bundle.go:5985 +0x64
  net/http.(*readTrackingBody).Close()
      /home/travis/.gimme/versions/go1.16.13.linux.amd64/src/net/http/transport.go:637 +0x6c
  net/http.(*http2clientStream).cleanupWriteRequest()
      /home/travis/.gimme/versions/go1.16.13.linux.amd64/src/net/http/h2_bundle.go:7942 +0x6a8
  net/http.(*http2clientStream).doRequest()
      /home/travis/.gimme/versions/go1.16.13.linux.amd64/src/net/http/h2_bundle.go:7723 +0x66

This builds in #1 and extends the proxy with a new abstract interface to handle a set of grace requests in the case that the Accept method returns false. In this case, if the backend is configured to support grace periods, an abstract database should be updated to reflect this free pad in order to reject/deny any future requests.

An draft version of such an interface follows:

type FreebieDB interface {
    CanPass(*http.Request, net.IPMask) (bool, error) 

    TallyFreebie(*http.Request, net.IPMask) (bool, error)
}

The first method should read the concrete database and gauge if the request is able to pass based on the prior history of that request, backend, and also IP range. The limiting (initially, unless we have a better suggestion), should be IP based, allowing a number of requests free from an IP before we reject their request and force them to carry out the authentication protocol.

The second method should tally the freebie request, updating the underlying concrete database implementation.

A concrete database for bbolt as well as etcd should be created. The schema for both should be designed in mind for several kirin proxies connected to the same backing store.

Hello, I am trying to get Aperture running on a server and I am receiving the following error when I run the aperture command after building:

unable to parse config file: invalid authenticator configuration

You can view the config I am using here and the diff from the defaults:
master...secondl1ght:aperture:master

I have the auth disabled so I am not sure why it is not liking this config. All the changes made to the config were from following the instructions about setting up Aperture + a LNC Mailbox in the Builder's Guide (https://docs.lightning.engineering/lightning-network-tools/aperture).

Any help to get this working would be appreciated. Thanks!

Hey guys! Thanks again for aperture, really interested mostly in using it as a library than a proxy over existing services. Not sure if that's the direction you want to be open to or not but there was some things I had in my local project that I thought would be good to discuss & potentially contribute back as feedback.

First is mostly around exports. There's some things in aperture that I'd like to not have to copy/paste into my project, so things like newStaticServiceLimiter and LndChallenger.client would be great to have exported.

Along these lines, another improvement that would be nice to have would be to allow the invoice checker to be optional. I'm kind of curious why that was introduced actually, shouldn't preimage always be good enough to prove an invoice was paid? Unless of course someone accessed the node and viewed the preimage themselves, or created the invoice with a preimage. I think those are edge cases that may be nice to ignore without having to create the classes myself with empty methods fulfilling the InvoiceChecker interface.

I created a PR here with some of these changes - mostly as a way to visualize what I mean and discuss. Happy to clean up or go a different direction with some of the code to produce a similar result. #44

And some examples of how I use the proposed code:

func (l *lightningAuth) VerifyLightningAuth(o *AuthOptions) (bool, error) {
	// challenger
	genInvoiceReq := func(price int64) (*lnrpc.Invoice, error) {
		return &lnrpc.Invoice{
			Memo:  "LSAT",
			Value: price,
		}, nil
	}
	challenger := &aperture.LndChallenger{
		Client:                  l.lndClient,
		GenInvoiceReq:           genInvoiceReq,
		VerifyInvoiceStatusFunc: VerifyInvoiceStatusNil,
	}

	// limiter
	s := lsat.Service{
		Name:  o.ServiceName,
		Tier:  lsat.BaseTier,
		Price: o.Price,
	}
	limiter := &aperture.StaticServiceLimiter{}
	if o.Capabilities != "" {
		limiter.Capabilities = map[lsat.Service]lsat.Caveat{
			s: lsat.NewCapabilitiesCaveat(s.Name, o.Capabilities),
		}
	}

	authenticator := auth.NewLsatAuthenticator(
		mint.New(&mint.Config{
			Secrets:        l.secretStorage,
			Challenger:     challenger,
			ServiceLimiter: limiter,
		}),
		challenger,
	)

	return authenticator.Accept(o.H, o.ServiceName), nil
}

func (l *lightningAuth) NewChallengeHeader(o *AuthOptions) (http.Header, error) {
	// challenger
	genInvoiceReq := func(price int64) (*lnrpc.Invoice, error) {
		return &lnrpc.Invoice{
			Memo:  "LSAT",
			Value: price,
		}, nil
	}
	challenger := &aperture.LndChallenger{
		Client:                  l.lndClient,
		GenInvoiceReq:           genInvoiceReq,
		VerifyInvoiceStatusFunc: VerifyInvoiceStatusNil,
	}

	// limiter
	s := lsat.Service{
		Name:  o.ServiceName,
		Tier:  lsat.BaseTier,
		Price: o.Price,
	}
	limiter := &aperture.StaticServiceLimiter{}
	if o.Capabilities != "" {
		limiter.Capabilities = map[lsat.Service]lsat.Caveat{
			s: lsat.NewCapabilitiesCaveat(s.Name, o.Capabilities),
		}
	}

	authenticator := auth.NewLsatAuthenticator(
		mint.New(&mint.Config{
			Secrets:        l.secretStorage,
			Challenger:     challenger,
			ServiceLimiter: limiter,
		}),
		challenger,
	)

	return authenticator.FreshChallengeHeader(o.R, o.ServiceName, o.Price)
}

I'm making the connection to LND myself, so exporting Client would be nice so I don't have to go through creating the challenger via NewLndChallenger way. Also I'm not going to be consistently connected to the possibly many lnd clients I'd be creating invoices from, so I'm simply only caring about preimage matching macaroon (so not constantly streaming invoice statuses or caring about invoice checking).

Anyways, thoughts? I'm okay keeping a fork for what could be very edge case/specialized behavior specific to me. Just thought I seen some thing that made me want to talk about contributing back. Going a different direction to establish the same goals is okay too, just got a quick mvp of what I thought I needed myself. I have some other ideas around capabilities and constraints that may be nice to chat about in another issue in the near future too.

The proxy should eventually grow its config to allow backend's to configure the precise value of the LSAT token for a particular endpoint. This value may even be dynamic, allowing an admin to increase/decrease the price via a programmatic interface. An example is the initial account creation for CLM.

This is a brain storming issue for future improvements to the hashmail server and LNC transport protocol to increase scalability.

Mailbox ID based horizontal scaling

In most cluster environments services are scaled horizontally by using load balancers with some kind of "session stickiness" mechanism turned on, that makes sure the same client always is forwarded to the same backend.
With LNC this is not feasible, because both the server (litd) and the client (browser or LNC client application) must connect to the same instance of aperture in order to be able to communicate.
Since both server and client derive the same mailbox or session ID (=SID), they can use that ID to look up the server instance to connect to by asking any of the instances:

1. LNC application (server or client) connects to load balanced aperture instance (e.g. mailbox.terminal.lightning.engineering), queries ServerForMailbox(SID=xyz)
2. The aperture instance that got the request checks the connected etcd instance if SID is already mapped.
   1. If the SID is mapped, return the direct server instance (e.g. srv-004.mailbox.terminal.lightning.engineering) as the response.
   2. If the SID is not yet mapped, pick a server at random (each aperture knows each other instance through etcd) and store the mapping in etcd. Return the server instance as the response.

The reason this extra call and then direct connection to a non-load balanced instance is necessary is the nature of the LNC transport connection: It's a long-lived gRPC (or WebSocket) connection. So it uses up one of max. 65k TCP client ports on a machine. If we would run all connections through the same Layer 4 load balancer, then we would still be limited to a maximum of 65k clients. With this extra call we increase the initial connection complexity a bit but allow each individual instance to accept a maximum of 65k clients.

Hashmail protocol improvements

The current protocol requires both the server and client of an LNC connection to open two half-duplex mailbox streams to the mailbox server, each with a unique (but related) SID to create the virtual full-duplex connection between server and client.
Because in practice the two SIDs are identical except for the last bit which is flipped, the protocol could quite easily be simplified to allow a single full-duplex stream to be opened. So basically merging two uni-directional gRPC (or WebSocket) streams into a single, bi-directional one.

A further improvement that could lead to better memory efficiency on the aperture side is to remove the requirement for a-synchronous mailbox usage. Originally the mailbox protocol was developed for sending a single piece of information from a sender to a receiver, potentially a-synchronous. So the sender would send their message and disconnect. Then the receiver would later connect to the server and retrieve the message. That is useful for sending updates of a Pool sidecar channel order from the buyer to the recipient, or for transferring Taproot asset proofs from sender to receiver. But for a protocol that requires both parties to be online in the first place, that message buffering feature is not strictly necessary.
So the LNC mailbox gRPC endpoint could be improved further by making writes blocking as well until there is a reader on the other end (instead of allowing writes up to a certain buffer size).

Versioning

Because all the improvements described above will largely be incompatible with older clients, some form of versioning system needs to be implemented at the LNC transport (=hashmail) layer.
There are version numbers available at the LNC transmission (=Go-Back-N) and LNC noise layer, but those won't be of any use, as they can only be exchanged once an initial transport layer connection was initialized.

Most likely some sort of new connection phrase encoding (either as an URL or QR code) would be required to indicate a new mailbox protocol needs to be used (so an old client wouldn't understand the new scheme and would give an error message).

(from chat)

Possible working path:

• modify LSAT to not read the services from disk anymore (well have the option, but a new interface), instead it accesses etcd to get the current working config set
• there's an entry for each service it proxies to (key hierachy ofc flexible), it then watches those key sets for changes, when they change it updates its in-memory forwarding/service table
• we have the normal 3f+1 instance of the naut server itself active
• upon start up, they each attempt to grab win the election by grabbing the underlying mutex w/ a lease (https://godoc.org/github.com/coreos/etcd/clientv3/concurrency#Election)
• the leader then sets a bit in their etcd service discovery config, which updates the LSAT proxy to direct traffic as needed
• anytime they instance die or become unavailable (fail the hearbeat, or their lease expires), then the election restarts

docs on their election API: https://github.com/etcd-io/etcd/blob/master/Documentation/dev-guide/api_concurrency_reference_v3.md

Right now we only pass along the path of the request to the backend: https://github.com/lightninglabs/aperture/blob/master/pricesrpc/rpc.proto#L12

This is useful or pricing distinct calls to diff paths, but isn't as useful for more advanced use cases. One example is proxying access to an LLM, but scaling the pricing per query, based on things like the chosen model, the context size, etc, etc. All information that will be sent along with the rest as JSON encoded body params.

We should pass along all/some of the other request level context: https://pkg.go.dev/net/http#Request

I have a very simple web service running locally. It has a POST endpoint at /hello that accepts a JSON body of the form

{
  "name": "John"
}

and it responds with a body like

{
  "message": "Hello John"
}

When I try using Aperture to proxy the request, I get an error. Here is the curl command I'm using

% curl -X POST -H "Content-Type: application/json" -d '{"name": "John"}' -k --http1.1 https://localhost:8081/hello

and the error I get is

curl: (18) transfer closed with 16 bytes remaining to read

However, if I don't send a body to the endpoint, I get a 402 as expected:

% curl -X POST -k --http1.1 https://localhost:8081/hello
payment required

FWIW, I am able to dump the request details right before the call to sendDirectResponse in the handlePaymentRequired method, and I can see the body I'm expecting.

POST /hello HTTP/1.1
Host: localhost:8081
Accept: */*
Access-Control-Allow-Headers: Authorization, Grpc-Metadata-macaroon, WWW-Authenticate
Access-Control-Allow-Methods: GET, POST, OPTIONS
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: WWW-Authenticate
Content-Length: 16
Content-Type: application/json
User-Agent: curl/8.1.2
Www-Authenticate: LSAT macaroon="...", invoice="..."

{"name": "John"}

Am I missing something in my request?

Otherwise, an attacker can just spam our end point, more or less executing a DoS attack.

A new package should be constructed, that in isolation, is able to perform first-level verification on any LSAT tokens, and also expose a state machine to handle the authentication flow. This package need not be aware of any transport or HTTP semantics. Instead, it should focus only on the chosen macaroon structure for verification.

The package should abstract away it's main external dependency (obtaining invoices from lnd):

type ChallengeDispenser interface {
    NewChallengeInvoice() (*zpay32.Invoice, error)
}

In this context, the challenge string is a BOLT 11 invoice. Verification can be done in many ways, but in order to reduce external calls, first level validation of minting should be done using the pre-image and payment hash within the invoice. Note that post-minting verification and pass through verification are one in the same.

The package should also define an interface for use by external services to define any additional constraints they may require for macaroons destined to a given service. For inspiration, one may look to the macaroons.Checker and macaroons.Constraint abstractions.

I see no reference to etcd in the README and if I comment it out in the sample-conf.yaml I receive a segmentation fault.

Go 1.19 was deprecated 10 months ago (source: Go | endoflife.date). To avoid security loopholes, we need to upgrade to the latest version as a routine matter.

A recent request from a develooper: they're using LSAT, but want to hand out "freebie" tokens to users to be able to play with. The first half here is being able to create the token itself (so just add+settle the invoice, them make an LSAT from that). The second half (with the larger design space), is how that token is actually imported into some application/CLI/service.

The demo link in the readme goes to this URL: https://test-staging.swap.lightning.today:8081/ but it seems that endpoint is currently down and can't be reached.

There is no error or cancellation at calling grpc.DialContext below, it is just put on hold.
The mailbox is mailbox.terminal.lightning.today:443
https://github.com/lightninglabs/aperture/blob/master/lnc/lnc.go#L284-L286

By the way, this issue may be due to be re-use passphrase from multiple containers under development, because it worked generating new phrase per container. However, I don't know because I haven't received a response and seen any documentation prohibiting passphrase reuse.

I noticed some breadcrumbs in the existing code that indicate there was the intention to build in some support for this but best I can tell they are mainly just placeholders for now.

token.go just sets a TimeCreated value when parsing from the challenge but that's just set to time.Now(). As best I can tell, this isn't really persisted anywhere "permanent" on the lsat itself. Then there is this code that sets an IsValid method on the Token. It's only a placeholder now that always returns true. It's not clear to me how the TimeCreated value on the token serves any use if it's not encoded directly in the macaroon. I might be missing where this is persisted though.

It seems to me like this would be pretty valuable functionality to have. Without the ability to use aperture as a library, the only way to add caveats via aperture as the author is hardcoded in the config file. As far as I can tell, this makes relative caveat restrictions like an expiration date impossible. The sample config shows how to add an absolute expiration with a valid_until constraint, but I can't see how given the current tooling you could do something like "valid_for=xx_days", which feels much more valuable and flexible to me.

Is there some way to achieve this that I'm missing? If not, what are the chances of getting this support in aperture?

The current default Go Loop client should grow to be aware of the LSAT minting+verification protocol. Namely, unary and streaming client interceptors should be created that are aware of the gRPC minting protocol. Note that the creation of such interceptors may warrant creating a distinct public facing repository which may contain a brief specification of our protocol, as well as other client side libraries.

https://docs.lightning.engineering/lightning-network-tools/lightning-terminal/mailbox shows the config options required to setup the hashmail mailbox proxy.

Wondering, why is port 443 recommended? Is it required to use https for this to be secure? If this is just a dumb proxy, it seems like port 80 could work and then no SSL certificate would be required? I thought a big reason for Lightning Node Connect was to get rid of the need for SSL certificates, yet it seems like aperture and the proxy may be adding them back in.

Also, can the hashmail mailbox proxy be used in addition to the other features of aperture in the same instance? Wondering what http path the hashmail proxy runs under so that if I want to also use aperture for doing the Lightning Service Authentication Token Reverse Proxy (that it is mainly designed for), I can avoid path conflicts.

When the proxy encounters an unknown route, it would be useful to have the option of responding with a redirect instead of a static site.

An example is when the proxied service has a main page/site hosted elsewhere, and it's undesirable to maintain a separate landing page for the proxy. For my use case I think only permanent (301) is needed, although maybe two versions could be useful:

• servepermanentredirect: return 301, overrides servestatic
• servetemporaryredirect: return 302, active when others are undefined (or static site missing?)

I don't have a specific case in mind for the temporary version, but it seems like it could be useful for maintenance. Or maybe a simple serveredirect would be plenty, and the code could be set another way if needed.

Related to #5, we should also create a Javascript/Node version of the interceptors for usage within the mobile/desktop applications.