GithubHelp home page GithubHelp logo

zllovesuki / specter Goto Github PK

View Code? Open in Web Editor NEW
26.0 6.0 1.0 7.29 MB

like ngrok, but more ambitious with DHT for flavor

License: MIT License

Go 96.81% Makefile 1.33% Dockerfile 0.15% HTML 1.72%
chord chord-dht chord-protocol ngrok quic tunnel networking

specter's Introduction

๐Ÿ‘ป Specter

GoDoc GitHub Workflow Status (with branch)

What is it?

Specter aims to create a distributed network of nodes that forms an overlay network, allowing users to create a reverse tunnel on any publicly exposed node, then redundantly route the traffic back to the user, without having the user's machine exposed to the Internet.

Why another tunnel project?

Specter is the spiritual successor of t, another similar reverse tunnel software written by me.

(Also this is an excuse for me to play with DHT ๐Ÿ˜›)

Specter has these improvements over t:

  1. Utilizes Chord DHT (with improvements) for distributed KV storage to store routing information;
    • t uses a simple periodic updates via hashicorp/memberlist
    • meaning that if you have 100 nodes each with 100 clients connected, every single node has to maintain an 100*100 list about where the client is connected to
  2. Redundant connections from client to edge nodes and self-healing to maintain the tunnel;
    • t only connects to 1 public node and can encounter downtime when a node is having an outage
    • this has caused headache in production usage and causes assets to be unreachable
  3. Robust testing to ensure correctness (make full_test).
    • t has zero tests whatsoever
    • development is difficult as I have no confidence on correctness

Similar to t, specter also:

  1. Uses quic and only quic for inter-nodes/node-client transport. However the transport (see spec/transport) is easily extensible;
  2. Supports tunneling L7(HTTP/S)/L4(TCP) traffic over a single TLS port;
  3. Manages Let's Encrypt certificate via dns01 challenge for gateway hostname.

specter also has some interesting features:

  1. Support tunneling TCP or HTTP to Unix socket and Windows named pipe (!);
  2. When connecting to the tunnel from gateway, error is propagated from the client.

Client Configuration

CLI

A sample barebone YAML config is needed for specter client tunnel:

version: 2
apex: specter.im:443
tunnels:
  # following are shown as examples of what is valid
  - target: tcp://127.0.0.1:22
  - target: http://127.0.0.1:5173
  - target: https://example.com
  - target: unix:///run/unicorn.sock
  - target: \\.\pipe\debugger-ipc

On initial connection with specter gateway server, your configuration file will be updated:

version: 2
apex: specter.im:443
certificate: |
  -----BEGIN CERTIFICATE-----
  ...
privKey: |
  -----BEGIN PRIVATE KEY-----
  ...
tunnels:
  - target: tcp://127.0.0.1:22
    hostname: overnight-graph-caboose-list-boney
  - target: http://127.0.0.1:5173
    hostname: dreamless-spirits-episode-gloomy-path
  # ...

API

To manage custom hostnames, unpublish, release, or list tunnels, the specter client tunnel subcommand accepts an optional argument to start a local API server.

Connecting to Tunnel (CLI)

HTTP/HTTPS

For HTTP/HTTPS target, you can access the tunnel securely and directly using the assigned hostname.

TCP/Unix/Named Pipe

For example, if your tunnel was assigned a hostname overnight-graph-caboose-list-boney.specter.im:

Using specter client (Recommended, Encrypted)

This command:

specter client connect overnight-graph-caboose-list-boney.specter.im

Will establish the tunnel via stdin/stdout. This can be used in combination with ssh -o ProxyCommand if the target is an SSH server:

ssh -o ProxyCommand="specter client connect overnight-graph-caboose-list-boney.specter.im" user@host

Using HTTP Connect (Unencrypted)

If the client environment does not allow you to run arbitary binary (therefore specter client is not available), HTTP Connect Proxy can be used as a fallback. For example:

nc -X connect -x specter.im:80 %h %p

Where specter.im is the apex of the specter server, and 80 is the http handler port (default to 80). This can be used in combination with ssh -o ProxyCommand if the target is an SSH server:

ssh -o ProxyCommand="nc -X connect -x specter.im:80 %h %p" [email protected]

!! Note !! The tunnel established using HTTP Connect is unencrypted. Therefore, the underlying protocol must support encryption (e.g. TLS, SSH, etc).

Status

Component Status Description
Chord DHT Stable Chord implementation is stable to be used as a dependency
Chord KV Stable Key consistency is maintained during concurrent Join/Leave
Tunnel Core Beta RTT based intelligent routing is in development
Tunnel Gateway Beta Unified multiplexing for TCP/TLS and QUIC with status feedback
Tunnel Server Beta Server now supports persisting client tunnels
Tunnel Client Beta Client can publish multiple tunnels with redundant links with RTT tracking

Roadmap

Please see issues under Roadmap.

Development

The following should be installed on your machine:

Windows development support is limited, you may have to run go test manually instead of make test. However WSL is a great environment.

Run make dev-server-acme to compile binary for your architecture via buildx, bring up Let's Encrypt test server pebble, and a 5-node specter cluster.

Run make dev-validate to verify that all nodes have the same certificate (validate atomic ring maintenance).

Run make dev-client to start a demo nginx server as proxy target, and a specter client connected to the cluster.

For changes unrelated to KV, make test should be sufficient. Any changes to KV must pass make concurrency_test.

References

Chord:

Key Consistency:

KV Persistence:

  • Inspired by Redis' AOF format, by appending log when mutation occurs, and replay mutation logs on start-up to restore state: Redis persistence

specter's People

Contributors

zllovesuki avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

Forkers

fossabot

specter's Issues

Investigate client connection redundancy

It seems that the client currently does not correctly connect to new specter endpoint correctly (happens when upgrades occur). Client should connect to new endpoints automatically and server rpc should remove outdated mapping.

chord: FindSuccessor returned the wrong node

Insert went to correct the nodes. At this point, no joins are happening:

        local_kv_test.go:390: message 289 inserted
        logger.go:130: 2023-01-26T09:40:09.987Z	DEBUG	chord/local_kv.go:62	KV Put	{"node": 1159167524425, "target": "Remote", "key": "key 290: ...", "id": 2690397138713}
        logger.go:130: 2023-01-26T09:40:09.987Z	DEBUG	chord/local_kv.go:62	KV Put	{"node": 2817335783655, "target": "Local", "key": "key 290: ...", "id": 2690397138713}
        local_kv_test.go:390: message 290 inserted
[...]
        logger.go:130: 2023-01-26T09:40:09.990Z	DEBUG	chord/local_kv.go:62	KV Put	{"node": 1159167524425, "target": "Remote", "key": "key 300: ...", "id": 2705985598441}
        logger.go:130: 2023-01-26T09:40:09.991Z	DEBUG	chord/local_kv.go:62	KV Put	{"node": 2817335783655, "target": "Local", "key": "key 300: ...", "id": 2705985598441}
        local_kv_test.go:390: message 300 inserted

However during validation, it went to the wrong node:

        logger.go:130: 2023-01-26T09:40:10.532Z	DEBUG	chord/local_kv.go:71	KV Get	{"node": 1159167524425, "target": "Remote", "key": "key 290: ...", "id": 2690397138713}
        logger.go:130: 2023-01-26T09:40:10.533Z	DEBUG	chord/local_kv.go:71	KV Get	{"node": 2714747531593, "target": "Local", "key": "key 290: ...", "id": 2690397138713}
[...]
        logger.go:130: 2023-01-26T09:40:10.536Z	DEBUG	chord/local_kv.go:71	KV Get	{"node": 1159167524425, "target": "Remote", "key": "key 300: ...", "id": 2705985598441}
        logger.go:130: 2023-01-26T09:40:10.536Z	DEBUG	chord/local_kv.go:71	KV Get	{"node": 2714747531593, "target": "Local", "key": "key 300: ...", "id": 2705985598441}
        local_kv_test.go:430: stale ownership counts: 15
        local_kv_test.go:431: missing indicies: [290 300]
        local_kv_test.go:432: mismatched indicies: []
        local_kv_test.go:443: missing key index 290 routed to node 2714747531593 but found in node 2817335783655
        local_kv_test.go:443: missing key index 300 routed to node 2714747531593 but found in node 2817335783655
        local_kv_test.go:449: 
            	Error Trace:	/home/jenkins/agent/workspace/specter/chord/local_kv_test.go:449
            	            				/home/jenkins/agent/workspace/specter/chord/local_kv_test.go:290
            	Error:      	Not equal: 
            	            	expected: 600
            	            	actual  : 598
            	Test:       	TestConcurrentJoinKV/test_with_64_nodes_and_600_keys
            	Messages:   	expect 600 keys to be found, but only 598 keys found with 2 missing and 0 mismatched

.fixK() and .closestPreceedingNode() may be broked

Support websocket over tunnel

{
  "level": "error",
  "ts": 1661767160.5492272,
  "caller": "gateway/http_handler.go:96",
  "msg": "forwarding to client",
  "component": "gateway",
  "error": "http2: invalid Upgrade request header: [\"websocket\"]",
  "stacktrace": "kon.nect.sh/specter/tun/gateway.(*Gateway).errorHandler\n\t/home/rachel/code/specter/tun/gateway/http_handler.go:96\nnet/http/httputil.(*ReverseProxy).ServeHTTP\n\t/usr/local/go/src/net/http/httputil/reverseproxy.go:312\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2947\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:1991"
}

chord/membership: incorrect successor when joining

        logger.go:130: 2023-01-26T07:58:28.720Z	INFO	chord/local_membership.go:55	Successfully joined Chord ring	{"node": 114622756829, "predecessor": 93693217140, "successor": 119592975162}
        logger.go:130: 2023-01-26T07:58:28.720Z	INFO	chord/local_membership.go:148	Join completed, joiner has requested to update pointers	{"node": 93693217140}
        logger.go:130: 2023-01-26T07:58:28.721Z	INFO	chord/local_tasks.go:133	FingerTable entries updated	{"node": 93693217140, "fixed": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35]}
        logger.go:130: 2023-01-26T07:58:28.721Z	INFO	chord/local_membership.go:156	Join completed, joiner has requested to release membership lock	{"node": 119592975162}
        logger.go:130: 2023-01-26T07:58:28.721Z	INFO	chord/local_membership.go:71	Joining Chord ring	{"node": 644634094637, "via": ""}
        logger.go:130: 2023-01-26T07:58:28.721Z	INFO	chord/local_membership.go:100	incoming join request	{"node": 703332983828, "joiner": 644634094637}
        !!! -> logger.go:130: 2023-01-26T07:58:28.722Z	DEBUG	chord/local_chord.go:216	skip transferring keys to predecessor because predecessor left	{"node": 703332983828, "prev": 688551194597, "new": 644634094637}
        logger.go:130: 2023-01-26T07:58:28.722Z	INFO	chord/local_tasks.go:133	FingerTable entries updated	{"node": 644634094637, "fixed": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42]}
        logger.go:130: 2023-01-26T07:58:28.722Z	INFO	chord/local_membership.go:55	Successfully joined Chord ring	{"node": 644634094637, "predecessor": 688551194597, "successor": 703332983828}
        logger.go:130: 2023-01-26T07:58:28.722Z	INFO	chord/local_tasks.go:133	FingerTable entries updated	{"node": 3930596034382, "fixed": [39]}
        logger.go:130: 2023-01-26T07:58:28.722Z	INFO	chord/local_membership.go:148	Join completed, joiner has requested to update pointers	{"node": 688551194597}
        logger.go:130: 2023-01-26T07:58:28.723Z	INFO	chord/local_chord.go:91	Discovered new predecessor via Notify	{"node": 703332983828, "previous": 644634094637, "predecessor": 688551194597}
        logger.go:130: 2023-01-26T07:58:28.723Z	INFO	chord/local_tasks.go:98	Discovered new successors via Stablize	{"node": 329708973517, "successors": [688551194597, 703332983828, 1008538930581, 1012275919287]}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.