progrium / qmux

wire protocol for multiplexing connections or streams into a single connection, based on a subset of the SSH Connection Protocol

qmux

qmux is a wire protocol for multiplexing connections or streams into a single connection. It is based on the SSH Connection Protocol, which is the simplest, longest running, most widely deployed TCP multiplexing protocol with flow control.

It is meant as a drop-in layer for any stream capable transport (TCP, WebSocket, stdio, etc) to provide basic multiplexing. This brings any connection API to rough semantic parity with QUIC multiplexing, so it can act as a stopgap or fallback when QUIC is not available. You can then design higher level protocols based on multiplexing semantics that sit on top of QUIC or any other streaming transport with qmux.

Spec

The specification is here. It is a simplified version of the Channel Mechanism in the SSH Connection Protocol.

Implementations

• Golang (best reference)
• TypeScript
• Python
• C# (help wanted)

Demos

• groktunnel: Ephemeral localhost public forwarding system for HTTP similar to Ngrok in less than 150 lines of Go.

About

Licensed MIT

qmux's Issues

encount error when i try to run my own server instance

Hi!
Thanks for this project! it looks awesome.

I'm trying to run my own instance on VPS using docker.
i can establish connection between client and server.
i get back a randomized url but when in try to use it i get this kind of error:

go_srv_1  | 2022/01/26 17:25:10 groktunnel server [grok.greenponik.com] ready!
go_srv_1  | 2022/01/26 17:25:11 75npf5e8b1.grok.greenponik.com:9999: start session
go_srv_1  | 2022/01/26 17:25:11 8m0bcmc5cm.grok.greenponik.com:9999: start session
go_srv_1  | Failed to extract vhost name: invalid method "\x16\x03\x01\x02\x00\x01\x00\x01\xfc\x03\x03\xf3\xf2V\x9f;dx\xb9y]\t&\x10Q\xcbL\x97\x0f\xd9"

On server side:
i just change default value for port, host and addr in main.go (args aren't takes when i use them on Dockerfile script)
main.go:

        var port = flag.String("p", "9999", "server port to use")
        var host = flag.String("h", "grok.greenponik.com", "server hostname to use")
        var addr = flag.String("b", "54.37.11.133", "ip to bind [server only]")
        flag.Parse()

here my Dockerfile:

FROM golang:1.16-alpine

WORKDIR /app

COPY ./go.mod .
COPY ./go.sum .
RUN go mod download

COPY . /app

RUN go build

CMD ["./groktunnel"]

here my docker-compose.yml:

version: "3.8"

services:
  go_srv:
    build:
      context: .
      dockerfile: Dockerfile.server
    image: go_srv
    network_mode: host
    restart: always

on client side:
main.go:

	var port = flag.String("p", "9999", "server port to use")
	var host = flag.String("h", "grok.greenponik.com", "server hostname to use")
	var addr = flag.String("b", "127.0.0.1", "ip to bind [server only]")
	flag.Parse()

Dockerfile:

WORKDIR /app

COPY ./go.mod .
COPY ./go.sum .
RUN go mod download

COPY . /app

RUN go build

CMD ["./groktunnel", "8000"]

docker-compose.yml:

version: "3.8"

services:
  tunnel:
    build:
      context: .
      dockerfile: Dockerfile
    image: go_srv
    restart: always

How an i debug this error?
is it about the vhost "github.com/inconshreveable/go-vhost" component?

Are there any security issues with the demo?

A few queries regarding using this tool

I switched my internet connection recently and they use Carrier Grade NAT.

1. If I install this on a Linode server, can I install the same code on my local server (Raspberry PI) to enable incoming connections by talking to my Linode server running the same?
2. Also does this mean, all traffic from outside my home via this grok server (Linode) to my home PI will go through Linode? Or is it just network translation? I am concerned about possible bandwidth usage.

typescript: split transport into deno and browser modules

this way transport/index.ts can check for deno and import/re-export the deno module transports, otherwise import/re-export the browser module transports.

Use memory-aligned data structures in qmux protocol

I am currently looking into implementing a TCP-in-TCP connection tunneling protocol very similar to qmux or OpenSSH in Devolutions Gateway + jetsocat, a project written in Rust. The cleaned up protocol specification used in qmux is exactly what I'm looking for, but I've noticed that the message headers haven't been modified to respect memory alignment rules commonly enforced in certain types of processors like ARM. To read a uint32 (4-bytes) then it should be at a memory address that is a multiple of 4 bytes, etc. This is the primary reason why a lot of protocols try hard to align their data structures, and why compilers automatically pad data structures that don't respect those rules. You can still read data structures that are not memory aligned, but it comes with a performance cost.

I know this would be a breaking change, but would you be interested in making the change anyway? I will likely implement the memory aligned variant on my side and keep everything else as-is. Here is what the current header structure looks like:

  byte      QMUX_MSG_CHANNEL_OPEN
  uint32    sender channel
  uint32    initial window size
  uint32    maximum packet size

And what a memory-aligned variant would look like:

  byte      QMUX_MSG_CHANNEL_OPEN
  byte     reserved[3]
  uint32    sender channel
  uint32    initial window size
  uint32    maximum packet size

This would free 1 byte and a uint16 for other use like flags, etc, if you have such needs. What do you think? Everything else that follows would be nicely memory-aligned with those 3 bytes.

typescript: move deno conn wrapper in tests into transport/deno/tcp.ts

typescript: add deno websocket transport

wrap https://deno.land/x/[email protected]

example with two sessions

the provided example uses just single session - wouldn't it make more sense to tunnel two ports, eg run two sessions in a multiplexer demo

Constant vhost errors from server

These are just from a single client connection:
Failed to extract vhost name: read tcp 45.76.255.14:4415->173.44.95.183:50518: i/o timeout
Failed to extract vhost name: read tcp 45.76.255.14:4415->173.44.95.183:50524: i/o timeout
Failed to extract vhost name: read tcp 45.76.255.14:4415->173.44.95.183:50528: i/o timeout
Failed to extract vhost name: read tcp 45.76.255.14:4415->173.44.95.183:50550: i/o timeout
Failed to extract vhost name: read tcp 45.76.255.14:4415->173.44.95.183:50552: i/o timeout
Failed to extract vhost name: read tcp 45.76.255.14:4415->173.44.95.183:50558: i/o timeout
Failed to extract vhost name: read tcp 45.76.255.14:4415->173.44.95.183:50562: i/o timeout

Client and Server running on two different linux systems

Server DNS has wildcard subdomains set
Server bound to 0.0.0.0
Server port 4415

Running client on two different PCs simultaneously
Client connects to server with port 3415
Running quick test using mini-httpd listening on port 3415 with simple 2-line output html file

Browsers using generated address to connect to other PC successfully - html displayed

Plan is to use groktunnel to do crypto wallet transfers (EPIC Cash) to wallet on LAN behind router (using ngrok currently but don't trust them to keep the free access going so we need our own option).

Load issue with server throwing constant errors even if sending to /dev/nul ?

Can Server handle thousands of concurrent client connections?

I tried a wallet transfer from a mining pool to local wallet (these transactions were fine with localtunnel). It's a very small data stream. The server threw a Failed to extract vhost name: EOF and ended the session. Client shows EOF and Exit. So it looks like groktunnel is very http specific and will not allow just the http header then an open tcp tunnel like localtunnel. Will have to use that instead.