GithubHelp home page GithubHelp logo

sprotly's Introduction

Overview

What's in a name? sprotly; proxy -> proxly, x -> t for transparent and s for splice, as it uses the Linux splice(2) system call.

sprotly is a tool to sit in front of a proxy such as squid and forward https client requests to the proxy converting them into CONNECT's.

This would be used for cases where you want to be able to perform some action on https requests based purely on destination within squid and don't want them run through SSLBump.

This is done by using an iptables rule to redirect port 443 traffic to a port that sprotly is listening on e.g

# ip6tables -t nat -A PREROUTING -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 3129
# iptables -t nat -A PREROUTING -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 3129
# sprotly -l localhost:3129 -p :9443

Thus any traffic destined for port 443 is redirected to port 3129 where sprotly turns the standard requests into CONNECT's as squid would be expecting them for https requests. The corresponding squid config for this would be

http_port 9443

It then connects to squid on port 9443 (it will try IPv6 first then IPv4) on localhost (by not specifying a host) and sends the CONNECT request, once it gets the OK from squid it then just shuffles packets back and forth between the client and squid.

Building

The only hard requirement is libac. On a rpm based system something like the following should work

$ git clone https://github.com/ac000/libac
$ cd libac
$ cp libac.spec ~/rpmbuild/SPECS
$ git archive --format=tar --prefix=libac-$(grep ^Version libac.spec | cut -f 2)/ -o ~/rpmbuild/SOURCES/libac-$(grep ^Version libac.spec | cut -f 2).tar HEAD
$ rpmbuild -bb ~/rpmbuild/SPECS/libac.spec
$ sudo dnf install ~/rpmbuild/RPMS/x86_64/libac-<VERSION>-?.<DIST>.x86_64.rpm

You can create a suitable rpmbuild tree with

$ mkdir -p ~/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}

sprotly can optionally include support for seccomp via libseccomp if you have it installed, e.g on Red Hat based distros this would be the

libseccomp
libseccomp-devel

packages.

This can be disabled at build time by setting the SPROTLY_SECCOMP shell environment variable to 0. e.g

$ SPROTLY_SECCOMP=0 make

For sprotly

$ git clone https://github.com/ac000/sprotly
$ cd src
$ make
$ sudo install -Dp 0755 sprotly /usr/local/sbin/

How to use

sprotly has no configuration file and has only a few options

$ ./sprotly -h
Usage: sprotly [-D] <-l [host]:port[,...]> <-p [proxy]:port> [-s] [-v] [-h]

  -D      - Run in debug mode. Log goes to terminal and runs in the
             foreground.
  -l      - Listens on the optionally specified host/address(es) and
             port(s). If no host is specified uses the unspecified address
             (::, 0.0.0.0). Listens on both IPv6 and IPv4.
  -p      - The optional host/address of the proxy and port to send
             requests to. If the host is unspecified uses localhost. Will
             try IPv6 first then IPv4.
  -s      - Disable TLS SNI extraction.
  -v      - Display the version.
  -h      - Display this text.

Example -

    sprotly -l localhost:3129 -p :9443

-l and -p are the only required options. And the example shown is generally how you'd want to run it.

-l can take a comma separated list of [host]:port pairs.

This will tell sprotly to listen on port 3129 on ::1 and 127.0.0.1 for client requests which have been redirected by the above ip{6}tables rules.

It will then send the CONNECT requests to the proxy running on ::1 or 127.0.0.1 on port 9443.

By default sprotly will try to extract the requested hostname as given in the TLS SNI field from the 'Client Hello' message and use this in the CONNECT requests. If there isn't one it will fall back to using the IP address as retrieved from the network stack.

-s is used to tell sprotly not to do this and just use the IP address.

Architecture

sprotly uses a multiprocess, multi-threaded architecture.

When started, sprotly will fork a listen process for each cpu in the system. These listen processes wait for new client connections and hand them off to a worker thread.

For the actual sending/receiving of data to/from client/proxy it uses the splice(2) system call which has the potential to allow for zero copy I/O.

However it does make sprotly somewhat file descriptor hungry requiring six fd's per connection. i.e

1 - peer socket
2 - peer pipe read end
3 - peer pipe write end
4 - proxy socket
5 - proxy pipe read end
6 - proxy pipe write end

although on modern 64bit systems with plenty of RAM this shouldn't really be an issue, hence at startup sprotly will attempt to increase RLIMIT_NOFILE to 65536. This limit is per-process and should allow for 10,000+ connections per worker.

When run as root, the worker processes will change to run as the sprotly user. Also sprotly will write to two log files; /var/log/sprotly/access_log and /var/log/sprotly/error_log.

If the sprotly user doesn't exist, then it will try using the 'nobody' user.

You can create a sprotly user like

# useradd -r -d / -s /sbin/nologin sprotly

Also by default sprotly will daemon(3)ize itself. When run in the foreground (-D) sprotly simply logs to the terminal.

sprotly can also optionally do syscall filtering with seccomp(2) via libseccomp if you have it installed.

License

sprotly is licensed under the GNU General Public License version 2. See GPL-2.0.

src/tls_sni.[ch] are licensed under the BSD 2-clause "Simplified" License, see BSD-2-Clause.

Credits

It should be noted that sprotly was heavily influenced by Andi Kleen's eproxy https://github.com/andikleen/eproxy

Contributing

Patches and/or pull requests should be emailed to the project maintainer

Andrew Clayton <[email protected]>

preferably using git-send-email(1) and git-request-pull(1)

Code should follow the coding style as outlined in CodingStyle.

Also, code should be signed-off. This means adding a line that says

Signed-off-by: Name <email>

at the end of each commit, indicating that you wrote the code and have the right to pass it on as an open source patch.

See: http://developercertificate.org/

sprotly's People

Contributors

ac000 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

sprotly's Issues

large file dowload stuck after several seconds

during tests I noticed that downloads are stuck when getting large files. My tests show problem appears with files larger then few tens on megabytes. Shows in almost every download of file with size ~100 MB

random download testing site: https://fastest.fish/test-files

I made tests in popular browsers FF and Chrome . on terminal I use wget to make the tests.
From client side it looks like, several seconds after download starts, wget stops to download given file.
Nothing special in sprotly access.log as well as in squid access.log

here is wget download report showing this problem:
devuan@dev4-20-tunapi:~/test$ wget -O /dev/null https://speed.hetzner.de/1GB.bin
--2023-10-03 14:05:43-- https://speed.hetzner.de/1GB.bin
Resolving speed.hetzner.de (speed.hetzner.de)... 88.198.248.254, 2a01:4f8:0:59ed::2
Connecting to speed.hetzner.de (speed.hetzner.de)|88.198.248.254|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1048576000 (1000M) [application/octet-stream]
Saving to: ‘/dev/null’

/dev/null 2%[ ] 25.61M --.-KB/s in 15m 2s

2023-10-03 14:20:45 (29.1 KB/s) - Read error at byte 26853054/1048576000 (Connection timed out). Retrying.

--2023-10-03 14:20:46-- (try: 2) https://speed.hetzner.de/1GB.bin
Connecting to speed.hetzner.de (speed.hetzner.de)|88.198.248.254|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 1048576000 (1000M), 1021722946 (974M) remaining [application/octet-stream]
Saving to: ‘/dev/null’

/dev/null 38%[======> ] 385.87M --.-KB/s in 15m 22s

2023-10-03 14:36:09 (400 KB/s) - Read error at byte 404618576/1048576000 (Connection timed out). Retrying.

Looks like squid isn't a problem here - without sprotly downloads are done properly.

Regards,
Greg

URL with plain IP - missing destination address

First, it's great piece of software, doing exactly what I need. Its only one transparent https proxy with http upstream, working out of the box. I think I found corner case. It's rare but sometimes appears.

URL using plain IP (expl. https://212.244.48.139/ ) - missing destination address - log looks like that
...
[2023-09-29 10:53:11 +0200] 3475: Proxying 192.168.118.11:52294->[192.168.118.11]:52294
[2023-09-29 10:53:11 +0200] 3475: Closed 192.168.118.11:52294->[192.168.118.11]:52294, bytes tx/rx 848/2471, 95ms
...

for regular FQDN URL (expl. https://ncloud.tunapi.pl/ ) - working properly - log looks like that
...
[2023-09-29 10:53:08 +0200] 3475: Proxying 192.168.118.11:33032->ncloud.tunapi.pl[192.168.118.11]:33032
[2023-09-29 10:53:08 +0200] 3475: Closed 192.168.118.11:33032->ncloud.tunapi.pl[192.168.118.11]:33032, bytes tx/rx 850/6589, 151ms
...

on client side, using plain IP URL disables SNI field, sprotly should fallback to original destination IP, or I'm wrong here and it's my misunderstanding

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.