• setup CI for all PRs:
  • build
  • test
  • lint
  • coverage (we don't want it to drop)
• setup required checks to pass in master branch protection
• create CONTRIBUTING.md, describing
• create Makefile so that make resembles CI run as closely as possible

The proxy.pac can derive multiple proxies, which might require different passwords. Alpaca should prompt for each new proxy, with affordances to reuse creds from proxies already seen.

sudo add-apt-repository ppa:graphics-drivers/ppa

gpg: keyring /tmp/tmpm22_n592/secring.gpg' created gpg: keyring /tmp/tmpm22_n592/pubring.gpg' created
gpg: requesting key 1118213C from hkp server keyserver.ubuntu.com
gpgkeys: key 2388FF3BE10A76F638F80723FCAE110B1118213C can't be retrieved
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0
gpg: keyserver communications error: keyserver helper general error
gpg: keyserver communications error: unknown pubkey algorithm
gpg: keyserver receive failed: unknown pubkey algorithm
Failed to add key.

tailf /var/log/syslog
Mar 4 12:07:49 m84116293-HP-ProDesk-600-G3-PCI-MT alpaca[21627]: 2020/03/04 12:07:49 proxyfinder.go:92: [3408] CONNECT //launchpad.net:443 via "PROXY proxy.lan:8080 "
Mar 4 12:07:51 m84116293-HP-ProDesk-600-G3-PCI-MT alpaca[21627]: 2020/03/04 12:07:51 proxyfinder.go:92: [3409] CONNECT //launchpad.net:443 via "PROXY proxy.lan:8080 "
Mar 4 12:07:51 m84116293-HP-ProDesk-600-G3-PCI-MT alpaca[21627]: 2020/03/04 12:07:51 proxyfinder.go:92: [3410] GET http://keyserver.ubuntu.com:80/pks/lookup?op=get&options=mr&search=0x2388FF3BE10A76F638F80723FCAE110B1118213C via "PROXY proxy.lan:8080 "

Alpaca releases should be given a version number and this version should be made visible via a -version flag and/or printed in the logs.

There is currently no concept of a "release" in Alpaca, so resolving this issue would require a build/release pipeline to be set up.

I'm using alpaca_v2.0.2_linux-amd64 and websites like figma.com and strigo.io aren't loading.

(NB: Just a note for now, as I've not got a machine running Big Sur to do any testing)

Overview

Seems one of the ABIs used by go-keychain has been recently deprecated - it presumedly still works (with warnings), but that likely won't last long:

(copied from another user)

~ % go get -v github.com/samuong/alpaca
# github.com/keybase/go-keychain
cgo-gcc-prolog:203:11: warning: 'SecTrustedApplicationCreateFromPath' is deprecated: first deprecated in macOS 10.15 - No longer supported [-Wdeprecated-declarations]
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Security.framework/Headers/SecTrustedApplication.h:59:10: note: 'SecTrustedApplicationCreateFromPath' has been explicitly marked deprecated here

Also see: keybase/go-keychain#54

What to do?

Hopefully just bump the go-keychain version, assuming they've worked out how to transparently support the older OS ABI alongside the new one

The following tests fail on macOS:

• TestNoMADNotConfigured
• TestNoMADNotUsingKeychain
• TestNoMADNoUserPrincipal
• TestNoMADInvalidUserPrincipal

The problem here is that they check for specific text in the error message, which changed as part of pull request #34.

These tests don't run as part of go test on other platforms; this only fails on macOS.

Maybe even error out, with a command-line option acknowledging the security vulnerability to suppress the error.

Hi!
Thanks for this great project!

It would be nice to have the feature to start alpaca in a mode where all requests are routed as "direct". This way I could always keep the proxy configured and start alpaca with this flag when I'm connected to a network without a proxy.

Does this support subscribing to the same symbol by different subscribers?

Currently, the PAC URL is read from the system settings before downloading the PAC file. This is done by calling out to a subprocess (gsettings or networksetup on Linux and macOS - but for Windows this is not yet implemented) and reading stdout. It takes around 50-60ms on my M1 MacBook Pro, which is too slow to do on every outgoing request. But if we don't check this value frequently, the configured PAC URL can change and Alpaca won't notice.

We can make the findPACURL() function faster by calling the native system libraries from Cgo. This will then allow us to check the PAC URL before Alpaca forwards each outgoing request.

Linux (GNOME)

https://wiki.gnome.org/DevGnomeOrg/Gnome3PortingGuide/ProxyConfiguration
https://docs.gtk.org/gio/class.Settings.html

macOS

https://developer.apple.com/documentation/systemconfiguration/1517088-scdynamicstorecopyproxies

Windows

https://docs.microsoft.com/en-us/windows/win32/api/winhttp/nf-winhttp-winhttpgetieproxyconfigforcurrentuser

Alpaca monitors the host's IP addresses to determine whether to use the PAC file, or proxy requests directly. For example, if a user moves from a corporate network to their home WiFi, their IP addresses will change, Alpaca will notice that the PAC server is no longer reachable, and conclude that requests should be sent directly.

This heuristic can fail when a new IP address is issued, but DNS resolution fails for some time until the network connection is fully established (which could happen in cases of slow 802.1x negotiation?)

Chrome works around this by waiting 2 seconds after the IP address change before attempting to fetch the PAC file. However Chrome seems to have a way to get notified of IP address changes, whereas Alpaca checks the IP addresses when it does a proxy request.

So another option might be to introduce a retry mechanism when downloading the PAC file, with an exponential backoff.

I created a systemd service unit file and started Alpaca as service and passed all the argument like username and password, for direct connection it works fine for proxied connection i get site unavailable error on browser, the app does not show any error or or logs

The pac works in my chrome, but in alpaca, the log shows that :

proxyfinder.go:128: [1] Couldn't parse proxy: "HTTPS ssl-xxxxxx.yyyyy.com:3389"

Is there anything wrong? Thanks.

For circumstances where users need to tailor their pac file from the standard, do testing etc, the ability to specify file:// type URI schemes would be very beneficial. Our team is working around this by locally serving a pac file

Extract from proxy.pac loaded by Alpaca:

function FindProxyForURL(url, host) {
  if (shExpMatch(url, "*://" + host + ":*") && (!shExpMatch(url, "*://" + host + ":443/*"))) {
    return "DIRECT";
  }
  return "PROXY myproxy:3128";
}

curl https://raw.githubusercontent.com/samuong/alpaca/master/README.md going through Alpaca will give:

proxy.go:138: [1] Error dialling host raw.githubusercontent.com:443: dial tcp 185.199.109.133:443: connect: connection refused
proxyfinder.go:113: [2] CONNECT //raw.githubusercontent.com:443 via "DIRECT"

Where I would expect:

proxyfinder.go:135: [1] CONNECT //raw.githubusercontent.com:443 via "PROXY myproxy:3128"

Loading the same proxy.pac from Firefox/Chrome gives me the expected behavior.

I narrowed it down to the ":443/*" part of the rule matching in Chrome but not in Alpaca.

Here's what I think is happening:
Alpaca strips the URL path before attempting to match the rules. The comment explains this is to match Firefox/Chrome behavior. However there seems to be a difference between Alpaca and Chrome when handling URLs with empty paths: Go's "net/url" won't add a trailing slash for an empty path, but Chrome does.

The following code uses a bare string as a context key. This is a bug.

I noticed that Apple silicon running on ARM64 is not a release target in the brew tap, is it possible to add a if statement in the brew tap script to support:
GOOS=darwin GOARCH=arm64 go build?

When Alpaca receives a 407 from an upstream proxy, it just assumes that it's an NTLM proxy and starts to do an NTLM handshake.

In the last few months, I noticed that some of the proxies that I was authenticating to began to send back a "Proxy-Authenticate: Negotiate" header instead of "Proxy-Authenticate: NTLM". This allows the client (Alpaca) to use either NTLM or Kerberos.

Alpaca should check the value of this header, and switch authentication mechanisms accordingly. More information about the Negotiate scheme can be found at https://tools.ietf.org/html/rfc4559 and https://docs.microsoft.com/en-us/dotnet/framework/wcf/feature-details/understanding-http-authentication.

A Kerberos library that might help is https://github.com/jcmturner/gokrb5.

If my AD account password changes, alpaca will keep retrying after a password failure. In such cases, alpaca should prompt for a new password. Same goes for mistyped passwords.

Here's what's going on:

1. Safari sends a CONNECT request to Alpaca
2. Alpaca responds with a 200 OK, and Go's ResponseWriter.WriteHeader() will automatically add Transfer-Encoding: chunked to the response header
3. It then blindly copies data back and forth between the client and the server, but this data is not chunk encoded, and Safari's chunk decoder appears to fail as a result

According to https://tools.ietf.org/html/rfc7231#section-4.3.6:

A server MUST NOT send any Transfer-Encoding or Content-Length header fields in a 2xx (Successful) response to CONNECT. A client MUST ignore any Content-Length or Transfer-Encoding header fields received in a successful response to CONNECT.

It's not clear to me why Safari isn't ignoring the Transfer-Encoding header from Alpaca, but I think it should.

This also needs to be fixed in Alpaca by writing the CONNECT response directly to the connection (using Conn.Write()), rather than using Go's ResponseWriter.

Hi,

Alpaca is a great tool, and replaces easily cntlm on my computer due to its ability to support proxypac.

Unfortunatly I'm facing one issue : I use docker inside wsl2, and because of the fact that alpaca only listens on the localhost interface, I can't use it from a container :(

tcp 0 0 127.0.0.1:3128 0.0.0.0:* LISTEN 111/alpaca

Is there a way to make alpaca listen on all interfaces (0.0.0.0) ?

BR

During the handling of a CONNECT request, if the response fails to be written, we will call net/http.ResponseWriter.WriteHeader() on an already-hijacked connection. The call should just be removed (the error is already being logged).

As mentioned in #62 (comment)

Instructions here: https://github.com/golangci/golangci-lint-action

A CONNECT verb used when proxying usually contains just a hostname:port, not a full URL - i.e. it does not contain a scheme.

However, certain code in alpaca expects this to be a URL - in particular, it is passed to shExpMatch as a URL but without the scheme, so any matches against a pattern with a scheme will fail. It seems that chrome does pass a URL here, from anecdotal evidence.

I think we need to add the https scheme if it is missing. I'm not sure it makes sense for any scheme other than https to be used for the tunnel.

The PAC "standard" as defined by Netscape includes a few functions that assume IPv4, so Alpaca currently only supports IPv4.

Microsoft has a few extensions which work better for IPv6 and Google has implemented some of these functions into Chrome (although they make these functions available in the regular FindProxyForURL function, rather than just scoping it to FindProxyForURLEx). These include dnsResolveEx() and myIpAddressEx(), and they both replace functions that return IPv4 addresses only with functions that could return either (or both).

There are also other functions that accept IPv4 addresses, and should be extended to accept IPv6 addresses as well. These include isResolvable() and isInNet().

The only remaining function is convert_addr(). There doesn't appear to be an IPv6 version in Mozilla, Microsoft or Google's implementations, and it doesn't make sense for Alpaca to implement its own extension to the standard, so I'll leave this as it is.

https://github.com/dop251/goja is much faster than Otto. Aside from speed, I have no idea if it's a better choice or even viable.

It would be super useful for alpaca to have a brew tap so it can be easily installed and updated without go installed.

Steps to reproduce:

1. Set up the proxy to go via Alpaca, by following the instructions at: https://cloud.google.com/sdk/docs/proxy-settings#proxy_configuration
2. Run gcloud compute addresses list. If you haven't already done so, it'll ask you to provide your credentials and project name.

The output should look like this:

$ gcloud compute addresses list
Listed 0 items.

But what actually happens is that it just sits there doing nothing.

A long-running Alpaca session ended up not being able to proxy traffic, and had the following error messages in the logs:

2019/06/03 09:15:53 server.go:2979: http: Accept error: accept tcp 127.0.0.1:3128: accept: too many open files; retrying in 5ms
2019/06/03 09:15:55 server.go:2979: http: Accept error: accept tcp 127.0.0.1:3128: accept: too many open files; retrying in 10ms
2019/06/03 09:15:55 server.go:2979: http: Accept error: accept tcp 127.0.0.1:3128: accept: too many open files; retrying in 20ms
2019/06/03 09:15:55 server.go:2979: http: Accept error: accept tcp 127.0.0.1:3128: accept: too many open files; retrying in 40ms
2019/06/03 09:15:55 server.go:2979: http: Accept error: accept tcp 127.0.0.1:3128: accept: too many open files; retrying in 80ms

It looks like Alpaca is leaking connections, maybe due to not closing one half of the CONNECT requests (ref ProxyHandler.handleConnect).

In addition to fixing the leak, there should be better visibility into what's going on in terms of open connections.

            	Error:      	Not equal: 
            	            	expected: false
            	            	actual  : true
            	Test:       	TestWeekdayRange/M-F_UTC_MON
    --- FAIL: TestWeekdayRange/M-F_UTC_SAT (0.00s)
        pacrunner_test.go:327: 
            	Error Trace:	pacrunner_test.go:327
            	Error:      	Not equal: 
            	            	expected: true
            	            	actual  : false
            	Test:       	TestWeekdayRange/M-F_UTC_SAT
    --- FAIL: TestWeekdayRange/SAT_UTC_SUN (0.00s)
        pacrunner_test.go:327: 
            	Error Trace:	pacrunner_test.go:327
            	Error:      	Not equal: 
            	            	expected: true
            	            	actual  : false
            	Test:       	TestWeekdayRange/SAT_UTC_SUN
    --- FAIL: TestWeekdayRange/SAT_UTC_SAT (0.00s)
        pacrunner_test.go:327: 
            	Error Trace:	pacrunner_test.go:327
            	Error:      	Not equal: 
            	            	expected: false
            	            	actual  : true
            	Test:       	TestWeekdayRange/SAT_UTC_SAT
FAIL
FAIL	github.com/samuong/alpaca	0.271s
ok  	github.com/samuong/alpaca/cancelable	0.003s

Will have to investigate more but this is what I'm seeing.

Originally posted by @jamesmoriarty in https://github.com/samuong/alpaca/pull/45/files

According to README.md, install using Go will fail.

$ go get -v -u github.com/samuong/alpaca
go: go.mod file not found in current directory or any parent directory.
        'go get' is no longer supported outside a module.
        To build and install a command, use 'go install' with a version,
        like 'go install example.com/cmd@latest'
        For more information, see https://golang.org/doc/go-get-install-deprecation
        or run 'go help get' or 'go help install'.
$ go install github.com/samuong/[email protected]
go: github.com/samuong/[email protected]: github.com/samuong/[email protected]: invalid version: module contains a go.mod file, so module path must match major version ("github.com/samuong/alpaca/v2")

but install alpaca v1.3.2 is well.

$ go install github.com/samuong/[email protected]
go: downloading github.com/samuong/alpaca v1.3.2
go: downloading golang.org/x/term v0.0.0-20220526004731-065cf7ba2467
go: downloading github.com/gobwas/glob v0.2.3
go: downloading github.com/Azure/go-ntlmssp v0.0.0-20211209120228-48547f28849e
go: downloading github.com/robertkrimen/otto v0.0.0-20211024170158-b87d35c0b86f
go: downloading golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e
go: downloading golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a
go: downloading gopkg.in/sourcemap.v1 v1.0.5

Hi, sorry if I have missed something, but is there a recommended way to store credentials and then run alpaca non-interactively - preferably at start up? Thanks.

/projects/alpaca/authenticator.go:30:2 package golang.org/x/crypto/md4 is deprecated: MD4 is cryptographically broken and should should only be used where compatibility with legacy systems, not security, is the goal. Instead, use a secure hash like SHA-256 (from crypto/sha256). (SA1019)

Is it too soon to depreciate NTLMv1 support?

Alpaca could cache the current proxy.pac and, when it detects that the one being served up is different to the cached version, present the user with a diff and confirm whether to accept the new one, continue with the old one, or exit.

At one level, you kind of expect proxy.pac to be trustworthy, however, in some environments it might be delivered over insecure http and is thus susceptible to MITM attacks. #3

If the proxy.pac is delivered over https, the default could be to always trust new versions, with an option for the paranoid to force diff-prompting even then.

Working out the best place to store logs is complicated.

If the user is running the program, from the console, I suspect that the right place is stdout / stderr (as appropriate).

If, however, they are running it in a "detached" (background, daemon) mode with no associated console (or via some agent), a more reasonable default would seem to be $XDG_STATE_HOME (see: https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html).

MS Teams is unable to make or receive calls through alpaca when used as a system wide proxy.

Serve a "copy" of the PAC file that alapca is using that is wrapped with:

function FindProxyForURL(url, host) {
  return FindProxyForURL(url, host) === "DIRECT" ? "DIRECT" : "PROXY localhost:{{.Port}}";
{{.PAC}}
}

where {{.Port}} is the port that alpaca is listening on and {{.PAC}} is the PAC file that alpaca has fetched and is using.

This can be served on /alpaca.pac by alpaca.

This allows you to configure your system to use http://localhost:3128/alpaca.pac as the PAC file and know what URLs should be not proxied (DIRECT). All others are sent to alpaca which runs the unwrapped PAC file to proxy the request to the right place.

If you were to direct all HTTP(s) requests to alpaca in the system proxy settings, the system would use the proxy protocol always, even if alpaca is just going to go DIRECT itself. Some programs don't seem to like that they go through a proxy, so this feature would allow them to continue to go direct to their URL target.

On a Mac, one specific use case that this feature resolves is Citrix remote desktop not working when you proxy all connections via alpaca, but does work when having the system use a PAC file that does the wrapping I've provided above.

Serving files and proxy requests on the same HTTP port is possible because proxy requests all use the absolute-form (https://tools.ietf.org/html/rfc7230#section-5.3.2) and resource requests to an origin server use the origin-form (https://tools.ietf.org/html/rfc7230#section-5.3.1). alpaca can differentiate between a proxy request that it should forward and a request for a resource it serves by looking at the request-target (https://tools.ietf.org/html/rfc7230#section-5.3) and determining the form.

This would also allow other features (to be raised as separate issues if desired) such as serving the raw pac file itself (on /proxy.pac) that alpaca is using and allowing custom wrapping of PAC files by supplying templates similar to above by users who wish to customise the PAC routing (on /wrapped.pac).

… where practicable.

Steps to reproduce:

1. On a Windows machine, open up Git Bash
2. Launch Alpaca using /path/to/alpaca -d mydomain

Expected behaviour:

Alpaca should prompt for the user's password and use that to authenticate to the proxy.

Actual behaviour:

The password prompt is printed, followed by an error message:

main.go:52: Error reading password from stdin: The handle is invalid.

Workaround:

Run Alpaca from a Command Prompt (cmd.exe) window.

This is broken even when the gateway URL resolves to "DIRECT" in the PAC file. The workaround is to add the gateway host to the proxy bypass list in macOS System Preferences.

Even when NoMAD is installed and configured to use the system keychain, Alpaca fails to read the NoMAD configuration and get the credentials out of the keychain. It disables proxy authentication as a result:

main.go:58: NoMAD configuration key not found. Disabling proxy authentication.

When NoMAD app preferences are managed by a configuration profile which means they get written to /Library/Managed Preferences/$USER/com.trusourcelabs.NoMAD.plist. When a user then logs into NoMAD, their particular details get written to /Users/$USER/Library/Preferences/com.trusourcelabs.NoMAD.plist. In this case, UseKeychain gets defined in Managed Preferences and UserPrincipal gets defined in the user's Preferences directory.

Alpaca uses the defaults command to read the configuration, and it looks like this is only reading the plist file in the user's Preferences directory.

A workaround is to manually use defaults write to set UseKeychain to 1 in the user's plist:

defaults write com.trusourcelabs.NoMAD UseKeychain 1

Prior to #41, some of the unit tests for NoMAD integration contained assertions on the message within an error. With these removed, the tests just check for the presence of an error, without checking what type.

This should be changed to check the error type, using the new errors.As function, introduced in Go 1.13.

Additional context can be found in the comments of #41.

Seeing a few of these...

2020/06/10 15:46:48 proxy.go:146: [16301] Error reading CONNECT response: read tcp xx.xx.xx.xx:xxxx->xx.xx.xx.xx:xxxx: read: operation timed out

They appear to block all other connections through alpaca when they occur.

Hello @samuong,

i tried to the latest release of alpaca up and running. it detects the proxy pac due -C command correctly and direct calls are working fine. call through the external proxy are giving us following error message:

2022/04/29 16:32:12 proxyfinder.go:125: [1] CONNECT //www.google.com:443 via "PROXY proxy.domain.com:8080"
2022/04/29 16:32:12 proxy.go:153: [1] Got "407 Proxy authentication required" response, retrying with auth
2022/04/29 16:32:12 authenticator.go:56: Error decoding NTLM Type 2 (Challenge) message: illegal base64 data at input byte 8

we are using alpaca on windows 10 (Version 20H2)

it seems like our proxy does not return any valid Proxy-Authenticate header... i will try to get more informations together on monday with the help of our proxy admins. maybe you already got an idea whats the problem.

is there any way to give more detailed informations about the response of the proxy?

kind regards,
julian