bug(windows): System DNS servers should be removed if connlib sends sentinels about firezone

[ReactorScram]

I'll ask ChatGPT about how DNS works on Windows. If we can shadow everything yeah that's probably best cause timeouts wouldn't get cached.

[jamilbk]

Yeah, this will happen any time a Gateway flaps, the Relays are being restarted, or the Portal is being restarted since those all can cause a timeout when the client tries a lookup.

[jamilbk]

I think it was done like this because we don't know how many of each IPv4/IPv6 dummy addresses to generate until we do an actual lookup, and also because we don't have a good way to reverse the mapping when it reaches a Gateway.

If we immediately generated 100.96.100.100 for example, we would still need some way to tell whichever Gateway we land on that 100.96.100.100 corresponds to the original DNS name requested, and then do the actual lookup.

We should probably review this architecture

cc @conectado

[ReactorScram]

Yeah that would explain some things I've seen lately when trying the Windows Client.

I'll see if there's any other way to handle it. I don't like removing the system DNS servers because then it'll be a mix of the iOS resolver dance and the /etc/resolv.conf thing, where we have to remove the servers, but also keep them safe, and revert them, and if we crash the user is in deep trouble, and if DHCP keeps in we have to re-remove them and update our stashed servers.

Could connlib's DNS have a more aggressive timeout and lie and say the domain doesn't exist? Then traffic can't escape the tunnel.

[jamilbk]

Ah I see, on other platforms the nameservers are reverted when bringing the tunnel back down. Maybe Windows has a similar option?

Could connlib's DNS have a more aggressive timeout and lie and say the domain doesn't exist?

Yeah maybe that's a better option. If a Gateway is flapping or no Gateways are online what should connlib do though? I think it's probably most appropriate to timeout instead of incorrectly returning an NXDOMAIN which the app might cache.

[ReactorScram]

I'm not sure exactly how it works on Windows but we want something that shadows the system resolvers without removing them from their own interfaces. That was frustrating about the /etc/resolv.conf thing on Linux.

[ReactorScram]

Yeah replicates for me on the dev laptop on c036d1a (tip of main)

Even after I enable the Policy again, it's stuck outside the tunnel.

[ReactorScram]

And there was a reason why we can't assign an IP and respond to the DNS query before the gateway responds to us?

I remember asking this and I think Gabi said there was a good reason, but I can't remember. If we could just always assign an IP before knowing whether the Resource was even reachable, it would avert this - The DNS query would come back in milliseconds and then the connect() call will just time out harmlessly.

[ReactorScram]

ChatGPT suggested blocking traffic to the system's other DNS servers 🤔
Maybe we could claim their routes. Would that cause a packet loop when we try to send traffic to an IP address we've already claimed?

[ReactorScram]

Yeah I think it was about the number of IPs we get back.

I could poke around in the DNS code and try things like returning NXDOMAIN or SERVFAIL if we run out of time. ChatGPT thought SERVFAIL might be treated as a temporary error and the resolver would try again, but I haven't found MS docs to prove it yet

[jamilbk]

Claiming routes or adding firewall rules might be a creative way to solve it.

I don't think it'd cause a loop -- we'd either drop the packet in connlib or if it's a resource it would be sent through a tunnel anyway

[ReactorScram]

I was thinking if we use the system resolvers and we also claim their routes, we won't be able to reach them. Unless we have a way to bypass our own routes

[jamilbk]

If we add them as routes and ensure our tun interface has metric priority, they should be effectively blackholed by connlib for the duration of the tunnel

[ReactorScram]

Yeah so connlib can't reach them if connlib is blackholing them
Like my 192.168.1.1 DNS cache on my home router will be unreachable because connlib tries to send it a query and it just comes back to connlib

[jamilbk]

Ah, I'm following now. Yeah this seems tricky on Windows. On Linux we have fwmark, Android protect, and Apple NECP that handles this nicely for us.

Can think more about it, but unless we make major changes in connlib's DNS proxy I don't know if responding with NXDOMAIN or SERVFAIL won't cause unexpected application behavior. How long are those cached for? If a DNS server doesn't respond, we're at least pretty confident the application will ask again and again on each request.

It might not be perfect but we could do this:

but also keep them safe, and revert them, and if we crash the user is in deep trouble, and if DHCP keeps in we have to re-remove them and update our stashed servers.

and just run the risk of them not being reverted.

I would imagine our likelihood of crashing is lower than the likelihood of customers' Gateways going down or flapping