some TCP services with proxyProtocol.trustedIPs broken in 3.0.0-rc1 about traefik

[dani]

Forgot to add : even with rc1, both the router and the service appear in Traefik dashboard as if everything were OK. Only the routing is broken

[dani]

mmm, after some more debugging, the problem is not with the service mesh support. But with proxyProtocol.trustedIPs on the entrypoint (I set this as 10.99.9.1 is my HAProxy which can expose services from the outside, working as an L4 proxy, so I need the proxyProtocol support to get the real client IP).

Not sure what's special about it, because the same proxyProtocol.trustedIPs is also used on other entrypoints (like one for a postgres service, and several http services), but removing it makes my mariadb instance accessible again

[rtribotte]

Hello @dani,

Thanks for reporting this!

We are unsure about a few things to be able to reproduce the issue:

mmm, after some more debugging, the problem is not with the service mesh support. But with proxyProtocol.trustedIPs on the entrypoint (I set this as 10.99.9.1 is my HAProxy which can expose services from the outside, working as an L4 proxy, so I need the proxyProtocol support to get the real client IP).

When you are trying to connect to MariaDB is this through HAPRoxy?

Not sure what's special about it, because the same proxyProtocol.trustedIPs is also used on other entrypoints (like one for a postgres service, and several http services), but removing it makes my mariadb instance accessible again

Did you remove the whole proxyProtocol configuration or just the trustedIPs configuration?

Can you please share the full DEBUG logs of Traefik when reproducing the problem?

[dani]

Ok, so here are some more info

• I encounter the issue when I connect directly, not using HAProxy (this particular service is not yet exposed by HAProxy, I just configure all my entrypoints like this so if I need to expose it, everything is ready)
• trustedIPs is the only attribute under proxyProtocol, so I have to remove the whole proxyProtocol node (as specifying an empty node as proxyProtocol: {} is invalid and Traefik fails to start)
• The only logs in DEBUG are (filtered, as this Traefik handles quite a lot of other, unrelated trafic)

2024-02-19T16:15:25+01:00 DBG github.com/traefik/traefik/v3/pkg/middlewares/ipallowlist/ip_allowlist.go:85 > Accepting IP 192.168.7.32 middlewareName=ip-trusted@file middlewareType=IPAllowLister
2024-02-19T16:15:26+01:00 DBG github.com/traefik/traefik/v3/pkg/tcp/proxy.go:41 > Handling TCP connection address=10.99.9.11:23953 remoteAddr=10.99.20.2:35772
2024-02-19T16:15:31+01:00 DBG github.com/traefik/traefik/v3/pkg/server/server_entrypoint_tcp.go:452 > IP 10.99.20.2 is not in trusted IPs list, ignoring ProxyProtocol Headers and bypass connection entryPointName=mariadb

Note :

• I did set a IPAllowList middleware on this service (omitted when I opened the issue because it's not relevant, the issue is the same with or without this middleware)
• 10.99.20.2 is the IP of the client trying to connect (and getting a timeout)
• 10.99.9.11:23953 is the correct address for the backend service (IP:port of the envoy proxy in the service mesh)

[rtribotte]

Thanks for your feedback!

We are able to reproduce the problem, configuring the readTimout option on the entryPoint should do the trick.

Stay tuned, we are working on a fix.