Support running Antrea in clusters created with Kind about antrea

[antoninbas]

Tentative support in #32

[antoninbas]

With the current support, the Antrea components (agent, controller) can come up. However, there is no connectivity between Nodes. At this time, I believe that this is because using VXLAN tunnels in OVS userspace mode requires some special configuration: http://docs.openvswitch.org/en/latest/howto/userspace-tunneling/. I will work on this.

[antoninbas]

It seems that using OVS in userspace mode also requires to explicitly disable TX offloading for each Pod's eth0 interface. Otherwise all TCP traffic going through the gateway is dropped (and probably Pod to Pod traffic as well). This is something I observed when working on support for Kind. All the packets going through the gateway were dropped and Pods couldn't reach the K8s API server.

Found this link, which is not part of OVS documentation: https://arthurchiao.github.io/blog/ovs-deep-dive-5-datapath-tx-offloading/

[antoninbas]

Found a good reference for the checksum issue: https://bugzilla.redhat.com/show_bug.cgi?id=1685616

The issue here is that OVS netdev datapath doesn't
support TX checksum offloading (this is not easy task with arguable profit).
i.e. if packet arrives with bad/no checksum it will be sent to the output port
with same bad/no checksum. Everything works in case of kernel datapth because
the packet doesn't leave the kernel space. In case of netdev datapath some
information (like CHECKSUM_VALID skb flags) is lost while receiving via
socket in userspace and subsequently kernel expects valid checksum while
receiving the packet from userspace because TX offloading is not enabled.

This kind of issues usually mitigated by disabling TX offloading on the
"right*" interfaces, or by setting iptables to fill the checksums like this:

iptables -A POSTROUTING -t mangle -p udp -m udp -j CHECKSUM --checksum-fill

Some related OpenStack bug: https://bugs.launchpad.net/neutron/+bug/1244589

Also, note that this happens only for virtual interfaces like veth/tap because
kernel always tries to delay checksum calculation/validation as much as possible.
Correct packets received from the wire will always have correct checksums.

[trozet]

@antoninbas Hi, I've been working on adding similar support into OVN. I wanted to ask specifically why multiple OVS in separate containers cannot utilize the same kernel path? If each OVS is in its own namespace with its own unique DPID, will there be conflicts in kernel path? Thanks.

[williamtu]

I think it doesn't work, but in reality, I do see people running multiple ovs-vswitcd in multiple
containers sharing one ovs kernel datapath, without any problem. I guess it depends on use cases.

There is a talk about this in 2015 mentioning a couple of issues
https://www.openvswitch.org/support/ovscon2015/17/1555-benc.pdf

[antoninbas]

@trozet I believe you can make it work, but I also think that wasn't the best option for the Antrea case:

• the OVS bridge for each Kind Node needs to have a different name (and I believe so does the host gateway interface), which means that we have to tinker with some Antrea configuration files to make this happen.
• I don't know how easy it would be to make it work on macOS (or if it's even possible with some reasonable effort), I doubt that the OVS kernel module is available out-of-the-box in HyperKit.

And as William pointed-out, there may be some other issues on top of that. Of course, using the userspace datapath also comes with its own issues :)