IPaddr2 may fail on LVS IPv6 configuration about resource-agents

[dmuhamedagic]

On Sun, Nov 18, 2012 at 11:04:07PM -0800, Keisuke MORI wrote:

IPaddr2 may failed to assign an IPv6 address with 'dadfailed' status on LVS configuration, which has the same IPv6 address on lo too.

I consider that this is a blocker for the release.

The steps to reproduce is below. Step 4a. or 4b. is a failed scenario. Step 4c. is a succeeded scenario. (thanks to @nozawatm for testing it)

Diagnosis:

It fails when send_ua is called if the VIP is still 'tentative' status (accomplishing the assignment in the kernel).

The cause seems that send_ua is sending ping for waiting the tentative flag is disappeared (i.e. assingment has finished) but the ping packet makes IPv6 DAD (duplicate address detection) protocol fail in the case of LVS configuration.

The kernel should know where the ICMP packets are coming from.
Did you watch the network with tcpdump?

(L317-L322 in IPaddr2.c)
17d9f6a#L0R317

One strange thing is that this problem was not observed on RHEL5.7 (send_ua suceeds even if it was tentative status). We are unsure that if the kernel behavior does matter or not, but anyway we have to fix it somehow.

Neither does it happen with SLE11SP2, kernel 3.0.42. I guess it
does depend on the kernel.

Solution:

I think we should take another way for waiting to finish to assign the IPv6 address. Probably we are going to remove the ping loop in send_ua and alternatively use ip command to check tentative flag in the RA.

I guess that that's also a possibility.

[kskmori]

The cause seems that send_ua is sending ping for waiting the tentative flag is disappeared (i.e. assingment has finished) but the ping packet makes IPv6 DAD (duplicate address detection) protocol fail in the case of LVS configuration.

The kernel should know where the ICMP packets are coming from. Did you watch the network with tcpdump?

Yes (attached below). Ping packets themselves work as usual. The problem is a neighbor solicitation packet for DAD can not be seen on the wire in the case of the failed scenario. I suspect that succeeding ping to lo makes the kernel think as the address has already assigned hence DAD fails.

One strange thing is that this problem was not observed on RHEL5.7 (send_ua suceeds even if it was tentative status).

Let me correct this; RHEL5.7 did not work properly either. 'dadfailed' was not displayed on RHEL5.7 but the kernel log reports that it has detected the address duplication and the 'tentative' status remains. The connection between other nodes may work but apparently it is not right status.

I think we should take another way for waiting to finish to assign the IPv6 address. Probably we are going to remove the ping loop in send_ua and alternatively use ip command to check tentative flag in the RA.

I guess that that's also a possibility.

I and @nozawatm are now testing the patch along with this.
Please allow us for one more day.

Regards,

---

Failed scenario. (4b.)

# tcpdump -i lo -n icmp6
10:16:54.490512 IP6 2001:db8:100::101 > 2001:db8:100::101: ICMP6, echo request, seq 0, length 64
10:16:54.490526 IP6 2001:db8:100::101 > 2001:db8:100::101: ICMP6, echo reply, seq 0, length 64

# tcpdump -i eth0 -n icmp6
10:16:54.491331 IP6 2001:db8:100::101 > ff02::1: ICMP6, neighbor advertisement, tgt is 2001:db8:100::101, length 32
(5 times)

Succeeded scenario. (4c.)

# tcpdump -i lo -n icmp6
10:18:06.904199 IP6 2001:db8:100::101 > 2001:db8:100::101: ICMP6, echo request, seq 0, length 64
10:18:06.904208 IP6 2001:db8:100::101 > 2001:db8:100::101: ICMP6, echo reply, seq 0, length 64

# tcpdump -i eth0 -n icmp6
10:18:04.534835 IP6 :: > ff02::1:ff00:101: ICMP6, neighbor solicitation, who has 2001:db8:100::101, length 24
10:18:06.904327 IP6 2001:db8:100::101 > ff02::1: ICMP6, neighbor advertisement, tgt is 2001:db8:100::101, length 32
(5 times)

[dmuhamedagic]

On Tue, Nov 20, 2012 at 02:47:48AM -0800, Keisuke MORI wrote:

I and @nozawatm are now testing the patch along with this.
Please allow us for one more day.

Of course.

[kskmori]

Closed as the patch is submitted in pull request #181.

[kskmori]

Just for an additional note:

According to our kernel experts, this issue is fixed by the commit below so the recent kernel should not be affected:

• http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=patch;h=bd015928bb1713691068c4d0d159afccbaf0f8c0

Looking at tentative status should be more reliable than pinging anyway.