What's the performance of xdpcap? about xdpcap

[arthurfabre]

Error: lost 1 packets is reported directly by the perf system, it indicates that the perf buffer was full when the XDP program tried to copy a packet into it.

You can try increasing the size of the buffer (-buffer), and also maybe the watermark (-watermark). man perf_event_open has more info on wakeup_watermark.

[marshalWS]

Error: lost 1 packets is reported directly by the perf system, it indicates that the perf buffer was full when the XDP program tried to copy a packet into it.

You can try increasing the size of the buffer (-buffer), and also maybe the watermark (-watermark). man perf_event_open has more info on wakeup_watermark.

Thanks very much !^_^. I get it and run success.

[marshalWS]

resolved!

[marshalWS]

Error: lost 1 packets is reported directly by the perf system, it indicates that the perf buffer was full when the XDP program tried to copy a packet into it.

You can try increasing the size of the buffer (-buffer), and also maybe the watermark (-watermark). man perf_event_open has more info on wakeup_watermark.

First of all thank you for taking the time to browse my questions！
I found another problem。
background: Xdpcap filters 100Mb packets from 1Gbps.

I add a var all in main.go for packets statistic.
such as:
var all int64 = 1
for{
_, err := filter.read()
all++
atomic.StoreInt64(&inall, all)
if err !=nil{
fmt.Fprintln(os. Stderr, "Error:", err)
}
}
。。。。

statistic code：
aa := atomic.LoadInt64(&inall)
fmt.Printf("--user model stats inall=%d pkts. avg=%f pa \n", aa, ....)

1:
Xdpcap can't read any packet while I set -watermark = 8191 (-buffer = 8192 )。
It seems that cilium code was blocked(cilium.ebpf.perf.reader.go func(pr* Reader)Read (Record, error) ).

root@xxxxxx:/xxx/xxx/xxx# ./xdpcap -watermark 8191 /sys/fs/bpf/xdp/globals/action_map pp.pcap "src 168.168.1.50"

[1] secs...
-- kernel mode> aborted: 0/0 drop: 243979/19717 pass: 0/0 tx: 0/0 redirect: 0/0 (received/matched packets)
--user model stats inall=0 pkts. avg=0.000000 ps

[2] secs...
-- kernel mode> aborted: 0/0 drop: 488084/39488 pass: 0/0 tx: 0/0 redirect: 0/0 (received/matched packets)
--user model stats inall=0 pkts. avg=0.000000 ps

2: There have lost packets while I set -waterbuffer = 8000
root@xxxxxx:/xxx/xxx/xxx# ./xdpcap -watermark 8000 /sys/fs/bpf/xdp/globals/action_map pp.pcap "src 168.168.1.50"
Error: lost 4 packets
Error: lost 1 packets
Error: lost 19 packets

3: I try to resize -buffer size. But a Error happened！

root@xxxxxx:/xxx/xxx/xxx##./xdpcap v -buffer 10240 -watermark 8191 /sys/fs/bpf/xdp/globals/action_map pp.pcap "src 168.168.1.50"
Error: creating filter: can't create perf event reader: failed to create perf ring for CPU 0: invalid argument

So, where is the problem.
Thanks！

[arthurfabre]

1:
Xdpcap can't read any packet while I set -watermark = 8191 (-buffer = 8192 )
It seems that cilium code was blocked(cilium.ebpf.perf.reader.go func(pr* Reader)Read (Record, error) ).

Thanks for the report, that's unexpected. I'll have a look.

2: There have lost packets while I set -waterbuffer = 8000

In this case with -watermark 8000, and the default -buffer 8192, xdpcap will only get notified there is something in the buffer once it contains 8000 bytes. This doesn't leave much time for xdpcap to empty the buffer before it becomes full. I don't have a good feel for how to tune these for the best performance, but I think setting -watermark to half of -buffer is a good starting point.

3: I try to resize -buffer size. But a Error happened

I'll have a look at this as well, thanks for letting me know.

[marshalWS]

1:
Xdpcap can't read any packet while I set -watermark = 8191 (-buffer = 8192 )
It seems that cilium code was blocked(cilium.ebpf.perf.reader.go func(pr* Reader)Read (Record, error) ).

Thanks for the report, that's unexpected. I'll have a look.

2: There have lost packets while I set -waterbuffer = 8000

In this case with -watermark 8000, and the default -buffer 8192, xdpcap will only get notified there is something in the buffer once it contains 8000 bytes. This doesn't leave much time for xdpcap to empty the buffer before it becomes full. I don't have a good feel for how to tune these for the best performance, but I think setting -watermark to half of -buffer is a good starting point.

3: I try to resize -buffer size. But a Error happened

I'll have a look at this as well, thanks for letting me know.

---

I'm sorry that I didn't reply in time. I spent a lot of time building the environment to test accurately.
I'll follow up on this.

After learning, I learned about the watermark parameter. If this parameter is equal to 4000, does it mean that xdpcap will not receive the notified message if the number of the last batch of packets does not reach 4000.

Does it mean that the last batch of packet is lost?
Is the last batch of packet not available in user mode ？

[arthurfabre]

No worries :)

1:
Xdpcap can't read any packet while I set -watermark = 8191 (-buffer = 8192 )
It seems that cilium code was blocked(cilium.ebpf.perf.reader.go func(pr* Reader)Read (Record, error) ).

It turns out this is expected. The kernel will only notify xdpcap that it can read a packet when the number of bytes in the buffer exceeds the watermark. With watermark = 8191 and buffer = 8192, the buffer would have to contain 8192 bytes for the kernel to notify xdpd. However the size of the captured packets is unlikely to add up to 8192 exactly. The buffer is likely to end up almost but not quite full, with not enough room to accept any new packets.

3: I try to resize -buffer size. But a Error happened

This should be fixed by #40 once it's merged

If this parameter is equal to 4000, does it mean that xdpcap will not receive the notified message if the number of the last batch of packets does not reach 4000.
Does it mean that the last batch of packet is lost?

Yes, that is correct. That's why by default watermark = 0. Userspace could read out the last batch of packets, but we don't currently support it.

[arthurfabre]

I forgot about this, but the PR is finally merged. Feel free to re-open this if it doesn't work as expected.