sethhall / bro-myricom

Bro plugin to receive packets through the Myricom Sniffer API

Bro::Myricom

This plugin provides native Myricom SNF v3+v4 support for Bro.

Bro-pkg Installation

Make sure you have the SNF library installed and then run:

bro-pkg install sethhall/bro-myricom

Manual Installation

Follow Myricom's instructions to get its kernel module and userspace libraries installed, then use the following commands to configure and build the plugin.

After building bro from the sources, change to the "aux/plugins/myricom" directory and run:

./configure --with-myricom=<path to sniffer sources> --bro-dist=<path to bro sources>
make && sudo make install

Note: In most cases, if you are building this plugin from the Bro source tree, you won't need any configure arguments.

If everything built and installed correctly, you should see this:

# bro -N Bro::Myricom
Bro::Myricom - Packet acquisition via Myricom SNF v3+v4 (dynamic, version 1.0)

You may run Bro as unprivileged user.

Usage

Once installed, you can use Myricom interfaces/ports by prefixing them with myricom:: on the command line. For example, to use Myricom SNF to monitor interface p2p1:

bro -i myricom::p2p1

To use it in production with multiple Bro processes, use a configuration similar to this in node.cfg:

[worker-1]
type=worker
host=localhost
lb_method=custom
lb_procs=<number of processes, like 16>
interface=myricom::<interface name, like p2p1>

If you would like to sniff all Myricom interfaces on a system and merge them together, there is a special interface name available of myricom::*. It uses a special feature of the Myricom SNF library for port aggregation.

To run a cluster sniffing all Myricom interfaces on a system, you can use the same configuration as above, but with the special interface name that aggregates all Myricom ports like this:

[worker-1]
type=worker
host=localhost
lb_method=custom
lb_procs=<number of processes, like 16>
interface=myricom::*

Tuning

You may wish to tune the amount of memory used for the global packet buffer. This setting is available in the Bro script interface to the plugin, but it's also available as a global option in broctl.cfg or as a per-node option in node.cfg. The following line in either config file will set the SNF packet buffer ring size to 16GB (the default is 1GB):

myricom.snf_ring_size=16384

Enjoy!

bro-myricom's Issues

Please tag 1.0.3 :-)

Since #2 was merged there hasn't been a new release tagged and the latest(1.0.2) is missing that fix.

Add compatibilitiy with the Myricom Sniffer v5 Drivers

Please add compatibility with the Myricom Sniffer v5 drivers API...this would be amazing as it would bring in support for better Myricom cards (4 port cards etc.) which would be extremely beneficial to many people. Thank you!

Below is the link to where you can find the Myricom SNFv5 driver API reference guide and SNFv5 user guide assuming that might be helpful:

https://www.cspi.com/ethernet-products/support/downloads/

odd dependency on .bro-pkg contents for workers in cluster environment.

So, as I mentioned earlier today, the plugin wasn't working on our cluster, giving several errors like these in 'broctl diag':

error in /usr/local/bro/spool/installed-scripts-do-not-touch/auto/broctl-config.bro, line 18: "redef" used but not previously defined (Myricom::snf_ring_size)
internal warning in /usr/local/bro/spool/installed-scripts-do-not-touch/auto/broctl-config.bro, line 18: Can't document redef of Myricom::snf_ring_size, identifier lookup failed

Looking deeper, I found that "bro -N Bro::Myricom" was failing on the workers. I resolved this by installing bro-pkg and 'failing' to install the myricom plugin, but wasn't sure about what actually fixed the problem until I tried this:

1. Delete the .bro-pkg directory... the check fails.
   [rootmd@bro-lbl3 ~bro]# rm -rf .bro-pkg/
   [rootmd@bro-lbl3 ~bro]# su - bro
   Last login: Mon Sep 18 15:58:18 PDT 2017 on pts/1
   [bro@bro-lbl3 ~]$ bro -N Bro::Myricom
   internal error: internal variable Myricom::snf_ring_size missing
   Aborted

2. Attempt to install the plugin, but don't let it proceed after the test suite fails:
   [bro@bro-lbl3 ~]$ bro-pkg refresh
   Refresh package source: bro
   No changes
   Refresh installed packages
   No new outdated packages
   [bro@bro-lbl3 ~]$ bro-pkg install bro/sethhall/bro-myricom
   The following packages will be INSTALLED:
   bro/sethhall/bro-myricom (1.0.2)

Proceed? [Y/n] y
Running unit tests for "bro/sethhall/bro-myricom"
error: failed to run tests for bro/sethhall/bro-myricom: package build_command failed, see log in /home/bro/.bro-pkg/logs/bro-myricom-build.log
Proceed to install anyway? [Y/n] n
Abort.
[bro@bro-lbl3 ~]$ bro -N Bro::Myricom
internal error: internal variable Myricom::snf_ring_size missing
Aborted

3. Attempt to install the plugin, but let it try to install anyway (which it claims fails):

[bro@bro-lbl3 ~]$ bro-pkg install bro/sethhall/bro-myricom
The following packages will be INSTALLED:
bro/sethhall/bro-myricom (1.0.2)

Proceed? [Y/n] y
Running unit tests for "bro/sethhall/bro-myricom"
error: failed to run tests for bro/sethhall/bro-myricom: package build_command failed, see log in /home/bro/.bro-pkg/logs/bro-myricom-build.log
Proceed to install anyway? [Y/n] y
Failed installing "bro/sethhall/bro-myricom": package build_command failed, see log in /home/bro/.bro-pkg/logs/bro-myricom-build.log
[bro@bro-lbl3 ~]$ bro -N Bro::Myricom
Bro::Myricom - Packet acquisition via Myricom SNF v3+v4 (dynamic, version 1.0)

Now it works.

plugin fails to build with 3.1.0

The configure script refers to bro-config, which fails.

Also, the iosource changes appear to break the build process :

bro-myricom/src/Plugin.cc:10:34: error: no type named 'PktSrcComponent' in namespace 'iosource'
AddComponent(new ::iosource::PktSrcComponent("MyricomReader", "myricom", ::iosource::PktSrcComponent::LIVE, ::iosource::pktsrc::MyricomSource::InstantiateM...

bro workers are very slow to stop on FreeBSD

Using FreeBSD 10.3 and the myricom plugin. Myricom driver version : 3.0.12.50830 .

When I attempt "broctl stop" I frequently have to try several times as workers do not stop on the first attempt (or even 2nd, or 3rd attempt). The processes also don't respond to kill signals when I try to kill them directly. I am using the default ring size.

Tests fail due to version number change

The tests run via bro-pkg install fail because the version number referenced changed from 1.0 to 1.0.0:

[zeek@dumbledore ~]$ bro-pkg install bro-myricom
The following packages will be INSTALLED:
  bro/sethhall/bro-myricom (1.0.4)

Proceed? [Y/n] y
Running unit tests for "bro/sethhall/bro-myricom"
[  0%] myricom.show-plugin ... failed
  % 'btest-diff output' failed unexpectedly (exit code 1)
  % cat .diag
  == File ===============================
  Bro::Myricom - Packet acquisition via Myricom SNF v3+v4 (dynamic, version 1.0.0)
      [Packet Source] MyricomReader (interface prefix "myricom"; supports live input)
      [Constant] Myricom::snf_ring_size
      [Constant] Myricom::snf_num_rings
      [Constant] Myricom::snf_app_id
      [Constant] Myricom::snf_aggregate
      [Constant] Myricom::snf_rss_mode
      [Type] Myricom::RssField
  
  == Diff ===============================
  --- /tmp/test-diff.448175.output.baseline.tmp 2019-01-08 01:55:03.133835830 +0000
  +++ /tmp/test-diff.448175.output.tmp  2019-01-08 01:55:03.142835927 +0000
  @@ -1,4 +1,4 @@
  -Bro::Myricom - Packet acquisition via Myricom SNF v3+v4 (dynamic, version 1.0)
  +Bro::Myricom - Packet acquisition via Myricom SNF v3+v4 (dynamic, version 1.0.0)
       [Packet Source] MyricomReader (interface prefix "myricom"; supports live input)
       [Constant] Myricom::snf_ring_size
       [Constant] Myricom::snf_num_rings
  =======================================

  % cat .stderr

1 of 1 test failed

I think just changing the baseline in this file should solve this issue, but I'll let someone with more experience than myself make that call.