rkosegi / netflow-collector Goto Github PK

I'm trying to come up with a dashboard for internal hosts' traffic monitoring using NetFlow data.

I have OPNsense sending NetFlow V5 to netflow-collector which exports metrics to Prometheus. At the same time I have ntopng installed and sending metrics to InfluxDB.

Somehow I have inconsistent results with NetFlow, so I'm wondering if I'm missing something here. With my bandwidth tests I get consistently 200-400 MBps down/up (down usually lower).

With InfluxDB this is the graph with already calculated speed (in mbps):

But NetFlow metrics are very inconsistent and all over the place (sometimes there are no spikes for up like on this graph):

This graph has raw values in bytes returned from Prometheus as they were sent by netflow-collector.

FWIW my scrape and evaluation intervals are 15 in Prometheus and flush interval is 120s in netflow-collector.

Is there a missing link here?

Hello! Is it possible for this exporter to show metrics by source addresses? Now with a simple config we get data by AS and countries. We would like to view information by source addresses. If this is possible, please provide an example of the config.

Hello, thanks for this collector!

I noticed a few panics with specific configuration.

Adding bytes or packets as labels like:

          - name: bytes
            value: bytes
            converter: uint32

results in:

panic: interface conversion: interface {} is uint64, not uint32

as well as when converter is invalid.

For some reason in my setup it doesn't work, even though it seems netflow-collector processes yaml config properly.

I have:

pipeline:
  filter:
    - local-to-local: true
    - match: source_ip
      is: 0.0.0.0
    - match: source_ip
      is: 255.255.255.255
    - match: destination_ip
      is: 0.0.0.0
    - match: destination_ip
      is: 255.255.255.255
  enrich:
    - interface_mapper
    - protocol_name
  metrics:
    prefix: netflow
    items:
      - name: traffic_detail
        description: Traffic detail
        labels:
          - name: sampler
            value: sampler
            converter: ipv4
          - name: source_ip
            value: source_ip
            converter: ipv4
          - name: destination_ip
            value: destination_ip
            converter: ipv4
          - name: protocol
            value: proto_name
            converter: str
          - name: input_interface
            value: input_interface
            converter: uint32
          - name: output_interface
            value: output_interface
            converter: uint32
extensions:
  interface_mapper:
    0: if0
    1: if1
    2: if2
    5: if5

input_interface_name and output_interface_name don't show up in metrics.