ahlashkari / cicflowmeter

CICFlowmeter-V4.0 (formerly known as ISCXFlowMeter) is an Ethernet traffic Bi-flow generator and analyzer for anomaly detection that has been used in many Cybersecurity datsets such as Android Adware-General Malware dataset (CICAAGM2017), IPS/IDS dataset (CICIDS2017), Android Malware dataset (CICAndMal2017) and Distributed Denial of Service (CICDDoS2019).

Install jnetpcap local repo

for linux, sudo is a prerequisite

//linux :at the pathtoproject/jnetpcap/linux/jnetpcap-1.4.r1425
//windows: at the pathtoproject/jnetpcap/win/jnetpcap-1.4.r1425
mvn install:install-file -Dfile=jnetpcap.jar -DgroupId=org.jnetpcap -DartifactId=jnetpcap -Dversion=1.4.1 -Dpackaging=jar

Run

IntelliJ IDEA

open a Terminal in the IDE

//linux:
$ sudo bash
$ ./gradlew execute

//windows:
$ gradlew execute

Eclipse

Run eclipse with sudo

1. Right click App.java -> Run As -> Run Configurations -> Arguments -> VM arguments:
-Djava.library.path="pathtoproject/jnetpcap/linux/jnetpcap-1.4.r1425"  -> Run

2. Right click App.java -> Run As -> Java Application

Make package

IntelliJ IDEA

open a Terminal in the IDE

//linux:
$ ./gradlew distZip
//window
$ gradlew distZip

the zip file will be in the pathtoproject/CICFlowMeter/build/distributions

Eclipse

At the project root

mvn package

the jar file will be in the pathtoproject/CICFlowMeter/target

cicflowmeter's Issues

can't find bin directory

I can't find bin directory ,how to execute? please help me

CIC-IDS-2017: clm is not extracting FWD HEADER LENGTH

Hi, I am using CIC-IDS-2017,
and I found that Provided csv files contain Forward header length,
However, it this feature is not calculated using current version of the code.

I just wonder if you remove this intentionally or it is a mistake

Partial Flows in the final CSV

CSV generated has flow entries with same ID, and few flows has Flow Duration == 0 micro seconds.
By default flow timeout is set to 120 sec, and few discussion I read it should be 600 seconds. Please suggest me what should be the idle flow timeout and activity timeout.
Is the timeout set in the code affecting in number of flows being generated.
what should be the SYN and FIN flag count in an idle flow?

CSV headers Document

Is there a detailed description of the headers for generating CSV for this tool ?

Can i use CICFlowMeter to convert PCAP -> XML? Where is manual?

I can't find manual to this utility. Where can i find it? Thank you in advance.

Can u please tell about your configuration how you collected more data within 5 seconds

Hai sir,
Currently, I am doing a final year project I am Electronics background in India. At the time when I want to do a final year project, I choose your base paper because it is unique and it is different from other projects and I am spending a lot more time to understanding.

Later I saw your personal Information in UNB I Inspired by your projects. My area of interest is Network domain these are the reason I choose your project.

Thank you for your kind response sir.

Sir, at last, one request I have two last doubts can u please answer. I understood all the procedures and technical terms.

(1)  As u mentioned you captured the tor and non-tor traffic data using Tcpdump and Wireshark. After capturing by this way I gave (.pcap files) to the CIC FlowMeter as offline mode. In CIC flow Meter after entering it shown It listened to (1 pcap) file that's it. In CIC FlowMeter it is not shown any features.

(2) I captured the data by using CICFlowMeter in ubuntu and Whonix gateway in the virtual box. As u mentioned in the paper (merged_5s) you collected more data within 5 seconds.
I collected as In ubuntu as a workstation (270(.pcap files) non-tor) and in the host after gateway, I collected (70 (.pacp files) tor) in the duration of 30 minutes with 330kbps wi-fi speed.

                                            IN HOST(WINDOWS)

                                           IN UBUNTU(IN VIRTUAL BOX)

Can u please tell about your configuration how you collected more data within 5 seconds?
Please answer sir.

Labels missing from extracted xml

Hello,

Using as input to the ISCX FlowMeter-4.0 the file: testbed-12jun.pcap i get as output the respective flow file.

The issue here is that the output file's feature 'Label' has values None, unlike the xml outputs that you provide in:

http://205.174.165.80/CICDataset/ISCX-IDS-2012/

I can see that you have uploaded the labeled xml outputs , but i am concerned about how to obtain the labeled xml file in first place.

So my question is:
If i use for instance testbed-jun12.pcap from your link with FlowMeter-4.0 i should get labeled flows in xml format? If the raw pcap files do not contain labels, is there a link to the labeled raw pcap files?

Thank you in advance.

Update:

I also did this:

1. merged pcap files into one from all 7 days.
2. run merged pcap file with CICFlowMeter-4.0

Again the result is the same as the column "Label" has rows with value: "No Label".

False calculation of total number of ACK, SYN and other flags

How to develop using tensorflow(1.14)

How to develop this using Tensorflow 1.13/1.14 for CICFlowmwter

https://cse-cic-ids2018.s3.ca-central-1.amazonaws.com/Processed+Traffic+Data+for+ML+Algorithms/Friday-02-03-2018_TrafficForML_CICFlowMeter.csv

I am new to this area

@Jumabek

Could not find org.jnetpcap:jnetpcap:1.4.1.

I get this error:
Could not find org.jnetpcap:jnetpcap:1.4.1.

Is the code accurate?

Hello, for the CICIDS2017 dataset, the CSV file extracted with your tool is not consistent with the CSV dataset you published. Is the code you posted accurate?

Missing columns information

Hi,

I am using the csv files from the following link: https://www.unb.ca/cic/datasets/ids-2018.html.
As it is explained in 3 - Feature Extraction, the csv files might contain 6 columns (FlowID, SourceIP, DestinationIP, SourcePort, DestinationPort, and Protocol) and other features (about 80). The main problem is that the csv files does not contain the first 6 columns, they just contain destinationPort, protocol and the 80 features.

Also, the protocol column contains a single number between 0 to 17, without knowing which protocol matches that number.

I attached you a single screenshot showing that:

Thanks in advance,

jnetpcap

CICFlowMeter generating empty CSV files with 1 flow and all packets discarded

Hi,
I have tested CIC flowmeter and sometimes doesn't work: all packets are discarded.
When this happens?

Thanks

Network Interface

Running the FlowMeter in IntelliJ, and keep getting the following errors in the DEBUG log after trying to load the available networks in GUI.

2019-06-25 17:27:08 DEBUG cic.cs.unb.ca.flow.ui.FlowMonitorPane java.lang.UnsatisfiedLinkError: com.slytechs.library.NativeLibrary.dlopen(Ljava/lang/String;)J
2019-06-25 17:28:26 DEBUG cic.cs.unb.ca.flow.ui.FlowMonitorPane java.lang.NoClassDefFoundError: Could not initialize class org.jnetpcap.Pcap

libawt_xawt.so: libXrender.so.1: cannot open shared object file

After running Maven success builds, I have got CICFlowMeterV3-0.0.4-SNAPSHOT.jar and when I wrote `"java -jar target/CICFlowMeterV3-0.0.4-SNAPSHOT.jar" to the command line of ubuntu 18.04. I am getting the below error.

java -jar target/CICFlowMeterV3-0.0.4-SNAPSHOT.jar

Exception in thread "main" java.lang.UnsatisfiedLinkError: /usr/local/java/jdk1.8.0_221/jre/lib/i386/libawt_xawt.so: libXrender.so.1: cannot open shared object file: No such file or directory at java.lang.ClassLoader$NativeLibrary.load(Native Method) at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941) at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824) at java.lang.Runtime.load0(Runtime.java:809) at java.lang.System.load(System.java:1086) at java.lang.ClassLoader$NativeLibrary.load(Native Method) at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941) at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1845) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1122) at java.awt.Toolkit$3.run(Toolkit.java:1636) at java.awt.Toolkit$3.run(Toolkit.java:1634) at java.security.AccessController.doPrivileged(Native Method) at java.awt.Toolkit.loadLibraries(Toolkit.java:1633) at java.awt.Toolkit.<clinit>(Toolkit.java:1670) at java.awt.EventQueue.invokeLater(EventQueue.java:1294) at cic.cs.unb.ca.ifm.App.main(App.java:34)

Discrepancies between offline and online flow packet reader

I suspected something might be off with the offline packet reader so I did a test to try it. I was on one side reading all my traffic live with pyshark and then saving it to a pcap file and on the other, I was reading the traffic flows live. Now on the live version, I get 36 flows whereas when I run the offline version on the same traffic I get 0 flows. what could this be?

note that the 1 flow it says it returns are in fact just the names of the fields

How flow direction of a packet is determined?

Hi,

FLOWMETER is generating FlowId with below source:

public String generateFlowId(){
    	boolean forward = true;
    	
    	for(int i=0; i<this.src.length;i++){           
    		if(((Byte)(this.src[i])).intValue() != ((Byte)(this.dst[i])).intValue()){
    			if(((Byte)(this.src[i])).intValue() >((Byte)(this.dst[i])).intValue()){
    				forward = false;
    			}
    			i=this.src.length;
    		}
    	}     	
    	
        if(forward){
            this.flowId = this.getSourceIP() + "-" + this.getDestinationIP() + "-" + this.srcPort  + "-" + this.dstPort  + "-" + this.protocol;
        }else{
            this.flowId = this.getDestinationIP() + "-" + this.getSourceIP() + "-" + this.dstPort  + "-" + this.srcPort  + "-" + this.protocol;
        }
        return this.flowId;
    }

What code says is "If src_ip of the packet is small then it is FWD packet, otherwise it is BWD packet"

Code refernce

However, the netflowmeter page says
Flow direction is determined when first packet of the flow arrives.

It seems to me above two contradicts each other.
Would you kindly explain more how generateFlowId() method is determining direction and generating flowid?

Possible miscalculations of flow bulk, sub flow and TCP flags-related features.

Greetings,

I noticed that a week ago you guys changed the "BasicFlow.java" file with the commit ed1d6bc. In this specific commit, three functions were uncommented:

updateFlowBulk(packet);
detectUpdateSubflows(packet);
checkFlags(packet);

Which in this case, I believe, means that they weren't taken into account before, when updating existent flows with the addPacket(BasicPacketInfo packet) method, only with the firstPacket(BasicPacketInfo packet).

Since I'm using the CICIDS2017's .csv files to train a few machine learning techniques and these files were previously generated by the CICFlowMeter, I'm concerned about possible miscalculations of certain features related to these three methods in the files provided w/ the dataset for machine learning purposes.

I upload the pcap file,and click 'ok' button, it shows Error while opening file for capture: bad dump file format

when i click the 'ok' button, it shows the Error above.

this file was processed by

and the size of the file I used is 19.8M
MondayWorkingHours.pcap is one of the file provided by CICIDS2017 dataset

Thank you for helping me!

Could you please provide the calculation formula using for the feature extraction?

I have some difficult reading the java code. Could you please provide the calculation method in python way?

More than 100mb

Why did you put a 100mb limit? and how can i remove this?

pom.xml seems forget tika dependecy

pom.xml seems forget tika dependecy

CIC-IDS-2017 - Init_win_bytes_forward = '-1'

Why the number of bytes of first packet send is '-1' in almost all entry in the Wednesday Dos Dataset?
Other fields also have this value. It makes it impossible for me to apply many algorithms, if not replacing these values ​​with '1', but is it correct? The number of bytes of the first request packet is impossible to be zero. I would need to have the number of bytes of the first packet sent, in order to recognize portscanning and bruteforce attacks

CICFlowMeter doesn't generate output files

Hi!
I tried executing from jar (https://www.unb.ca/cic/_assets/documents/cicflowmeter-4.zip) and also tried to build it in IntelliJ IDEA with gradle.
When I select the input pcap file/direstory, the output directory, the logger pane shows a message "CICFlowMeter has received 1 pcap file" and then nothing happens.
I've tried on CSE-CIC-IDS2018 dataset.

debug_logs:
2019-04-15 17:32:35 DEBUG cic.cs.unb.ca.flow.ui.FlowOfflinePane offline select input /Users/.../.../.../Original Network Traffic and Log data/Friday-02-03-2018/pcap
2019-04-15 17:32:41 DEBUG cic.cs.unb.ca.flow.ui.FlowOfflinePane offline select output /Users/.../.../.../processed_by_me
2019-04-15 17:32:43 DEBUG cic.cs.unb.ca.jnetpcap.worker.ReadPcapFileWorker CICFlowMeter found :442 pcap files

if i'm trying to make it through terminal the following exception i get:
Exception in thread "main" java.lang.UnsatisfiedLinkError: com.slytechs.library.NativeLibrary.dlopen(Ljava/lang/String;)J
at com.slytechs.library.NativeLibrary.dlopen(Native Method)
at com.slytechs.library.NativeLibrary.(Unknown Source)
at com.slytechs.library.JNILibrary.(Unknown Source)
at com.slytechs.library.JNILibrary.loadLibrary(Unknown Source)
at com.slytechs.library.JNILibrary.register(Unknown Source)
at com.slytechs.library.JNILibrary.register(Unknown Source)
at com.slytechs.library.JNILibrary.register(Unknown Source)
at org.jnetpcap.Pcap.(Unknown Source)
at cic.cs.unb.ca.jnetpcap.PacketReader.config(PacketReader.java:58)
at cic.cs.unb.ca.jnetpcap.PacketReader.(PacketReader.java:52)
at cic.cs.unb.ca.ifm.Cmd.readPcapFile(Cmd.java:128)
at cic.cs.unb.ca.ifm.Cmd.main(Cmd.java:80)
i'm working on macOS

how you have labelled port scan csv file.

on what basis you have labelled that this flow is port scan and that flow is benign.
how can we classify Port Scan label into different types of port scans in the CSV file?

libpcap-dev needed

On Linux, libpcap-dev is a mandatory dependency.

A suggestion on the part to install jnetpcap

I don't think it is a big deal, but it would be helpful for beginners of your repository.

Your README is saying

//windows: at the pathtoproject/jnetpcap/win/jnetpcap-1.4.r1425
mvn install:install-file -Dfile=jnetpcap.jar -DgroupId=org.jnetpcap -DartifactId=jnetpcap -Dversion=1.4.1 -Dpackaging=jar

but, there is no pom.xml at the specified path (pathtoproject/jnetpcap/win/jnetpcap-1.4.r1425).

So I think it would be better if you write it like below.

//windows: at root directory of this repository
mvn install:install-file -Dfile=".\jnetpcap\win\jnetpcap-1.4.r1425\jnetpcap.jar" -DgroupId="org.jnetpcap" -DartifactId=jnetpcap -Dversion="1.4.1" -Dpackaging=jar

Windows cmd cannot recognize the complex path and strings containing ".", so I think it would be better to use quotation marks on them.

Thanks.

Please help with jnetpcap linkage error

I am faced with linkage error when running ./cfm and still cannot fix it after spending a day. Could you see the root cause?

Exception in thread "main" java.lang.UnsatisfiedLinkError: com.slytechs.library.NativeLibrary.dlopen(Ljava/lang/String;)J
at com.slytechs.library.NativeLibrary.dlopen(Native Method)
at com.slytechs.library.NativeLibrary.(Unknown Source)
at com.slytechs.library.JNILibrary.(Unknown Source)
at com.slytechs.library.JNILibrary.loadLibrary(Unknown Source)
at com.slytechs.library.JNILibrary.register(Unknown Source)
at com.slytechs.library.JNILibrary.register(Unknown Source)
at com.slytechs.library.JNILibrary.register(Unknown Source)
at org.jnetpcap.Pcap.(Unknown Source)
at cic.cs.unb.ca.jnetpcap.PacketReader.config(PacketReader.java:58)
at cic.cs.unb.ca.jnetpcap.PacketReader.(PacketReader.java:52)
at cic.cs.unb.ca.ifm.Cmd.readPcapFile(Cmd.java:128)
at cic.cs.unb.ca.ifm.Cmd.readPcapDir(Cmd.java:100)
at cic.cs.unb.ca.ifm.Cmd.main(Cmd.java:73)

How to get the .jar file for this CICFlowMeter

how to install this to get gui

CFM executable JVM heap shortage

When using the cfm script to run the software from the command line, the JVM crashes with an out of memory error for the heap. The input file I used is 114MB. The JVM had a max heap size of -XX:MaxHeapSize=4190109696 bytes =~ 4GB. Surely processing a 114MB file shouldn't cause this much memory usage.

The stacktrace

Working on... c91e8614-8654-4bac-ac2a-e73a54742e7f.pcap
Exception in thread "DisposableGC" Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at org.jnetpcap.nio.LinkSequence.iterator(Unknown Source)
	at org.jnetpcap.nio.DisposableGC.sortGenerations(Unknown Source)
	at org.jnetpcap.nio.DisposableGC.drainRefQueueLoop(Unknown Source)
	at org.jnetpcap.nio.DisposableGC$2.run(Unknown Source)
	at java.base/java.lang.Thread.run(Thread.java:834)
java.lang.OutOfMemoryError: Java heap space
	at cic.cs.unb.ca.jnetpcap.FlowGenerator.addPacket(FlowGenerator.java:120)
	at cic.cs.unb.ca.ifm.Cmd.readPcapFile(Cmd.java:144)
	at cic.cs.unb.ca.ifm.Cmd.readPcapDir(Cmd.java:100)
	at cic.cs.unb.ca.ifm.Cmd.main(Cmd.java:73)

How to install CIC flow meter in whonix workstation.

Hai sir I have confusion. I installed the virtual box in windows os and in the virtual box I installed whonix Linux distribution. I created the whonix gateway and workstation. I need CIC flow meter for collecting the data so that I want to install CIC flow meter.

1)(((https://www.unb.ca/cic/_assets/documents/cicflowmeter-4.zip))) Through these link I downloaded the CIC flowmeter.
2)(((sudo apt-get install libcap-dev ))) Through these I installed the libcap package in whonix workstation.
3) I Installed CIC flow meter in whonix workstation but it is in .bat format. I think (.bat) is only work in DOS and windows. I am unable to open it in whonix workstation.
4)((sudo ./CICFlowMeter)) for these command I am getting command not found in whonix wokstation.
5) (./cfm "inputFolder" "outputFolder") for these command I am getting command not found in whonix wokstation.

Please reply ...

cannot load network interface: UnsatisfiedLinkError in com.slytechs.library.NativeLibrary.dlopen

Built from source without major problems. I start CICFlowMeter and get the dialog for Realtime captures but cannot load any network interface because of a UnsatisfiedLinkError in libjnetpcap library runtime error. The full stack trace is below.

My setup:

$ which java
/usr/lib/jvm/java-8-openjdk-amd64/bin/java

$ java -version
openjdk version "1.8.0_151"
OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-0ubuntu0.16.04.2-b12)
OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.3 LTS
Release:        16.04
Codename:       xenial

$ echo $LD_LIBRARY_PATH
/usr/lib/x86_64-linux-gnu:/home/oem/Devel/CICFlowMeter/jnetpcap-1.3.0

$ echo $CLASSPATH
/home/oem/Devel/CICFlowMeter/jnetpcap-1.3.0

$ echo $JAVA_HOME
/usr/lib/jvm/java-8-openjdk-amd64/

Started CICFlowMeter as follows:

$ sudo java  -Djava.library.path=/home/oem/Devel/CICFlowMeter/jnetpcap-1.3.0  -jar CICFlowMeterV3-0.0.4-SNAPSHOT.jar

I have installed jnetpcap from their tarball and tried several things. Not a big Java user though and I may have missed some obvious setup.

Any help would be appreciated.

Full stack trace:

Mar 09, 2018 9:54:14 AM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
WARNING: org.xerial's sqlite-jdbc is not loaded.
Please provide the jar on your classpath to parse sqlite files.
See tika-parsers/pom.xml for the correct version.
java.util.concurrent.ExecutionException: java.lang.UnsatisfiedLinkError: com.slytechs.library.NativeLibrary.dlopen(Ljava/lang/String;)J
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at javax.swing.SwingWorker.get(SwingWorker.java:602)
        at cic.cs.unb.ca.flow.ui.FlowMonitorPane.lambda$loadPcapIfs$5(FlowMonitorPane.java:271)
        at java.beans.PropertyChangeSupport.fire(PropertyChangeSupport.java:335)
        at java.beans.PropertyChangeSupport.firePropertyChange(PropertyChangeSupport.java:327)
        at javax.swing.SwingWorker$SwingWorkerPropertyChangeSupport.firePropertyChange(SwingWorker.java:854)
        at javax.swing.SwingWorker$SwingWorkerPropertyChangeSupport$1.run(SwingWorker.java:860)
        at javax.swing.SwingWorker$DoSubmitAccumulativeRunnable.run(SwingWorker.java:832)
        at sun.swing.AccumulativeRunnable.run(AccumulativeRunnable.java:112)
        at javax.swing.SwingWorker$DoSubmitAccumulativeRunnable.actionPerformed(SwingWorker.java:842)
        at javax.swing.Timer.fireActionPerformed(Timer.java:313)
        at javax.swing.Timer$DoPostEvent.run(Timer.java:245)
        at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
        at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
        at java.awt.EventQueue.access$500(EventQueue.java:97)
        at java.awt.EventQueue$3.run(EventQueue.java:709)
        at java.awt.EventQueue$3.run(EventQueue.java:703)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
        at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
        at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
        at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
        at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Caused by: java.lang.UnsatisfiedLinkError: com.slytechs.library.NativeLibrary.dlopen(Ljava/lang/String;)J
        at com.slytechs.library.NativeLibrary.dlopen(Native Method)
        at com.slytechs.library.NativeLibrary.<init>(Unknown Source)
        at com.slytechs.library.JNILibrary.<init>(Unknown Source)
        at com.slytechs.library.JNILibrary.loadLibrary(Unknown Source)
        at com.slytechs.library.JNILibrary.register(Unknown Source)
        at com.slytechs.library.JNILibrary.register(Unknown Source)
        at com.slytechs.library.JNILibrary.register(Unknown Source)
        at org.jnetpcap.Pcap.<clinit>(Unknown Source)
        at cic.cs.unb.ca.jnetpcap.worker.LoadPcapInterfaceWorker.doInBackground(LoadPcapInterfaceWorker.java:27)
        at cic.cs.unb.ca.jnetpcap.worker.LoadPcapInterfaceWorker.doInBackground(LoadPcapInterfaceWorker.java:14)
        at javax.swing.SwingWorker$1.call(SwingWorker.java:295)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at javax.swing.SwingWorker.run(SwingWorker.java:334)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
java.util.concurrent.ExecutionException: java.lang.UnsatisfiedLinkError: com.slytechs.library.NativeLibrary.dlopen(Ljava/lang/String;)J
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at javax.swing.SwingWorker.get(SwingWorker.java:602)
        at cic.cs.unb.ca.flow.ui.FlowMonitorPane.lambda$loadPcapIfs$5(FlowMonitorPane.java:271)
        at java.beans.PropertyChangeSupport.fire(PropertyChangeSupport.java:335)
        at java.beans.PropertyChangeSupport.firePropertyChange(PropertyChangeSupport.java:327)
        at javax.swing.SwingWorker$SwingWorkerPropertyChangeSupport.firePropertyChange(SwingWorker.java:854)
        at javax.swing.SwingWorker$SwingWorkerPropertyChangeSupport$1.run(SwingWorker.java:860)
        at javax.swing.SwingWorker$DoSubmitAccumulativeRunnable.run(SwingWorker.java:832)
        at sun.swing.AccumulativeRunnable.run(AccumulativeRunnable.java:112)
        at javax.swing.SwingWorker$DoSubmitAccumulativeRunnable.actionPerformed(SwingWorker.java:842)
        at javax.swing.Timer.fireActionPerformed(Timer.java:313)
        at javax.swing.Timer$DoPostEvent.run(Timer.java:245)
        at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
        at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
        at java.awt.EventQueue.access$500(EventQueue.java:97)
        at java.awt.EventQueue$3.run(EventQueue.java:709)
        at java.awt.EventQueue$3.run(EventQueue.java:703)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
        at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
        at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
        at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
        at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Caused by: java.lang.UnsatisfiedLinkError: com.slytechs.library.NativeLibrary.dlopen(Ljava/lang/String;)J
        at com.slytechs.library.NativeLibrary.dlopen(Native Method)
        at com.slytechs.library.NativeLibrary.<init>(Unknown Source)
        at com.slytechs.library.JNILibrary.<init>(Unknown Source)
        at com.slytechs.library.JNILibrary.loadLibrary(Unknown Source)
        at com.slytechs.library.JNILibrary.register(Unknown Source)
        at com.slytechs.library.JNILibrary.register(Unknown Source)
        at com.slytechs.library.JNILibrary.register(Unknown Source)
        at org.jnetpcap.Pcap.<clinit>(Unknown Source)
        at cic.cs.unb.ca.jnetpcap.worker.LoadPcapInterfaceWorker.doInBackground(LoadPcapInterfaceWorker.java:27)
        at cic.cs.unb.ca.jnetpcap.worker.LoadPcapInterfaceWorker.doInBackground(LoadPcapInterfaceWorker.java:14)
        at javax.swing.SwingWorker$1.call(SwingWorker.java:295)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at javax.swing.SwingWorker.run(SwingWorker.java:334)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Unable to obtain UDP Flows

Hello

I am unable to obtain UDP Flows using CICFlowMeter in both Online/Offline mode. I made a pcap capture with Wireshark to try and extract UDP flows using CICFlowMeter but it only creates the TCP flows and not the UDP ones. Also, is there a way to obtain ICMP flows between two IP addresses. I mostly use Python, not Java so I don't know how or where I can make changes for controlling UDP timeouts and capturing ICMP flows but if someone can point me in the right direction, I will be extremely grateful. More important is the UDP issue. (I unfortunately cannot upload the exact files)

All captures are carried out on a Virtual Machine running Ubuntu 16.04 LTS

I thank you for your help

How linux os use realtime function

How linux os use realtime function

Can this utilite convert PCAP to XML?

Installation Issues

Hi, I was trying to install CICFlowMeter but facing the following problems:

Step 1: Extract CICFlowMeterV3.zip. The file available for download from this site is CICFlowMeter-master.zip and not CICFlowMeterV3.zip

I still downloaded CICFlowMeter-master.zip and extracted

Step 2: Go to the extracted directory, enter the 'bin' folder

No bin folder is found in the extracted directory CICFlowMeter-master

Please fix the issue or let me know how to install from other sources.

Thanks and Regards,
-Hari

no feature labels on each feature columns

hi,
i executed CICFlowmeter, when i run the GUI (APP.JAVA) file, it give correct output CSV file with features but when i run the CICFLOWMETER.java file and give path to input pcap files and output folder, it generates flows but the feature labels are missing. you can check from the screnshots here;

Load Network Device Button

On linux install: when pressing the "load" button, nothing happens.

IPv6

I have a dataset containing both IPv4 and IPv6 flows and as far as I can see the CICFlowMeter includes code to handle IPv6 packets but it is disabled in the class function
`public class CICFlowMeter {

public static final Logger logger = LoggerFactory.getLogger(CICFlowMeter.class);
public static void main(String[] args) {
	
	PacketReader    packetReader;
	BasicPacketInfo basicPacket = null;
	FlowGenerator   flowGen; //15000 useconds = 15ms
	
	boolean readIP6 = false;
	boolean readIP4 = true;`

so I wanted to ask if there is a reason for that. Is something not correctly working with IPv6 or can it simply be activated by setting readIP6 to true?

Why are you commenting out this line? Is there a problem with the logic?

Why is this line of code commented? finishedFlows.put(getFlowCount(), flow); According to the logic of the code, shouldn't the flow with the fin flag be placed in the completed hashmap?

else if(packet.hasFlagFIN()){
    	    	logger.debug("FlagFIN current has {} flow",currentFlows.size());
    	    	flow.addPacket(packet);
                if (mListener != null) {
                    mListener.onFlowGenerated(flow);
                } /*else {
                    finishedFlows.put(getFlowCount(), flow);
                }*/
                currentFlows.remove(packet.getFlowId());

CICFlowMeter generating empty CSV files with 0 flows

Hi,
I tried executing the jar, as well as from Eclipse, and in both cases I face the following issue: The generated CSV file is empty with 1.1 Ko
I have tested CIC flowmeter with other PCAP files and it worked fine.

Please advise

Possible incorrect calculation of Tot Fwd Pkts and Tot Bwd Pkts

Greetings,

I have noticed that csv generated by the tool have incorrect calculation of Tot Fwd Pkts and Tot Bwd Pkts. "Tot Fwd Pkts" is always one less than actual and "Tot Bwd Pkts" is always one more than actual. I have cross verified with output of conversations in wireshark.

Please verify from your end and update the main source code.

Thanks

CICFlowMeter not generating the outputs

Hi,
This is my initial attempt at running CICFlowMeter on pcaps.
I tried executing the jar, as well as from Eclipse, and in both cases I face the following issue:
When I select the input pcap file, the output directory, the logger pane shows a message "CICFlowMeter has received 1 pcap file" and then nothing happens.

Am I doing something wrong?

Please advise

Does it have CLI version?

Does it only have GUI version? I want to run it in pure CLI environment... How should I do?

What units are used for flow timeout and activity timeout in offline mode?

Hi, I was using the offline mode and saw the default flow timeout and activity timeout were 120000000 and 5000000, respectively. Just wondering what these stand for and what unit is used?
Sorry if I missed this in the documentation.

How to instal CICFlowGenerator?

Hi,

I want to use CICFlowGenerator and generate som data, but I can't find some information about this feature. Can you help me and give me some information how can I install it?

Why is this line of code commented?According to the logic of the code, shouldn't the flow with the fin flag be placed in the completed hashmap?

Why is this line of code commented? finishedFlows.put(getFlowCount(), flow); According to the logic of the code, shouldn't the flow with the fin flag be placed in the completed hashmap?

// Flow finished due FIN flag (tcp only):
    		// 1.- we add the packet-in-process to the flow (it is the last packet)
        	// 2.- we move the flow to finished flow list
        	// 3.- we eliminate the flow from the current flow list   	
    		}else if(packet.hasFlagFIN()){
    	    	logger.debug("FlagFIN current has {} flow",currentFlows.size());
    	    	flow.addPacket(packet);
                if (mListener != null) {
                    mListener.onFlowGenerated(flow);
                } /*else {
                    finishedFlows.put(getFlowCount(), flow);
                }*/
                currentFlows.remove(packet.getFlowId());
    		}else{
    			flow.updateActiveIdleTime(currentTimestamp,this.flowActivityTimeOut);
    			flow.addPacket(packet);
    			currentFlows.put(packet.getFlowId(), flow);
    		}

Installation and execution of CICFlowmeter

Hi,
I downloaded the CICFlowmeter from here: [https://github.com/ISCX/CICFlowMeter]. But i don't know how to run and get the output CSV files. could you please show me step by step execution process? files are mixed up. I don't know where and which one is the main file to run the code.
Thanks

What's the difference between flow timeout and activity timeout in offline mode?

I guessed flow timeout is for both TCP and UDP, is that right?