In this project, we build a real-time system which is able to capture network intrusion and then predict its type of attack by an AI model.

• Viet-Sang Nguyen
• Phuong-Hoa Nguyen
• Ngoc-Nhat-Huyen Tran

sudo ./install.sh

Kafka server needs this one. It will run on localhost, port 2181

kafka_2.13-2.4.1/bin/zookeeper-server-start.sh kafka_2.13-2.4.1/config/zookeeper.properties

It will run on localhost, port 9092. When we publish messages to this server through producers, messages will be stored on /tmp/kafka-logs/

kafka_2.13-2.4.1/bin/kafka-server-start.sh kafka_2.13-2.4.1/config/server.properties

brew install snort

Snort needs a config file (snort.config) and a folder to store log. Normally, they are stored in /etc/snort/snort.config and /var/logs/snort.

In this project, we store them in folder snort and use the full paths to point where they are.

In this file, we point to the file rules/icmp.rules

include rules/icmp.rules

We write some rules to capture packets in config file. In this example, sort will alert all ping packets.

alert icmp any any -> any any (msg:"ICMP Packet"; sid:477; rev:3;)

snort -A console -q -c ~/.../src/snort/snort.config -b -i en0 -L ~/.../src/snort/logs/log.pcap

ping google.com

python3 producer_pcap.py

This consumer retrieves pcaps from kafka, then transform to readable data thanks to Zeek. After that, KDD99 data is generated by spark. TensorFlow models then predict whether data is normal or attacked. The results are sent back to kafka.

python3 consumer_spark.py

This consumer retrieves and shows predicted data from kafka

python3 consumer_warning.py

Notice: all commands should be run from NIDS-DL-Project folder

python3 dl_src/preprocessing.py

python3 dl_src/train.py

python3 dl_src/test.py