- Tal Malka
- Yann Chicheportiche
This project aims to develop a Network Intrusion Detection System (NIDS) leveraging machine learning techniques to detect and classify diverse network intrusions in real-time.
Which machine learning algorithm provides the best performance for real-time intrusion detection and classification on modern datasets (CIC-IDS2017), considering accuracy, detection speed, robustness, and false positive management?
- Global cyberattack losses exceed $6 trillion annually.
- Increasing sophistication of attacks (zero-day vulnerabilities, DDoS, infiltration attempts).
- Signature-based/rule-based systems struggle to detect novel (zero-day) attacks.
- Polymorphic malware and evolving threats evade static detection methods.
- Real-time analysis of complex network traffic is crucial.
- Machine learning techniques offer adaptability to emerging threats.
- Develop a machine learning-based NIDS that accurately detects and classifies cyber threats.
- Evaluate multiple ML algorithms (Random Forest, KNN, Logistic Regression, Voting Classifier) on modern datasets (CIC-IDS2017).
Extensive research has been conducted on ML techniques for intrusion detection. Some key findings:
- Random Forest & Decision Trees: Strong performance but limited scalability.
- Hybrid/Ensemble Methods: Improved accuracy but lacked real-time adaptability.
- Advanced Techniques (SVM, Neural Networks): High accuracy but require extensive preprocessing.
The CIC-IDS2017 dataset is utilized for training and testing, containing realistic cyberattack scenarios, including:
- Portscan - Identifying open or vulnerable ports.
- DoS/DDoS Attacks - Flooding services with excessive requests.
- Infiltration - Exploiting open ports for unauthorized access.
- Botnet & Brute Force Attacks - Compromising machines and guessing login credentials.
- Web Attacks (SQL Injection, XSS) - Exploiting web applications.
- Checked for missing data and removed irrelevant variables.
- Exploratory data analysis performed to understand network traffic behavior.
- Random Forest (RF)
- K-Nearest Neighbors (KNN)
- Logistic Regression (LR)
- Voting Classifier (Ensemble Approach)
| Algorithm | False Positives | False Positive Rate |
|---|---|---|
| Random Forest | 61 | 0.013% |
| KNN | 699 | 0.14% |
| Logistic Regression | 68,522 | 10.876% |
| Voting Classifier | 699 | 0.036% |
- Use of Modern Datasets - CIC-IDS2017 captures contemporary attack patterns.
- Algorithmic Comparison - Evaluated multiple models beyond traditional methods.
- False Positive Reduction - Focused on reducing false alarms for practical deployment.
- Real-Time Adaptability - Ensured models could scale to handle large network traffic volumes.
This study highlights the importance of machine learning in modern NIDS solutions. Unlike legacy IDS approaches, our ML-powered NIDS leverages modern datasets to enhance detection accuracy while minimizing false positives. The Voting Classifier and Random Forest models demonstrated strong performance, making them suitable for real-world cybersecurity applications.
- Implement deep learning techniques for further improvement.
- Optimize real-time processing for large-scale network environments.
- Integrate adaptive learning models to keep up with evolving threats.
- Install Dependencies:
pip3 install -r requirements.txt
- Run the Model Training:
python3 <Model>.py