nani1-glitch / llm-for-logging-

LLM for Logging 🚀

Overview 📚

This repository showcases the use of Language Models for log data processing, categorization, and analysis. By leveraging BERT, the project aims to enhance log data handling, making it efficient and insightful.

Features ✨

• Data Preprocessing: Extract relevant information from raw log files.
• Log Parsing & Labeling: Identify and label error messages.
• Model Training: Train BERT for sequence classification.
• Evaluation: Compute metrics like accuracy, precision, recall, and F1-score.

Results 📊

• Accuracy: 63.97%
• Precision: 63.01%
• Recall: 67.65%
• F1 Score: 65.25%

Getting Started 🛠️

Clone the Repository

git clone https://github.com/Nani1-glitch/LLM-for-Logging-.git cd LLM-for-Logging

🚀 Run the Notebooks

• 📘 Final Implementation: Final.ipynb
• 📓 Manual Log Categorization Examples: Manual Approach(EX).ipynb
• 📙 Training BERT from Scratch: Trained Bert Model from scratch.ipynb

🔑 Key Components 🔍

📂 Data Files:

• 🗂️ cisco_log.txt
• 🗃️ sample_logs.txt
• 📑 enhanced_cisco_logs.csv
• 📄 structured_logs.csv
• 📝 structured_output.csv

🛠️ Libraries:

• 🐼 Pandas
• 🐍 PyTorch
• 🤗 Transformers
• 📊 Scikit-Learn

📜 License 📄

This project is licensed under the MIT License. See the LICENSE file for details.

🤝 Contributing 🤝

Contributions are welcome! Feel free to open an issue or submit a pull request.

📬 Contact 📬

For any questions or feedback, please contact Nani1-glitch.