Log-analyzer

A robust server log analysis tool. Parses mixed-format log files (multiple timestamp styles, JSON-formatted lines, varying response-time units, missing fields, malformed lines) and produces either a terminal report or a self-contained HTML dashboard.

Quick Start (fresh machine)

Requirements: Python 3.10+ (standard library only — no pip installs needed)

git clone <repo-url>
cd log-analyzer

1. Generate a test log file

python scripts/generate_logs.py --lines 5000 --output test.log

Options:

Flag	Default	Description
`--lines N`	5000	Number of log entries to generate
`--output PATH`	test.log	Output file path

2. Analyse the log

Text report (stdout):

python analyze.py test.log

HTML dashboard (self-contained file):

python analyze.py test.log --html
# writes to test_report.html by default

python analyze.py test.log --html -o dashboard.html
# custom output path

Quiet mode (suppress progress messages):

python analyze.py server.log --html -q

What the tool produces

Text report includes:

Overview: total requests, error count, error rate, missing-status count
Status code distribution with ASCII bar chart
Top 10 endpoints by volume
Top 10 slowest endpoints (p50 / p95 / p99 / max)
Highest error-rate endpoints (min 5 requests)
Top IPs by volume
Suspicious IP list (high volume or >40% error rate)
Format anomaly counts (non-standard timestamps, bare response times, etc.)
First 10 skipped/malformed lines with line numbers and reasons

HTML dashboard adds:

Request volume over time (hourly bar chart)
Interactive status code distribution chart
Skipped line viewer (first 100 with line numbers and reasons)
Dark terminal-aesthetic UI, no external dependencies

Log formats handled

Format	Example
Standard	`2024-03-15T14:23:01Z 192.168.1.1 GET /api/users 200 142ms`
Slash date	`2024/03/15 14:23:01 10.0.0.1 POST /api/login 401 89ms`
Named month	`15-Mar-2024 14:23:01 10.0.0.1 GET /api/health 200 53ms`
Unix epoch	`1710512581 192.168.1.1 GET /api/users 200 142ms`
Response in seconds	`2024-03-15T14:23:01Z 10.0.0.1 GET /api/users 200 0.142s`
Bare response time	`2024-03-15T14:23:01Z 10.0.0.1 GET /api/users 200 142`
Missing status	`2024-03-15T14:23:01Z 10.0.0.1 GET /api/users - 142ms`
Extra fields	`... 200 142ms "Mozilla/5.0 ..."`
JSON lines	`{"timestamp":"2024-03-15T14:23:01Z","method":"GET",...}`
Malformed / blank	silently skipped, counted, shown in report

Project structure

log-analyzer/
├── analyze.py          # CLI entry point
├── parser.py           # Multi-format log parser
├── analytics.py        # Stats engine (percentiles, top-N, error rates)
├── report.py           # Text + HTML renderers
├── templates/
│   └── report.html     # HTML dashboard template
├── scripts/
│   └── generate_logs.py  # Test data generator
└── README.md

Running against your own files

python analyze.py /path/to/any/server.log
python analyze.py /path/to/any/server.log --html -o report.html

The tool never assumes a specific filename, line count, or value range. It degrades gracefully on any input: malformed lines are counted and reported, never silently dropped.

Sahitya05/Log-Analyzer

README