Log Analysis Project

🔍 Project Overview

This Python-based CLI tool automatically detects the type of log files (Apache, Nginx, Syslog, Auth.log), parses them using regex, and outputs results as CSV, Excel, or charts using Matplotlib. It includes detection for brute-force SSH attacks, suspicious sudo activity, geo-based anomalies, and odd-hour logins.

🧰 Tools & Technologies

Languages: Python 3
Libraries: re, pandas, matplotlib, openpyxl, geoip2
OS: Linux (Ubuntu)

✨ Key Features

Auto-detects Apache, Nginx, Syslog, and Auth.log files
Detects brute-force attempts and other suspicious patterns
Visualizes login patterns and anomalies
Exports to CSV, Excel, or graphical reports
Lightweight and modular CLI interface

📦 Output Formats

.json – Raw structured output for APIs or advanced analysis
.csv – Clean tabular data for spreadsheets or log reviews
.xlsx – Formatted Excel sheets for business or audit use
.png / .svg – Graphs and charts (matplotlib-based) for anomaly detection
CLI – Real-time terminal output for quick visibility

📂 File Structure


log-parser/
├── parser.py
├── requirements.txt
├── detections/
│   ├── apache_detections.py
│   ├── authlog_detections.py
│   ├── nginx_detections.py
│   ├── syslog_detections.py
├── log_patterns/
│   ├── __init__.py
│   ├── apache.py
│   ├── authlog.py
│   ├── common.py
│   ├── nginx.py
│   ├── syslog.py
├── utils/
|   ├── detector.py
|   ├── output.py
|   ├── visualizer.py

📥 Sample Input (auth.log)

Jan  1 02:00:01 myhost sshd[1201]: Accepted password for chandru from 192.168.1.10 port 50001
Jan  1 02:00:02 myhost sshd[1202]: Failed password for chandru from 192.168.1.10 port 50002
Jan  1 02:00:02 myhost sshd[1203]: Failed password for chandru from 192.168.1.10 port 50003
Jan  1 02:00:03 myhost sshd[1204]: Failed password for chandru from 192.168.1.10 port 50004
Jan  1 02:00:03 myhost sshd[1205]: Failed password for chandru from 192.168.1.10 port 50005
Jan  1 02:00:04 myhost sshd[1206]: Accepted password for chandru from 192.168.1.10 port 50006  
Jan  1 03:00:05 myhost sshd[1207]: Accepted password for anthony from 10.0.0.20 port 22
Jan  1 04:00:10 myhost sshd[1208]: Failed password for prince from 203.0.113.15 port 22
Jan  1 04:00:15 myhost sshd[1209]: Accepted password for sanjay from 192.168.1.25 port 22
Jan  1 05:00:01 myhost sshd[1210]: Accepted password for chandru from 192.168.1.10 port 22
Jan  1 06:00:05 myhost sshd[1211]: Accepted password for anthony from 10.0.0.20 port 22
Jan  1 07:00:10 myhost sshd[1212]: Failed password for prince from 203.0.113.15 port 22
Jan  1 07:00:15 myhost sshd[1213]: Accepted password for sanjay from 192.168.1.25 port 22
Jan  1 08:00:01 myhost sshd[1214]: Accepted password for chandru from 192.168.1.10 port 22
Jan  1 09:00:05 myhost sshd[1215]: Accepted password for anthony from 10.0.0.20 port 22
Jan  1 10:00:10 myhost sshd[1216]: Failed password for prince from 203.0.113.15 port 22
Jan  1 10:00:15 myhost sshd[1217]: Accepted password for sanjay from 192.168.1.25 port 22
Jan  1 11:00:01 myhost sshd[1218]: Accepted password for chandru from 192.168.1.10 port 22
Jan  1 12:00:05 myhost sshd[1219]: Accepted password for anthony from 10.0.0.20 port 22
Jan  1 13:00:10 myhost sshd[1220]: Failed password for prince from 203.0.113.15 port 22
Jan  1 13:00:15 myhost sshd[1221]: Accepted password for sanjay from 192.168.1.25 port 22
Jan  2 01:00:01 myhost sshd[1222]: Accepted password for chandru from 192.168.1.10 port 22 
Jan  2 01:00:05 myhost sshd[1223]: Accepted password for anthony from 10.0.0.20 port 22   
Jan  2 03:00:10 myhost sshd[1224]: Accepted password for prince from 172.16.0.5 port 22   
Jan  2 03:00:15 myhost sshd[1225]: Accepted password for sanjay from 192.168.1.25 port 22
Jan  2 05:00:01 myhost sshd[1226]: Accepted password for chandru from 192.168.1.10 port 22
Jan  2 06:00:05 myhost sshd[1227]: Accepted password for anthony from 10.0.0.20 port 22
Jan  2 07:00:10 myhost sshd[1228]: Accepted password for prince from 172.16.0.5 port 22
Jan  2 07:00:15 myhost sshd[1229]: Accepted password for sanjay from 192.168.1.25 port 22
Jan  3 00:00:01 myhost sshd[1230]: Accepted password for chandru from 192.168.1.10 port 22 
Jan  3 02:00:05 myhost sshd[1231]: Accepted password for anthony from 10.0.0.20 port 22   
Jan  3 04:00:10 myhost sshd[1232]: Accepted password for prince from 172.16.0.5 port 22  
Jan  3 04:00:15 myhost sshd[1233]: Accepted password for sanjay from 192.168.1.25 port 22
Jan  4 08:00:01 myhost sshd[1234]: Accepted password for chandru from 192.168.1.10 port 22
Jan  4 09:00:05 myhost sshd[1235]: Accepted password for anthony from 10.0.0.20 port 22
Jan  4 10:00:10 myhost sshd[1236]: Accepted password for prince from 172.16.0.5 port 22
Jan  4 10:00:15 myhost sshd[1237]: Accepted password for sanjay from 192.168.1.25 port 22
Jan  5 12:00:01 myhost sshd[1238]: Accepted password for chandru from 192.168.1.10 port 22
Jan  5 13:00:05 myhost sshd[1239]: Accepted password for anthony from 10.0.0.20 port 22
Jan  5 14:00:10 myhost sshd[1240]: Accepted password for prince from 172.16.0.5 port 22
Jan  5 14:00:15 myhost sshd[1241]: Accepted password for sanjay from 192.168.1.25 port 22
Jan  6 16:00:01 myhost sudo:   chandru : TTY=pts/2 ; PWD=/home/chandru ; USER=root ; COMMAND=/bin/su - root  # Sudden sudo
Jan  6 16:00:05 myhost sudo:   anthony : TTY=pts/3 ; PWD=/var/log/apache2 ; USER=root ; COMMAND=/usr/sbin/apache2ctl graceful
Jan  6 08:00:10 myhost sudo:   prince : TTY=pts/4 ; PWD=/var/www ; USER=root ; COMMAND=/usr/bin/git pull origin main
Jan  6 08:00:15 myhost sudo:   sanjay : TTY=pts/5 ; PWD=/etc/nginx/sites-available ; USER=root ; COMMAND=/usr/sbin/nginx -s reload
Jan  6 17:00:01 myhost sudo:   chandru : TTY=pts/6 ; PWD=/root ; USER=root ; COMMAND=/usr/bin/apt-get update
Jan  6 18:00:05 myhost sudo:   anthony : TTY=pts/7 ; PWD=/etc/ssh ; USER=root ; COMMAND=/usr/bin/vim /etc/ssh/sshd_config
Jan  6 19:00:10 myhost sudo:   prince : TTY=pts/8 ; PWD=/home/prince/backups ; USER=root ; COMMAND=/usr/bin/tar -czvf backup.tar.gz /var/www
Jan  6 20:00:15 myhost sudo:   sanjay : TTY=pts/9 ; PWD=/var/log/mysql ; USER=root ; COMMAND=/usr/bin/tail -f /var/log/mysql/error.log

🚨 Detected Alerts Output

{
    "Brute-Force SSH Attack": [
        "192.168.1.10 failed to login 4 times (possible brute-force)",
        "203.0.113.15 failed to login 4 times (possible brute-force)"
    ],
    "Suspicious Login Times": [
        "chandru logged in from 192.168.1.10 at odd hour 2:00 (2025-01-01 02:00:01)",
        "chandru logged in from 192.168.1.10 at odd hour 2:00 (2025-01-01 02:00:04)",
        "anthony logged in from 10.0.0.20 at odd hour 3:00 (2025-01-01 03:00:05)",
        "sanjay logged in from 192.168.1.25 at odd hour 4:00 (2025-01-01 04:00:15)",
        "chandru logged in from 192.168.1.10 at odd hour 5:00 (2025-01-01 05:00:01)",
        "chandru logged in from 192.168.1.10 at odd hour 1:00 (2025-01-02 01:00:01)",
        "anthony logged in from 10.0.0.20 at odd hour 1:00 (2025-01-02 01:00:05)",
        "prince logged in from 172.16.0.5 at odd hour 3:00 (2025-01-02 03:00:10)",
        "sanjay logged in from 192.168.1.25 at odd hour 3:00 (2025-01-02 03:00:15)",
        "chandru logged in from 192.168.1.10 at odd hour 5:00 (2025-01-02 05:00:01)",
        "chandru logged in from 192.168.1.10 at odd hour 0:00 (2025-01-03 00:00:01)",
        "anthony logged in from 10.0.0.20 at odd hour 2:00 (2025-01-03 02:00:05)",
        "prince logged in from 172.16.0.5 at odd hour 4:00 (2025-01-03 04:00:10)",
        "sanjay logged in from 192.168.1.25 at odd hour 4:00 (2025-01-03 04:00:15)"
    ],
    "Logins from New or Rare IPs": [
        "New IP 192.168.1.10 used by chandru at 2025-01-01 02:00:01",
        "New IP 10.0.0.20 used by anthony at 2025-01-01 03:00:05",
        "New IP 192.168.1.25 used by sanjay at 2025-01-01 04:00:15",
        "New IP 172.16.0.5 used by prince at 2025-01-02 03:00:10"
    ],
    "Multiple Failed Then Successful Login": [
        "192.168.1.10 user chandru succeeded login after 4 failures"
    ],
    "Sudden Sudo Access": [
        "chandru ran sudo command '/bin/su - root  # Sudden sudo' at 2025-01-06 16:00:01",
        "anthony ran sudo command '/usr/sbin/apache2ctl graceful' at 2025-01-06 16:00:05",
        "prince ran sudo command '/usr/bin/git pull origin main' at 2025-01-06 08:00:10",
        "sanjay ran sudo command '/usr/sbin/nginx -s reload' at 2025-01-06 08:00:15",
        "chandru ran sudo command '/usr/bin/apt-get update' at 2025-01-06 17:00:01",
        "anthony ran sudo command '/usr/bin/vim /etc/ssh/sshd_config' at 2025-01-06 18:00:05",
        "prince ran sudo command '/usr/bin/tar -czvf backup.tar.gz /var/www' at 2025-01-06 19:00:10",
        "sanjay ran sudo command '/usr/bin/tail -f /var/log/mysql/error.log' at 2025-01-06 20:00:15"
    ]
}

⚙️ How It Works

1. Detect Log Type

The tool scans the first few lines of the log and identifies its format automatically.

2. Parse the File

Using regex and custom handlers for each log type, the tool extracts IPs, timestamps, status codes, and more.

3. Analyze & Export

Data is analyzed for threats (e.g., brute-force patterns, odd-hour logins) and exported in user-specified formats.

🚀 How to Use

Step 1: Clone the Repository

$ git clone https://github.com/chandruthehacker/log-parser.git

Step 2: Chande directory to log-parser

$ cd log-parser

Step 3: Install Dependencies

$ pip install -r requirements.txt

Step 4: Run the Tool

$ python parser.py logs/auth.log [-o json] [-t authlog]

💻 Source Code

log-parser

👨‍💻 Author Info

Created by Chandraprakash C – BSc Computer Science | Cybersecurity Enthusiast
cyberchandru87@gmail.com | GitHub | LinkedIn