Log Parser
A log parsing and analysis tool for security operations and DevOps.
Features
- Multi-format Support - nginx, apache, syslog, application logs
- Auto-detection - Automatically detect log format
- Key Extraction - Extract IPs, timestamps, error codes, URLs
- Filtering - Filter logs by IP, status code, time range
- Statistics - Generate summary reports
- Error Detection - Identify and highlight error entries
Usage
python3 skills/log-parser/scripts/log_parser.py <action> [options]
Actions
| Action | Description |
|---|---|
parse | Parse log file and extract fields |
stats | Generate statistics report |
filter | Filter logs by criteria |
errors | Extract error entries only |
top | Top N items (IPs, URLs, etc.) |
Options
| Option | Type | Default | Description |
|---|---|---|---|
--file | string | - | Log file path |
--format | string | auto | Log format (auto, nginx, apache, syslog) |
--limit | int | 100 | Max results to return |
--filter-ip | string | - | Filter by IP address |
--filter-status | string | - | Filter by status code |
--top-field | string | - | Field for top N (ip, url, status) |
Supported Log Formats
nginx
192.168.1.1 - - [22/Mar/2026:14:00:00 +0800] "GET /api/test HTTP/1.1" 200 1234
apache
192.168.1.1 - - [22/Mar/2026:14:00:00 +0800] "GET /api/test HTTP/1.1" 200 1234
syslog
Mar 22 14:00:00 server sshd[12345]: Failed password for root from 192.168.1.1
application logs (JSON)
{"timestamp": "2026-03-22T14:00:00Z", "level": "ERROR", "message": "..."}
Examples
# Parse log file
python3 skills/log-parser/scripts/log_parser.py parse --file /var/log/nginx/access.log
# Generate statistics
python3 skills/log-parser/scripts/log_parser.py stats --file /var/log/nginx/access.log
# Filter by IP
python3 skills/log-parser/scripts/log_parser.py filter --file /var/log/nginx/access.log --filter-ip 192.168.1.1
# Get top 10 IPs
python3 skills/log-parser/scripts/log_parser.py top --file /var/log/nginx/access.log --top-field ip --limit 10
# Extract errors
python3 skills/log-parser/scripts/log_parser.py errors --file /var/log/nginx/access.log
Use Cases
- Security Analysis - Identify suspicious IPs, failed logins
- Performance Monitoring - Find slow requests, errors
- Traffic Analysis - Top URLs, user agents
- Debugging - Extract error entries quickly
Output Format
All results are returned in JSON format:
{
"success": true,
"total": 1000,
"parsed": 998,
"entries": [...],
"stats": {...}
}
Current Status
In development.