How to filter logs by time range with logdissect?

Use the --range option with a format like YYYYMMDDhhmm-YYYYMMDDhhmm, or shortcuts like 2017020412 for 20170204120000. For relative times, --last with units (e.g., 10m) works for recent periods.

Can logdissect parse Apache access logs?

Yes, it includes a 'webaccess' parser for standard web server logs. You can specify it with -p webaccess or rely on defaults for common file patterns.

Logdissect vs grep for log analysis?

Logdissect offers structured parsing and filtering by log attributes like time or process, while grep is better for simple text searches. Use logdissect when you need context-aware filtering beyond raw text matching.

How to export logs to JSON using logdissect?

Use --linejson for line-by-line JSON or --sojson for single-object JSON, with --pretty for formatted output. This converts logs into structured data for further processing or visualization.

Does logdissect support real-time log monitoring?

No, it only processes static log files in batch mode. For real-time analysis, you'd need to integrate it with other tools or consider streaming-focused alternatives.

What happens if I copy log files without preserving timestamps?

Year assignment may fail, causing incorrect date parsing. The README recommends using cp -p or scp -p to maintain modification times, or use JSON output for reliable re-parsing.

Open-Awesome

Logdissect

MITPythonv3.1.1

A CLI utility and Python library for parsing, filtering, and analyzing log files and other structured data.

GitHub

161 stars23 forks0 contributors

What is Logdissect?

Logdissect is a command-line utility and Python library for analyzing log files and other structured data. It parses, filters, merges, and exports log data from various sources, helping developers and system administrators extract meaningful insights from logs efficiently. It solves the problem of manual log inspection by providing powerful filtering and export capabilities.

Target Audience

System administrators, DevOps engineers, and developers who need to analyze log files from servers, applications, or network devices. It's particularly useful for those working with syslog, web server logs, or other structured log formats.

Value Proposition

Developers choose Logdissect for its combination of a simple CLI interface and a stable Python module, allowing both quick ad-hoc analysis and programmatic integration. Its modular parser system and flexible filtering options make it adaptable to various log formats without complex configuration.

Overview

CLI utility and Python module for analyzing log files and other data.

Use Cases

Best For

Filtering system logs by specific time ranges or processes
Parsing and analyzing web server access logs for traffic patterns
Extracting specific events from Cisco IOS or network device logs
Converting log files to JSON for further processing or visualization
Batch processing multiple log files with consistent filtering rules
Quickly searching logs for specific IP addresses or error patterns

Not Ideal For

Environments requiring real-time log streaming or live monitoring
Projects with custom or proprietary log formats not in the built-in parser list
Teams that prefer graphical interfaces or web-based dashboards over CLI tools
Situations where log files are frequently moved without preserving modification timestamps

Pros & Cons

Pros

Flexible Filtering Options

Supports grep-like patterns and multiple criteria like time ranges, hosts, and processes, as detailed in the extensive options list for precise log extraction.

Modular Parser System

Includes built-in parsers for common formats such as syslog, web access logs, and Cisco IOS, enabling versatile analysis without custom code for standard sources.

Robust Time Handling

Intelligently assigns years to timestamps using file modification times, which helps parse logs spanning multiple years, though it requires careful file handling as noted.

Dual Interface Support

Provides a straightforward CLI for quick ad-hoc analysis and a stable Python module for programmatic integration, catering to both interactive and automated workflows.

Cons

Metadata Dependency for Years

Relies on file modification times to assign years, which can fail if logs are copied without preserving timestamps, leading to incorrect parsing as warned in the notes.

Limited Parser Extensibility

The parser list is fixed; adding support for new log formats requires modifying the source code or contributing, which may not be user-friendly for non-developers.

No Real-Time Capabilities

Focuses solely on batch file processing without features for streaming or live log analysis, making it unsuitable for dynamic monitoring needs.

Frequently Asked Questions

Related Projects

Bat

A cat(1) clone with wings.

Stars60,892

Forks4,462

Last commit2 days ago

Maigret

🕵️‍♂️ Collect a dossier on a person by username from 6K websites

Stars39,282

Forks3,096

Last commit3 days ago

btop

A monitor of resources

Glances an Eye on your system. A top/htop alternative for GNU/Linux, BSD, macOS and Windows operating systems.

Stars33,825

Forks1,837