stenographer
A high-performance packet capture solution that buffers all network traffic to disk for fast retrieval of specific subsets.
What is stenographer?
Stenographer is a full-packet-capture solution that buffers all network traffic to disk for security monitoring purposes. It's designed to capture packets at very high speeds while managing disk space automatically, then provide fast access to specific subsets of packets for intrusion detection and incident response investigations.
Security analysts, network engineers, and incident response teams who need to capture and quickly query network traffic for forensic analysis and threat detection.
Developers choose Stenographer for its exceptional performance in high-speed packet capture and efficient disk management, combined with a simple query interface that enables rapid retrieval of specific traffic subsets without the overhead of complex packet processing.
Overview
Stenographer is a packet capture solution which aims to quickly spool all packets to disk, then provide simple, fast access to subsets of those packets. Discussion/announcements at stenographer@googlegroups.com
Use Cases
Best For
- Capturing network traffic at multi-gigabit speeds for security monitoring
- Storing packet history for intrusion detection and forensic investigations
- Quickly retrieving specific packets based on IP addresses, ports, or time ranges
- Incident response teams needing rapid access to historical network data
- Security operations centers requiring continuous full-packet capture
- Network forensic analysis where disk space management is critical
Not Ideal For
- Real-time network analysis requiring TCP stream reassembly or deep packet inspection
- Scenarios where analysts need to extract more than 1% of captured traffic for batch processing or data mining
- Teams seeking an integrated GUI-based packet analysis solution or out-of-the-box IDS features
- Environments with limited disk I/O or single-core systems, as it's optimized for multi-disk, high-performance setups
Pros & Cons
Pros
Can write packets at speeds up to ~10Gbps on multi-core, multi-disk machines, making it suitable for high-volume network monitoring as described in the README.
Automatically deletes oldest packets when disk limits are reached, ensuring continuous capture without manual intervention, which is critical for long-term security logging.
Quickly reads small subsets (<1%) of stored packets based on analyst queries, ideal for incident response where speed is paramount.
Uses a subset of BPF with time-based filters, allowing analysts to specify packet subsets easily without complex syntax, as detailed in the Query Language section.
Stenoread CLI automates packet retrieval and passes data to tcpdump for advanced filtering and output formatting, enhancing flexibility for security workflows.
Cons
Explicitly not designed for complex processing like TCP stream reassembly, limiting analysis to raw packet capture and requiring complementary tools for deeper insights.
Reading back large amounts (>1%) of packets is discouraged due to disk read-write competition, as admitted in the README, making it unsuitable for data-heavy extractions.
Requires installing specific packages (e.g., libaio-dev) and building with Go, which can be non-trivial and time-consuming compared to drop-in solutions.
Query language lacks features like hostname resolution or advanced BPF primitives, restricting filtering capabilities for complex security investigations.
Frequently Asked Questions
Related Projects
MaltrailMalicious traffic detection system
Tsunami Security ScannerTsunami is a general purpose network security scanner with an extensible plugin system for detecting high severity vulnerabilities with high confidence.
ArkimeArkime is an open source, large scale, full packet capturing, indexing, and database system.
MolochArkime is an open source, large scale, full packet capturing, indexing, and database system.
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.