ThreatIngestor
An extendable Python tool to extract and aggregate Indicators of Compromise (IOCs) from various threat intelligence feeds.
What is ThreatIngestor?
ThreatIngestor is an open-source Python tool that automates the extraction and aggregation of Indicators of Compromise (IOCs) from various online threat intelligence feeds. It monitors sources like Twitter, RSS feeds, and web pages to collect malicious IPs, domains, and YARA signatures, then forwards them to security analysis platforms. The tool helps security teams efficiently gather and process threat data to improve their defensive posture.
Security analysts, threat intelligence teams, and SOC (Security Operations Center) personnel who need to automate the collection and aggregation of threat indicators from diverse online sources.
Developers choose ThreatIngestor for its extensible plugin architecture, which allows seamless integration with existing workflows and tools like MISP and ThreatKB. Its ability to handle multiple input sources and output destinations, combined with continuous polling, makes it a flexible and powerful solution for threat intelligence automation.
Overview
Extract and aggregate threat intelligence.
Use Cases
Best For
- Automating IOC collection from security blogs via RSS feeds
- Monitoring Twitter for real-time threat intelligence and C2 indicators
- Integrating extracted IOCs into MISP or ThreatKB for analysis
- Extracting text and IOCs from images using OCR capabilities
- Building custom threat intelligence pipelines with plugin extensions
- Aggregating threat data from GitHub repositories and gists
Not Ideal For
- Real-time threat detection requiring sub-minute IOC ingestion
- Organizations using commercial SIEMs with built-in threat feed integrations (e.g., Splunk ES, IBM QRadar)
- Teams without Python development resources to manage plugins and dependencies
- Environments with strict data privacy laws restricting web scraping or social media monitoring
Pros & Cons
Pros
Supports custom source and operator plugins, allowing integration with any workflow, as detailed in the developing documentation for flexibility.
Monitors diverse sources like Twitter, RSS, GitHub, and web pages, with pre-configured examples such as InQuest's Twitter list for quick setup.
Forwards IOCs to systems like MISP, ThreatKB, SQL databases, and AWS SQS, enabling seamless data flow into existing analysis pipelines.
Uses OpenCV and Tesseract to extract text from images, useful for IOCs in screenshots, though it requires Python 3.7+ and additional dependencies.
Cons
Requires Python 3.6+ with development headers, optional dependencies for plugins like image extraction, and YAML configuration, making deployment non-trivial.
Needs a Twitter developer account with an application process, adding overhead and potential delays for real-time monitoring setups.
Runs on configurable intervals (default 15 minutes), lacking real-time event-driven ingestion, which may miss fast-evolving threats.
Frequently Asked Questions
Related Projects
Sigma RulesMain Sigma Rule Repository
YARAThe pattern matching swiss knife
ViperBinary analysis and management framework
GRASSMARLINProvides situational awareness of Industrial Control Systems (ICS) and Supervisory Control and Data Acquisition (SCADA) networks in support of network security assessments. #nsacyber
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.