How to deploy SpiderFoot with only the core services?

Use Docker Compose without profiles: 'docker compose -f docker-compose.yml up --build -d' to run just PostgreSQL, Redis, API, worker, and frontend, reducing resource usage for basic setups.

SpiderFoot or Maltego for threat intelligence?

SpiderFoot is better for automated, self-hosted OSINT with extensive source integration and AI analysis, while Maltego excels in interactive, graph-based visualization for manual investigation. SpiderFoot's microservices offer more scalability but require more setup.

How to add custom modules to SpiderFoot?

Follow the Module Guide in the documentation: create Python modules in the plugins directory, using the ModernPlugin base class, and ensure they subscribe to appropriate event types for integration into the scan pipeline.

Does SpiderFoot support real-time alerting?

Yes, via GraphQL subscriptions for live scan progress and events, and the observability stack (Grafana, Prometheus) can be configured for alerts, but built-in notification systems require additional setup as per the monitoring section.

How to backup and restore SpiderFoot data?

Use the included MinIO object storage for logs and reports, and the pg-backup service for PostgreSQL daily dumps; refer to the storage and maintenance sections in the README for configuration details.

SpiderFoot

MITPythonv6.1.0

SpiderFoot is an open-source intelligence (OSINT) automation platform that integrates with 309+ data sources for threat intelligence and attack surface mapping.

Visit Website

What is SpiderFoot?

SpiderFoot is an open-source intelligence (OSINT) automation platform that automates the collection and analysis of data from over 309 sources for threat intelligence and attack surface mapping. It helps security teams discover exposed assets, identify threats, and gather intelligence on targets like domains, IPs, emails, and usernames. The platform supports both passive data gathering and active reconnaissance with integrated external tools.

Target Audience

Security professionals, including red teams, blue teams, penetration testers, and threat intelligence analysts, who need to automate OSINT collection, map organizational attack surfaces, and conduct reconnaissance.

Value Proposition

Developers choose SpiderFoot for its extensive module library, modern microservices architecture, and comprehensive feature set including AI analysis, vector search, and robust APIs. Its self-hostable, Docker-based deployment and strong security hardening make it a reliable and scalable alternative to commercial OSINT tools.

Overview

SpiderFoot automates OSINT for threat intelligence and mapping your attack surface. Codename Mirage

Use Cases

Best For

Automating reconnaissance and attack surface mapping for penetration testing

Related Projects

Community-curated · Updated weekly · 100% open source

Found a gem we're missing?

Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.

Submit a project Star on GitHub

GitHub

182 stars28 forks0 contributors

Monitoring organizational assets and detecting shadow IT for defensive security teams

Enriching threat intelligence with data from hundreds of OSINT sources

Conducting subdomain discovery and credential exposure checks

Generating automated cyber threat intelligence (CTI) reports with AI analysis

Self-hosting a full-featured OSINT platform with observability and scalability

Not Ideal For

Teams performing quick, ad-hoc OSINT checks without automation needs
Organizations with limited infrastructure or budget for running 23+ Docker containers
Use cases requiring strict data sovereignty and minimal external API dependencies
Projects needing a simple, monolithic application without microservices complexity

Pros & Cons

Pros

Extensive Data Source Integration

Integrates with 309+ modules for passive and active intelligence gathering, covering DNS, social media, threat intel, and more, as listed in the module categories.

Modern Microservices Architecture

Deployable via Docker Compose or Kubernetes with 23+ optional services, enabling scalability, modularity, and observability, as shown in the architecture diagram.

AI-Powered Analysis Features

Includes six LLM-powered agents for automated validation, summarization, and threat intelligence reporting, enhancing analysis efficiency without manual intervention.

Comprehensive Security Hardening

Implements JWT authentication, input validation, Docker security opts, and frontend protections like CSP, with a 9.5+ composite security score mentioned in the README.

Cons

High Resource Requirements

The full stack deployment involves 23+ Docker containers, which can be resource-intensive and complex to manage, especially for smaller teams or low-budget environments.

Steep Initial Setup Complexity

Requires configuring environment variables, Docker profiles, and multiple services, which might be daunting for new users, as indicated in the Quick Start section with detailed steps.

Dependency on External APIs

Many modules require API keys for premium data sources, and the active scan worker depends on 33+ external tools, adding operational overhead and potential cost.

Frequently Asked Questions

Home

IAM

SecLists

SecLists is the security tester's companion. It's a collection of multiple types of lists used during security assessments, collected in one place. List types include usernames, passwords, URLs, sensitive data patterns, fuzzing payloads, web shells, and many more.

Information gathering framework for phone numbers

Stars17,101

Forks5,181

Last commit6 months ago

Awesome Threat Intelligence

A curated list of Awesome Threat Intelligence resources

Stars10,471

Forks1,803

Last commit1 month ago

PhishingKitTracker

PROJECT PAUSED 1/11/22, Tracking Threat Actor Emails in Phishing Kits. CC @PhishKitTracker on twitter if you find a #threatactoremail in #phishingkit , maintained by @neonprimetime

Stars101

Forks16

Last commit4 years ago

#attack-surface-mapping