TorBot
An open-source intelligence (OSINT) tool for crawling and analyzing websites on the dark web and beyond.
What is TorBot?
TorBot is an open-source intelligence (OSINT) tool built for crawling and analyzing websites, particularly those on the dark web (.onion domains). It helps security professionals and researchers map site structures, validate links, and export data for further investigation. The tool supports both standard web crawling and anonymous access via Tor.
Security researchers, penetration testers, digital forensics investigators, and OSINT analysts who need to explore and analyze dark web or other website structures programmatically.
TorBot provides a specialized, open-source alternative for dark web OSINT with features like visual link tree generation, Tor integration, and data export, all built on a modular Python foundation that encourages community contributions and customization.
Overview
Dark Web OSINT Tool
Use Cases
Best For
- Mapping the link structure of dark web (.onion) sites
- Conducting OSINT investigations on hidden services
- Validating live links during security assessments
- Exporting website crawl data as JSON for analysis
- Visualizing relationships between web pages in a tree format
- Automating dark web reconnaissance tasks
Not Ideal For
- Projects requiring real-time monitoring or automated alerts for dynamic content
- Users needing advanced data analysis like sentiment detection or pattern recognition beyond link mapping
- Teams preferring graphical user interfaces over command-line tools
- Investigations where high anonymity via Tor is unnecessary, as simpler web crawlers might suffice
Pros & Cons
Pros
Optimized for .onion sites with Tor integration via SOCKS5 proxy, making it a focused tool for dark web OSINT, as highlighted in the features list.
Exports crawl results as JSON files or visual link trees, facilitating further analysis and reporting, as specified in the options.
Generates visual trees and tables of link relationships, aiding in quick data comprehension, which is a key feature in the README.
Built on Python with a modular design, encouraging community contributions and customization, as stated in the philosophy.
Includes a built-in updater via the --update flag, allowing users to easily fetch the latest stable version without manual intervention.
Cons
The README explicitly marks features like saving links to a database and outputting HTML as 'Not done,' indicating gaps in promised functionality.
Requires manual Tor setup and configuration for anonymity, which can be cumbersome for users unfamiliar with proxy settings and network setups.
Lacks integrated features for keyword search, social media analysis, or screenshot capture, as noted in the curated features list as pending.
No graphical user interface, which may deter users who prefer visual, point-and-click tools for OSINT tasks.
Frequently Asked Questions
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.