Highly extensible, highly scalable web crawler for production environments
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
A scalable web crawler framework for Java.
Open Source Web Crawler for Java
SDK for building low-latency and scalable web crawlers
Scrapes, parses, manipulates and cleans HTML