Showing 5 of 5 projects
A fast and elegant scraping and crawling framework for Go, designed for extracting structured data from websites.
An open-source Java web crawler that provides a simple interface for multi-threaded web crawling.
A sitemap generator for Next.js that creates sitemaps and robots.txt for static, pre-rendered, dynamic, and server-side pages.
An open-source, extensible, web-scale, archival-quality web crawler from the Internet Archive.
An open-source, extensible, web-scale, archival-quality web crawler from the Internet Archive.
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.