Web Spider
4 projects
Showing 4 of 4 projects
SpidrRuby
A versatile Ruby web spidering library for crawling sites, domains, or specific links with extensive filtering and callback support.
#web-crawling#spider#crawler
Stars835
Forks107
Last commit6 months ago
crawleyGo
A fast, Unix-style command-line web crawler that extracts links, resources, and API endpoints from web pages.
#api-discovery#resource-discovery#link-extraction
Stars340
Forks18
Last commit6 days ago
antchGo
A fast, powerful, and extensible web crawling and scraping framework for Go, inspired by Scrapy.
#web-crawling#concurrent#crawler
Stars266
Forks40
Last commit6 years ago
SpiderlessJavaScript
A serverless web spider service for monitoring websites and extracting content via CSS selectors.
#vue-js#api#content-extraction
Stars185
Forks24
Last commit3 years ago
Related Tags
Community-curated · Updated weekly · 100% open source
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.