Node-Crawler
A Node.js web crawler with server-side jQuery, rate limiting, and proxy support for efficient scraping.
What is Node-Crawler?
Node Crawler is a web crawling and scraping library for Node.js that provides server-side DOM manipulation using Cheerio, a jQuery-like API. It enables developers to efficiently extract data from websites with features like automatic charset conversion, priority queuing, and configurable concurrency.
Node.js developers building web scrapers, crawlers, or data extraction tools who need a robust, feature-rich solution with jQuery-like DOM querying capabilities.
Developers choose Node Crawler for its balance of ease of use with advanced features like rate limiting, proxy rotation, HTTP/2 support, and automatic charset handling, making it a reliable and efficient tool for web scraping without unnecessary complexity.
Overview
Web Crawler/Spider for NodeJS + server-side jQuery ;-)
Use Cases
Best For
- Scraping dynamic websites with server-side jQuery-like DOM manipulation using Cheerio.
- Building scalable web crawlers with configurable concurrency and rate limiting to avoid overwhelming target sites.
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.