Web Scraping
Showing 36 of 53 projects
A JavaScript library providing a high-level API to control Chrome or Firefox for browser automation, testing, and web scraping.
A JavaScript library providing a high-level API to control Chrome or Firefox browsers for automation and testing.
An open-source Python library and cloud service that enables AI agents to automate web browsing and task completion.
A framework for web testing and automation that drives Chromium, Firefox, and WebKit with a single API.
An open-source web crawler and scraper that converts web content into clean, LLM-ready Markdown for RAG, agents, and data pipelines.
A system for building agents that monitor the web and automate tasks, giving you full control over your data.
A system for building agents that perform automated tasks online, like a self-hosted IFTTT or Zapier.
A Model Context Protocol server that enables LLMs to automate web browsers using Playwright's accessibility tree.
A fast and elegant scraping and crawling framework for Go, designed for extracting structured data from websites.
A pure-JavaScript implementation of web standards like DOM and HTML for Node.js, enabling browser-like environments for testing and scraping.
Collects a dossier on a person by checking for accounts on 3000+ websites using only a username.
CLI tool and library for saving complete web pages as a single, self-contained HTML file.
A jQuery-like HTML manipulation and traversal library for Go, built on net/html and cascadia CSS selectors.
A free, self-hostable RSS feed aggregator that is lightweight, customizable, and supports multi-user access with instant push notifications.
Python library to automate Chromium, Firefox, and WebKit browsers with a single API for testing and automation.
A Node.js library for automating Chrome locally or headless on AWS Lambda with a simple API.
A Go library for driving browsers via the Chrome DevTools Protocol without external dependencies.
A Go library for driving browsers via the Chrome DevTools Protocol without external dependencies.
A Python web mining module with tools for scraping, NLP, machine learning, network analysis, and visualization.
A high-level Python wrapper for Selenium that simplifies web automation with a more intuitive API.
A navigation scripting and testing utility for PhantomJS and SlimerJS, easing web automation and functional testing.
A high-level Go driver for Chrome DevTools Protocol, designed for web automation and scraping.
A Node.js web crawler with server-side jQuery, rate limiting, and proxy support for efficient scraping.
A Python library and CLI tool for web crawling, scraping, and extracting main text, metadata, and comments from web pages.
A .NET library for parsing HTML5, SVG, MathML, and CSS with a standards-compliant DOM.
A pure-C HTML5 parsing library implementing the HTML5 parsing algorithm.
A pure Swift HTML parser with DOM, CSS, and jQuery-like methods for parsing, manipulating, and cleaning HTML across Apple platforms and Linux.
A simple yet powerful Go HTTP client with automatic decoding, debugging, retry, and HTTP fingerprinting support.
An open-source Java web crawler that provides a simple interface for multi-threaded web crawling.
A Ruby library for automating web interaction, handling cookies, redirects, forms, and navigation.
A Slack bot that reads and summarizes webpages, documents, and videos using ChatGPT, with voice chat capabilities.
A lightweight, efficient, and fast high-level web crawling and scraping framework for .NET.
A Ruby HTTP client library that wraps libcurl to make fast and reliable requests with parallel execution support.
Unofficial Python port of Puppeteer for headless Chrome/Chromium browser automation.
A .NET port of the official Node.js Puppeteer API for headless browser automation.
A .NET port of the official Node.js Puppeteer API for headless browser automation.
Related Tags
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.