Web Scraping
Showing 36 of 191 projects
An Elixir library for extracting and curating the primary readable content from webpages.
A Node.js library to interact with Google Play APIs, enabling app details retrieval and downloads by emulating an Android device.
A declarative struct-tag-based HTML unmarshaling and web scraping library for Go built on goquery.
A fast, powerful, and extensible web crawling and scraping framework for Go, inspired by Scrapy.
A universal Python API for solving CAPTCHAs using multiple backend services including 2captcha, RuCaptcha, and browser automation.
A Vue component that generates link preview cards by fetching metadata from URLs.
An open-source project that uses Amazon Echo to control home automation software via a web scraper and Sinatra server.
A Java client library for instrumenting, inspecting, debugging, and profiling Chromium-based browsers via the Chrome DevTools Protocol.
R bindings to libcurl for performing fully customizable HTTP/FTP/SCP requests with flexible response handling.
A simple, flexible, extensible, and liberal RSS and Atom reader for Ruby with zero dependencies.
A comprehensive Python wrapper and execution manager for Chrome/Chromium using the Chrome Debugging Protocol.
A deterministic, skills-based browser agent that automates web interactions using predefined action recipes for popular sites.
A collection of Python applications demonstrating various use cases of ChatGPT, including chatbots, automation, and voice assistants.
An auto-generated Go client library for the Chrome DevTools Protocol, enabling programmatic browser control.
A Go tool and library for downloading URLs and files from Common Crawl and Wayback Machine web archives.
A serverless web spider service for monitoring websites and extracting content via CSS selectors.
A PHP client library for interacting with Chrome DevTools Protocol to automate and control Chrome/Chromium browsers.
A curated guide to R packages for web scraping, APIs, web services, and web technologies.
A D language HTTP client library inspired by Python's requests, offering a simple high-level API with low memory footprint and high performance.
Error-recovering streaming HTML5 and XML parsers for OCaml with lazy, non-blocking, and one-pass processing.
Python type wrappers and WebSocket client for the Chrome DevTools Protocol, enabling browser automation and debugging.
An open-source price tracker that automatically monitors product prices from e-commerce sites and sends alerts when prices drop.
XPath/XQuery 3.1 interpreter for Pascal with HTTP/S, JSON, HTML, and web scraping capabilities.
An async/await friendly Chrome debugging client with TypeScript support, designed for automation.
A Clojure library for controlling headless Chrome via the Chrome DevTools Protocol, with auto-generated low-level commands and higher-level automation utilities.
A reliable, flexible, and fast Rust framework for web crawling and request-response services.
A scriptable HTTP service and asyncio driver for headless Chrome, compatible with the Splash API.
An Elixir client library implementing the WebDriver protocol for browser automation with PhantomJS, Firefox, Chrome, and remote servers.
A fast, memory-efficient HTML5 parser with CSS selector support for Crystal, successor to myhtml.
A Go library that parses and deserializes HTML pages into structs using goquery and struct tags for web crawlers.
A sample project to search startup jobs scraped from various websites, built with Elixir/Phoenix backend and React/Redux frontend.
A Python script using BeautifulSoup to scrape video game sales data from VGChartz and save it as a CSV dataset.
An AI-native browser automation library using snapshot + ref targeting for deterministic interaction, built for AI agents.
Generates strongly-typed .NET Core wrappers for the Chrome DevTools Protocol from its JSON definition.
A DOM parser for React Native and Titanium that enables HTML parsing and manipulation using a standard DOM API.
A PHP library for programmatically interacting with the Steam Community website and its APIs.
Related Tags
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.