Web Scraping
Showing 36 of 191 projects
A low-level Chrome DevTools Protocol client written in Kotlin with RxJava3 for reactive composability.
A Delphi HTML parser library with read/write capabilities, based on wr960204's HtmlParser.
A Node.js module for interacting with the Steam store website, enabling programmatic account and store operations.
A Go library for extracting readable content from webpages using Open Graph metadata and arc90's readability algorithm.
A Node.js library to generate HTML embeds from URLs using oEmbed, Open Graph, and meta tags.
A multiplatform Kotlin client for the Chrome DevTools Protocol using coroutines and code generation.
NPM package and CLI tool for saving web pages as single HTML files, implemented in TypeScript.
A GitHub pinned gist that displays your Medium follower count and latest articles with claps.
A web application that scrapes CTF writeups from CTFtime.org and helps users organize which ones to read first.
An Elixir library for pipeable HTML transformations using CSS selectors, powered by the Lexbor C library.
A Go library for scraping recipe data from websites using Schema.org and custom scrapers.
A fast HTML parser for Free Pascal and Delphi that enables DOM-like element access without regular expressions.
An AutoHotkey WebDriver library for automating Chromium-based browsers without requiring Selenium or WebSocket.
A high-level Rust API to control headless Chrome or Chromium over the DevTools Protocol, equivalent to Puppeteer.
A fast high-level web crawling and scraping framework for Elixir, built on Broadway.
A Crystal language client library for Selenium WebDriver to automate browser interactions.
A Go library for multiplexing HTTP clients based on source IP addresses to bypass rate limits and manage network interfaces.
A Go library for parsing Netscape cookies.txt format files used by browsers and tools.
A tool to upload all Pokémon images as custom emoji to a Slack workspace.
A Go library for generating realistic, up-to-date user-agent strings from a real-world database.
A lightweight package manager that scrapes AppImageHub to install, update, and manage AppImage applications.
Elixir/Erlang bindings for the myhtml C library, providing fast and safe HTML parsing.
A fast, stealth-first browser automation library for Go using direct Chrome DevTools Protocol over WebSocket.
A TypeScript library that converts XPath expressions to CSS selectors using a recursive parser.
A Python-based web scraper that searches for AppImage releases across GitHub and other sources.
A Crystal library for fetching and validating HTTP/HTTPS proxies from multiple providers.
An Elixir HTTP client for making requests through the Tor network with SOCKS5 proxy support and IP rotation.
An Elixir library for discovering and extracting favicon URLs from websites.
A PHP tool that checks target websites for links to your site or competitors' sites for SEO analysis.
An Elixir module for decoding and encoding XML into a tree structure with browser-like tolerance.
A fast, JSoup-inspired HTML5 parser and DOM manipulation library for the V programming language, powered by Lexbor.
A Python CLI tool for converting web pages to clean, readable Markdown format, optimized for feeding documentation into LLMs.
A Go package for traversing, navigating, filtering, and processing trees of html.Node objects.
An Elixir library for unrolling (un-shortening) URLs by following redirects.
An Elixir library that pulls headlines and links from Reddit subreddits via JSON API.
Search npm packages via web scraping without requiring full package indexing.
Related Tags
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.