Scraping
Showing 20 of 20 projects
Scrapy, a fast high-level web crawling & scraping framework for Python.
A fast and elegant scraping and crawling framework for Go, designed for extracting structured data from websites.
A scalable Java framework for building web crawlers, covering downloading, URL management, content extraction, and persistence.
A Ruby library for automating web interaction, handling cookies, redirects, forms, and navigation.
A PHP and Symfony library for browser testing and web scraping using real browsers via the WebDriver protocol.
A PHP library to extract metadata, embed codes, and structured data from any web page using multiple protocols.
A modern, fast, privacy-respecting meta search engine written in Rust, offering a secure and ad-free search experience.
A high-level web crawling and scraping framework for Elixir, designed for data extraction and processing.
A modular Rust service framework for building programmable network proxies, clients, and servers with fine-grained control over packet flow.
A Python tool to automatically archive web content (videos, images, social media) from Google Sheets and other sources in a secure, verifiable way.
A high-performance, multithreaded command-line tool for downloading images from webpages.
A Go web scraping framework that extracts structured data from websites using CSS selectors, including JavaScript-rendered pages.
A functional HTML scraping and manipulation library for OCaml with CSS selector support.
An Elixir library for structured data extraction from websites, articles, and RSS/Atom feeds using information-retrieval techniques.
A fast, powerful, and extensible web crawling and scraping framework for Go, inspired by Scrapy.
A reliable, flexible, and fast Rust framework for web crawling and request-response services.
A sample project to search startup jobs scraped from various websites, built with Elixir/Phoenix backend and React/Redux frontend.
Fetch npm user profile information by scraping the npm website since no official API exists.
A Go package for fetching song lyrics from the Wikia (Lyrics.wikia.com) website.
A Node.js API wrapper for accessing kebab-frites.info data programmatically.
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.