hyphe
A research-driven web crawler for building and analyzing curated web corpora as networks of web entities.
What is hyphe?
Hyphe is a web crawler and corpus curation tool designed for researchers in the social sciences and digital humanities. It allows users to build curated collections of web pages (corpora) by defining 'web entities'—custom units like websites or page groups—and crawling links between them to generate analyzable networks. It solves the problem of conducting structured, reproducible web studies beyond simple list-based scraping.
Researchers, academics, and analysts in social sciences, digital humanities, and media studies who need to systematically collect and analyze web data as networks for projects like controversy mapping, community detection, or hyperlink analysis.
Developers and researchers choose Hyphe for its unique curation-first approach, built-in web interface for iterative exploration, and focus on generating meaningful network data from the web rather than just raw page collection. It bridges the gap between automated crawling and manual, methodologically-driven corpus building.
Overview
Websites crawler with built-in exploration and control web interface
Use Cases
Best For
- Mapping hyperlink networks within a specific online community or discourse
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.