test-crawler

test-crawler is an end-to-end testing tool that crawls websites and performs snapshot comparisons to detect visual regressions. It helps developers ensure UI consistency by identifying unintended visual changes across web pages. The tool is fully open-source and supports both self-hosting and direct use via GitHub.

Key Features

• Spider Bot Crawling — Automatically discovers and crawls all links from a starting URL, recursively exploring child pages.
• URLs List Crawling — Crawls a predefined list of URLs provided via an endpoint, offering precise control over tested pages.
• Visual Snapshot Comparison — Generates screenshots and highlights differences with visual indicators (yellow/red/green rectangles).
• Code Injection — Allows injecting custom JavaScript (using Puppeteer) to manipulate pages, remove dynamic content, or simulate user interactions before capturing screenshots.
• Pins Management — Enables creating reference screenshots (pins) for comparison, with tools to validate or report differences.
• CLI & CI Integration — Includes a command-line interface for automated testing and supports continuous integration workflows (e.g., GitHub Actions).
• Storybook Support — Provides code snippets to facilitate crawling and testing Storybook components via iframe URLs.

Philosophy

test-crawler emphasizes flexibility and control in visual regression testing, allowing users to tailor crawling behavior and page interactions through code injection while maintaining a simple, self-hostable setup.