A durable and asynchronous PHP data importer for consuming APIs and web data at scale with memory-efficient processing.
Porter is a PHP library for durable and asynchronous data imports, designed to fetch data from APIs, web scraping, or any external source and serve it as iterable record collections. It solves the problem of consuming data at scale by enabling memory-efficient processing and automatic recovery from network failures. The library provides a clean abstraction through providers, resources, and connectors, making it easy to build testable SDKs.
PHP developers building applications that need to import data from external APIs or web services at scale, especially those requiring robust error handling, asynchronous operations, and memory-efficient data processing.
Developers choose Porter for its combination of durability features, asynchronous capabilities, and memory-efficient design, which together provide a reliable and performant solution for data imports. Its interface triad promotes testability and clean architecture, distinguishing it from simpler HTTP client libraries.
:lipstick: Durable and asynchronous data imports for consuming data at scale and publishing testable SDKs.
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Processes data as iterable collections, enabling stream-like handling of large datasets without loading everything into memory, as emphasized in the record collections section.
Uses PHP fibers to run multiple imports concurrently, transforming network-bound operations to CPU-bound for optimal performance, with built-in throttling to manage connection rates.
Provides durability with configurable retry attempts and exponential backoff for failed fetches, handling intermittent network errors transparently via the ExponentialSleepFetchExceptionHandler.
Separates providers, resources, and connectors, allowing easy mocking and testing of data import layers, which maps well to API SDK development and is highlighted in the benefits section.
The README explicitly states a limitation: 'No end-to-end data streaming interface,' restricting use for continuous data flow scenarios.
Asynchronous features rely on PHP 8.1+ fibers; PHP 7 requires bug-prone coroutines with potential segfaults, making upgrades necessary for stable async performance.
Requires integration with a PSR-11 compatible container for provider registration, adding setup complexity and dependency management for lightweight applications.