A blazingly fast, highly scalable graph-based stream processing framework for latency-critical applications like electronic trading and real-time AI.
Wingfoil is a stream processing framework designed for latency-critical applications like electronic trading and real-time AI systems. It simplifies receiving, processing, and distributing streaming data using an efficient directed acyclic graph (DAG) based execution engine. The framework handles execution complexity, allowing developers to focus on defining their data processing graphs.
Developers and quantitative engineers building high-frequency trading systems, real-time AI inference pipelines, or any application requiring ultra-low latency stream processing. It is suited for teams needing multi-language support (Rust/Python) and integration with financial data sources like KDB+.
Developers choose Wingfoil for its combination of ultra-low latency, high throughput, and simplicity in defining complex processing graphs. Its unique selling points include production-ready I/O adapters for systems like KDB+ and ZeroMQ, built-in backtesting capabilities, and seamless async/multi-threading support out of the box.
graph based stream processing framework
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Benchmarks demonstrate blazingly fast execution with a DAG-based engine optimized for high-throughput, latency-critical applications like electronic trading, as highlighted in the README.
Developers define calculation graphs while Wingfoil handles execution, making complex stream processing accessible, as shown in examples like the order book pipeline with minimal code.
Available as a Rust crate and Python package, with plans for WASM/JavaScript/TypeScript, enabling cross-language integration for diverse teams.
Historical data replay functionality allows strategy optimization without separate tools, seamlessly integrating with real-time graphs for testing.
Includes adapters for KDB+ tick data, CSV, etcd, and ZeroMQ (beta), facilitating quick integration with common data sources in financial and real-time systems.
Python support is in beta, and planned languages like JavaScript/TypeScript are not yet available, restricting immediate adoption for non-Rust/Python projects.
I/O adapters target specific systems like KDB+; for broader data sources, users may need to build custom adapters, increasing development effort.
Features such as ZeroMQ messaging are marked as beta, which could lead to instability or breaking changes in production environments.
Defining and debugging complex DAGs requires familiarity with stream processing paradigms, which may be challenging for teams new to this architecture.