A Go library for writing Storm spouts and bolts that communicate with Storm shells via the multilang protocol.
GoStorm is a Go library that implements the communications protocol required to write Apache Storm spouts and bolts in Go. It solves the problem of integrating non-Java components into Storm topologies by providing a multilang protocol implementation, allowing Go code to communicate with Storm shells. The library supports various encoding schemes for optimized performance and includes testing utilities for development without a Storm cluster.
Go developers building real-time stream processing applications within Apache Storm ecosystems, particularly those needing to write high-performance spouts or bolts in Go rather than Java.
Developers choose GoStorm because it provides a robust, protocol-compliant way to write Storm components in Go with support for multiple encodings, including high-performance Protocol Buffers. It offers flexibility for testing and integrates seamlessly with existing Storm topologies.
GoStorm is a Go library that implements the communications protocol required to write Storm spouts and Bolts in Go that communicate with the Storm shells.
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Fully implements and enforces the Storm multilang protocol, ensuring reliable communication between Go components and Java-defined topologies through shell spouts or bolts.
Supports multiple encoding schemes, from JSON for compatibility to Protocol Buffers for high throughput, allowing developers to balance performance and ease of integration.
Provides mock output collectors for local testing of spouts and bolts without a running Storm cluster, though the README admits these are immature.
Enables binary encoding with Protocol Buffers for reduced latency and improved throughput, especially with the hybrid and protobuf schemes for Storm 0.9.2+.
Requires Java-defined topologies and shell components for each Go spout or bolt, adding setup complexity and dependency on the Storm ecosystem.
The README states that mock collectors lack full Storm semantics and are a basic way to test, making comprehensive local debugging challenging.
For best performance with protobuf encoding, it relies on external projects like protoshell and specific Storm versions, increasing maintenance and compatibility risks.