A Node.js client for Apache Kafka 0.9 and later, providing producers, consumers, and administrative APIs.
Kafka-node is a Node.js client library for Apache Kafka that allows JavaScript applications to interact with Kafka clusters. It provides APIs for producing messages to Kafka topics, consuming messages with various consumer models, and performing administrative tasks like topic management. It solves the problem of integrating Kafka's distributed streaming platform into Node.js-based microservices and data processing applications.
Node.js developers building real-time data pipelines, event-driven microservices, or stream processing applications that need to integrate with Apache Kafka. It's also suitable for teams migrating Java-based Kafka clients to Node.js environments.
Developers choose Kafka-node because it offers a comprehensive, production-ready Node.js interface to Kafka with support for modern Kafka features (0.9+), multiple consumption patterns, and built-in stream integration. Its active maintenance and detailed troubleshooting guide make it a reliable choice for Node.js Kafka integration.
Node.js client for Apache Kafka 0.8 and later.
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Supports multiple producer types (Producer, HighLevelProducer, ProducerStream) and consumer models (Consumer, ConsumerGroup, ConsumerStream), offering flexibility for various use cases as detailed in the API section.
Provides ProducerStream and ConsumerGroupStream for seamless integration with Node.js streams, enabling real-time data pipelines without manual buffer management, with examples in the README.
Includes Admin APIs for topic creation, group management, and configuration inspection directly from Node.js, reducing reliance on external tools, as shown in the createTopics and describeConfigs methods.
Supports SSL connections, SASL/PLAIN authentication, and message compression (GZip, snappy), ensuring secure and efficient communication, with configuration options documented in KafkaClient.
Optional dependencies like snappy can cause node-gyp errors during npm install, often requiring the --no-optional flag or manual fixes, as highlighted in the troubleshooting FAQ.
Numerous options for clients, producers, and consumers (e.g., fetchMaxBytes, autoCommit) make setup error-prone and require careful tuning to avoid issues like stuck partitions.
As a Node.js-specific library, it's unsuitable for projects using other runtimes or languages, and lacks built-in TypeScript support compared to newer alternatives.