A high-performance, columnar, in-memory storage engine with bitmap indexing and zero-allocation queries, written in Go.
Column is a Go library that provides a high-performance, columnar, in-memory storage engine. It organizes data in a structures-of-arrays (SoA) layout to minimize cache misses and uses bitmap indexing for rapid filtering and querying, making it suitable for applications requiring fast data access and real-time analytics.
Go developers building applications that need fast in-memory data storage and querying, such as real-time analytics systems, high-throughput data processing pipelines, or game servers managing large numbers of entities.
Developers choose Column for its combination of columnar storage and bitmap indexing, which enables zero-allocation queries, SIMD-enabled operations, and ACID-like transactions, offering a balance of speed, efficiency, and transactional integrity in a lightweight package.
High-performance, columnar, in-memory store with bitmap indexing in Go
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Queries and iterations avoid heap allocations, with benchmarks showing operations like count taking 2057 ns/op and updates at 202.8 ns/op for 100k rows.
Uses bitmap indexes for fast filtering, enabling up to 100x speed improvements for indexed queries over full scans, as demonstrated in the examples.
Supports ACID-like transactions with commit and rollback, ensuring data integrity during concurrent writes using sharded latches.
Includes change data capture (CDC) capabilities via commit logging, allowing for easy replication and synchronization between collections.
Features TTL columns and a cleanup goroutine for automatic row eviction, useful for time-sensitive data without manual deletion.
Columns must be explicitly defined with types, and schema changes aren't dynamic, making it less flexible for evolving data models compared to NoSQL stores.
Using primary keys requires additional hash table lookups, adding latency compared to internal offsets, as noted in the README under 'Using Primary Keys'.
Snapshots do not carry schema information, requiring collections to be pre-initialized with identical schemas before restoration, which can be error-prone.
Limited to Go applications, restricting use in polyglot environments and lacking built-in network interfaces for client-server architectures.