How do I set up peer-to-peer replication with Hypercore?

You need to integrate a peer discovery system like Hyperswarm and use the replicate method to create streams between peers. This involves setting up servers or clients, handling the replication protocol manually, and managing connections with events like 'peer-add' and 'peer-remove'.

Hypercore vs IPFS: which is better for distributed storage?

Hypercore focuses on secure, append-only logs with real-time updates and sparse replication, ideal for data streams and collaborative apps. IPFS is more general-purpose for content-addressed, immutable file storage. Choose Hypercore for verifiable, sequential data; IPFS for decentralized file sharing.

How to encrypt data in Hypercore?

Hypercore supports encryption via custom modules that implement the HypercoreEncryption interface. You can set an encryption key using setEncryptionKey, and blocks are encrypted before storage and replication, with options for block-level encryption in the API.

Can Hypercore handle large video files efficiently?

Yes, but it requires chunking files into blocks, as Hypercore deals with data blocks up to MAX_SUGGESTED_BLOCK_SIZE (15MB). You'd append blocks sequentially and use sparse replication to download only needed parts, optimizing for large datasets.

What's the difference between Hypercore and Hyperbee?

Hypercore is a low-level append-only log for raw data streams, while Hyperbee is a B-tree built on top, providing key-value storage and querying. Use Hypercore for foundational data distribution; Hyperbee adds database-like functionality with indexing.

How to migrate from Hypercore 9 to version 10?

Migration is challenging due to incompatibility in storage format and wire protocol. You'll likely need to export data from version 9 cores and re-import into new version 10 cores, or rebuild from scratch, as there's no direct upgrade path.

hypercore — Secure Distributed Append-Only Log

What is hypercore?

Hypercore is a secure, distributed append-only log built for sharing large datasets and real-time data streams across a network. It allows peers to replicate data efficiently while ensuring integrity through cryptographic signatures and Merkle trees. The system supports sparse replication, enabling users to download only the specific data blocks they need.

Target Audience

Developers building decentralized applications, peer-to-peer systems, or distributed data pipelines that require verifiable, real-time data sharing.

Value Proposition

Developers choose Hypercore for its strong security guarantees, efficient sparse replication, and real-time capabilities, providing a reliable foundation for decentralized data streams without sacrificing performance.

Hypercore is a secure, distributed append-only log.

Use Cases

Best For

Building decentralized file-sharing or data synchronization tools
Creating real-time collaborative applications with verifiable data history
Implementing secure audit logs in distributed systems
Developing peer-to-peer databases or data pipelines
Sharing large scientific or sensor datasets across a network
Building blockchain-like data structures with append-only semantics

Not Ideal For

Projects requiring frequent data updates or deletions beyond append-only operations
Teams looking for an all-in-one distributed database with built-in querying and indexing
Applications where simplified data sync is preferred over cryptographic verification overhead
Environments with strict compatibility requirements, as version 10 breaks backward compatibility

Pros & Cons

Pros

Sparse Replication Efficiency

Allows downloading only specific blocks of interest, saving bandwidth and storage for large datasets, as highlighted in the sparse replication feature.

Real-Time Data Streaming

Supports live updates as data is appended, enabling secure and fast distribution of real-time data streams across peers, with events like 'append' for immediate notifications.

High Performance I/O

Uses a flat file structure to maximize I/O performance, ensuring efficient handling of data blocks without unnecessary overhead, as noted in the performant design.

Cryptographic Integrity

Employs signed Merkle trees for real-time verification of log integrity, providing strong security guarantees against tampering, with features like treeHash and proof generation.

Cons

Breaking Version Changes

Hypercore 10 is not compatible with earlier versions (9 and below), requiring migration and potentially disrupting existing deployments, as admitted in the README's release notes.

Complex Setup and Integration

Requires manual configuration for storage, replication, and peer discovery (e.g., with Hyperswarm), adding development overhead compared to plug-and-play solutions.

Advanced Storage Management

Features like mark-and-sweep for clearing blocks are intricate and error-prone, with caution notes in the README about caching issues, not suited for casual use.

What is hypercore?

Target Audience

Developers building decentralized applications, peer-to-peer systems, or distributed data pipelines that require verifiable, real-time data sharing.

Value Proposition

Use Cases

Best For

Building decentralized file-sharing or data synchronization tools
Creating real-time collaborative applications with verifiable data history
Implementing secure audit logs in distributed systems
Developing peer-to-peer databases or data pipelines
Sharing large scientific or sensor datasets across a network
Building blockchain-like data structures with append-only semantics

Not Ideal For

Projects requiring frequent data updates or deletions beyond append-only operations
Teams looking for an all-in-one distributed database with built-in querying and indexing
Applications where simplified data sync is preferred over cryptographic verification overhead
Environments with strict compatibility requirements, as version 10 breaks backward compatibility

Pros & Cons

Pros

Sparse Replication Efficiency

Allows downloading only specific blocks of interest, saving bandwidth and storage for large datasets, as highlighted in the sparse replication feature.

Real-Time Data Streaming

Supports live updates as data is appended, enabling secure and fast distribution of real-time data streams across peers, with events like 'append' for immediate notifications.

High Performance I/O

Uses a flat file structure to maximize I/O performance, ensuring efficient handling of data blocks without unnecessary overhead, as noted in the performant design.

Cryptographic Integrity

Employs signed Merkle trees for real-time verification of log integrity, providing strong security guarantees against tampering, with features like treeHash and proof generation.

Cons

Breaking Version Changes

Hypercore 10 is not compatible with earlier versions (9 and below), requiring migration and potentially disrupting existing deployments, as admitted in the README's release notes.

Complex Setup and Integration

Requires manual configuration for storage, replication, and peer discovery (e.g., with Hyperswarm), adding development overhead compared to plug-and-play solutions.

Advanced Storage Management

Features like mark-and-sweep for clearing blocks are intricate and error-prone, with caution notes in the README about caching issues, not suited for casual use.

hypercore

What is hypercore?

Overview

Use Cases

Best For

Not Ideal For

Pros & Cons

Pros

Cons

Frequently Asked Questions

Related Projects

Found a gem we're missing?

hypercore

What is hypercore?

Overview

Use Cases

Best For

Not Ideal For

Pros & Cons

Pros

Cons

Frequently Asked Questions

Related Projects

Found a gem we're missing?