How to set up an etcd cluster for Kubernetes?

etcd is the default data store for Kubernetes; follow Kubernetes documentation for etcd configuration, typically using kubeadm or manual setup with multiple nodes, TLS, and proper peer communication on port 2380. Ensure high availability by deploying odd-numbered nodes across zones.

etcd vs Redis for distributed locking?

etcd uses Raft for strong consistency, making it reliable for critical distributed locks, while Redis with RedLock offers faster but potentially less consistent locking. Choose etcd for guaranteed consistency in systems like Kubernetes, Redis for high-speed, less critical scenarios.

How to secure etcd with TLS certificates?

Generate certificates using tools like cfssl or OpenSSL, then configure etcd with --cert-file, --key-file, and --client-cert-auth flags. The documentation provides detailed steps for enabling automatic TLS and peer authentication in clustered setups.

What is the maximum data size etcd can handle efficiently?

etcd is optimized for small to medium-sized key-value pairs, typically under a few megabytes per key. Large datasets can degrade performance; it's best for metadata and configuration, not bulk data storage like files or logs.

How to backup and restore etcd data?

Use etcdctl snapshot save to create backups and etcdctl snapshot restore to recover data. Schedule regular backups and store them securely, as etcd holds critical cluster state, and restoration requires cluster reconfiguration in multi-node setups.

Open-Awesome

etcd

Apache-2.0Gov3.7.1

A distributed reliable key-value store for the most critical data of a distributed system.

Visit Website GitHub

52.2k stars10.5k forks0 contributors

What is etcd?

etcd is a distributed key-value store designed to hold the most critical data in a distributed system, providing a reliable foundation for coordination and configuration management. It focuses on simplicity, security, speed, and reliability, using the Raft consensus algorithm to ensure high availability and consistency. It is widely used in production, notably as a core component of platforms like Kubernetes.

Target Audience

Developers and operators building or managing distributed systems that require reliable coordination, configuration storage, or service discovery, such as those deploying Kubernetes clusters or similar cloud-native platforms.

Value Proposition

Developers choose etcd for its proven reliability in critical deployments, simple gRPC API, automatic TLS security, and high performance (benchmarked at 10,000 writes per second). Its use of the Raft consensus algorithm ensures strong consistency and fault tolerance, making it a trusted foundation for essential system data.

Overview

Distributed reliable key-value store for the most critical data of a distributed system

Use Cases

Best For

Storing configuration data for distributed applications that require consistency across multiple nodes.
Implementing service discovery mechanisms in microservices architectures.
Managing leader election and distributed locking in clustered systems.
Serving as the backing store for Kubernetes cluster state and coordination.
Building reliable distributed systems that need a highly-available key-value store with strong consistency guarantees.
Securing critical system data with automatic TLS and optional client certificate authentication.

Not Ideal For

Applications requiring only ephemeral, in-memory data storage without persistence or strong consistency guarantees.
Single-server deployments where the overhead of distributed consensus and clustering is unnecessary.
Projects needing complex data querying, relational features, or rich data models beyond simple key-value pairs.

Pros & Cons

Pros

Simple gRPC API

Offers a well-defined, user-facing gRPC API that simplifies integration, as highlighted in the project's focus on simplicity and ease of use.

Built-in Security

Supports automatic TLS with optional client certificate authentication, ensuring secure communication out of the box without extra configuration.

High Performance

Benchmarked at 10,000 writes per second, making it suitable for fast data operations in critical distributed systems, as stated in the README.

Proven Reliability

Uses the Raft consensus algorithm and undergoes rigorous robustness testing, ensuring high availability and consistency for production deployments.

Cons

Operational Complexity

Setting up and maintaining a multi-node cluster requires additional tools and configuration, such as using goreman for local setups, increasing management overhead.

Limited Data Model

As a key-value store, it lacks support for complex queries or data relationships, which may necessitate extra application logic for advanced use cases.

Development Instability

The README warns that the main branch may be in an unstable or broken state during development, posing risks for those using non-release versions.

Frequently Asked Questions

Related Projects

Caddy

Fast and extensible multi-platform HTTP/1-2-3 web server with automatic HTTPS

Stars75,560

Forks4,946

Last commit5 hours ago

minio

MinIO is a high-performance, S3 compatible object store, open sourced under GNU AGPLv3 license.

Stars61,372

Forks7,858

Last commit4 months ago

istio

Connect, secure, control, and observe services.

Stars38,374

Forks8,377

Last commit14 hours ago

SFTPGo

Full-featured and highly configurable SFTP, HTTP/S, FTP/S and WebDAV server - S3, Google Cloud Storage, Azure Blob

Stars12,498

Forks955

Last commit13 hours ago

Community-curated · Updated weekly · 100% open source

Found a gem we're missing?

Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.

Submit a project Star on GitHub