Skillversitas

TiKV (Storage Engine)

TiKV is the distributed transactional key-value storage engine that forms the foundation of data storage in TiDB. TiKV guarantees strong data consistency and high availability.

TiKV Architecture

TiKV implements a multi-raft-group mechanism based on the design of Google Spanner. Each TiKV node stores multiple Regions (the basic unit of data movement).

TiKV Node
┌─────────────────────────────────┐
│  Raft Log RocksDB Instance      │
├─────────────────────────────────┤
│  Region Data RocksDB Instance   │
│  ┌──────┐ ┌──────┐ ┌──────┐   │
│  │Region│ │Region│ │Region│   │
│  │  A   │ │  B   │ │  C   │   │
│  └──────┘ └──────┘ └──────┘   │
├─────────────────────────────────┤
│  Raft Group (3 peers per Region)│
└─────────────────────────────────┘

Region

A Region is the basic unit of data storage in TiKV. Each Region has a key range (StartKey to EndKey) that is left-closed, right-open. The default Region size is 96 MB, and it is split into 2 new Regions when it exceeds 256 MB.

RocksDB

Each TiKV node uses two RocksDB instances:

  • Raft RocksDB — stores raft logs for data consistency
  • KV RocksDB — stores business data (key-value pairs)

This separation is done because sequential I/O is faster than random I/O. By separating raft logs and data, TiDB combines all write operations into a single I/O operation.

Raft Consensus Algorithm

Raft is a consensus protocol that guarantees:

  • Leader Election — one node is elected as the leader that serves reads and writes
  • Log Replication — all writes must be replicated to a majority of nodes before being committed
  • Safety — as long as a majority of replicas is alive, data will not be lost

Each Region has at least 3 peers: 1 leader and 2 followers. The leader handles all write operations. A write is committed only if it has been written to a majority of replicas.

Distributed Transaction

TiDB uses two-phase commit (2PC) for distributed transactions:

  1. Prewrite — write the data to all involved nodes
  2. Commit — commit the data globally with a timestamp from PD

By default, TiDB uses the Snapshot Isolation level.

TiKV Coprocessor

TiDB pushes part of the computation down to the TiKV Coprocessor. The Coprocessor handles filters, aggregations, and other operations directly in the storage layer, reducing data transfer between nodes.

In the next chapter, we will study TiFlash as the columnar storage engine.