TiFlash & HTAP

TiFlash is a columnar storage engine designed to speed up analytical queries in TiDB. Together with TiKV, TiFlash enables an HTAP (Hybrid Transactional and Analytical Processing) architecture.

What is HTAP?

HTAP is a data processing paradigm that combines OLTP (Online Transactional Processing) and OLAP (Online Analytical Processing) in a single system.

• OLTP — everyday transactional operations: INSERT, UPDATE, DELETE
• OLAP — data analysis for reporting and business intelligence

Before HTAP, companies had to build a data pipeline: MySQL → ETL → Data Warehouse. With TiDB HTAP, analysis can be done directly on the transactional database.

TiFlash Architecture

TiFlash runs as a learner replica of TiKV. TiFlash uses the Multi-Raft Learner protocol to replicate data asynchronously from TiKV.

TiDB Server
    │
    ├──► TiKV (row-based, OLTP)
    │       │
    │       └──► TiFlash (column-based, OLAP)
    │
    └──► TiFlash (direct read)

Advantages of TiFlash

• Real-time Replication — data in TiFlash is always up-to-date with TiKV (typically a delay of < 2 seconds)
• Columnar Storage — speeds up aggregations and scan queries
• Resource Isolation — TiFlash can be deployed on nodes separate from TiKV
• MPP Engine — Multiple Parallel Processing for complex analytical queries

Usage Examples

Creating a TiFlash Replica

ALTER TABLE orders SET TIFLASH REPLICA 1;

Forcing TiFlash for Specific Queries

SELECT /*+ read_from_storage(tiflash[orders]) */
  category, SUM(amount)
FROM orders
GROUP BY category;

When to Use TiFlash?

TiFlash Pipeline Model

TiFlash uses the Pipeline Model to process queries in parallel. Each query is broken into several pipelines that run concurrently across multiple threads.

In the next chapter, we will study the Placement Driver (PD) as the cluster scheduler.