Module 2 · Indexes

Episode 16 Compaction

10 min · SQL · Postgres

Title card of Episode 16: Compaction

▶ This video is coming to YouTube soon.

Why does RocksDB rewrite your data again and again? This episode animates the k-way merge, size-tiered and leveled compaction, and measures read, write and space amplification on one 6,000-write workload, then shows Postgres paying for old row versions with VACUUM and VACUUM FULL.

Chapters

  1. 0:00Intro
  2. 0:15Every flush adds a file
  3. 0:38Without compaction, files pile up
  4. 1:24Compaction is a merge of sorted files
  5. 2:23The code: merge sorted runs
  6. 2:54Size-tiered: merge files of similar size
  7. 4:02Leveled: one sorted run per level
  8. 5:29Read, write, space: pick two
  9. 6:08Measured: one workload, three strategies
  10. 6:44A tombstone drops only at the last level
  11. 7:34Proof: Postgres pays for old versions too
  12. 8:36Who uses what
  13. 9:01Recap
  14. 9:31Next: Bloom Filters