Log Compaction Theory

The video tutorial explains log compaction in Kafka, focusing on its purpose, functionality, and configuration. It uses an example of employee salary data to illustrate how log compaction retains the latest updates for keys, ensuring efficient data storage. The tutorial also addresses common myths and potential issues, such as duplicate data and compaction failures, and provides guidance on configuring log compaction settings.

What is the primary purpose of log compaction in Kafka?

• To store the entire history of all messages

• To keep only the latest value for each key

• To compress data for storage efficiency

• To reorder messages for better performance

How does log compaction affect message ordering?

• It reorders messages to optimize performance

• It orders messages by key value

• It maintains the original order of messages

• It randomly shuffles messages

What is the default duration for which a deleted record can still be seen by a consumer?

• 24 hours

• 12 hours

• 48 hours

• 72 hours

Which of the following is NOT a myth about log compaction?

• It prevents duplicate data from being pushed

• It automatically deduplicates data

• It requires manual triggering through an API

• It can sometimes fail due to memory issues

What should you ensure to avoid log compaction failures?

• Low CPU usage

• Enough disk space

• Sufficient RAM

• High network bandwidth

What is the default maximum size of a segment in Kafka?

• 5 GB

• 500 MB

• 1 GB

• 2 GB

How does the min cleanable dirty ratio affect log compaction?

• Higher ratio results in less frequent compaction

• Higher ratio leads to more frequent compaction

• Lower ratio results in more efficient cleaning

• Lower ratio increases the segment size