Obsidian

Search

Search IconIcon to open search

Backfill

Last updated Jan 12, 2023

# What is backfill?

In Kafka, backfill refers to the process of re-processing and re-ingesting historical data in order to make it available for real-time streaming or batch processing. This can be useful in scenarios where the data pipeline or the processing logic has been updated and the historical data needs to be re-processed to match the new schema or to apply the new logic. It can also be used for disaster recovery or to fix data inconsistencies.