Question
You have an Azure Databricks workspace that is enabled for Unity Catalog.
You have an Apache Spark Structured Streaming job that writes data to a Delta table.
After the cluster restarts, the streaming job reprocesses previously ingested data.
You need to prevent the streaming job from reprocessing the data after the cluster restarts.
What should you do?
- A. Configure a checkpoint location for the streaming query. Correct
- B. Increase the trigger interval of the streaming query.
- C. Enable change data feed (CDF) for the target table.
- D. Configure a watermark for the streaming query.
Explanation
A checkpoint stores the source offsets processed in each micro-batch and a record of the batches committed to the sink, so the query resumes where it left off with exactly-once semantics. The trigger interval only controls how often batches run. Change data feed records changes made to a table and does not track what a stream has already read. A watermark bounds how long a stateful query waits for late data and does not record progress across restarts.
