Question
You plan to develop a dataset named Purchases by using Azure Databricks. Purchases will contain the following columns:
- ProductID
- ItemPrice
- LineTotal
- Quantity
- StoreID
- Minute
- Month
- Hour
- Year
- Day
You need to store the data to support hourly incremental load pipelines that will vary for each StoreID. The solution must minimize storage costs.
How should you complete the code? To answer, select the appropriate options in the answer area.
```python
df.write
[Dropdown 1] [Dropdown 2]
.mode("append")
[Dropdown 3]
```
Explanation
Dropdown 1 is .partitionBy, because it writes the data into a folder for each partition value, so a pipeline can read only the folders it needs. The .bucketBy and .sortBy methods do not create that folder structure, and .range is not a DataFrameWriter method.
Dropdown 2 is ("Year", "Month", "Day", "Hour", "StoreID"). The date parts down to the hour isolate each hourly load, and StoreID as the last level lets each pipeline read just its own store within that hour. Partitioning by StoreID and Hour alone would mix the same hour from different days, and "*" is not a valid partition specification.
Dropdown 3 is .parquet("/Purchases"), because Parquet is a compressed columnar format that needs less storage than CSV or JSON. The .saveAsTable method registers a table rather than simply writing files to the path.
