Question
You have an Azure Databricks workspace that contains a cluster named Cluster1.
Performance monitoring shows that Cluster1 is consistently overprovisioned for its batch workload:
- CPU utilization remains below 20 percent, including peak processing periods.
- The workload is highly predictable and does not spike.
- The current node type already meets the workload requirements.
You need to reduce compute costs without increasing job duration.
What should you do?
- A. Enable Photon acceleration.
- B. Configure Cluster1 to use a larger node type.
- C. Decrease the autotermination timeout of Cluster1.
- D. Disable autoscaling and reduce the number of worker nodes. Correct
Explanation
CPU stays below 20 percent even at peak and the load is predictable, so the cluster simply has more workers than the job needs. A smaller fixed number of workers removes the idle capacity you are paying for, and autoscaling adds little because its value is in following workloads whose requirements change. Photon is mainly a performance feature for SQL and complex DataFrame work and does not address spare capacity. A larger node type adds more unused capacity, and a shorter autotermination timeout only affects idle time after the work finishes, not the oversizing during the run.
