Question
You have an Azure Databricks workspace named Workspace1.
You create a compute cluster named Cluster1 that will be used to ingest data.
You need to install the required libraries on Cluster1. The solution must use Unity Catalog for access control.
What should you do?
- A. Install the libraries by using pip3.
- B. Create a custom dependency management script and run the script from a Databricks notebook.
- C. Upload the libraries to Workspace1 and install the libraries on Cluster1. Correct
- D. Install the libraries on Cluster1 and manually restart the cluster.
Explanation
Databricks recommends uploading libraries, including Python wheels and JAR files, to workspace files or Unity Catalog volumes and then installing them as compute-scoped libraries, so every notebook and job on the cluster can use them. When a library is installed from one of these locations, an identity is associated with the installation and must have read access on the file. On Unity Catalog compute, a volume with READ granted to the necessary users is the preferred location. Running pip3 or a custom script from a notebook does not use this managed installation path or its access checks. A manual restart is not part of installing a library: a restart is needed to uninstall one, while a notebook only needs a new session to see a new library.
