You have an Azure Databricks workspace.
You are creating a Lakeflow Spark Declarative Pipelines (SDP) pipeline that scales automatically.
You need to configure compute for the pipeline. The solution must minimize operational costs and effort.
What should you use?
A.the existing SQL warehouse
B.an all-purpose cluster that uses autoscaling
C.a job cluster that uses autoscalingCorrect
D.a single-node, all-purpose cluster
Explanation
Pipeline compute is created for the update and shut down automatically when it is not in use, which is the job-style lifecycle that keeps cost and effort low, and autoscaling adds and removes workers as the workload changes. A SQL warehouse serves SQL queries and is not the compute that runs a pipeline update. An all-purpose cluster is meant for interactive work and keeps costing money between runs. A single node cannot scale at all, so it fails the main requirement.
You have an Azure Databricks workspace that contains an all-purpose compute cluster named Cluster1. Cluster1 is used for interactive development.
You need to configure Cluster1 to meet the following requirements:
- Automatically add and remove worker nodes based on workload demand.
- Automatically shut down when the cluster has been idle for a specific period.
What should you configure for each requirement? To answer, drag the appropriate options to the correct requirements. Each option may be used once, more than once, or not at all.
NOTE: Each correct selection is worth one point.
Explanation
Autoscaling lets Azure Databricks add workers during demanding phases and remove them when they are no longer needed, within the range you set. Min workers and Max workers only define the bounds of that range and do not turn the behaviour on by themselves. Auto termination shuts the cluster down after the inactivity period you specify, measured from the last command run. Photon acceleration, spot instances and the driver node type affect performance, price or the driver, not scaling or idle shutdown.
You have an Azure Databricks workspace that is attached to a Unity Catalog metastore named metastore1. metastore1 contains a catalog named catalog1.
You need to create a new schema named schema2 that meets the following requirements:
- Is contained in catalog1
- Uses abfss://container@storageaccount.dfs.core.windows.net/data as the managed location
Which SQL statement should you execute?
D.CREATE SCHEMA catalog1.schema2 WITH DBPROPERTIES (LOCATION='abfss://container@storageaccount.dfs.core.windows.net/data');
Explanation
In Unity Catalog, a schema's storage root is set with MANAGED LOCATION, and the qualified name catalog1.schema2 places the schema in catalog1. LOCATION is not supported in Unity Catalog; it applies to workspace-level Hive or third-party metastores. CREATE CATALOG would create a catalog named schema2 rather than a schema. WITH DBPROPERTIES only stores key-value properties and does not set the storage location.
You have an Azure Databricks workspace named Workspace1.
You create a compute cluster named Cluster1 that will be used to ingest data.
You need to install the required libraries on Cluster1. The solution must use Unity Catalog for access control.
What should you do?
A.Install the libraries by using pip3.
B.Create a custom dependency management script and run the script from a Databricks notebook.
C.Upload the libraries to Workspace1 and install the libraries on Cluster1.Correct
D.Install the libraries on Cluster1 and manually restart the cluster.
Explanation
Databricks recommends uploading libraries, including Python wheels and JAR files, to workspace files or Unity Catalog volumes and then installing them as compute-scoped libraries, so every notebook and job on the cluster can use them. When a library is installed from one of these locations, an identity is associated with the installation and must have read access on the file. On Unity Catalog compute, a volume with READ granted to the necessary users is the preferred location. Running pip3 or a custom script from a notebook does not use this managed installation path or its access checks. A manual restart is not part of installing a library: a restart is needed to uninstall one, while a notebook only needs a new session to see a new library.
You have an Azure Databricks workspace that is enabled for Unity Catalog and contains a catalog named finance. finance contains two schemas named default and procurement.
You need to create a table named assets in the procurement schema. assets must contain the following columns:
- asset_id
- asset_type
- asset_name
How should you complete the SQL statement? To answer, drag the appropriate values to the correct targets. Each value may be used once, more than once, or not at all.
```sql
USE [Value 1];
USE [Value 2];
CREATE [Value 3] (
asset_id INT,
asset_name STRING,
asset_type STRING
);
```
NOTE: Each correct selection is worth one point.
Explanation
USE CATALOG finance comes first because setting a catalog resets the current schema to default, and the schema must exist within the current catalog. USE SCHEMA procurement then makes unqualified names resolve to the procurement schema, so CREATE TABLE assets creates the table in the right place. CATALOG hive_metastore points at the legacy metastore, not the finance catalog. A bare finance after USE would be treated as a schema name, and CREATE needs the TABLE keyword, so a bare assets is invalid.
You have an Azure Databricks workspace that contains an all-purpose cluster named Cluster1.
You need to configure Cluster1 to meet the following requirements:
- Scale up automatically when workloads increase
- Scale down automatically when workloads decrease
- Minimize costs
Which two actions should you perform? Each correct answer presents part of the solution.
NOTE: Each correct selection is worth one point.
A.Disable Photon acceleration.
B.Enable autoscaling for Cluster1.Correct
C.Apply a compute policy that enables users to manage the cluster settings.
D.Specify a fixed number of workers.
E.Configure Cluster1 to terminate after 30 minutes of inactivity.Correct
Explanation
Autoscaling adds workers when demand rises and removes them when it falls, and it can reduce cost compared to a statically sized cluster. Automatic termination stops an all-purpose cluster after a period of inactivity, so you stop paying for it while it is idle. A fixed number of workers is the opposite of autoscaling. Disabling Photon has no effect on scaling, and a policy that lets users manage cluster settings neither scales the cluster nor limits its cost.
You have an Azure Databricks workspace that is enabled for Unity Catalog.
You have an Apache Spark Structured Streaming job that writes data to a Delta table.
After the cluster restarts, the streaming job reprocesses previously ingested data.
You need to prevent the streaming job from reprocessing the data after the cluster restarts.
What should you do?
A.Configure a checkpoint location for the streaming query.Correct
B.Increase the trigger interval of the streaming query.
C.Enable change data feed (CDF) for the target table.
D.Configure a watermark for the streaming query.
Explanation
A checkpoint stores the source offsets processed in each micro-batch and a record of the batches committed to the sink, so the query resumes where it left off with exactly-once semantics. The trigger interval only controls how often batches run. Change data feed records changes made to a table and does not track what a stream has already read. A watermark bounds how long a stateful query waits for late data and does not record progress across restarts.
You have an Azure Databricks workspace that is enabled for Unity Catalog and contains a managed Delta table named Table1.
Table1 is written by batch jobs every hour and is queried frequently by filtering two columns named CustomerId and EventDate.
You expect Table1 to grow significantly over time.
The rows in Table1 are frequently updated and deleted to support compliance requests.
You need to keep query performance consistent as Table1 grows. The solution must minimize update and deletion effort.
What should you include in the solution? To answer, select the appropriate options in the answer area.
NOTE: Each correct selection is worth one point.
Explanation
To optimize query performance, implement: Liquid clustering. Liquid clustering replaces partitioning and Z-ordering, suits fast-growing tables and queries that filter on high-cardinality columns such as CustomerId, and clusters data incrementally so upkeep stays low as the table grows. Partitioning on CustomerId would create far too many small partitions and is hard to change later, and Z-ordering is not compatible with liquid clustering and has to be re-applied as data changes.
To optimize updates and deletions: Enable deletion vectors. Without them, changing a single row rewrites the entire Parquet file that holds it, whereas deletion vectors mark changed rows in metadata and speed up DELETE, UPDATE and MERGE. Disabling them does the opposite, and VACUUM only removes old unreferenced files, so it does nothing to make updates or deletions cheaper.
You have an Azure Databricks workspace that contains a cluster named Cluster1.
Performance monitoring shows that Cluster1 is consistently overprovisioned for its batch workload:
- CPU utilization remains below 20 percent, including peak processing periods.
- The workload is highly predictable and does not spike.
- The current node type already meets the workload requirements.
You need to reduce compute costs without increasing job duration.
What should you do?
A.Enable Photon acceleration.
B.Configure Cluster1 to use a larger node type.
C.Decrease the autotermination timeout of Cluster1.
D.Disable autoscaling and reduce the number of worker nodes.Correct
Explanation
CPU stays below 20 percent even at peak and the load is predictable, so the cluster simply has more workers than the job needs. A smaller fixed number of workers removes the idle capacity you are paying for, and autoscaling adds little because its value is in following workloads whose requirements change. Photon is mainly a performance feature for SQL and complex DataFrame work and does not address spare capacity. A larger node type adds more unused capacity, and a shorter autotermination timeout only affects idle time after the work finishes, not the oversizing during the run.
You have an Azure Databricks workspace.
You are creating a Lakeflow Spark Declarative Pipelines (SDP) pipeline that scales automatically.
You need to configure compute for the pipeline. The solution must minimize operational costs and administrative effort.
What should you use?
A.serverless computeCorrect
B.a single-node, all-purpose cluster
C.an all-purpose cluster that uses autoscaling
D.an existing SQL warehouse
E.a job cluster that uses autoscaling
Explanation
Serverless pipelines run on Databricks-managed compute, which removes most infrastructure configuration, and Databricks recommends serverless for new pipelines. They add vertical autoscaling, choosing the most cost-efficient instance types, on top of horizontal autoscaling. An all-purpose or job cluster with autoscaling still leaves you to size and maintain the compute, and a single-node cluster cannot scale out. A SQL warehouse is compute for SQL queries, not a compute option you assign to run a pipeline.
You have an Azure Databricks solution that was developed by multiple engineers and deployed to development, test, and production environments.
You need to implement an application lifecycle management (ALM) process that meets the following requirements:
- Deploys and provides source control for Databricks notebooks, workflows, and jobs in a consistent and repeatable way
- Parameterizes the target deployment environment
What should you include in the solution?
A.the export and import of Databricks workspaces
B.Databricks resources managed by using the Databricks Terraform provider
C.Git folders and manual promotion
D.Declarative Automation Bundles in GitCorrect
Explanation
Declarative Automation Bundles (formerly Databricks Asset Bundles) describe jobs, pipelines and their source files, such as notebooks, as YAML kept alongside the code, so everything is source controlled in Git and deployed as one unit. Each deployment goes to a chosen target environment, which gives a repeatable process across development, test and production. Exporting and importing workspaces and manual promotion from Git folders are manual and not repeatable. Terraform is suited to provisioning infrastructure and does not package notebook code and job definitions into a project deployed per target the way bundles do.
You have an Azure Databricks workspace that contains a cluster named Cluster1.
Cluster1 has the following characteristics:
- Configured as a single node cluster
- Uses a general purpose virtual machine node type
The cluster runtime environment has the following configurations:
- Uses Databricks Runtime for Machine Learning (Databricks Runtime ML)
- Includes common machine learning libraries
- Supports Python workloads
For each of the following statements, select Yes if the statement is true. Otherwise, select No.
NOTE: Each correct selection is worth one point.
Explanation
Statement 1 (No): GPU compute requires the worker type to be a GPU instance type, and Cluster1 uses a general purpose virtual machine that has no GPU. Using Databricks Runtime ML does not add GPU hardware.
Statement 2 (No): A single node cluster has a driver that acts as both master and worker, with no worker nodes, and it runs Spark locally. It is intended for non-distributed workloads, and it cannot be converted to a multi-node cluster.
Statement 3 (Yes): Databricks Runtime ML comes with common machine learning libraries preinstalled, and the cluster supports Python. Such workloads run on the single node without any extra library installation.
You have an Azure Databricks workspace that contains the objects shown in the following table.
| Name | Type |
| --- | --- |
| Catalog1 | Catalog |
| Schema1 | Schema |
| Sales1 | Table |
| Notebook1 | Notebook |
| Space1 | AI/BI Genie space (now called a Genie Agent) |
Users often use the following words to refer to a sale: transaction, event, order, and invoice.
You need to create a knowledge store. The solution must ensure that when the users use any of the words in Space1, Genie queries the Sales1 table. Any other Genie spaces must remain unaffected.
To which object should you add the instructions?
A.Sales1
B.Schema1
C.Space1Correct
D.Catalog1
E.Notebook1
Explanation
Instructions and the knowledge store belong to a Genie space and apply only to that space, so adding the synonyms to Space1 teaches Genie these words there and leaves other spaces alone. Metadata added to Sales1, Schema1 or Catalog1 lives in Unity Catalog and is visible to any Genie space that uses those objects, which breaks the requirement that other spaces stay unaffected. Notebook1 is not read by Genie as a source of instructions. Current documentation calls Genie spaces Genie Agents.
You have an Azure Databricks workspace named Workspace1 that is attached to a Unity Catalog metastore named metastore1.
You need to register an Azure Storage account named account1 that has a hierarchical namespace enabled as an external location. The external location must use a managed identity to authenticate to account1 and the solution must follow the principle of least privilege.
Which three actions should you perform in sequence? To answer, move the appropriate actions from the list of actions to the answer area and arrange them in the correct order.
Explanation
First create the access connector for Azure Databricks, because it holds the managed identity that will authenticate to the storage account. Next, grant that identity access to account1 with the Storage Blob Data Contributor role, which gives the read and write access required; Storage Blob Data Owner is more than needed and breaks least privilege, and the role must go to the connector's identity rather than to Workspace1. Finally, create the storage credential that references the connector. A storage credential is a Unity Catalog object governed at the metastore level, so it belongs in metastore1 rather than in Workspace1, and the external location then references it.
You have an Azure Databricks workspace that is enabled for Unity Catalog.
You need to ensure that data lineage is captured and can be reviewed for tables accessed by Databricks notebooks and jobs. The solution must minimize administrative effort.
Which compute configuration should you use to capture the data lineage and what should you use to review the data lineage? To answer, select the appropriate options in the answer area.
NOTE: Each correct selection is worth one point.
Explanation
Capture: Unity Catalog records lineage automatically for queries against tables registered in the metastore, so what matters is that the compute works with Unity Catalog, and no separate lineage setup is needed. A single-user or interactive cluster describes how compute is used or shared, not whether it is governed by Unity Catalog, so those labels alone do not guarantee capture.
Review: Catalog Explorer has a Lineage tab on each table, with an interactive lineage graph and links to the related notebooks and jobs. Log Analytics and Azure Data Explorer are not where Unity Catalog lineage is presented, and a DAG only describes the task order inside a job or pipeline.
You have an Azure Databricks workspace.
You have an Azure key vault named kv-secure that stores a secret named storageKey. The value of storageKey is managed and updated by the cloud security team at your company.
You need to enable a Databricks notebook named Notebook1 to retrieve the value of storageKey securely at runtime. The solution must follow the principle of least privilege and always retrieve the latest value.
What should you do? To answer, select the appropriate options in the answer area.
NOTE: Each correct selection is worth one point.
Explanation
Expose storageKey to Databricks by using: a Key Vault-backed secret scope is a read-only interface to the key vault, so the security team keeps managing the secret in Azure and Databricks reads the value held there. A Databricks-backed scope stores its own copy in a Databricks database, which would not follow updates made in the key vault. An access connector provides a managed identity for Unity Catalog storage access; it does not expose key vault secrets to notebooks.
Retrieve storageKey from within Notebook1 by using: dbutils.secrets.get returns the secret value for a given scope and key, and redacts it in notebook output. dbutils.secrets.list returns only metadata such as key names, not values. spark.conf.get reads Spark configuration properties and is not the call for reading a secret from a scope.
You have an Azure Databricks workspace that is enabled for Unity Catalog and contains a catalog named Catalog1. Catalog1 contains a schema named Schema1 and a table named Table1.
You need to ensure that access to the data in Table1 is controlled by using attribute-based access control (ABAC).
What should you apply to Table1, and how should you control access for users? To answer, select the appropriate options in the answer area.
NOTE: Each correct selection is worth one point.
Explanation
Apply to Table1: in Unity Catalog ABAC, the attributes that drive access are represented by governed tags on securable objects. A table privilege is a direct grant rather than an attribute, and a workspace permission does not govern Unity Catalog data.
To control user access: a Unity Catalog policy uses the governed tags in its conditions and takes effect automatically on objects that carry those attributes. A dynamic view is a separate mechanism that holds its logic in the view definition and is not ABAC. Granting SELECT only gives direct access and evaluates no attributes.
You have an Azure Databricks workspace that is enabled for Unity Catalog.
You need to implement a data lifecycle and expiration solution that meets the following requirements:
Transaction logs and deleted data files that are older than 90 days must be removed from Delta tables to reclaim storage.
All the tables must remain available for querying during the cleanup process.
Administrative effort must be minimized.
What should you do for each requirement? To answer, select the appropriate options in the answer area.
NOTE: Each correct selection is worth one point.
Explanation
Data lifecycle control: delta.deletedFileRetentionDuration sets the threshold VACUUM uses to remove data files the current table version no longer references, so 90 days is the value that drives this cleanup. logRetentionDuration only controls how long table history is kept, and log files are removed automatically after checkpoints; it must also be at least as long as the deleted file retention. timeUntilArchived relates to archival rather than deletion, and a tag only adds metadata and removes nothing.
Expiration cleanup: VACUUM permanently removes unreferenced data files older than the retention threshold, and the current version of the table stays in place for queries. OPTIMIZE compacts files without purging expired ones, dropping and recreating makes the tables unavailable, and DELETE changes rows and creates more files to clean up.
You have an Azure Databricks workspace that is enabled for Unity Catalog and contains a managed Delta table named Table1. Table1 stores customer data.
You need to implement a data retention solution that meets the following requirements:
Deleted data must be retained for 30 days to support audits.
Deleted data that is older than 30 days must be removed permanently.
The solution must minimize administrative effort.
Which two properties should you configure? Each correct answer presents part of the solution.
NOTE: Each correct selection is worth one point.
A.delta.timeUntilArchived
B.delta.enableDeletionVectors
C.delta.logRetentionDurationCorrect
D.delta.deletedFileRetentionDurationCorrect
E.delta.autoOptimize.autoCompact
Explanation
Keeping a past table version available requires both its log and its data files, so both retention properties must be set to 30 days. delta.deletedFileRetentionDuration sets the threshold VACUUM uses to permanently remove unreferenced data files, and delta.logRetentionDuration controls how long table history is kept. Predictive optimization can run VACUUM automatically on Unity Catalog managed tables, which keeps administrative effort low. delta.timeUntilArchived concerns archival rather than removal, deletion vectors only change how deletes are recorded, and auto compaction merges small files.
You have an Azure Databricks workspace that is enabled for Unity Catalog and contains two catalogs named Catalog1 and Catalog2.
An external application uses a service principal named SP1 to connect to a SQL warehouse.
You need to ensure that SP1 can query the data in Catalog1 and Catalog2. The solution must follow the principle of least privilege.
Which permissions should you grant to SP1 for the catalogs?
A.USE SCHEMA and SELECT
B.USE CATALOG and SELECT
C.USE CATALOG, USE SCHEMA, and SELECTCorrect
D.USE CATALOG and USE SCHEMA
Explanation
Reading a table in Unity Catalog needs all three privileges: USE CATALOG on the catalog, USE SCHEMA on the schema, and SELECT on the table. USE CATALOG and USE SCHEMA are required to interact with objects in those containers but do not themselves grant access to data, so option D cannot return any rows. Options A and B each leave out a required usage privilege. Granted at catalog level, USE SCHEMA and SELECT are inherited by the schemas and tables in the catalog.
Verifying access...
Access Required
You need to purchase this exam to access practice mode
Unlock a few more questions every day for 300 points you earn free on
Godlike Academy. Everything you unlock with the Daily Climb stays open for 7 days after your last unlock.