Question
You have an Azure Synapse Analytics Apache Spark pool named Pool1.
You plan to load JSON files from an Azure Data Lake Storage Gen2 container into the tables in Pool1. The structure and data types vary by file.
You need to load the files into the tables. The solution must maintain the source data types.
What should you do?
- A. Load the data by using PySpark. Correct
- B. Load the data by using the OPENROWSET Transact-SQL command in an Azure Synapse Analytics serverless SQL pool.
- C. Use a Get Metadata activity in Azure Data Factory.
- D. Use a Conditional Split transformation in an Azure Synapse data flow.
Explanation
PySpark runs directly on the Spark pool, and its JSON reader infers each file's schema, including its data types, so it handles files whose structure varies and can write them to the pool's tables. OPENROWSET runs in the serverless SQL pool, not in Pool1, and requires you to define how the JSON is parsed. A Get Metadata activity only returns information about files, and a Conditional Split only routes rows, so neither loads the data while keeping its types.
