Manage Data with Azure Databricks and Azure Data Lake
Connect Databricks to ADLS Gen2 securely, ingest with Auto Loader and govern with Unity Catalog.
Level: Intermediate / Advanced | Platform: Azure Databricks + ADLS Gen2
Sign in to read this course
A free account unlocks all 514 courses. 20 are readable without one.
What's inside
23 sections- 1 Table of Contents
- 2 ADLS Gen2 Architecture and Foundations
- 3 Azure Databricks Integration with ADLS Gen2
- 4 Security with RBAC and Managed Identities
- 5 Authentication Mechanisms
- 6 Azure Key Vault — Secure Secret Management
- 7 Connecting to ADLS Gen2 from Databricks
- 8 Mount Points vs Direct Access
- 9 Troubleshooting Connectivity Issues
- 10 PySpark vs SQL — Decision Guide
- 11 Data Ingestion from ADLS Gen2
- 12 Schemas: Inference and Evolution
- 13 Auto Loader — Incremental Ingestion
- 14 Optimization: Parquet vs Delta Lake
- 15 Unity Catalog — Centralized Governance
- 16 Configuring Unity Catalog in a Workspace
- 17 Sharing Data Between Workspaces
- 18 Data Lineage with Unity Catalog
- 19 Access Control Models Compared
- 20 Delta Sharing — Cross-Organization Sharing
- 21 Fine-Grained Permissions: Rows and Columns
- 22 Summary and Best Practices
- 23 Glossary
More Azure Databricks & Spark courses
View all 14Administering Clusters and Configuring Policies with Databricks
Databricks architecture, cluster types and runtimes, autoscaling, cluster policies, pools and init scripts.
ETL Pipelines with Azure Databricks and Data Factory
Build ETL with Spark and PySpark, Unity Catalog governance, Delta Lake and Databricks vs Data Factory.
Optimize Storage and Performance with Delta Lake
Delta Lake internals, ACID, OPTIMIZE, Z-Order, liquid clustering, caching and Photon acceleration.
Real-Time Data Processing with Azure Databricks
Spark Structured Streaming with Event Hubs — windowing, stateful processing and real-time anomaly detection.
Machine Learning with Azure Databricks
The Databricks ML lifecycle: MLflow tracking, tuning with Ray, the model registry, serving and AutoML.
Building Deep Learning Models on Databricks
Build, train, tune and serve deep-learning models on Databricks with TensorBoard integration.
Interested in this course?
Contact us to book it or get a custom training plan for your team.