In this course, you'll learn to set up a live data processing system from scratch using Databricks. You'll explore real-time data processing techniques, configure automated data ingestion, and work with SQL for analytics, all through practical examples. The course is designed to fill the gap between understanding these tools individually and using them together in a production environment.
What you'll learn
Set up Spark Structured Streaming for real-time data processing
Configure Auto Loader for automatic data ingestion
Build a Delta Lake for central data storage
Create live dashboards and alerts using SQL and AI