Migrating a Data Warehouse to a Lakehouse on Azure Databricks

Vattenfall is a Swedish energy and utility company. Their Azure-based data warehouse — built on Analysis Services, Synapse and Databricks — was troubled by performance, maintenance and cost problems. I was part of a new scrum team responsible for migrating this data warehouse to a modern lakehouse architecture based on Databricks and dbt.

As data engineer and solution architect I was responsible for designing the new architecture, determining the technology stack, setting up the infrastructure and CI/CD pipelines, training the team in the chosen technologies, extracting the data sources using Databricks notebooks (PySpark) and quality assurance.