Redesigning and Rebuilding a Data Platform

During the implementation of Nationale-Nederlanden’s reusable data platform for a business unit, it became clear that the platform did not meet the requirements. Our team designed and built a new data platform with Terraform, Databricks, Synapse, Purview, Great Expectations and Data Lake Storage Gen2, implemented automated infrastructure testing and improved the CI/CD pipelines. The project ended with a successful implementation of the platform.

As a data engineer I was responsible for designing the infrastructure, rebuilding the ETL, improving CI/CD, knowledge sharing and implementing the data quality component.