Job Description
On behalf of NDA, Databricks-native platform, SD Solutions is looking for an experienced Data Engineer to design and build the data infrastructure that powers our Master Data Management platform, built natively on the Databricks Data Intelligence Platform.
In this role, you will develop scalable, high-performance data pipelines that support entity resolution, data quality, and real-time analytics!
You will take a hands-on role in building and optimizing batch and streaming pipelines using modern Lakehouse technologies, including Delta Live Tables and Change Data Capture patterns. This includes ensuring data reliability, consistency, and performance through robust pipeline design, testing, and optimization of Spark workloads.
Working closely with AI/ML Engineers, Product Managers, and Data Scientists, you will translate data requirements into effici...