🎯
Full-Time Opportunity: This is a permanent, full-time position with a competitive package and real career growth potential.
Job Description
Roles and Responsibilities
- Data Platform Architecture: Design and implement scalable data platform architectures on Azure and Databricks, including medallion architecture (Bronze/Silver/Gold), data lakehouse patterns, and integration layers that serve analytics, reporting, and downstream consumers.
- Pipeline Development: Build and maintain robust ETL/ELT pipelines using Databricks (PySpark/Spark SQL), and related Azure data services. Ensure pipelines handle batch and streaming workloads reliably at scale.
- Legacy Modernisation: Lead the migration of legacy data pipelines to modern cloud-native solutions on Azure and Databricks. Define migration strategies, manage cutover plans, and ensure zero data loss during transition.
- Data Quality & Governance: Establish and enforce data quality frameworks, validation rules, and monitoring across all pipelines. Implement data lineage tracking, cataloguing (Unity Catalog), ...