Job Description
Key Deliverables:
- Build and optimize scalable data pipelines using Apache Spark on Databricks
- Implement ETL/ELT processes with Databricks Notebooks and Delta Lake
- Ensure data quality, integrity, and compliance across platforms
- Support CI/CD, cloud migrations, and infrastructure automation
- Collaborate with data scientists and analysts to deliver data products
- Maintain and optimize large-scale distributed data jobs
- Use Azure DevOps, GitHub Actions, and Terraform for workflow automation
- Work with structured and unstructured data using Lakehouse architecture