Job Description
Role : Sr. Data Engineer-Databricks
Description
We are looking for an experienced Senior Databricks Data Engineer with strong hands-on expertise in Py Spark, Delta Lake, and cloud data pipelines. The ideal candidate will have experience integrating SAP systems (SAP BDC / SAP Datasphere / SAP ECC / SAP S/4 HANA) with Databricks and a solid engineering background.
Responsibilities
- Design, develop, and maintain scalable data pipelines on Databricks using Py Spark and Delta Lake.
- Build and optimize ETL/ELT workflows integrating SAP systems with Databricks (using OData, SAP BTP, CDC tools, DB Connectors, or partner solutions).
- Collaborate with AI teams to integrate LLMs and advanced ML models into data workflows.
- Ingest structured and unstructured data from multiple SAP and non-SAP sources into the lakehouse.
- Implement data modeling (Medallion architecture – Bronze/Silver/Gold).
- Optimize Databricks clusters, jobs, workflows, and performance tuning of P...
Description
We are looking for an experienced Senior Databricks Data Engineer with strong hands-on expertise in Py Spark, Delta Lake, and cloud data pipelines. The ideal candidate will have experience integrating SAP systems (SAP BDC / SAP Datasphere / SAP ECC / SAP S/4 HANA) with Databricks and a solid engineering background.
Responsibilities
- Design, develop, and maintain scalable data pipelines on Databricks using Py Spark and Delta Lake.
- Build and optimize ETL/ELT workflows integrating SAP systems with Databricks (using OData, SAP BTP, CDC tools, DB Connectors, or partner solutions).
- Collaborate with AI teams to integrate LLMs and advanced ML models into data workflows.
- Ingest structured and unstructured data from multiple SAP and non-SAP sources into the lakehouse.
- Implement data modeling (Medallion architecture – Bronze/Silver/Gold).
- Optimize Databricks clusters, jobs, workflows, and performance tuning of P...