Job Description
Duties and Responsibilities
• Skilled Offshore ETL/Data Engineer Contractor to support enterprise data integration, transformation, and modernization initiatives.
• Will build scalable data pipelines and contribute to next-gen data solutions incorporating AI/ML capabilities along with legacy ETL modernization.
• Design, develop, and maintain ETL pipelines using Informatica, Azure Data Factory (ADF), and Databricks
• Perform migration of legacy ETL workflows (Informatica) to Databricks using Python/PySpark
• Analyze existing ETL workflows and re-engineer into optimized Spark-based transformations
• Develop data processing and transformation solutions using Python and PySpark
• Apply AI/ML techniques for data enrichment, anomaly detection, and predictive insights (where applicable)
• Build and optimize SQL queries, data models, and transformations
• Schedule and monitor jobs using AutoSys • Integrate data...