Job Description
Description
Design, develop and implement ingestion framework from Oracle source to Azure Data Lake - initial load and incremental ETL.
Tools
- Oracle Golden Gate (knowledge and experience are an asset but not required) for data ingestion and change data capture (currently in final stages of proof of concept)
- Azure Data Factory (expert knowledge) to maintain pipeline from Oracle to Azure Data Lake
- Azure Databricks/PySpark (good Python knowledge required) to build transformations of raw data into curated zone in the data lake
- Azure Databricks/Python/SQL (good SQL knowledge required) to develop and/or troubleshoot transformations of curated data into datamart model
General Responsibilities
- Review the requirements, database tables, and database relationships. Identify gaps and inefficiencies in current production reporting environment and provide ...