Job Description
What the role is
To design, build, and lead the development of reliable data ingestion and transformation pipelines for the Enterprise Data and AI platform. The Senior Data Engineer will ensure that data from multiple sources is ingested, governed, secured, and made available for analytics, AI, and business use in alignment with NP’s data architecture and governance frameworks. This is a 2-year contract position with the Digital Services & Technology Office.
What you will be working on- Design, build and lead the development of ingestion pipelines for structured and unstructured data (Excel, MSSQL, Postgres, APIs, SharePoint, PDFs, etc.).
- Architect and optimise transformation workflows across Bronze, Silver, and Gold layers using Delta Lake and Delta Live Tables.
- Define and implement data quality frameworks (schema, dedupe, lineage, metadata enrichment) and work with Governance team to enforce standards.
- Collaborate with the Securi...