Job Description
Job Description
Contribute to building and evolving the platform (infrastructure + reusable abstractions) that standardises data engineering workloads (batch/streaming pipelines, data processing) and traditional ML workflows (feature engineering, training, batch/real-time serving) across teams
Implement platform-level IaC, CI/CD, and environment management to support consistent, reproducible workloads across dev/test/prod
Build and maintain components using Python and Spark for data processing, shared datasets, and platform services
Contribute to shared services for data and ML lifecycle management (data pipelines, experiment tracking, versioning, lineage, permissions), aligned to enterprise governance ( Unity Catalog)
Support the implementation and operation of a centralised AgentOps capability (LLM gateway, tool integration, prompt and version management)
Contribute to agent-spe...