Job Description
We are seeking a Principal Data Engineer with 5+ years of hands‑on experience in the modern cloud data stack. This role is ideal for someone passionate about building robust, scalable data solutions and who thrives in agile, collaborative environments.
The core technical stack centers around Databricks / PySpark for large‑scale data processing, DBT for transformation, Astronomer / Airflow for orchestration (with Cosmos to run DBT DAGs natively in Airflow), and Snowflake as the cloud data warehouse.
Key Responsibilities
- Build and maintain scalable data pipelines using DBT and Astronomer/Airflow for batch and incremental data processing.
- Develop modular DBT transformation layers (staging → intermediate → mart) with robust testing, documentation, and incremental model strategies.
- Orchestrate end‑to‑end workflows on Astronomer / Airflow , using Cosmos to trigger ...