Data Engineer – Azure & Databricks

About the role

We are looking for a Data Engineer to design, build and optimise data pipelines on the Azure platform. The role involves developing, testing and maintaining code that powers data products, while ensuring high performance and reliability.

Key responsibilities

• Write, test and optimise code for data products using Python, Spark and PySpark.
• Design and implement batch and streaming pipelines on Azure Data Factory, Databricks and related services.
• Apply the Medallion architecture (Bronze‑Silver‑Gold) and manage data in Delta, Iceberg or Parquet formats.
• Lead a team of developers, define technical designs and establish coding standards.
• Mentor junior engineers and coordinate delivery of features or epics.
• Collaborate with external stakeholders and ensure documentation, unit testing and version control best practices.

Required profile

• Proven ability to lead development teams and deliver complex data platform components.
• Strong communication skills with stakeholders and experience mentoring junior staff.
• Advanced knowledge of data engineering concepts, ETL/ELT, data lakes and warehouses.
• Expertise in Azure Data Platform services and security, automation and cost‑management considerations.

Required skills

• Azure Data Lake Gen2, Azure Storage, Azure Data Factory, Azure SQL, Azure Databricks, Azure Key Vault, Azure Cosmos DB, Azure App Services, Azure Analysis Services.
• Docker, Kubernetes, Azure Stream Analytics.
• Databricks, ADL, ADF, PySpark, Spark, Hive, SQL‑DW (Synapse), Python, C# (optional).
• Data modelling techniques (DataVault, Kimball) and table formats (Delta, Iceberg, Parquet).
• CI/CD pipelines, Azure DevOps, Git, unit testing, observability, security and linting practices.
• Orchestration tools such as Airflow, Luigi or Prefect.