Data Engineer – Databricks & Spark Specialist

About the role

The Data Engineer will design, develop, and maintain robust data pipelines and infrastructure using Databricks and Apache Spark. You will enable reliable data integration, transformation, and delivery for enterprise users while ensuring data quality and governance.

Key responsibilities

• Design, build, and maintain data pipelines that extract, transform, and load data from multiple sources.
• Implement and manage advanced data products within a medallion architecture.
• Develop dimensional, relational, and Data Vault data models.
• Integrate data from databases, data warehouses, APIs, and external systems, ensuring consistency and integrity.
• Optimize pipelines for performance, scalability, and cost efficiency.
• Monitor, tune, and troubleshoot data systems, applying caching and indexing strategies.
• Implement data quality checks and validation rules.
• Administer Databricks Unity Catalog, including security, access control, and cost management.
• Configure clusters, manage policies, and ensure high‑availability environments.
• Build CI/CD pipelines using GitHub or Azure DevOps.
• Collaborate with cross‑functional teams in an agile setting.

Required profile

• Minimum 5 years of data engineering experience (8 years for senior level).
• Hands‑on experience with Databricks (2 years for mid‑level, 4 years for senior).
• Strong background in data integration, modeling, optimization, and data quality.

Required skills

• Databricks
• Apache Spark
• Data integration and ETL
• Dimensional, relational, and Data Vault modeling
• Unity Catalog administration
• CI/CD pipelines (GitHub, Azure DevOps)
• Cluster configuration and policy management
• Data quality validation

What we offer

• Opportunity to work on cutting‑edge data platforms.
• Collaborative, agile environment.
• Competitive freelance compensation.