JustJoin.IT Hybrydowo Mid

Data Engineer

emagine Polska

⚲ Lisbon

Wymagania

  • Microsoft Azure
  • Git
  • CI/CD
  • ETL
  • Governance
  • Artificial Intelligence (AI)
  • DataStage (ETL)
  • SQL
  • Python
  • Microsoft Platform

Opis stanowiska

The project: • International project in the financial sector • Data platform in Azure, with ADF and Databricks as core • Strong focus on data engineering, governance and quality • Critical platform in production, with high reliability and stability requirements Responsibilities • Develop and maintain data pipelines in Azure Databricks (PySpark / Spark SQL). • Implement and optimize Delta Lake tables (ETL/ELT, medallion architecture). • Collaborate in the design of analytical datasets. • Integrate Databricks workloads with Azure Data Factory (orchestration). • Apply good practices in data performance, quality and reliability. • Contribute to the evolution of the platform for analytics and AI scenarios. Requirements • At least 4 years of experience as a Data Engineer. • Hands-on experience with Azure Databricks and Apache Spark. • Good knowledge of Python (PySpark) and SQL. • Experience with Delta Lake. • Experience with Azure services:• Azure Data Lake Storage (ADLS Gen2), • Azure Data Factory • fluent English (oral and written). Preferred: • Experience with CI/CD for data platforms (Azure DevOps, Git). • Notions of data governance (e.g., Unity Catalog, Purview). • Sensitivity to safety, quality and regulated environments. • Experience in international projects or distributed teams.