Lead Data Engineer

co.brick talents — powered by AI, powered by people.   Join a global tourism leader as a Lead Data Architect to spearhead an extensive cloud data modernization program. The primary focus of this long-term mission is a complex migration to Databricks Unity Catalog, involving the transition of all data layers (landing, raw, prepared, reporting, services) from Hive Metastore. You will be responsible for re-architecting the Lakehouse structure, standardizing governance, and modernizing ETL/ELT pipelines in a high-stakes, professional environment. Szczegóły wspólpracy • Start Date: ASAP • Duration: Long-term. • Working Hours: 9:00 – 17:00 CET/CEST. • English Level: Minimum B2+. Responsibilities • Architectural Leadership: Designing and developing data architecture within Azure and Databricks environments, serving as the technical lead for the project team. • Unity Catalog Migration: Leading the migration of infrastructure, pipelines (DLT, Python/SQL jobs, Synapse/ADF), and metadata frameworks to Unity Catalog. • Lakehouse Design: Implementing Medallion Architecture and Metadata-Driven ETL standards across the organization. • Optimization & Standards: Fine-tuning data processing workflows (Python, SQL, Spark) and implementing robust logging, monitoring, and data quality frameworks. • DevOps Mastery: Creating and implementing CI/CD processes and branching strategies using Azure DevOps and Git. • Stakeholder Collaboration: Partnering with business and technical stakeholders to deliver top-tier solutions and mentoring junior team members. Requirements • Cloud Infrastructure: Advanced knowledge of Microsoft Azure (networking, authorization, data environment design). • Databricks Expert: Deep proficiency in Unity Catalog, Delta Live Tables (DLT), Workflows, and workspace administration. • Synapse Stack: Strong experience with Azure Synapse (Serverless and Pipelines). • Development: Excellent Python skills and experience with Apache Spark (workflow design and code optimization). • Data Modeling: Advanced SQL (query optimization, SQL Server) and experience designing scalable data warehouses. • Modern Practices: Proven track record with Medallion Architecture, Metadata-Driven ETL, and Lakehouse design. • Leadership: Previous experience in a Lead Engineer role, including technical decision-making and mentoring. Nice-to-Have • Familiarity with Power BI for reporting and analytics integration. • Strong documentation skills and a passion for knowledge sharing.