Principal Data Platform Engineer

O projekcie: Project Overview We are a global, collaborative team of engineers, data scientists, and product managers leveraging Generative AI to solve high-impact challenges. Our mission is to evolve Sightline, a premier Databricks-based Lakehouse platform. This is not a traditional BI or data architect role. We are looking for a developer-oriented platform leader who sits at the intersection of software engineering, streaming systems, and DevOps. You will not just consume the platform; you will shape it, building the abstraction layers and frameworks that allow our global engineering teams to deliver data products at scale. Wymagania: Background & Leadership- 10+ years of professional experience in global IT projects, with a proven track record of managing senior stakeholders and business end-users.- Software Engineering DNA: You should come from a strong coding background rather than a pure analytics or data science profile.- Language Proficiency: B2+ or higher in written and spoken English.- Ambiguity Navigator: Comfortable shaping a vision and building a platform from the ground up in a fast-paced environment.Technical Expertise- Databricks Mastery: Deep experience with Lakehouse architecture, Structured Streaming, and Delta Live Tables.- Azure Cloud Stack: Practical knowledge of Azure Functions, Service Bus, Event Hub, App Services, and databases (Cosmos DB, MongoDB).- Modern DevOps: Hands-on experience with CI/CD pipelines and managing cloud infrastructure via Terraform or similar IaC tools.- AI-Driven Development: Proficiency using modern AI agents and tools such as GitHub Copilot, Cursor, Claude Code, or Gemini CLI to accelerate delivery.- Integration Knowledge: Understanding of integration patterns between data platforms and consumption tools like Power BI. Codzienne zadania: - Platform Evolution: Act as the Tech Lead for the Databricks Lakehouse, designing abstraction layers that simplify developer workflows and move beyond "batch-only" ETL thinking. - Streaming-First Strategy: Drive the transition from legacy warehouse paradigms to event-driven, near-real-time architectures using Delta Live Tables and Structured Streaming. - Engineering Leadership: Manage a small, specialized team of DevOps and Platform Engineers while mentoring cross-functional teams on best practices and toolsets. - Operational Excellence: Own the "DataOps" lifecycle, including CI/CD, Infrastructure as Code (IaC), and the optimization of Databricks consumption/costs. - Strategic Integration: Bridge the gap between the platform and downstream technologies like Microsoft Fabric, Power BI, and Azure ecosystem services.