Lead Data Engineer
⚲ Warszawa
170 - 180 PLN netto (B2B)
Wymagania
- Databricks
- Python
- PySpark
- Data Lakehouse
Opis stanowiska
SNI is serving as a trusted IT Outsourcing partner in line with the needs of World's most prestigious firms and carried out successful projects worldwide.
Responsibilities:
• Lead data engineering work streams and hands-on technical implementation across multi-functional squads.
• Design and deploy resilient, scalable batch and streaming pipelines and data processing frameworks.
• Drive platform migrations from legacy schedulers to modern, cloud-native workflow orchestration tools.
• Establish data governance policies, lineage tracking, metadata management, and role-based security controls (e.g., Unity Catalog).
• Implement data virtualization and unified integration patterns across disparate, multi-source enterprise systems.
• Lead performance tuning, query optimization, cost management, and operational reliability across the data platform.
Skills:
• 6–9 years of professional experience in data engineering and enterprise data platform delivery.
• Expert-level hands-on experience with Databricks and Apache Spark.
• Deep knowledge of Data Lakehouse architecture (Delta Lake, medallion architecture, schema enforcement).
• Advanced proficiency in Python, SQL, and PySpark for data transformation and distributed computing.
• Strong background in workflow orchestration tools (e.g., Airflow, Databricks Workflows, Dagster) and scheduler migration projects.
• Practical experience implementing data governance, cataloging, metadata, and data access security frameworks.
• Proven track record architecting data virtualization and integration solutions across heterogeneous cloud and on-premise sources.
Follow us on Linkedin! https://www.linkedin.com/company/snisourcing/posts/?feedView=all
PLDE2628
Responsibilities:
• Lead data engineering work streams and hands-on technical implementation across multi-functional squads.
• Design and deploy resilient, scalable batch and streaming pipelines and data processing frameworks.
• Drive platform migrations from legacy schedulers to modern, cloud-native workflow orchestration tools.
• Establish data governance policies, lineage tracking, metadata management, and role-based security controls (e.g., Unity Catalog).
• Implement data virtualization and unified integration patterns across disparate, multi-source enterprise systems.
• Lead performance tuning, query optimization, cost management, and operational reliability across the data platform.
Skills:
• 6–9 years of professional experience in data engineering and enterprise data platform delivery.
• Expert-level hands-on experience with Databricks and Apache Spark.
• Deep knowledge of Data Lakehouse architecture (Delta Lake, medallion architecture, schema enforcement).
• Advanced proficiency in Python, SQL, and PySpark for data transformation and distributed computing.
• Strong background in workflow orchestration tools (e.g., Airflow, Databricks Workflows, Dagster) and scheduler migration projects.
• Practical experience implementing data governance, cataloging, metadata, and data access security frameworks.
• Proven track record architecting data virtualization and integration solutions across heterogeneous cloud and on-premise sources.
Follow us on Linkedin! https://www.linkedin.com/company/snisourcing/posts/?feedView=all
PLDE2628
🔍 Dekoder Ogłoszenia
✓ Ogłoszenie wygląda transparentnie — brak typowych czerwonych flag.