Senior Data Engineer (Databricks)
⚲ Warszawa
Do uzgodnienia
Wymagania
- Python
- Databricks
- Kafka
- SQL
Opis stanowiska
Nasze wymagania:
Minimum 5 years of experience in Data Engineering or a similar software engineering role.
Strong, hands-on experience in building data solutions with Databricks and Apache Spark.
High proficiency in programming with Python or Scala, coupled with advanced SQL skills.
Deep understanding of Delta Lake concepts and modern Data Lakehouse architectures.
Solid experience working with major cloud platforms (Azure, AWS, or GCP).
Experience with version control (Git) and CI/CD pipelines (e.g., GitHub Actions, Azure DevOps).
Excellent analytical and problem-solving skills with a high degree of autonomy.
Fluency in English (both spoken and written).
Mile widziane:
Official Databricks certifications (e.g., Databricks Certified Data Engineer Professional).
Hands-on experience with modern data orchestration and transformation tools such as dbt, Apache Airflow, or Databricks Workflows.
Practical knowledge of Infrastructure as Code (IaC) principles using Terraform.
O projekcie:
Join us as a Senior Databricks Data Engineer! You will play a key role in designing, building, and maintaining robust data architectures. Utilizing the power of Databricks, Apache Spark, and cloud technologies, you will help us process massive amounts of data and drive data-driven decision-making across the organization.
Zakres obowiązków:
Design, build, and maintain highly scalable and reliable ETL/ELT data pipelines using Databricks and Apache Spark.
Architect and optimize Data Lakehouse solutions leveraging Delta Lake.
Write clean, efficient, and well-documented code in Python, Scala, and SQL.
Collaborate closely with Data Scientists, Data Analysts, and business stakeholders to understand data requirements and deliver optimal solutions.
Monitor, troubleshoot, and optimize data processing performance and cloud infrastructure costs.
Drive engineering best practices, including CI/CD, automated testing, and code reviews.
Mentor junior team members and actively share knowledge within the engineering team.
Minimum 5 years of experience in Data Engineering or a similar software engineering role.
Strong, hands-on experience in building data solutions with Databricks and Apache Spark.
High proficiency in programming with Python or Scala, coupled with advanced SQL skills.
Deep understanding of Delta Lake concepts and modern Data Lakehouse architectures.
Solid experience working with major cloud platforms (Azure, AWS, or GCP).
Experience with version control (Git) and CI/CD pipelines (e.g., GitHub Actions, Azure DevOps).
Excellent analytical and problem-solving skills with a high degree of autonomy.
Fluency in English (both spoken and written).
Mile widziane:
Official Databricks certifications (e.g., Databricks Certified Data Engineer Professional).
Hands-on experience with modern data orchestration and transformation tools such as dbt, Apache Airflow, or Databricks Workflows.
Practical knowledge of Infrastructure as Code (IaC) principles using Terraform.
O projekcie:
Join us as a Senior Databricks Data Engineer! You will play a key role in designing, building, and maintaining robust data architectures. Utilizing the power of Databricks, Apache Spark, and cloud technologies, you will help us process massive amounts of data and drive data-driven decision-making across the organization.
Zakres obowiązków:
Design, build, and maintain highly scalable and reliable ETL/ELT data pipelines using Databricks and Apache Spark.
Architect and optimize Data Lakehouse solutions leveraging Delta Lake.
Write clean, efficient, and well-documented code in Python, Scala, and SQL.
Collaborate closely with Data Scientists, Data Analysts, and business stakeholders to understand data requirements and deliver optimal solutions.
Monitor, troubleshoot, and optimize data processing performance and cloud infrastructure costs.
Drive engineering best practices, including CI/CD, automated testing, and code reviews.
Mentor junior team members and actively share knowledge within the engineering team.
🔍 Dekoder Ogłoszenia
🔴
Excellent analytical and problem-solving skills with a high degree of autonomy.
Oczekuje się, że będziesz samodzielnie rozwiązywać problemy bez ciągłego nadzoru, co może oznaczać mniejszą pomoc ze strony zespołu.
🔴
High proficiency in programming with Python or Scala, coupled with advanced SQL skills.
Oczekuje się biegłości na poziomie, który pozwala na samodzielne tworzenie złożonych rozwiązań, a nie tylko podstawowe skrypty.
🔴
Deep understanding of Delta Lake concepts and modern Data Lakehouse architectures.
Wymaga to nie tylko znajomości narzędzia, ale też teoretycznego i praktycznego zrozumienia jego zastosowania w architekturze danych.
🔴
Design, build, and maintain highly scalable and reliable ETL/ELT data pipelines using Databricks and Apache Spark.
Może oznaczać pracę nad istniejącymi, potencjalnie problematycznymi systemami, a nie tylko tworzenie nowych od zera.