Senior Data Engineer (Databricks)
⚲ Wrocław
18 480 - 25 200 PLN (B2B)
Wymagania
- Data engineering
- PySpark
- Spark
- SQL
- Python
- ETL
- Cloud
- Azure
- AWS (nice to have)
- Git (nice to have)
- Communication skills (nice to have)
- Databricks (nice to have)
- Performance tuning (nice to have)
Opis stanowiska
O projekcie:
About the role
We are looking for a Senior Data Engineer with strong Databricks experience to build and optimize modern data platforms. You will develop scalable ETL/ELT pipelines, process large datasets and contribute to the evolution of cloud-native Data Engineering solutions.
About Spyrosoft
Spyrosoft is an authentic, cutting-edge software engineering company, established in 2016. In 2021 and 2022, we were among the fastest growing technology companies in Europe, according to the Financial Times. We were founded by a group of tech experts with established backgrounds in software engineering, who created an ‘engineer-to-engineer’ workplace, powered by enthusiasm, fairness and authentic relationships. Having a unique offering, which bridge the gap between technology and business, we specialise in technology solutions for industry 4.0, automotive, geospatial, healthcare & life sciences, employee experience & education and financial services industries.
Wymagania:
- 5+ years of experience in Data Engineering- 2+ years of commercial experience with Databricks- Strong knowledge of PySpark and Spark SQL- Excellent SQL skills- Strong Python programming skills- Experience building ETL/ELT pipelines- Experience working with Delta Lake- Good understanding of Medallion Architecture- Experience with cloud environments (Azure, AWS or GCP)- Experience with Git and CI/CD- Understanding of data modelling principles- Good communication skills- Fluent English - C1Nice to have:- Unity Catalog experience- dbt- Kafka or Structured Streaming- Airflow or Databricks Workflows- Databricks certification- Experience with performance tuning and cost optimization
Codzienne zadania:
- Design, develop and maintain scalable ETL/ELT pipelines
- Build reliable batch and streaming data processing solutions
- Develop data pipelines using Databricks and Apache Spark
- Optimize Spark jobs and improve platform performance
- Work with Delta Lake and Medallion Architecture
- Build and maintain data models for analytics and reporting
- Ensure data quality, monitoring and observability
- Collaborate with Architects, Data Scientists and Analytics teams
- Participate in code reviews and technical improvements
- Contribute to CI/CD processes and engineering best practices
About the role
We are looking for a Senior Data Engineer with strong Databricks experience to build and optimize modern data platforms. You will develop scalable ETL/ELT pipelines, process large datasets and contribute to the evolution of cloud-native Data Engineering solutions.
About Spyrosoft
Spyrosoft is an authentic, cutting-edge software engineering company, established in 2016. In 2021 and 2022, we were among the fastest growing technology companies in Europe, according to the Financial Times. We were founded by a group of tech experts with established backgrounds in software engineering, who created an ‘engineer-to-engineer’ workplace, powered by enthusiasm, fairness and authentic relationships. Having a unique offering, which bridge the gap between technology and business, we specialise in technology solutions for industry 4.0, automotive, geospatial, healthcare & life sciences, employee experience & education and financial services industries.
Wymagania:
- 5+ years of experience in Data Engineering- 2+ years of commercial experience with Databricks- Strong knowledge of PySpark and Spark SQL- Excellent SQL skills- Strong Python programming skills- Experience building ETL/ELT pipelines- Experience working with Delta Lake- Good understanding of Medallion Architecture- Experience with cloud environments (Azure, AWS or GCP)- Experience with Git and CI/CD- Understanding of data modelling principles- Good communication skills- Fluent English - C1Nice to have:- Unity Catalog experience- dbt- Kafka or Structured Streaming- Airflow or Databricks Workflows- Databricks certification- Experience with performance tuning and cost optimization
Codzienne zadania:
- Design, develop and maintain scalable ETL/ELT pipelines
- Build reliable batch and streaming data processing solutions
- Develop data pipelines using Databricks and Apache Spark
- Optimize Spark jobs and improve platform performance
- Work with Delta Lake and Medallion Architecture
- Build and maintain data models for analytics and reporting
- Ensure data quality, monitoring and observability
- Collaborate with Architects, Data Scientists and Analytics teams
- Participate in code reviews and technical improvements
- Contribute to CI/CD processes and engineering best practices
🔍 Dekoder Ogłoszenia
🔴
build and optimize modern data platforms
Może oznaczać zarówno budowanie od zera, jak i optymalizację istniejących, potencjalnie przestarzałych rozwiązań.
🔴
contribute to the evolution of cloud-native Data Engineering solutions
Może oznaczać pracę nad nowymi funkcjonalnościami, ale też wspieranie istniejących rozwiązań w ich rozwoju, co może być mniej ekscytujące.
🔴
engineer-to-engineer’ workplace
Sugestia, że komunikacja i zrozumienie techniczne będą na wysokim poziomie, ale może też oznaczać brak silnego wsparcia ze strony managementu nietechnicznego.
🔴
bridge the gap between technology and business
Oczekuje się, że będziesz tłumaczyć wymagania biznesowe na rozwiązania techniczne i odwrotnie, co może oznaczać dodatkowe obowiązki komunikacyjne.
🟡
Good understanding of Medallion Architecture
Wymagane jest zrozumienie, ale niekoniecznie aktywne projektowanie lub implementowanie tej architektury od podstaw.