Senior Azure Data Engineer
⚲ Wrocław
25 200 - 26 880 PLN netto (B2B)
Wymagania
- Databricks
- Python
- PySpark
- ADF
Opis stanowiska
SNI is serving as a trusted IT Outsourcing partner in line with the needs of World's most prestigious firms and carried out successful projects worldwide.
Responsibilities:
• Design and construct data structures, DDLs, tables, and views across MS Cloud SQL Server and Data Lake environments.
• Develop Azure Databricks notebooks and PySpark/Spark SQL jobs to aggregate, join, and output standardized data formats (SFFs).
• Configure automated Ingestion Frameworks to streamline data loading across staging and Data Lake Core (DLC) layers.
• Build, optimize, and maintain large-scale batch and real-time data pipelines using Azure Data Factory and Databricks.
• Monitor performance, tune complex SQL queries, and implement automation across data integration workflows.
• Support data governance, tracking data consumption patterns, testing pipeline outputs, and promoting curated datasets to the data catalog.
• Apply CI/CD practices and collaborate within cross-functional Agile/Scrum teams to deliver project estimates and technical solutions.
Skills:
• 5+ years of data engineering experience with a focus on enterprise ETL/ELT pipelines and cloud data platforms.
• Mandatory certifications: Certified Azure Data Engineer and Databricks Certified Data Engineer.
• Core Azure expertise: Azure Data Factory (ADF), Azure Databricks, Azure Data Lake Storage (ADLS), and MS Cloud SQL Server.
• Strong programming proficiency in Python, PySpark, Spark SQL, or Scala/Java.
• Advanced SQL skills for data manipulation, query optimization, and relational database management.
• Experience with legacy MS SQL stack technologies (SSIS, SSRS) and reporting tools (Power BI).
• Experience with automated ingestion frameworks, big data processing, and CI/CD delivery practices in Agile environments.
• Full professional fluency in both English and Polish.
Follow us on Linkedin! http://linkedin.com/company/snisourcing/
Responsibilities:
• Design and construct data structures, DDLs, tables, and views across MS Cloud SQL Server and Data Lake environments.
• Develop Azure Databricks notebooks and PySpark/Spark SQL jobs to aggregate, join, and output standardized data formats (SFFs).
• Configure automated Ingestion Frameworks to streamline data loading across staging and Data Lake Core (DLC) layers.
• Build, optimize, and maintain large-scale batch and real-time data pipelines using Azure Data Factory and Databricks.
• Monitor performance, tune complex SQL queries, and implement automation across data integration workflows.
• Support data governance, tracking data consumption patterns, testing pipeline outputs, and promoting curated datasets to the data catalog.
• Apply CI/CD practices and collaborate within cross-functional Agile/Scrum teams to deliver project estimates and technical solutions.
Skills:
• 5+ years of data engineering experience with a focus on enterprise ETL/ELT pipelines and cloud data platforms.
• Mandatory certifications: Certified Azure Data Engineer and Databricks Certified Data Engineer.
• Core Azure expertise: Azure Data Factory (ADF), Azure Databricks, Azure Data Lake Storage (ADLS), and MS Cloud SQL Server.
• Strong programming proficiency in Python, PySpark, Spark SQL, or Scala/Java.
• Advanced SQL skills for data manipulation, query optimization, and relational database management.
• Experience with legacy MS SQL stack technologies (SSIS, SSRS) and reporting tools (Power BI).
• Experience with automated ingestion frameworks, big data processing, and CI/CD delivery practices in Agile environments.
• Full professional fluency in both English and Polish.
Follow us on Linkedin! http://linkedin.com/company/snisourcing/
🔍 Dekoder Ogłoszenia
🔴
SNI is serving as a trusted IT Outsourcing partner in line with the needs of World's most prestigious firms and carried out successful projects worldwide.
Praca w firmie outsourcingowej, która może delegować Cię do różnych klientów, nawet jeśli ogłoszenie jest wystawione przez SNI.
🔴
Mandatory certifications: Certified Azure Data Engineer and Databricks Certified Data Engineer.
Posiadanie tych certyfikatów jest bezwzględnym wymogiem, a nie tylko mile widziane, co może ograniczyć pulę kandydatów.
🟡
Apply CI/CD practices and collaborate within cross-functional Agile/Scrum teams to deliver project estimates and technical solutions.
Oczekuje się aktywnego udziału w procesach CI/CD i pracy w metodyce Agile, co może oznaczać częste spotkania i iteracyjne podejście do zadań.
🟡
Monitor performance, tune complex SQL queries, and implement automation across data integration workflows.
Oprócz budowania, będziesz odpowiedzialny za optymalizację i utrzymanie istniejących rozwiązań, co może wymagać analizy i rozwiązywania problemów.
🟡
Support data governance, tracking data consumption patterns, testing pipeline outputs, and promoting curated datasets to the data catalog.
Zakres obowiązków obejmuje nie tylko techniczne aspekty budowy potoków danych, ale także aspekty związane z zarządzaniem danymi i ich jakością.