Azure Data Engineering Expert

O projekcie: Insights & Data delivers state-of-the-art Data solutions. Our expertise primarily lies in Cloud & Big Data engineering, where we develop robust systems capable of processing extensive and complex datasets, utilizing specialized Cloud Data services across platforms like AWS, Azure, and GCP. We oversee the entire Software Development Life Cycle (SDLC) of these solutions, which involves not only leveraging data processing tools such as ETL but also extensive programming in languages like Python, Scala, or Java, coupled with the adoption of DevOps tools and best practices. The processed data is then made accessible to downstream systems through APIs, outbound interfaces, or is visualized via comprehensive reports and dashboards. Additionally, within our AI Center of Excellence, we undertake Data Science and Machine Learning projects with a focus on cutting-edge areas such as Generative AI, Natural Language Processing (NLP), Anomaly Detection, and Computer Vision. Project aims at integrating and harmonize data from strategic platforms into our global data platform. We look for a data engineer to create, maintain and improve pipelines from source to harmonized layer and from harmonized layer to consumption layer for Power BI reporting. Data engineers are responsible for building reliable and scalable data infrastructure that enables organizations to derive meaningful insights, make data-driven decisions, and unlock the value of their data assets. Business domains covered are around commercial insurance pricing & underwriting. Wymagania: - strong experience in data engineering- very good experience with data-oriented Python (Polars, Pandas, Pyspark) very good experience with Microsoft Azure, especially Synapse is a plus- Solid understanding of cloud architecture ITIL- Strong analytical skills and customer focus- Can rapidly adapt and respond to changes in environment and priorities- Advanced English is a must- Excellent communication skills with high customer focus- Desirable: Database knowledge, PowerBI knowledge, experience in Bl Projects & DataVault ✔️Nice-to-have knowledge and experience: - ITIL Foundation certified or proven work experience in IT Service Management Codzienne zadania: - Developing new functionality within the Python-based ETL pipeline - Ongoing maintenance and optimization of existing pipelines - Planning the capacity and sizing of service components, as well as proactively managing cloud components - Initiating the planning and setup of new service components - Coordinating the operations and maintenance of related software, complex server infrastructure, and databases (such as Synapse, PowerBI, and local development environments) - Service-related documentation must be maintained in accordance with customer technology standards - Monitoring services and managing SLA performance will be an ongoing task - Processes and configurations related to the environments must be documented, maintained, and optimized - Collaboration with various stakeholders, including development, architecture, rollout, and operations teams, as well as customers, will be necessary for managing releases, upgrades, and changes