Senior Data Engineer
Avenga
⚲ Wrocław
28 000 - 31 000 PLN netto (B2B)
Wymagania
- Python
- SQL
- Databricks
- MS Azure
Opis stanowiska
Senior Data Engineer for the MDM Data Product Team The Data Product Team within the MDM unit is responsible for provisioning data from our master data customer and product systems. We make sure that high quality and well-structured Master Data is available in our cloud platforms at any speed and in any format required for downstream consumption in the whole company. We manage the whole lifecycle of the MDM Core Data Products in close collaboration with source system teams, downstream consumers, and data management networks. We use modern cloud-based toolsets to ingest, cleanse, standardize, merge, transform, document and provision data. Senior Data Engineer - Role Mission As a member of the Stable Team, You are participating in development, maintenance and support of data ingestions / integrations to source systems and data products with API solutions. • You are a self-starter that actively searches for relevant information and builds cooperation with stakeholders, subject-matter experts and other team members. • You proactively provide input to stakeholders to guide them into the best solution. • You actively strive to learn and understand relevant business processes and their corresponding legacy IT systems and data contents in relevant parts of the company landscape • You can independently cooperate with data stewards and architects to implement non-functional requirements and architecture, including Infrastructure, Integration, and Software Architecture, quality, security and stability. • You can independently make software design decisions and technology choices regarding digital products, services and solutions in collaboration with the Digital Product Owner and Enterprise / Solution Architects. • You support business stakeholders and your team members on technical questions and implement best practices and modern ways of developing. • You create patterns and routines for managing and deploying high-quality code by developing test routines and CI/CD automation. Competence and Experience • You have several years of relevant experience from designing data products / data warehouse solutions, including logical and physical design for data loading and transformations as well as data modelling. You have good understanding of various design patterns and architectures for data processing and storage. • You have expert knowledge in design and development of large-scale data processing solutions, using various tools such as Apache Spark, Databricks, Azure Data Factory, Delta Lake or equivalent. • You have expert knowledge in Python and SQL languages. • You have a very good understanding of technology choices, cost impact and trade-offs for data storage, processing and provisioning in Azure solutions. • You have very good knowledge in developing data solutions in Azure. • Competence in master data management tools and practices is considered a large merit. • Competence in developing and maintaining API’s is considered a large merit.