Semantic Data Engineer
⚲ Warszawa, Olsztyn, Gdańsk, Białystok, Szczecin, Poznań, Łódź, Kraków, Wrocław, Lublin
16 800 - 22 680 PLN netto (B2B)
Wymagania
- Data
- SQL
- ETL
- Python
- NoSQL
Opis stanowiska
We are looking for a Data Engineer to join an international project supporting pharmaceutical research through the integration and standardization of laboratory data. In this role, you will design and maintain scalable data pipelines, enabling seamless access to high-quality, analysis-ready data for scientists and research teams.
Responsibilities
• Design, develop, and maintain ETL pipelines for integrating laboratory instrument data.
• Build and optimize SQL queries to support efficient data processing and analytics.
• Develop and extend data models and JSON Schemas based on industry standards.
• Collaborate with DevOps teams to implement and maintain cloud-based data solutions.
• Ensure data quality, consistency, and compliance with FAIR data principles.
• Support continuous improvements in data integration processes and platform capabilities.
Requirements
• Experience in Data Engineering, including SQL, ETL development, and data integration.
• Strong Python programming skills.
• Hands-on experience with NoSQL databases (document or graph databases).
• Familiarity with DevSecOps practices, including CI/CD, Git, Infrastructure as Code (Terraform or CloudFormation), and Docker.
• Good communication skills and experience working in cross-functional Agile teams.
• Professional proficiency in English.
Nice to Have
• Experience with Semantic Web technologies (RDF, SPARQL, Knowledge Graphs).
• Hands-on experience with AWS, particularly Lambda, Glue, and Step Functions.
• Familiarity with Jira and Confluence.
• Experience working with scientific, laboratory, or pharmaceutical data environments.
Responsibilities
• Design, develop, and maintain ETL pipelines for integrating laboratory instrument data.
• Build and optimize SQL queries to support efficient data processing and analytics.
• Develop and extend data models and JSON Schemas based on industry standards.
• Collaborate with DevOps teams to implement and maintain cloud-based data solutions.
• Ensure data quality, consistency, and compliance with FAIR data principles.
• Support continuous improvements in data integration processes and platform capabilities.
Requirements
• Experience in Data Engineering, including SQL, ETL development, and data integration.
• Strong Python programming skills.
• Hands-on experience with NoSQL databases (document or graph databases).
• Familiarity with DevSecOps practices, including CI/CD, Git, Infrastructure as Code (Terraform or CloudFormation), and Docker.
• Good communication skills and experience working in cross-functional Agile teams.
• Professional proficiency in English.
Nice to Have
• Experience with Semantic Web technologies (RDF, SPARQL, Knowledge Graphs).
• Hands-on experience with AWS, particularly Lambda, Glue, and Step Functions.
• Familiarity with Jira and Confluence.
• Experience working with scientific, laboratory, or pharmaceutical data environments.
🔍 Dekoder Ogłoszenia
🔴
seamless access to high-quality, analysis-ready data for scientists and research teams
Dane będą prawdopodobnie wymagały znaczącego czyszczenia i transformacji przed faktycznym użyciem przez naukowców.
🔴
Develop and extend data models and JSON Schemas based on industry standards
Może oznaczać pracę z istniejącymi, ale niekoniecznie dobrze udokumentowanymi lub elastycznymi standardami.
🔴
Collaborate with DevOps teams to implement and maintain cloud-based data solutions
Może oznaczać, że będziesz musiał przejąć część obowiązków DevOps, jeśli zespół nie jest wystarczająco duży lub kompetentny.
🔴
Ensure data quality, consistency, and compliance with FAIR data principles
W praktyce może oznaczać walkę z chaotycznymi danymi i brakiem jasnych procesów ich wprowadzania.
🔴
Support continuous improvements in data integration processes and platform capabilities
Oczekuje się, że będziesz aktywnie szukać i wdrażać usprawnienia, co może oznaczać dodatkowe obowiązki poza podstawowymi zadaniami.