Senior Data Engineer
DCG sp. z o.o.
⚲ Warszawa, Gdynia, Gdańsk
25 200 - 29 400 PLN (B2B)
Wymagania
- Data engineering
- Cloud
- OLAP
- Python
- PL/SQL
- ETL
- IBM DataStage
- Azure Data Factory
- Oracle
- MS SQL Server
- Snowflake
- DevOps
- Git
- Docker
- Kubernetes
- Terraform
- Jenkins
- Bamboo
- Azure DevOps
- Test automation
- Robot Framework
- GitHub
- GitLab
- Airflow
- Kafka
Opis stanowiska
O projekcie: Offer: - Private medical care - Co-financing for the sports card - Constant support of dedicated consultant - Employee referral program Wymagania: Requirements: - Experience with designing OLAP systems and applying techniques such as STAR schema and denormalization - Proficiency in Python and PL/SQL development - Experience with ELT/ETL development using GUI-based tools such as IBM DataStage, Azure Data Factory (ADF), Stitch, and Matillion - Knowledge of database management systems including Oracle, MS SQL Server, and Snowflake - Understanding of DevOps and CI/CD practices, including Git, Docker, Kubernetes, Terraform, and CI/CD tools such as Jenkins, Bamboo, and Azure DevOps - Experience with test automation tools such as Robot Framework and Selenium, including knowledge of different test scenarios - Knowledge of version control systems including Git, GitHub, and GitLab - Experience with workflow orchestration tools such as Airflow and Prefect - Knowledge of streaming and batch processing concepts, including Kafka - Very good command of English, both spoken and written Codzienne zadania: - Design, build and maintain scalable ELT pipeline - Develop and optimize PLSQL queries and database objects for high performance data storage in Oracle - Write python-based data engineering scripts to process data - Implement CI/CD pipelines for data infrastructure and workflow automation - Understand and have experience of database change management - Work one or more of the major cloud vendor - Integrate and manage modern data stack - Monitor, debug and optimize data workflows and infrastructure performance - Ensure data quality, governance, and compliance with good practices