Data Engineer (On-Prem)
emagine Polska
⚲ Warsaw
Wymagania
- Scalability
- Deployment
- Operations
- Artificial Intelligence (AI)
- DataStage (ETL)
- Spark
- Cloud
- Kubernetes
- ETL
- DevOps
Opis stanowiska
Summary: The core purpose of the on-prem Data Engineer role is to design, manage, and optimize scalable data pipelines while supporting the technical decision-making process in building AI-driven services for Nordic clients. Main Responsibilities: • Build, maintain, and optimize scalable data pipelines (Kubernetes-based). • Work with distributed data processing tools (e.g., Spark). • Design and implement robust data architectures in hybrid/on-prem environments. • Work with storage technologies including Cloudera and MinIO. • Deliver end-to-end code across multiple environments. • Collaborate closely with a small, senior engineering team. • Participate in building AI-driven data products for Nordic clients. Key Requirements: • Strong hands-on experience with on-prem data platforms. • Proficiency in designing and operating Kubernetes. • Experience managing distributed data processing frameworks (e.g., Spark). • Familiarity with Cloudera for storage and data services. • Ability to deliver end-to-end code in hybrid environments. Nice to Have: • Hands-on experience with Airbyte. • Experience with dbt for development & data transformations. • Familiarity with ETL/ELT orchestration and modern data tooling. Other Details: • Team Structure: Small, remote-first team with a flexible culture. • Work Environment: Offers autonomy and responsibility.