Senior Data Engineer (Ai&ML)
Harvey Nash Technology
⚲ Warszawa
30 000 - 40 000 PLN (PERMANENT)
Wymagania
- Python
- AWS
- Databricks
- Spark (nice to have)
- Scala (nice to have)
Opis stanowiska
O projekcie: Data Engineer – Cloud & AI Platforms We’re looking for a Data Engineer to design and scale high-throughput data pipelines supporting advanced analytics and AI-driven products. What You’ll Do - Architect and maintain distributed data pipelines in Databricks and AWS (Glue, EMR, Fargate, Step Functions) - Ingest and process large volumes of structured and unstructured data (internal, market, third-party, alternative sources) - Collaborate with AI/ML and engineering teams to design scalable data architectures and APIs - Optimize performance and cost using Spark and cloud-native best practices - Implement data governance, privacy, lineage, and access controls - Build automated validation, monitoring, and data quality frameworks - Evaluate emerging GenAI and data tooling to enhance platform capabilities What You Bring - 3+ years of experience in data engineering - Strong Python and experience with Spark or Scala - Proven experience building distributed pipelines in cloud environments - Solid understanding of data modeling, architecture, and warehousing principles - Innovative problem-solving mindset - Bachelor’s or Master’s degree in Computer Science or Engineering Nice to have: Experience with graph databases.