Data Engineer
Toro Performance Sp. z o.o.
⚲ Poland (Remote)
Wymagania
- NoSQL
- German
- Data Mesh
- SQL
- Data Lakes
- English
- OOP
- ETL
Opis stanowiska
We are currently looking for Senior Data Engineer to support an LLM/NPL Project. Data Engineer must-have experiences with: · Data management and integration, including Data Mesh, Data Lakes, and integration with external services · Core cloud concepts, with a special focus on databases (e.g., AWS RDS /Kinesis /Glue /EC2 /EKS /ECS) · Optimization of NoSQL and SQL databases in a cloud environment · Software engineering, especially in object-oriented programming (OOP) · SQL and database query optimization techniques · Implementing ETL and data ingestion pipelines for both initial and update loads, including batch processing of data (structured and unstructured sources) · Performing database benchmarking for latency and performance optimization · Good programming practices, particularly in implementing data cleansers and parsers · Understands the differences between various database solutions and can recommend appropriate ones, including document databases and vector databases · Big Data technologies (Hadoop, Spark, or Apache Kafka) · Message queues and asynchronous processing Location: 100% remote