Senior Data Engineer (with Backend Experience)

Nasze wymagania: Strong experience with Snowflake and dbt Strong Python skills Ability to write clean, scalable, and testable code (including unit tests) Solid understanding and applying object-oriented programming (OOP) Experience with version control systems: Git Experience with AWS or Azure Good knowledge of English (minimum C1 level) Mile widziane: Experience with Apache Spark (preferably Databricks) Experience with Azure AI Search or AWS OpenSearch Experience designing ETL/ELT pipelines Familiarity with LLMs, Azure OpenAI, or agentic AI systems O projekcie: You will work on a large-scale data platform built for a global consulting firm, supporting M&A (Mergers and Acquisitions) analysis. The project consists of: • A cloud-based platform for data visualization (APIs and services) • A large, integrated dataset (300M+ company records from 10+ sources) Your primary focus will be on data engineering (approx. 80%), including building and improving data pipelines and dataset quality. Additionally, you will contribute to backend and platform development (approx. 20%), including features, APIs, and integration with the data layer. Technology stack: • Languages: Python, SQL • Data: Snowflake, dbt • Workflow Orchestration: Apache Airflow (complex DAGs) • Data Processing: Apache Spark - Platform • Cloud: AWS (EKS, S3, Lambda, ECR, EMR, Opensearch) – Platform, Azure (AKS, Blob Storage, Azure Functions, ACR, Databricks, AI Search) - Dataset • API: API Gateway, FastAPI (REST, async) • CI/CD: GitHub Actions • Data Scale: 300M+ records across multiple sources • Future Direction: Agentic AI systems, LangChain and cloud-native AI libraries and frameworks, Azure OpenAI integration Team: Data Architecture Lead, Data Engineers, DataOps Engineers, Frontend Engineer, Backend Engineer, Product Owner Distributed team across Europe and India Agile, collaborative, proactive and impact-driven environment We work on multiple projects at the same time, so we may suggest a different project if it better matches your experience and profile. Zakres obowiązków: This is a hybrid role combining Data Engineering and Backend Engineering, with a stronger focus on the data layer. Build and maintain scalable ETL/data pipelines (Python, SQL, dbt, Snowflake, Airflow) Design and optimize data models, transformations and workflows Ensure data quality, consistency, and performance Integrate multiple data sources into a unified dataset Contribute to backend development (APIs, data access layer, backend-frontend contracts) Support platform integrations and cloud-based solutions Write clean, testable code and participate in code reviews Document data processes and system architecture Oferujemy: Flexible working hours and approach to work: fully remotely, in the office or hybrid Professional growth supported by internal training sessions and a training budget Solid onboarding with a hands-on approach to give you an easy start A great atmosphere among professionals who are passionate about their work