Senior Data Engineer with Python

Senior Data Engineer (Azure / Databricks / Python) Location: Remote Type: Full-time Level: Senior Duration: 4 months Start: ASAP About the role We are looking for a Senior Data Engineer to design and implement scalable data pipelines supporting advanced, customizable data processing workflows. You will be responsible for building end-to-end solutions that ingest data from streaming sources, process it in distributed analytics environments, and expose curated datasets through well-defined interfaces. The role also involves integrating external machine learning workloads into an existing cloud-native architecture. This position is well-suited for engineers who enjoy working at the intersection of data engineering, cloud platforms, and applied machine learning. Responsibilities • Design and implement scalable ETL/ELT data pipelines for streaming and batch workloads • Ingest data from cloud-based event streaming platforms into distributed processing environments • Build and maintain data processing workflows using Azure-native services • Expose processed and curated data according to architectural specifications • Integrate external machine learning or analytical workloads into existing data pipelines • Ensure reliability, performance, and scalability of data processing solutions Core Skills • Extensive hands-on experience with Azure Databricks • Strong experience with Azure Event Hubs and Azure Functions • Proven expertise in designing and operating robust, scalable ETL/ELT pipelines • Strong Python programming skills for data processing and orchestration • Hands-on experience integrating LLM APIs into production systems Highly Desirable • Experience integrating machine learning or custom analytics models into production data pipelines • Familiarity with streaming and time-series data processing • Knowledge of data formats commonly used in large-scale monitoring or analytics systems • Experience designing data solutions for high-volume, high-velocity data What we offer • Work on complex, large-scale data processing systems • Real impact on architecture and data platform design • Collaboration with experienced engineers and architects • Competitive compensation