Senior Python Data Engineer – Data Platforms and Cloud Solutions
ITDS
⚲ Warszawa
23 100 - 27 300 PLN netto (B2B)
Wymagania
- Apache Airflow
- REST APIs
- fastapi
- SQL
- Python
- Google Cloud Platform
- BigQuery
- ETL
- CI/CD
- Distributed systems
Opis stanowiska
Unleash the power of data — develop innovative platforms that shape the future of healthcare and agriculture! Warsaw-based opportunity with remote work options (up to 5 days per week).As a Senior Python Data Engineer, you will be working for our client, a global leader in life sciences dedicated to advancing healthcare and agriculture. You will contribute to a large-scale project focused on building and maintaining high-performance data-driven platforms, leveraging modern cloud technologies to process and analyze vast amounts of critical information. This role offers a unique chance to influence architecture, optimize data pipelines, and drive innovation in environments demanding scalability and performance, all while adhering to industry standards and best practices in software engineering. Your main responsibilities: • Design and implement scalable backend services with Python 3.x • Develop secure REST APIs using FastAPI and Pydantic • Optimize SQL queries and design efficient data models in BigQuery • Build and maintain ETL pipelines with orchestrators such as Airflow • Integrate and manage solutions within GCP services like Pub/Sub, Cloud Functions, and Cloud Storage • Ensure cost-effective cloud resource usage through monitoring and tuning • Collaborate with teams to define technical designs, RFCs, and runbooks • Implement robust authentication, authorization, and security practices • Handle high-load system design challenges such as throughput and backpressure • Support CI/CD pipelines and automation with Cloud Build or equivalent tools You're ideal for this role if you have: • 7+ years of professional software engineering experience • 4+ years of production experience with Python 3.x • Strong knowledge of FastAPI, Pydantic, and asynchronous programming • Proven expertise in SQL, data modeling, and BigQuery optimization • Hands-on experience with GCP services (Pub/Sub, Cloud Functions, Cloud Storage, IAM) • Track record designing and operating high-load distributed systems • Experience with ETL processes and orchestration tools like Airflow • Familiarity with CI/CD pipelines and automation practices • Excellent communication and ownership skills in cross-functional environments • Strong focus on security best practices and data protection Nice-to-have skills: (optional) • Experience with file parsing at scale (CSV/Excel/XML), schema evolution/versioning • Data quality frameworks (assertions, expectations), dbt/Dataform familiarity • Event-driven or streaming backgrounds; Beam/Dataflow experience • Concurrency/performance profiling in Python (asyncio, multiprocessing) • IaC (Terraform), SRE practices Language Required for the role: • English — Communicative level Eligibility to work on this role: • Only candidates with an existing legal right to work in the European Union will be considered for this role. #MAKEYourCareerBETTER Interested? Apply now and include your CV (preferably in English) along with a statement confirming your consent to the processing and storage of your personal data.