Senior / Lead Data Software Engineer (Python, Spark, Azure)
EPAM Systems (Poland) sp. z o.o.
⚲ Wrocław, Krzyki
Do uzgodnienia
Wymagania
- Pytest
- Spark
- Microsoft Azure
- PySpark
- Docker
- Kubernetes
- Terraform Cloud
- Gtilab
Opis stanowiska
Nasze wymagania:
Proficiency in Python and Spark with at least 3 years in data engineering roles
Strong experience with Azure Databricks and PySpark
Proven expertise in designing and implementing ETL/ELT solutions
Experience migrating big data platforms to Azure-native services
Proficiency with Delta tables for model tuning
Knowledge of data governance and regulatory compliance frameworks
Familiarity with Docker, Kubernetes (AKS), and Terraform for infrastructure automation
Ability to manage large data volumes with high efficiency
Excellent problem-solving and analytical skills
Strong communication and collaboration abilities
English proficiency at B2 level or higher
O projekcie:
We are seeking a Senior/Lead Data Software Engineer to join our team working on a scalable, ML-ready platform that enhances portfolio model development and deployment with advanced data governance and AI capabilities.
You will play a key role in migrating from an IaaS Big Data platform to Azure-native Databricks, optimizing data workflows and improving data quality. Join us to contribute to innovative solutions that boost client services and regulatory compliance.
Zakres obowiązków:
Migrate and optimize over 500 data jobs using Azure Databricks optimization techniques
Manage and process 12 TB of data efficiently across platforms
Tune machine learning models for Azure environments using Java Spark and Delta tables
Update and maintain libraries to address security vulnerabilities
Develop and maintain ETL/ELT pipelines using PySpark and related technologies
Collaborate with cross-functional teams to integrate GenAI capabilities into data workflows
Monitor data quality and implement improvements to ensure accuracy and reliability
Automate deployment and operational tasks using Terraform and GitLab CI/CD
Support data governance initiatives to comply with regulatory standards
Troubleshoot and resolve performance issues in data processing systems
Document system processes and provide technical guidance to junior engineers
Implement best practices for code quality and data security
Participate in code reviews and knowledge sharing sessions
Optimize costs associated with data storage and processing
Oferujemy:
We gather like-minded people:
• Top tech minds driving innovation in AI, cloud and digital platform modernization
• Supportive team and agile, startup-like culture
• Hybrid by design mode and opportunity to work remotely within Poland
• Chance to work abroad for up to 60 days annually
• Business-driven relocation opportunities
We provide growth opportunities:
• Career development programs
• Thought leadership, mentoring, soft skills and well-being programs
• Certification (Anthropic, Gemini, GCP, Azure, AWS)
• English classes
We cover it all:
• Stable pay
• Participation in the Employee Stock Purchase Plan with a 15% discount
• Benefits package (health insurance, multisport, shopping vouchers)
• Referral bonuses up to $2,000
• Offices featuring entertainment and relaxation zones, table tennis and football, free snacks, coffee and more
• Corporate, social and well-being events
Please, note:
• Benefits listed above are available to employees only
• We are open for working with Contractors. Terms of B2B cooperation agreements are agreed individually
• We will reach out to selected candidates exclusively
Proficiency in Python and Spark with at least 3 years in data engineering roles
Strong experience with Azure Databricks and PySpark
Proven expertise in designing and implementing ETL/ELT solutions
Experience migrating big data platforms to Azure-native services
Proficiency with Delta tables for model tuning
Knowledge of data governance and regulatory compliance frameworks
Familiarity with Docker, Kubernetes (AKS), and Terraform for infrastructure automation
Ability to manage large data volumes with high efficiency
Excellent problem-solving and analytical skills
Strong communication and collaboration abilities
English proficiency at B2 level or higher
O projekcie:
We are seeking a Senior/Lead Data Software Engineer to join our team working on a scalable, ML-ready platform that enhances portfolio model development and deployment with advanced data governance and AI capabilities.
You will play a key role in migrating from an IaaS Big Data platform to Azure-native Databricks, optimizing data workflows and improving data quality. Join us to contribute to innovative solutions that boost client services and regulatory compliance.
Zakres obowiązków:
Migrate and optimize over 500 data jobs using Azure Databricks optimization techniques
Manage and process 12 TB of data efficiently across platforms
Tune machine learning models for Azure environments using Java Spark and Delta tables
Update and maintain libraries to address security vulnerabilities
Develop and maintain ETL/ELT pipelines using PySpark and related technologies
Collaborate with cross-functional teams to integrate GenAI capabilities into data workflows
Monitor data quality and implement improvements to ensure accuracy and reliability
Automate deployment and operational tasks using Terraform and GitLab CI/CD
Support data governance initiatives to comply with regulatory standards
Troubleshoot and resolve performance issues in data processing systems
Document system processes and provide technical guidance to junior engineers
Implement best practices for code quality and data security
Participate in code reviews and knowledge sharing sessions
Optimize costs associated with data storage and processing
Oferujemy:
We gather like-minded people:
• Top tech minds driving innovation in AI, cloud and digital platform modernization
• Supportive team and agile, startup-like culture
• Hybrid by design mode and opportunity to work remotely within Poland
• Chance to work abroad for up to 60 days annually
• Business-driven relocation opportunities
We provide growth opportunities:
• Career development programs
• Thought leadership, mentoring, soft skills and well-being programs
• Certification (Anthropic, Gemini, GCP, Azure, AWS)
• English classes
We cover it all:
• Stable pay
• Participation in the Employee Stock Purchase Plan with a 15% discount
• Benefits package (health insurance, multisport, shopping vouchers)
• Referral bonuses up to $2,000
• Offices featuring entertainment and relaxation zones, table tennis and football, free snacks, coffee and more
• Corporate, social and well-being events
Please, note:
• Benefits listed above are available to employees only
• We are open for working with Contractors. Terms of B2B cooperation agreements are agreed individually
• We will reach out to selected candidates exclusively
🔍 Dekoder Ogłoszenia
✓ Ogłoszenie wygląda transparentnie — brak typowych czerwonych flag.