Python & Apache Spark Engineer / Data Processing Specialist

O projekcie:
About the ProjectYou will join a dynamic technology team responsible for designing, developing, and deploying innovative enterprise technology and AI-driven solutions supporting the delivery of tax services.The team combines expertise across software engineering, data engineering, AI, tax technology, change management, and project management. The projects cover the full technology lifecycle — from solution design and development through deployment and optimization, to training, adoption, and stakeholder engagement.You will work in a modern, cloud-based environment and contribute to the development of scalable data processing solutions, with a strong focus on Python, Apache Spark, cloud technologies, microservices, and modern data platforms.Technology Stack- Cloud: Microsoft Azure- Data Processing: Apache Spark, Databricks, Azure Synapse / Microsoft Fabric- Backend: Python, .NET 8, ASP.NET Core- Data: MongoDB, Azure SQL, Parquet, Delta Tables- Frontend: Angular 18, Kendo- AI / GenAI: LangGraph, LangChain, RAG Pipelines, Multi-modal LLMs- Architecture: Microservices- DevOps & Collaboration: GitHub Enterprise, GitHub Copilot- Containerization: Docker, Kubernetes

Wymagania:
Professional Skills- Strong analytical and problem-solving capabilities.- Detail-oriented approach with the ability to identify potential technical issues and bottlenecks.- Practical and pragmatic mindset - able to balance standardized processes with the flexibility required to achieve project objectives.- Strong organizational and time-management skills.- Ability to independently manage workload, prioritize tasks, and meet tight deadlines.- Excellent communication skills and the ability to explain technical topics clearly to both technical and non-technical stakeholders.- Comfortable working independently while also being an active member of a collaborative, international team.- Willingness to take ownership and proactively drive technical topics forward.Nice to Have- Experience with Microsoft Azure or other major cloud platforms.- Practical knowledge of Azure services, particularly:- Azure Service Bus- Azure Data Lake- Azure Blob Storage- Azure Redis- Other Azure SaaS/PaaS services- Experience building APIs and backend services with FastAPI.- Hands-on experience with Docker.- Experience with Kubernetes and container orchestration.- Familiarity with modern AI/GenAI technologies, including LangChain, LangGraph, RAG pipelines, or LLM-based applications.- Experience with Azure Synapse Analytics or Microsoft Fabric.- Experience working in enterprise environments with complex data processing requirements.

Codzienne zadania:
- Software & Data Engineering
- Design, develop, and maintain robust, scalable, and production-ready Apache Spark applications.
- Develop efficient data processing, transformation, and analysis workflows using Python and Spark.
- Build solutions capable of handling large-scale datasets while maintaining high performance and reliability.
- Work with columnar data formats, particularly Parquet, and implement data solutions using Delta Tables.
- Develop clean, maintainable, testable, and efficient code in accordance with established engineering standards.
- Optimize Spark applications, data processing pipelines, and code for performance, scalability, and efficient resource utilization.
- Analyze and resolve technical issues affecting application reliability, performance, and data processing.
- Contribute to the design and implementation of scalable microservices-based solutions.
- Technical Leadership & Code Quality
- Define, promote, and enforce software engineering best practices and coding standards across the project.
- Conduct detailed code reviews, ensuring code quality, maintainability, security, performance, and compliance with project standards.
- Identify potential technical risks and proactively propose improvements.
- Support the team in solving complex technical and architectural problems.
- Provide technical guidance to other team members and help establish consistent development practices.
- Contribute to continuous improvement of development processes, workflows, and engineering standards.
- Collaboration & Stakeholder Management
- Work both independently and collaboratively within a distributed, cross-functional team.
- Cooperate closely with software engineers, data specialists, architects, project managers, and business stakeholders.
- Communicate clearly with team members, coordinate day-to-day technical activities, and help ensure smooth delivery.
- Maintain regular communication with the client and participate in technical discussions and project meetings.