Senior Data Engineer (Python/Azure)

O projekcie:
The team you will join designs, develops, and deploys innovative enterprise technology and AI-driven tools to support the delivery of tax services. It is a dynamic group combining expertise in tax, software engineering, change management, and project management, working on initiatives from tool design and deployment to training development and engagement management

Wymagania:
Our requirements- 5+ years of hands-on experience in software development- Extensive experience working with Apache Spark, including platforms such as Databricks and/or Azure Synapse/Fabric- Proficient in Python, with strong skills in data manipulation using Pandas/Polars and similar- Solid understanding of columnar data storage formats, particularly Parquet, with practical experience using Delta Tables- Proven expertise in data processing, analysis, and transformation workflows- Strong analytical and problem-solving abilities with a detail-oriented mindset- Practical and pragmatic approach to balancing standardized processes with flexibility to meet project goals effectively- Excellent organizational skills with the ability to self-manage, prioritize tasks, structure workload, and meet tight deadlines- Solid understanding of microservices architecture and its implementation in scalable systemsOptional- Experience working with Azure Cloud services (or other major cloud platforms), including a range of SaaS offerings such as Service Bus, Data Lake, Blob Storage, Redis, and more- Familiarity with FastAPI- Expertise in containerization and orchestration tools such as Docker and Kubernetes

Codzienne zadania:
- Define and enforce best practices and coding standards across the project
- Conduct thorough code reviews to ensure adherence to established guidelines and maintain high code quality
- Working both independently and in close collaboration with others in the team
- Communicating clear instructions to team members and helping manage the flow of day-to-day operations
- Communicating with the client regularly
- Design, develop, and maintain robust and scalable Spark applications
- Write clean, maintainable, and efficient code following best practices and coding standards
- Optimize code for performance and scalability, ensuring efficient data handling
- Work closely with cross-functional teams to deliver high-quality software solutions
- Identify and resolve technical issues, ensuring the reliability and performance of applications
- Create and maintain comprehensive documentation for code, processes, and workflows