Big Data Engineer

For a client, one of the most recognizable airlines in the world, we are looking for a Big Data Engineer. The person in this role will be involved in working with large volumes of data, including data modeling, data integrations, and ensuring data quality. Key responsibilities • Designing and implementing robust ETL/ELT pipelines • Implementing efficient Spark/PySpark transformations, managing schemas and Delta Lake versioning • Integrating streaming and batch data sources • Implementing data quality control frameworks, monitoring, schema validation, and test suites • Collaborating on data modeling and metadata management • Working with technical documentation  Ideal candidate profile • Minimum 7 years of professional experience in data engineering, building large-scale data pipelines, or related roles • Strong knowledge of Azure Databricks, Spark (PySpark and/or Scala), and Delta Lake. • Experience with ADLS Gen2, data ingestion patterns, and streaming technologies. • Experience with streaming technologies such as Event Hubs and Kafka. • Knowledge of CI/CD for data artifacts and deployment practices in Databricks. • Experience in Python and/or Java, using test-driven development (TDD). Nice to have: • Experience with tools such as Unity Catalog, Purview, or other metadata management and governance systems. • Knowledge of Apache Beam or other data processing frameworks. • Databricks or Azure Data Engineering certifications. Conditions • Working mode: Remote • Form of cooperation: B2B • Duration: long-term cooperation • Salary depends on your experience (160-190 PLN net/h) • Benefits: private medical care, life insurance, sport card Recruitment steps • Screening call with Recruiter (30 min.) • Technical interview with Client • Decision and feedback