Mid Flink Developer

O projekcie: Project- The project focuses on the continuous development and modernization of a large-scale streaming data platform.- The main objective is to evolve existing Apache Flink-based streaming pipelines while executing complex data migrations and platform upgrades.Key responsibilities include:- Developing and enhancing real-time streaming pipelines using Apache Flink- Migrating existing Flink jobs using the DataStream API and adapting them to newer platform standards- Leading and executing the upgrade of the Flink platform to version 2.0- Designing, optimizing, and maintaining high-throughput, fault-tolerant streaming architectures- Migrating large-scale datasets from BigQuery (BQ) to Data Cloud Storage (DCS)- Scaling and automating ongoing data migration processes to support growing data volumes- Converting datasets from Avro to Parquet format with attention to performance, schema evolution, and storage optimization- Leveraging AI-powered tools to accelerate migration, validation, and transformation workflows- Ensuring data quality, integrity, and minimal downtime during migrations- Collaborating with cross-functional teams and clearly communicating technical concepts to non-technical stakeholders Wymagania: Technical Skills: - Streaming & Platform Engineering- Strong hands-on experience with Apache Flink, including development using the DataStream API- Proven experience maintaining and upgrading Flink environments, ideally including exposure to Flink 2.0- Deep understanding of streaming pipeline architecture, performance tuning, state management, and fault tolerance- Data Migration & Transformation- Experience migrating large-scale datasets from BigQuery (BQ) to Data Cloud Storage (DCS)- Strong proficiency in data format conversion, particularly Avro to Parquet- Ability to design, scale, and automate migration workflows while ensuring data integrity and minimal service disruption- Cloud & Big Data Ecosystem- Solid knowledge of Google Cloud Platform (GCP) and its data services- Good understanding of distributed systems, schema evolution, and storage optimization strategies