Senior GCP Data Engineer – BigQuery & Data Pipeline Development
⚲ Kraków
18 700–31 500 zł / mies. (zal. od umowy)
Wymagania
- SQL
- BigQuery
- Google Cloud Platform
- Python
- Terraform
- Git
- Jenkins
- Ansible
- SoapUI
Opis stanowiska
Nasze wymagania:
At least 3 years of hands-on experience with SQL querying and optimization in BigQuery.
Proven expertise in SQL Data Transformation, ETL/ELT pipeline development in GCP DataFusion or similar tools.
Strong experience with Data Vault modeling principles.
Skilled in Cloud Composer/Airflow, Cloud Run, Pub/Sub, Python, and Terraform.
Proficiency with version control (Git) and CI/CD processes, including tools like Jenkins or Ansible.
Experience working in a DataOps environment and Agile methodologies.
Excellent problem-solving, organizational, and multi-tasking skills.
Mile widziane:
Experience designing data ingestion pipelines with GCP Data Fusion, CDAP, or comparable tools (handling CSV, JSON, XML, RESTful & SOAP APIs, SFTP).
Knowledge of Java development for custom Data Fusion plugins.
Familiarity with continuous testing and deployment tools for cloud data solutions.
O projekcie:
As a Senior GCP Data Engineer – BigQuery & Data Pipeline Development, you will be working for our client, a leading financial services organization, on a transformative data integration initiative. Your expertise will support the global MSS Operations team, enhancing data processing and analytics capabilities across multiple sources and enabling data-driven decision-making at scale. This role offers a platform for innovation and professional growth in the rapidly evolving cloud data landscape.
Unleash the Power of Data – Transform Business Insights with Cutting-Edge Cloud Engineering!
Krakow-based opportunity with hybrid work model.
Only candidates with an existing legal right to work in the European Union will be considered for this role.
Zakres obowiązków:
Design, build, test, and deploy GCP data models and transformations in BigQuery (including stored procedures, indexes, clusters, partitions, triggers).
Create and manage ETL/ELT data pipelines transforming raw or unstructured data into a Data Vault model.
Review, interpret, and implement business and technical requirements, ensuring compliance with IT and security standards.
Monitor data pipelines for failures or performance issues, implementing solutions for scalability and robustness.
Optimize ETL/ELT processes to handle large data volumes efficiently.
Integrate data from various sources, maintaining consistency and accuracy.
Manage code artifacts and CI/CD pipelines using Git, Jenkins, Terraform, with a focus on GCP cloud applications.
Collaborate within an Agile environment, applying DataOps best practices to deliver high-quality solutions.
Oferujemy:
Stable and long-term cooperation with very good conditions
Enhance your skills and develop your expertise in the financial industry
Work on the most strategic projects available in the market
Define your career roadmap and develop yourself in the best and fastest possible way by delivering strategic projects for different clients of ITDS over several years
Participate in Social Events, training, and work in an international environment
Access to attractive Medical Package
Access to Multisport Program
Access to Pluralsight
Flexible hours
At least 3 years of hands-on experience with SQL querying and optimization in BigQuery.
Proven expertise in SQL Data Transformation, ETL/ELT pipeline development in GCP DataFusion or similar tools.
Strong experience with Data Vault modeling principles.
Skilled in Cloud Composer/Airflow, Cloud Run, Pub/Sub, Python, and Terraform.
Proficiency with version control (Git) and CI/CD processes, including tools like Jenkins or Ansible.
Experience working in a DataOps environment and Agile methodologies.
Excellent problem-solving, organizational, and multi-tasking skills.
Mile widziane:
Experience designing data ingestion pipelines with GCP Data Fusion, CDAP, or comparable tools (handling CSV, JSON, XML, RESTful & SOAP APIs, SFTP).
Knowledge of Java development for custom Data Fusion plugins.
Familiarity with continuous testing and deployment tools for cloud data solutions.
O projekcie:
As a Senior GCP Data Engineer – BigQuery & Data Pipeline Development, you will be working for our client, a leading financial services organization, on a transformative data integration initiative. Your expertise will support the global MSS Operations team, enhancing data processing and analytics capabilities across multiple sources and enabling data-driven decision-making at scale. This role offers a platform for innovation and professional growth in the rapidly evolving cloud data landscape.
Unleash the Power of Data – Transform Business Insights with Cutting-Edge Cloud Engineering!
Krakow-based opportunity with hybrid work model.
Only candidates with an existing legal right to work in the European Union will be considered for this role.
Zakres obowiązków:
Design, build, test, and deploy GCP data models and transformations in BigQuery (including stored procedures, indexes, clusters, partitions, triggers).
Create and manage ETL/ELT data pipelines transforming raw or unstructured data into a Data Vault model.
Review, interpret, and implement business and technical requirements, ensuring compliance with IT and security standards.
Monitor data pipelines for failures or performance issues, implementing solutions for scalability and robustness.
Optimize ETL/ELT processes to handle large data volumes efficiently.
Integrate data from various sources, maintaining consistency and accuracy.
Manage code artifacts and CI/CD pipelines using Git, Jenkins, Terraform, with a focus on GCP cloud applications.
Collaborate within an Agile environment, applying DataOps best practices to deliver high-quality solutions.
Oferujemy:
Stable and long-term cooperation with very good conditions
Enhance your skills and develop your expertise in the financial industry
Work on the most strategic projects available in the market
Define your career roadmap and develop yourself in the best and fastest possible way by delivering strategic projects for different clients of ITDS over several years
Participate in Social Events, training, and work in an international environment
Access to attractive Medical Package
Access to Multisport Program
Access to Pluralsight
Flexible hours
🔍 Dekoder Ogłoszenia
🔴
transformative data integration initiative
Projekt może być w początkowej fazie, z potencjalnymi problemami i niejasnościami, które trzeba będzie rozwiązać.
🔴
enabling data-driven decision-making at scale
Oczekuje się, że Twoja praca będzie miała znaczący wpływ na strategiczne decyzje firmy, co może oznaczać dużą presję i odpowiedzialność.
🔴
platform for innovation
Może oznaczać, że będziesz miał swobodę w proponowaniu nowych rozwiązań, ale też będziesz musiał samodzielnie je wdrażać i udowadniać ich wartość.
🔴
rapidly evolving cloud data landscape
Technologie mogą się szybko zmieniać, co wymaga ciągłego uczenia się i adaptacji, a obecne rozwiązania mogą być szybko przestarzałe.
🟡
Unleash the Power of Data – Transform Business Insights with Cutting-Edge Cloud Engineering
To marketingowy slogan, który podkreśla wagę projektu, ale nie dostarcza konkretnych informacji o codziennych zadaniach czy wyzwaniach.