Senior Data Engineer with Databricks Azure

O projekcie: You own the day-to-day operations and continuous improvement of an Azure Databricks-based data platform. You bridge cloud infrastructure and data products/business applications, ensuring the platform is stable, secure, monitored, and production-ready, while delivering targeted enhancements (new interfaces, new data products). Wymagania: Tech Stack - Terraform- Bitbucket/Github (CI/CD -> pytest, static code analysis etc, databricks CLI),- Bash,- Python,- APIM,- familiarity with JIRA integrations (Modern Design Patterns for DBx - no legacy knowledge and familiarity with DBx) Nice to have - Azure ecosystem around Databricks (e.g., storage/data lake, networking, identity)- Delta/Delta Lake optimization concepts and operational tuning- Familiarity with service request / change processes (ITSM-style)- Data governance exposure (metadata, lineage, quality reporting) Codzienne zadania: - Run & Operate (≈70%) Own CDP operations: stability, controls, security, reliability, and cost awareness - Monitor pipelines/platform health; improve alerting and Data Health Monitoring - Handle service requests: access, configuration, troubleshooting, incident triage/support - Coordinate production changes/releases for platform and data product updates - Maintain secure connectivity to the application landscape (identity/access, secrets, network patterns) - Drive operational improvements: runbooks, automation, root-cause analysis, prevention actions Improve & Extend (≈30%) - Deliver data integrations (build/test/release) across multiple data products - Implement/extend interfaces for downstream consumers (APIs and/or consumption patterns) - Extend platform capabilities for new data interfaces and new data products - Support metadata/discovery and data quality enablement (pragmatic, operational tooling) - Role level security and authentications for Databricks apps and underlying data Required skills & experience - Hands-on experience operating a cloud data platform in production - Strong practical knowledge of Azure Databricks (jobs/workflows, clusters/compute, permissions, troubleshooting) - Data pipeline/integration experience (batch and/or streaming), production hardening mindset - Monitoring/observability skills (dashboards, alerts, health checks) - Security fundamentals: access management, secrets, least-privilege, secure connectivity - Strong ownership and communication; comfortable coordinating across infra/security/app/data teams