Databricks Data Platform Engineer/Architect

We are seeking a highly skilled Databricks Platform Engineer with strong experience in data engineering. The candidate will have a deep understanding of both data platforms and software engineering, enabling them to effectively integrate and operate the platform within a broader IT ecosystem. This role requires a hands-on individual contributor who takes full ownership of deliverables end-to-end, including design, development, testing, deployment, and ongoing support. - Manage and optimize Databricks data platform including workspace setup, cluster policies, job orchestration, Unity Catalog, cost controls, multi-tenancy. - Design, write and maintain APIs used for Platform automation, Serverless workflows, Deployment pipelines, release management and repository management - Ensure high availability, security, and performance of data systems which includes access control, secrets management, RBAC, monitoring, alerting, RLS, incident handling, performance tuning. - Provide valuable insights about the data platform (Databricks) usage which includes cost attribution, usage analytics, workload patterns, telemetry. - Implementing new features of Databricks, including serverless, Declarative Pipelines, Agents, lakebase , etc. - Design and maintain system libraries (Python) used in ETL pipelines and platform governance (Databricks). - Optimize ETL Processes - Enhance and tune existing ETL processes for better performance, scalability, and reliability.