Job Description
Senior Consultant - Platform Engineer
Responsibilities
• Design, build, and optimize platform solutions in Azure Databricks to support scalable, high-performance, and secure data engineering workflows.
• Implement and maintain platform automation for infrastructure, monitoring, and deployment pipelines using modern DevOps practices.
• Align with best practices for platform architecture, governance, and security within Databricks.
• Collaborate with data engineers, modellers, and testers to ensure the platform meets technical and business requirements.
• Proactively monitor, troubleshoot, and resolve issues to maintain platform stability and performance.
Role Requirements
• Strong platform engineering experience with Azure Databricks and related Azure services.
• Familiarity with DevOps principles and CI/CD pipeline tools (e.g., Azure DevOps, Jenkins, GitHub Actions).
- Background in system Azure architecture, networking, Windows and Linux configuration, DevOps or SRE roles
- • Ability to design scalable solutions using Databricks features such as Delta Lake, Unity Catalog, and cluster policies.
- Write and deploy infrastructure as Code (IaC) /Configuration as Code (CaC) (Azure Resource Manager, Puppet, Ansible, Terraform or similar)
- • Strong problem-solving skills, with the ability to think strategically and avoid tactical or short-term solutions when possible.
- • Experience working in Agile/Scrum teams.
Experience/Skillset
• 8+ years of experience in platform engineering or a related field.
• Strong knowledge of Databricks, including Spark optimization, cluster management, and governance frameworks.
- Good knowledge of Azure DevOps based toolchains
- • Proficiency in Python, SQL, and PySpark, with the ability to implement robust and efficient solutions in Databricks.
- • Expertise in Azure services such as Azure Data Factory, Azure Storage, and Azure Key Vault.
- • Strong knowledge of SDLC and Agile methodologies.
- • Excellent verbal and written communication skills.
- • Proven ability to manage multiple projects and priorities effectively while delivering to tight deadlines.