Sr Technical Architect - I_Cloud Architecture

We are seeking an experienced AI-Enabled Azure & Databricks Infrastructure Engineer to design, build, automate, secure, and operate cloud-native data and analytics platforms on Microsoft Azure.

The successful candidate will bring deep Azure infrastructure engineering and Azure Databricks administration experience, with strong capabilities in Infrastructure as Code (IaC), DevOps automation, platform operations, and AI-assisted engineering tools.

Key Responsibilities

Azure Infrastructure Engineering

• Design and manage Azure landing zones, subscriptions, resource groups, and governance frameworks.
• Provision and manage Azure services, including VNets, Private Endpoints, Azure Storage, Key Vault, Azure SQL, AKS, Azure Monitor, and App Services.
• Implement cloud security, high availability, disaster recovery, backup, and resiliency solutions.
• Manage Azure networking, access controls, and infrastructure governance.

Azure Databricks Platform Engineering

• Deploy, configure, and administer Azure Databricks workspaces.
• Manage clusters, cluster policies, job scheduling, and workload optimization.
• Implement Unity Catalog, access controls, workspace security, and data governance.
• Support integration with Azure Data Lake Storage (ADLS Gen2).
• Optimize Databricks performance and cloud costs.
• Implement CI/CD pipelines for Databricks deployments and assets.

DevOps & Infrastructure Automation

• Develop Infrastructure as Code (IaC) solutions using Terraform, Bicep, and ARM Templates.
• Build and maintain CI/CD pipelines using Azure DevOps and GitHub Actions.
• Automate infrastructure provisioning, configuration management, and operational activities.
• Implement GitOps and automated deployment practices.
• Write scripts using Python, PowerShell, and Bash.

AI-Enabled Engineering

• Use AI-assisted engineering tools to speed up infrastructure development and troubleshooting.
• Use GenAI for infrastructure code generation, configuration validation, root cause analysis, and documentation.
• Apply AI tools to deliver monitoring insights, operational runbooks, and knowledge management.
• Recognize opportunities to enhance automation and operational efficiency through AI.

Platform Operations & SRE

• Monitor platform availability, performance, and overall health.
• Implement observability using Azure Monitor, Application Insights, and Log Analytics.
• Perform incident management, problem management, and root cause analysis.
• Enhance operational excellence through automation and self-healing mechanisms.
• Support production environments and participate in on-call activities as required.

Mandatory Skills

• 8–10 years of experience in Azure infrastructure engineering and administration.
• 5+ years of hands-on Azure Databricks administration experience.
• Strong expertise in Microsoft Azure services and cloud infrastructure.
• Azure Networking, Security, Storage, Azure SQL, and Key Vault.
• Azure Monitor, Log Analytics, Azure Policy, and Entra ID.
• Azure Databricks administration, cluster management, and workspace governance.
• Unity Catalog, Delta Lake, and Databricks job orchestration.
• Terraform, Bicep, or ARM Templates.
• Azure DevOps, GitHub Actions, and CI/CD pipelines.
• Python, PowerShell, and Bash scripting.
• Git, monitoring, troubleshooting, and production support.
• Practical experience using AI-assisted engineering tools.

AI Tools – Preferred

• GitHub Copilot
• Claude
• Microsoft Copilot
• AI-based infrastructure automation assistants
• AI-powered monitoring and observability platforms

Preferred Skills

• ServiceNow and IT service management.
• Enterprise-scale cloud platform management.
• Cloud security and governance frameworks.
• Site Reliability Engineering (SRE) practices.
• Experience with self-healing and intelligent operations.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• Microsoft Certified: Azure Administrator Associate (preferred).
• Terraform Associate Certification (preferred).

Key Success Metrics

• Achieve more than 90% infrastructure deployment automation.
• Reduce infrastructure provisioning effort through AI-assisted automation.
• Improve platform availability and operational SLAs.
• Reduce incident resolution time using AI-enabled diagnostics.
• Optimize Azure and Databricks consumption costs.
• Maintain compliance with enterprise security and governance standards.