PE

Databricks Admin

Persistent
Pune4-8 LPA Posted 16 Apr 2025
FULL TIME
Docker
Kubernetes
Aws
Databricks

Job Description

What You'll Do:

  • Install, configure, and maintain Databricks clusters and workspaces.
  • Monitor cluster performance, resource utilization, and troubleshoot issues to ensure optimal performance.
  • Implement and manage access controls and security policies to protect sensitive data.
  • Perform regular software updates and patches.
  • Provide troubleshooting and issue resolution.
  • Optimize cluster configurations for performance and cost-effectiveness.
  • Monitor and manage auto-scaling to ensure resources are allocated efficiently.
  • Implement and enforce security policies, access controls, and encryption mechanisms.
  • Stay up-to-date with security best practices and compliance requirements.
  • Develop and maintain backup and disaster recovery strategies to ensure data integrity and availability.
  • Collaborate with data engineers to integrate Databricks with other data sources, data warehouses, and data lakes.
  • Monitor and manage platform costs, including optimizing resource utilization to control expenses.
  • Maintain detailed documentation of configurations, procedures, and best practices.
  • Collaborate with cross-functional teams, including data scientists, data engineers, and business analysts,
  • To understand their requirements and provide technical solutions

Expertise You'll Bring:

  • Managing Cluster at Scale, troubleshooting within Databricks and Working with Databricks System tables
  • Strong knowledge of Databricks infrastructure and administration to support the Analyst community in addressing Databricks-related queries and troubleshooting issues in notebooks,
  • Good knowledge of AWS, Databricks
  • Unity catalog, managing cluster, deployment, strong fundamental of DW/DH
  • Strong in troubleshooting's
  • Spark/Python either coding skills
  • Docker, Kubernetes
  • AWS(S3, Lambda, EMR)
  • Certified Databricks Engineer

Join WhatsApp Channel