Azure databricks Platform Admin

Job Overview:

• Design and enhance large-scale data workflows utilizing Apache Spark Delta Lake and native tools within Databricks
• Develop scalable processes for data ingestion and transformation, incorporating both batch and streaming methodologies
• Utilize Kafka and Spark Structured Streaming for efficient data handling
• Establish integration protocols to align Databricks with existing security solutions pertinent to data access control
• Implement data governance and security measures using Unity Catalog for access regulations and data classification
• Facilitate the transition of legacy systems to Databricks on cloud platforms such as Azure, AWS, or GCP
• Oversee operations within cloud platforms, emphasizing FinOps support to optimize resource usage, cost transparency, and governance across multiple cloud environments

Primary Duties:

• Collaborative Advisory
• Serve as a technical consultant for data engineering and analytics teams, providing guidance on best practices and performance tuning
• Collaborate with architects and business leaders to ensure Databricks solutions align with enterprise objectives
• Lead proof of concept (PoC) projects to showcase the capabilities of Databricks for targeted use cases
• Strategic Leadership Engagement
• Guide junior engineers and foster knowledge sharing across teams
• Contribute to strategies for platform adoption, including training, documentation, and advocacy within the organization
• Keep abreast of advancements in Databricks and propose improvements to existing architectures

Desired Qualifications:

Technology->Data Engineering->Databricks, Technology->Cloud Platform->Azure Networking Services->Azure Bastion, Technology->Storage Administration->IBM Admin