Databricks Architect

Job Overview:

• We are in search of a seasoned Architect specializing in Databricks, Snowflake, and PySpark to conceptualize, structure, and implement scalable cloud-based data platforms along with advanced analytics solutions.
• The perfect candidate will possess extensive knowledge in data architecture, cloud-native technologies, data engineering practices, and large-scale data transformation initiatives.
• This position requires collaboration with business stakeholders, solution architects, and engineering teams to create high-performance, secure, and cost-effective data ecosystems utilizing Databricks, Snowflake, and PySpark.

Key Duties:

• Establish and lead enterprise-level data architecture and modernization strategies.
• Design comprehensive cloud data platforms incorporating Databricks, Snowflake, and contemporary data engineering methodologies.
• Architect scalable data processing pipelines capable of handling both batch and real-time large datasets.
• Oversee the implementation of data lakes, lakehouses, and data warehouses.
• Offer architectural insights on performance enhancement, scalability, governance, security, and reliability.
• Engage closely with business stakeholders to grasp analytical and reporting needs.
• Evaluate technical designs, perform architecture assessments, and establish best practices.
• Provide mentorship to data engineers and technical leads throughout project engagements.
• Direct technology assessments, proof of concepts (POCs), and solution accelerators.
• Facilitate cloud migration and modernization efforts from legacy data environments.
• Ensure compliance with enterprise security, data governance standards, and regulatory requirements.
• Collaborate with DevOps and platform teams to deploy CI/CD and automate infrastructure for data solutions.

Technical Qualifications:

• 11 years of experience in Data Engineering, Analytics, or Data Platform Architecture.
• In-depth expertise in the Databricks Lakehouse Platform.
• Extensive practical knowledge of the Snowflake Data Cloud.
• Advanced proficiency in PySpark and distributed data processing techniques.
• Significant experience in designing data lakes, data warehouses, and lakehouse architectures at an enterprise scale.
• Hands-on experience with cloud providers such as:
• Azure (ADF, ADLS Gen2, Synapse, Azure Databricks)
• AWS (S3, Glue, EMR, Redshift)
• GCP (BigQuery, DataProc, Cloud Storage)
• Strong SQL and data modeling capabilities, including Dimensional Modeling, Data Vault, and Star Schema.
• Experience constructing ETL and ELT frameworks along with reusable data engineering components.
• Familiarity with Data Governance, Metadata Management, Data Lineage, and Data Quality frameworks.
• Experience with orchestration tools like Airflow, Azure Data Factory, or similar solutions.
• Expertise in performance tuning and cost optimization in Databricks and Snowflake environments.
• Experience in implementing CI/CD pipelines using Azure DevOps, GitHub Actions, Jenkins, or equivalent tools.

Additional Responsibilities:

• Enterprise Data Architecture
• Solution Design Governance
• Technology Strategy Roadmap
• Presales Client Advisory
• Architecture Reviews
• Data Platform Modernization
• Technical Leadership and Mentoring
• Cost Optimization and Non-Functional Requirements
• Stakeholder Engagement

Preferred Skills:

Technology->Data Engineering->Databricks