Sr. Developer

Roles and Responsibilities:

• Create, enhance, and optimize data solutions using Delta Sharing and Databricks Unity Catalog Administration.
• Set up, operate, and maintain Databricks CLI alongside Delta Live Pipelines to support reliable and efficient data processing.
• Use Structured Streaming to enable real-time data processing and analysis.
• Apply risk management approaches to protect data integrity, security, and compliance.
• Integrate Apache Airflow to manage workflow orchestration and automation.
• Use Amazon S3 and Amazon Redshift to support scalable data storage and retrieval.
• Develop and maintain Python scripts to automate data processing activities.
• Leverage Databricks SQL and Delta Lake for effective data querying, transformation, and storage optimization.
• Design and implement Databricks Workflows to automate data pipelines.
• Build PySpark applications to process and transform large datasets.
• Partner with cross-functional teams to gather requirements and deliver high-impact data solutions.
• Monitor, diagnose, and resolve issues in data workflows to maintain stable operations.
• Continuously track emerging industry trends and technologies to advance data solutions.

Qualifications:

• Strong experience in Delta Sharing and Databricks Unity Catalog Administration.
• Proficiency in Databricks CLI and Delta Live Pipelines.
• Good understanding of Structured Streaming and Risk Management.
• Experience with Apache Airflow, Amazon S3, and Amazon Redshift.
• Proficiency in Python and PySpark.
• Strong problem-solving and analytical skills.