Data Engineer

Roles & Responsibilities:

• Design and develop scalable and efficient data pipelines using Azure Databricks, PySpark, and related technologies
• Work with Azure Integration Services to orchestrate data workflows across various data sources
• Collaborate with data scientists, analysts, and business teams to understand data requirements and deliver solutions
• Develop scripts and automation solutions using Python for data ingestion, transformation, and validation
• Integrate back-end systems with APIs and other data sources to ensure seamless data flow
• Ensure solutions adhere to best practices in performance, security, and compliance
• Leverage RESTful APIs and microservices for seamless data integration
• Participate in code reviews, testing, and debugging to improve data pipeline reliability and performance
• Maintain up-to-date knowledge of emerging cloud and data technologies