Data Engineer

Roles and Responsibilities

• Design, develop, and maintain large-scale data pipelines using Azure Data Factory (ADF) to extract, transform, and load data from various sources into Azure Data bricks.
• Develop complex ETL processes using Python scripts and SQL queries to process large datasets stored in Azure Data Lake Storage.
• Collaborate with cross-functional teams to gather requirements for new data warehousing solutions and implement scalable architectures on Azure Data Warehouse.
• Troubleshoot issues related to ADF pipeline failures, Azure Synapse connections, and SQL queries.