Senior Data Engineer

Pro Arch is seeking a Senior Data Engineer with 3-5 years of experience to join our Data & Analytics practice. In this role, you will design and deliver scalable data solutions, optimize performance, and enable advanced analytics across cloud platforms. You will work with large-scale datasets, modern cloud-native technologies, and collaborate with business stakeholders to build reliable and secure data ecosystems. This position also involves mentoring junior engineers and driving best practices in DataOps and performance optimization.

Responsibilities:

• Design, develop, and optimize data pipelines using Azure Synapse Analytics, Spark Pools, and Azure Data Factory.
• Write efficient Python and SQL code with a focus on scalability, performance, and optimization.
• Manage Azure Data Lake for structured and unstructured data ingestion at scale (volume, variety, velocity).
• Implement CI/CD pipelines and follow Data Ops practices for automation and reliability.
• Optimize data partitioning, versioning, and storage/compute costs for large-scale workloads.
• Ensure high-quality solutions through unit testing frameworks and automated validation.
• Collaborate with business teams to support analytics, reporting, and data science initiatives.
• Mentor junior engineers, sharing knowledge and promoting engineering excellence.