Data integration engineer

Key Responsibilities

• Design and implement data ingestion and integration pipelines across multiple source systems
• Develop and maintain ETL/ELT workflows on Databricks platform
• Use Apache Airflow to schedule, automate, and monitor data ingestion jobs
• Integrate data from diverse systems and tailor ingestion strategies per source system
• Collaborate with technical data stewards to understand business and data requirements
• Document end-to-end data ingestion flows, pipelines, and integration architectures
• Manage metadata and master data within technical data catalogs
• Ensure pipeline compliance with security standards, including PII identification and risk handling
• Monitor data pipelines and ensure performance, reliability, and data quality
• Ensure adherence to SDLC, Agile practices, and CI/CD pipelines
• Maintain and optimize Spark-based data processing jobs and workflows