Senior Data Engineer

What the role is about

We are looking for a Senior Data Engineer to join our Data Management team. In this position, you will lead the engineering capability for our data platform and data integration efforts. You will be responsible for defining and setting Data Engineering standards within the client environment. This includes designing and maintaining robust data architecture, pipelines, and integrations across multiple source systems, establishing data engineering standards, and delivering technical diagnosis for data pipeline incidents. You will collaborate closely with business and use-case owners, IT/cybersecurity stakeholders, source-system owners, and vendors to convert business requirements into maintainable data solutions. You may also provide technical guidance to a junior/mid-level Data Engineer and help ensure business continuity through production runbooks, recovery procedures, and knowledge transfer.

What you will be working on

Data Architecture and Pipeline Engineering

• Design and maintain data architecture, models, pipelines, and integration patterns across multiple source systems to a centralized Data Platform and governed analytical datasets.
• Build, review, and enhance automated ingestion and transformation workflows, ensuring robust failure handling, reconciliation, testing, and observability.
• Serve as the primary data engineering subject matter expert for Data Engineering activities.
• Manage data pipeline incidents, conduct root-cause analysis, and coordinate remediation.
• Establish engineering standards for code structure, version control, testing, documentation, deployment, monitoring, and long-term maintainability.
• Operationalize Data Governance policies, including automated data-quality controls and monitoring for incomplete, inconsistent, duplicated, or unexpected data.
• Define and review APIs, file/data-transfer specifications, schemas, and technical acceptance criteria with source-system owners and vendors.
• Review vendor-developed pipeline designs, code, documentation, and change proposals to ensure reliability, security, supportability, and minimized technical debt.
• Convert validated analytical and business requirements into reusable datasets and technical designs in partnership with Business Analysts.
• Work hands-on in the AWS environment, including AWS Glue, and coordinate access-control, security, logging, and governance requirements with IT/cybersecurity stakeholders.
• Own production runbooks, recovery procedures, technical documentation, and knowledge transfer activities required for business continuity.
• Provide guidance to any junior/mid-level Data Engineer and review technical work, without formal people-management responsibility.

3LoDGovernance Responsibilities

• Operationalize 3LoDgovernance controls. Design and implement automated data governance controls, including data quality monitoring, classification enforcement, access logging, and reconciliation checks. Ensure these controls are embedded from inception in the ADAPT platform and across all governed datasets, consistent with the governance-by-design principle.
• Compliance monitoring support. Provide the CDO-led data governance function with the technical monitoring mechanisms required to track adherence to data policies and standards across the client's data platform, including dashboards or alerts for data quality thresholds, pipeline failures, and access anomalies, in support of Line 2's compliance monitoring obligations.
• Line 3 audit enablement. Own and maintain production runbooks, data lineage documentation, pipeline architecture records, and change logs to a standard that enables Line3 to conduct periodic audits of data governance processes, controls, and compliance. Act as the primary technical point of contact during data governance audits and support root-cause analysis for audit findings.
• Risk escalation and issue resolution. Proactively identify, document, and escalate data engineering risks—including architecture risks, data quality failures, and security or access control gaps—to the CDO-led data governance function and IT/cybersecurity stakeholders, aligned with Line 2's issue resolution and strategic recommendations obligations under the 3LoD framework.
• Governance by design. Embed governance and control measures into pipeline and platform design from the beginning, ensuring business-as-usual engineering workflows incorporate data classification, quality, and access controls rather than treating them as post-hoc additions.
• Capability contribution. Support the data governance function’s capability-building efforts by providing technical guidance to the Data Engineer and contributing to the development of engineering standards that reflect evolving governance requirements and regulatory changes, supporting Line 2's continuous improvement mandate.

Operational expectations

• The role delivers first-line technical Production support during normal operations. Regular after-hours or on-call work is not expected, though planned after-hours support may be required for scheduled deployments.

What we are looking for

Must-have

• Possess knowledge or experience in Computer Science, Information Systems, Data Engineering, Software Engineering, and/or subjects with substantial technical or quantitative content.
• Demonstrated end-to-end ownership of production data pipelines and integrations, with approximately 5-8 years of relevant professional experience in data engineering or a closely related field.
• Demonstrated end-to-end ownership of production or business-critical data pipelines and integrations.
• Strong SQL, data modelling, and Python/data-engineering programming fundamentals.
• Strong technical fundamentals in SQL, data modelling, and Python, with hands-on experience building and maintaining ETL/ELT pipelines, automated tests, data-quality checks, and failure-handling approaches in a production environment.
• Hands-on experience with AWS services in a production context, including AWS Glue, with the ability to operate safely and coordinate access-control, security, and governance requirements across services.
• Ability to design maintainable ETL/ELT pipelines, automated tests, data-quality checks, logging/monitoring, and failure-handling approaches.
• Experience integrating multiple source systems through APIs, files, databases, or equivalent interfaces, along with the ability to define and review technical specifications and acceptance criteria with system owners and vendors.
• Ability to establish and uphold engineering standards—including code structure, version control, testing, documentation, and monitoring—in an environment where data engineering practices are still maturing.
• Good communication skills, including the ability to clearly communicate technical risk, incidents, and trade-offs to both technical and non-technical audiences, including business stakeholders and vendors.
• Demonstrated ability to operationalize data governance policies through technical controls, including automated data quality monitoring, access logging, classification enforcement, and audit-ready documentation, in support of a structured governance framework such as the Three Lines of Defense (3LoD).

Good-to-have

• Exposure to data platform or architecture design beyond pipeline-level work—relevant because the role involves setting standards as the first Data Engineer hire.
• Experience reviewing vendor-delivered data platforms or integrations.
• Experience working in or with public sector or heritage/cultural institutions
• Exposure to data platform or architecture design beyond pipeline-level work—relevant because the role involves setting standards as the first Data Engineer hire.
• Able to deploy via CI/CD, with experience in data observability and data governance.
• Experience with visualization tools such as Tableau, Amazon QuickSight, and Power BI to understand the end-to-end data pipeline and visualization.
• Hands-on experience with large-scale distributed data processing using Apache Spark or its Python variant, PySpark.
• Experience working with regulated/public-sector environments.
• Experience mentoring or technically guiding another engineer.
• Experience working within or alongside a formal data governance function, including supporting internal or external audit activities related to data pipelines, data quality, or access management.