Data Engineer

Hello,

Role Title: Data Engineer

Experience: 5 - 10 years

Location: Singapore

Contract Duration: 12 months & extendable (Extension is subject to project requirements.)

Work Mode: Work From Office (WFO) role.

Note: Candidates must have hands-on AWS and Databricks operations experience.

Role Overview

We are seeking a Data Engineer to join an existing data engineering team and help accelerate the implementation and operationalization of an enterprise data platform built on Databricks. The role includes migrating legacy data pipelines to Databricks, setting up operational excellence frameworks, and enabling seamless integration with healthcare information systems and clinical data workflows.

This is a hands-on, operations-focused position. You will be responsible for the reliability, performance, security, and cost efficiency of production data pipelines running on AWS. You will also mentor junior team members and collaborate closely with infrastructure and compliance teams.

Key Projects

• Healthcare Data Analytics Platform modernization

• Migration of existing ETL processes from traditional data warehouse systems to a Databricks Delta Lake architecture

• Real-time streaming analytics for clinical monitoring systems

• Advanced analytics infrastructure supporting machine learning operations (MLOps)

Responsibilities

Platform Operations

• Monitor and maintain production data pipelines to ensure 99.9% uptime and optimal performance

• Implement logging, alerting and monitoring using application monitoring tools

• Conduct regular health checks on pipeline performance, job execution times and resource utilization, and proactively address bottlenecks

• Manage incident response for pipeline failures, including root cause analysis, resolution, and post-incident reviews

• Set up and maintain disaster recovery procedures and backup strategies for critical data assets within the Databricks environment

• Perform ongoing performance tuning of Spark jobs and Databricks cluster configurations to optimize cost and execution efficiency

• Coordinate scheduled maintenance windows and system upgrades with minimal disruption to business operations

• Manage user access controls, workspace configurations and security policies

• Maintain documentation for operational procedures, runbooks, and troubleshooting guides

Delivery and Collaboration

• Lead the migration of legacy data pipelines to Databricks

• Support integration with existing healthcare information systems and clinical data workflows

• Collaborate closely with infrastructure teams and compliance officers

• Mentor junior team members

Required Qualifications

• Degree in Computer Science or Computer Engineering

• Minimum 5 years of experience in system operations, compliance and management

• Hands-on project experience on the AWS platform (primary requirement)

• Project experience in cloud operations or cloud architecture

• AWS certification (Associate or Professional level)

• Hands-on Databricks operations experience on AWS

Core Technical Skills

• Proficiency in the Databricks platform, including workspace management, cluster configuration and job orchestration

• Strong expertise in Apache Spark within Databricks (Spark SQL, DataFrames, RDDs)

• In-depth understanding of data warehouse concepts, data profiling, data verification and advanced analytics techniques

• Solid knowledge of monitoring, incident management and cloud cost control

Technology Stack

• Databricks

• AWS cloud services and architecture

• IDMC (Informatica Data Management Cloud)

• Tableau

• Oracle Database management

• MLOps practices within Databricks

• Amazon SageMaker integration with Databricks

• DataRobot platform integration

• STATA (advantage)

If this role isn't the right fit for you at the moment, but you know friends or colleagues who might be interested, please feel free to pass this opportunity along!

Important Note:

> Please share your resume in word format with [HIDDEN TEXT]

> Important Note: If this requirement is not a match for you, please refer to your friends.

> Interested professionals can reach out to me for Confidential Discussion @ +65- 6950 5535