Data Engineer (AWS + Databricks)

Job Title: Data Engineer (AWS + Databricks)

Work Location: Singapore

Experience Level: 4 to 8 Years

Essential Skills

• Proficient hands-on experience with Databricks
• In-depth knowledge of PySpark and Spark SQL
• Solid experience with AWS Cloud Services, encompassing:
• S3
• Glue
• Lambda
• Redshift
• Athena
• IAM
• Familiarity with Delta Lake and Lakehouse Architecture
• Strong capabilities in SQL and Data Modeling
• Experience in designing and implementing scalable ETL/ELT pipelines
• Understanding of CI/CD, Git, and best practices in cloud-based data engineering

Key Responsibilities

• Design, construct, and enhance scalable data pipelines utilizing AWS and Databricks.
• Create ETL/ELT processes for handling large-scale data efficiently.
• Establish and manage Delta Lake/Lakehouse solutions.
• Work collaboratively with both business and technical stakeholders to provide effective data solutions.
• Monitor, diagnose, and optimize data processing tasks.

Desired Certifications

• AWS Certified Data Engineer / Solutions Architect

Databricks Certified Data Engineer