Data Engineer-Data Platforms-AWS

Introduction

A career in IBM Consulting is grounded in enduring client relationships and hands-on collaboration with teams around the world. You’ll partner with leading organizations across industries, supporting their hybrid cloud and AI initiatives. Backed by strategic partner support, robust IBM technology, and Red Hat, you’ll use the right tools to help clients drive meaningful transformation and speed up business impact. At IBM Consulting, curiosity is central to success—you’ll be encouraged to challenge conventional thinking, test new ideas, and build solutions that produce measurable outcomes. Our growth- and empathy-focused culture supports your long-term career development while recognizing and valuing your distinctive skills and experiences.

Your Role And Responsibilities

Job Summary

IBM Consulting is looking for an experienced AWS Data Engineer to design, build, and support scalable, secure, high-performance data platforms on AWS. This position calls for strong, practical experience with AWS services, distributed data processing frameworks, and SQL-based analytics.

The successful candidate will collaborate closely with architects, application teams, and business stakeholders to deliver dependable, enterprise-ready data solutions.

Key Responsibilities

• Design, develop, and maintain data pipelines and data platforms on AWS
• Create and run serverless and container-based workloads using AWS Lambda, EC2, and EKS
• Deliver batch and large-scale data processing solutions using Apache Spark
• Perform data ingestion, transformation, and analytics using AWS Glue and Athena
• Produce advanced, high-performance SQL queries for analytics, reporting, and data validation
• Oversee and optimize data storage systems using AWS S3 and RDS
• Design and manage AWS infrastructure through Infrastructure as Code (IaC) using CloudFormation
• Apply AWS security best practices across IAM, VPC networking, data encryption, and access controls
• Develop clean, scalable, and maintainable code using Python and/or Scala
• Monitor, troubleshoot, and optimize performance, cost, and reliability for data workloads
• Partner with cross-functional teams to support analytics, reporting, and regulatory use cases
  
  

Mandatory Skills & Qualifications

• Strong hands-on experience with AWS services, including o S3, Lambda, EC2, EKS o AWS Glue, Athena o RDS o VPC, IAM o CloudFormation
• Strong SQL skills, including o Writing complex queries, joins, aggregations, and user defined functions o Performance tuning and query optimization
• Good knowledge of AWS Security, including IAM policies, VPC security, encryption, and least-privilege access
• Hands-on programming experience with Python and/or Scala
• Strong experience with Apache Spark for large-scale data processing
• Good communication skills to explain technical concepts to non-technical stakeholders
• Solid understanding of data engineering concepts such as ETL/ELT, data modeling, and distributed systems
  
  

Good To Have / Preferred Skills

• Experience with DBT for data transformations
• Hands-on exposure to Snowflake
• Experience working with Databricks
• Familiarity with modern data lake / Lakehouse architectures
• Experience working in enterprise or regulated environments
  
  

Nice-to-Have Competencies

• Strong analytical and problem-solving skills
• Ability to work independently and in distributed teams
  
  

Preferred Education

Master's Degree

Required Technical And Professional Expertise

• Exposure to AWS Toolset: Experience working with AWS services such as AWS EMR, AWS Glue, Glue Catalog, and Kinesis to design, build, and operate batch and real-time data pipelines.
• Data Pipeline Development: Exposure to developing batch and real-time data pipelines for Data Warehouse and Datalake, utilizing AWS Kinesis and Managed Streaming for Apache Kafka.
• Data Layer Creation: Experience working with AWS RedShift, Aurora, and DynamoDB to create data layers and migrate data using AWS DMS.
• Open Source Technologies: Exposure to utilizing open source technologies like Apache Airflow and dbt, Spark / Python or Spark / Scala on AWS Platform to support data engineering solutions.
• Data Service Management: Experience scheduling and managing data services on the AWS Platform, ensuring efficient operation of data engineering solutions.
  
  

Preferred Technical And Professional Experience

• Proficiency with AWS Databrew: Experience working with AWS Glue Databrew to support data engineering solutions, including data preparation and data quality tasks.
• Knowledge of Lambda Functions: Exposure to using Lambda functions with Python to support data engineering solutions, including data processing and data transformation tasks.
• Familiarity with RedShift Spectrum: Experience working with RedShift Spectrum to support data engineering solutions, including data warehousing and data analytics tasks.