Data Engineer-Data Platforms-AWS

Introduction

A career at IBM Consulting is grounded in long-lasting client partnerships and day-to-day collaboration with teams across the globe. In this role, you’ll work with leading organizations across industries, supporting the development of their hybrid cloud and AI strategies. Backed by strategic partners, strong IBM technology capabilities, and Red Hat, you’ll use advanced tooling to drive meaningful change and increase client outcomes. At IBM Consulting, curiosity is a core driver of success—you’ll be encouraged to question the status quo, experiment with new approaches, and deliver innovative solutions that produce measurable impact. Our culture of growth and empathy emphasizes long-term career development, while also recognizing and valuing your individual skills and experiences.

Your Role And Responsibilities

Job Summary

We are looking for an experienced AWS Data Engineer who will design, build, and support scalable, secure, high-performance data platforms in AWS. This position requires strong, hands-on knowledge of AWS services, distributed data processing frameworks, and SQL-based analytics.

In this role, you will partner closely with architects, application teams, and business stakeholders to deliver dependable, enterprise-grade data solutions.

Key Responsibilities

• Design, develop, and maintain data pipelines and data platforms on AWS
• Build and run serverless and containerized workloads using AWS Lambda, EC2, and EKS
• Create batch and large-scale data processing solutions using Apache Spark
• Implement data ingestion, transformation, and analytics using AWS Glue and Athena
• Develop advanced, high-performance SQL queries for analytics, reporting, and data validation
• Oversee and optimize data storage solutions using AWS S3 and RDS
• Create and manage AWS infrastructure using Infrastructure as Code (IaC) with CloudFormation
• Use AWS security best practices across IAM, VPC networking, data encryption, and access controls
• Write clean, scalable, and maintainable code using Python and/or Scala
• Monitor, troubleshoot, and optimize performance, cost, and reliability of data workloads
• Work with cross-functional teams to support analytics, reporting, and regulatory use cases
  
  

Mandatory Skills & Qualifications

• Hands-on experience with AWS services, including o S3, Lambda, EC2, EKS o AWS Glue, Athena o RDS o VPC, IAM o CloudFormation
• Strong SQL expertise, including o Writing complex queries, joins, aggregations, and user defined functions o Performance tuning and query optimization
• Knowledge of AWS Security, including IAM policies, VPC security, encryption, and least-privilege access
• Practical programming experience with Python and/or Scala
• Demonstrated experience with Apache Spark for large-scale data processing
• Good communication skills to explain technical concepts to non-technical stakeholders
• Solid understanding of data engineering fundamentals such as ETL/ELT, data modeling, and distributed systems
  
  

Good To Have / Preferred Skills

• Experience with DBT for data transformations
• Hands-on exposure to Snowflake
• Experience working with Databricks
• Familiarity with modern data lake / Lakehouse architectures
• Experience working in enterprise or regulated environments
  
  

Nice-to-Have Competencies

• Strong analytical and problem-solving skills
• Ability to work independently and in distributed teams
  
  

Preferred Education

Master's Degree

Required Technical And Professional Expertise

• Exposure to AWS Toolset: Experience working with AWS services such as AWS EMR, AWS Glue, Glue Catalog, and Kinesis to design, build, and operate batch and real-time data pipelines.
• Data Pipeline Development: Exposure to building batch and real-time data pipelines for Data Warehouse and Datalake, using AWS Kinesis and Managed Streaming for Apache Kafka.
• Data Layer Creation: Experience working with AWS RedShift, Aurora, and DynamoDB to create data layers and migrate data using AWS DMS.
• Open Source Technologies: Exposure to using open source technologies such as Apache Airflow and dbt, Spark / Python or Spark / Scala on AWS Platform to support data engineering solutions.
• Data Service Management: Experience scheduling and managing data services on the AWS Platform to ensure efficient operation of data engineering solutions.
  
  

Preferred Technical And Professional Experience

• Proficiency with AWS Databrew: Experience working with AWS Glue Databrew to support data engineering solutions, including data preparation and data quality tasks.
• Knowledge of Lambda Functions: Exposure to using Lambda functions with Python to support data engineering solutions, including data processing and data transformation tasks.
• Familiarity with RedShift Spectrum: Experience working with RedShift Spectrum to support data engineering solutions, including data warehousing and data analytics tasks.