Data Engineer-Data Platforms-AWS

Introduction

Build a career at IBM Consulting by strengthening long-term client relationships and working closely with teams worldwide. You’ll partner with leading organizations across industries to shape their hybrid cloud and AI initiatives. Backed by strategic partners, dependable IBM technology, and Red Hat, you’ll gain the capabilities to create meaningful change and help clients accelerate their outcomes. At IBM Consulting, curiosity drives performance—so you’ll be encouraged to challenge conventional thinking, test new ideas, and develop solutions that produce tangible results. Our growth- and empathy-centered culture supports your long-term career development while recognizing the value of your individual skills and experiences.

Your Role And Responsibilities

As a Data Engineer specializing in Data Platforms on AWS, you will advise on, develop, and support data engineering solutions across the AWS Cloud environment. You’ll design, implement, and run both batch and real-time data pipelines using a range of AWS services.

Your Primary Responsibilities Will Include

• Design and Develop Data Pipelines: Design, build, and operate batch and real-time data pipelines using AWS services such as AWS EMR, AWS Glue, Glue Catalog, and Kinesis, ensuring seamless integration and day-to-day operation of data engineering solutions.
• Create Data Layers: Build data layers on AWS RedShift, Aurora, and DynamoDB, and perform data migration using AWS DMS.
• Manage Data Services: Schedule and manage data services on the AWS Platform to ensure efficient operation of data engineering solutions.
• Develop Batch and Real-time Pipelines: Create batch and real-time data pipelines for Data Warehouse and Datalake, leveraging AWS Kinesis and Managed Streaming for Apache Kafka.
• Utilize Open Source Technologies: Apply open source technologies such as Apache Airflow and dbt, along with Spark / Python or Spark / Scala on the AWS Platform, to enable data engineering solutions.
  
  

Preferred Education

Master's Degree

Required Technical And Professional Expertise

• Exposure to AWS Toolset: Experience with AWS services including AWS EMR, AWS Glue, Glue Catalog, and Kinesis to design, build, and run batch and real-time data pipelines.
• Data Pipeline Development: Exposure to developing batch and real-time data pipelines for Data Warehouse and Datalake using AWS Kinesis and Managed Streaming for Apache Kafka.
• Data Layer Creation: Experience with AWS RedShift, Aurora, and DynamoDB to create data layers and migrate data using AWS DMS.
• Open Source Technologies: Exposure to open source technologies such as Apache Airflow and dbt, along with Spark / Python or Spark / Scala on the AWS Platform, to support data engineering solutions.
• Data Service Management: Experience scheduling and managing data services on the AWS Platform to ensure efficient operation of data engineering solutions.
  
  

Preferred Technical And Professional Experience

• Proficiency with AWS Databrew: Experience using AWS Glue Databrew to support data engineering solutions, including data preparation and data quality activities.
• Knowledge of Lambda Functions: Exposure to using Lambda functions with Python to support data engineering solutions, including data processing and data transformation tasks.
• Familiarity with RedShift Spectrum: Experience using RedShift Spectrum to support data engineering solutions, including data warehousing and data analytics activities.