Introduction

A career in IBM Consulting is built on long-term client relationships and close collaboration worldwide. You'll work with leading companies across industries, helping them shape their hybrid cloud and AI journeys. With support from our strategic partners, robust IBM technology, and Red Hat, you'll have the tools to drive meaningful change and accelerate client impact. At IBM Consulting, curiosity fuels success. You'll be encouraged to challenge the norm, explore new ideas, and create innovative solutions that deliver real results. Our culture of growth and empathy focuses on your long-term career development while valuing your unique skills and experiences.

Your Role And Responsibilities

Job Summary

We are seeking an experienced AWS Data Engineer to design, build, and support scalable, secure, and high-performance data platforms on AWS. The role requires strong hands-on experience with AWS services, distributed data processing frameworks, and SQL-driven analytics.

The ideal candidate will work closely with architects, application teams, and business stakeholders to deliver reliable enterprise-grade data solutions.

Key Responsibilities

Design, develop, and maintain data pipelines and data platforms on AWS
Build and operate serverless and containerized workloads using AWS Lambda, EC2, and EKS
Develop batch and large-scale data processing solutions using Apache Spark
Implement data ingestion, transformation, and analytics using AWS Glue and Athena
Write complex and high-performance SQL queries for analytics, reporting, and data validation
Manage and optimize data storage solutions using AWS S3 and RDS
Design and manage AWS infrastructure using Infrastructure as Code (IaC) with CloudFormation
Apply AWS security best practices across IAM, VPC networking, data encryption, and access controls
Develop clean, scalable, and maintainable code using Python and/or Scala
Monitor, troubleshoot, and optimize performance, cost, and reliability of data workloads
Collaborate with cross-functional teams to support analytics, reporting, and regulatory use cases

Mandatory Skills & Qualifications

Strong hands-on experience with AWS services, including o S3, Lambda, EC2, EKS o AWS Glue, Athena o RDS o VPC, IAM o CloudFormation
Strong SQL skills, including o Writing complex queries, joins, aggregations, and user defined functions o Performance tuning and query optimization
Good knowledge of AWS Security, including IAM policies, VPC security, encryption, and least-privilege access
Hands-on programming experience with Python and/or Scala
Strong experience with Apache Spark for large-scale data processing
Good communication skills to explain technical concepts to non-technical stakeholders
Solid understanding of data engineering concepts such as ETL/ELT, data modeling, and distributed systems

Good To Have / Preferred Skills

Experience with DBT for data transformations
Hands-on exposure to Snowflake
Experience working with Databricks
Familiarity with modern data lake / Lakehouse architectures
Experience working in enterprise or regulated environments

Nice-to-Have Competencies

Strong analytical and problem-solving skills
Ability to work independently and in distributed teams

Preferred Education

Master's Degree

Required Technical And Professional Expertise

Exposure to AWS Toolset: Experience working with AWS services such as AWS EMR, AWS Glue, Glue Catalog, and Kinesis to design, build, and operate batch and real-time data pipelines.
Data Pipeline Development: Exposure to developing batch and real-time data pipelines for Data Warehouse and Datalake, utilizing AWS Kinesis and Managed Streaming for Apache Kafka.
Data Layer Creation: Experience working with AWS RedShift, Aurora, and DynamoDB to create data layers and migrate data using AWS DMS.
Open Source Technologies: Exposure to utilizing open source technologies like Apache Airflow and dbt, Spark / Python or Spark / Scala on AWS Platform to support data engineering solutions.
Data Service Management: Experience scheduling and managing data services on the AWS Platform, ensuring efficient operation of data engineering solutions.

Preferred Technical And Professional Experience

Proficiency with AWS Databrew: Experience working with AWS Glue Databrew to support data engineering solutions, including data preparation and data quality tasks.
Knowledge of Lambda Functions: Exposure to using Lambda functions with Python to support data engineering solutions, including data processing and data transformation tasks.
Familiarity with RedShift Spectrum: Experience working with RedShift Spectrum to support data engineering solutions, including data warehousing and data analytics tasks.

Data Engineer-Data Platforms-AWS