Search by job, company or skills

AWS, Pyspark

Cognizant Consulting
6-12 Years
Chennai, India
Early Applicant
  • Posted 15 days ago
  • Be among the first 10 applicants

Job Description

Skill: Pyspark, AWS

Exp: 6 to 12 years

Location: Pune

Job Summary

We are seeking an experienced AWS PySpark Developer to design, develop, and optimize large-scale data processing pipelines in the cloud. The ideal candidate will have strong expertise in PySpark, AWS services, and ETL workflows, with a focus on building scalable, high-performance data solutions for analytics and reporting.

Key Responsibilities

  • Design, develop, and maintain ETL pipelines using PySpark on AWS.
  • Work with large datasets to perform data cleansing, transformation, and aggregation.
  • Optimize Spark jobs for performance and cost efficiency.
  • Integrate data from multiple sources including AWS S3, RDS, Redshift, DynamoDB, and APIs.
  • Implement data quality checks and ensure compliance with governance policies.
  • Collaborate with data analysts, data scientists, and business stakeholders to deliver data solutions.
  • Deploy and monitor jobs using AWS Glue, EMR, Step Functions, or Lambda.
  • Troubleshoot and resolve issues in production data pipelines.
  • Maintain documentation for data flows, transformations, and architecture.

Required Skills & Qualifications

  • 3–7 years of experience in data engineering or big data development.
  • Strong proficiency in PySpark and Spark SQL.
  • Hands-on experience with AWS services such as:
    • S3, Glue, EMR, Lambda, Athena, Redshift, IAM
  • Solid understanding of ETL concepts and data warehousing.
  • Proficiency in Python for scripting and automation.
  • Experience with version control (Git) and CI/CD pipelines.
  • Strong problem-solving skills and ability to work in an agile environment.
Preferred Skills

  • Experience with AWS Glue Studio and Glue Catalog.
  • Knowledge of Airflow or other orchestration tools.
  • Familiarity with streaming data using Kafka or Kinesis.
  • Exposure to Terraform or CloudFormation for infrastructure as code.
  • Understanding of data security and encryption in AWS.

More Info

Job Type:
Permanent Job
Industry:
Other
Function:
Data Engineering
Employment Type:
Full time

Job ID: 151729367

Last Updated: 29-07-2026 05:53:10 PM
Homejobs in ChennaiAWS, Pyspark

Similar Jobs

Java Architect with Cloud Exp Azure/AWS

Tata Consultancy Services
T
Chennai, India

Skills:

Java, Devops, Spring Boot, Azure, Microservices, AWS, cloud-native technologies

AWS, Pyspark

Cognizant Softvision
Chennai, India

Skills:

Spark SQL, Git, Pyspark, AWS Glue, Python, Aws S3, AWS, Etl

Python, Pyspark, AWS

Cognizant Consulting
Chennai, India

Skills:

S3, Pyspark, Emr, Redshift, Sql, Lambda, Git, Kinesis, Python, AWS, Glue, Athena

Pyspark Developer

P.R.Glolinks Consulting Private Limited
P
Bengaluru, Chennai

Skills:

Pyspark, Python

Early Applicant

AWS Data Engineer Lead

softview infotech
Chennai, India

Skills:

S3, Cloudformation, Pyspark, AWS Glue, Kafka, Emr, Redshift, Sql, Jenkins, Lambda, Kms, Kinesis, Cloudwatch, Terraform, Iam, Spark, Lake Formation, CodePipeline, Athena, Glue Streaming, CloudTrail

Beware of Scammers

We don’t charge money for job offers

Know More