Senior Data Engineer

Responsibilities:

. Enhance the existing data pipeline using industry-standard tools and mechanisms.

1. Redesign and implement data pipeline to use AWS managed components.

. Implement data pipeline monitoring to alert us if there is any issue with the pipeline.

. Investigate and resolve any issue with data pipeline.

. Document the data pipeline solution and how to operate the data pipeline.

. Implement unit testing and integration testing for data pipeline component.

Requirements:

. At least 3 years hands-on working experience in data engineering, designing and building ETL pipelines.

. Experience in project execution and demonstrated technical expertise.

. Proficient in Python.

. SQL Knowledge.

. Schell scripts (Linux).

. Airflow or similar tool to create pipelines.

. AWS tools like S3, Glue, AWS EMR, Lambda, Step functions.

. Familiar with tools like AWS Batch, Athena, file formats like Parquet, NoSQL knowledge and tools like AWS OpenSearch.

. The engineer should be able to physically work from MOHT premises whenever requested by the project team.