Senior Data Engineer

Key Responsibilities

. Build and test data pipelines on modern data platforms

. Support data engineering, ingestion, transformation and consumption activities

. Develop new pipelines and data products, and automation

. Enhance existing pipelines and data products, and provide support

. Maintain all the data pipelines and data products of these Solutions

. Design conceptual, logical, and physical data models

. Design data lake to support analytical and reporting needs

. Provide technical services like development, data preparation, data transformation, data

encryption, data masking, data optimization and data testing preparation

. Adhere to the standards and guidelines defined by the Authority for devops processes, provide

deployment plan, checklist and assist in production implementation/ roll out

. Provide support and integration with capability on latest AI tool stack to expedite the

development, testing and deployment process

Required Skills

. Python

. PySpark

. Snowflake

. Databricks

. Cloudera

. Migration from Cloudera to Snowflake or Databricks with meta data-driven design

Required Experience

. Minimum 5+ years of experience in the software used by the Solution

. Good understanding of data warehouse implementations

. Hands-on experience in big data engineering jobs using Python, PySpark to build and test data

pipelines on modern data platforms such as Snowflake, Databricks and Cloudera