Job Role: - Senior Data Engineer
Job Location: - Singapore
Experience: -8+ Years
Key Responsibilities: -- Design and architect data storage solutions such as data lakes, warehouses, and databases using AWS services (S3, RDS, Redshift, DynamoDB) and Delta Lake on Databricks.
- Build, manage, and optimize data pipelines for ingestion, processing, and transformation using AWS Glue, AWS Lambda, Databricks, and Informatica IDMC.
- Integrate internal and external data sources ensuring high data quality, consistency, and governance.
- Develop ETL/ELT workflows using Databricks (Spark), Python, and Informatica IDMC.
- Monitor and improve data pipeline performance, scalability, and reliability.
- Implement data security, encryption, and compliance best practices across cloud environments.
- Automate routine data tasks using AWS (Step Functions, Lambda), Databricks Jobs, and IDMC workflows.
- Maintain clear documentation for data pipelines, architecture, and configurations.
- Collaborate with cross-functional teams (data scientists, analysts, engineering teams) to meet data requirements.
- Troubleshoot data issues, ensure data availability, and manage performance optimization.
- Optimize resource usage to ensure cost-efficiency.
- Stay updated on cloud data engineering tools, services, and best practices.
Required Skills & Qualifications:-- Bachelors or Masters degree in Computer Science, Data Engineering, or related field.
- 8+ years of data engineering experience.
- Strong hands-on experience with Informatica IDMC, Databricks, and AWS services.
- Proficiency in Python, Java, or Scala for building data pipelines.
- Strong SQL/NoSQL knowledge and data modeling experience.
- Ability to evaluate technical solutions and optimize complex data transformations.
- Strong analytical, problem-solving, and communication skills.
- Experience working as a Team Lead.
- Experience with Git and CI/CD pipelines.
Preferred Skills- Hands-on experience with PySpark on Databricks.
- Knowledge of data governance and cataloging tools (especially IDMC).
- Experience with dashboards/visualization tools (Tableau, OAS).
- Familiarity with Docker, Kubernetes, and cloud DevOps practices.
- Relevant certifications (AWS, Databricks, Informatica).
