Job Role: -Data Engineer( Lead)
Job Location: -Singapore
Experience: -10+ Years
Role SummaryLead Data Engineer responsible for cloud migration and data platform modernization using AWS, Databricks, Informatica IDMC, and Tableau. The role focuses on designing scalable data architectures, building pipelines, ensuring governance, and leading technical decisions.
Key Responsibilities- Design and architect data storage solutions (databases, data lakes, data warehouses) using AWS services and Databricks Delta Lake.
- Build, manage, and optimize data pipelines using AWS Glue, Lambda, Databricks, and Informatica IDMC.
- Integrate data from multiple internal and external sources ensuring quality, consistency, and governance.
- Develop ETL/ELT workflows using PySpark, Databricks, and Informatica IDMC.
- Monitor and optimize data processing performance across AWS and Databricks environments.
- Implement data security, compliance, and governance standards.
- Automate data ingestion, transformation, and workflow tasks.
- Maintain documentation of data architectures, pipelines, and configurations.
- Collaborate with cross-functional teams to gather requirements and deliver data solutions.
- Troubleshoot data issues, ensure data integrity, and support production pipelines.
- Optimize resource usage for cost efficiency and scalability.
- Stay updated on cloud, data engineering, and analytics platform best practices.
Skills & Requirements- Bachelors/Masters degree in Computer Science, Data Engineering, or related field.
- 10+ years of experience in data engineering.
- Strong hands-on expertise with AWS cloud services, Databricks, and/or Informatica IDMC.
- Proficiency in Python, PySpark, Java, or Scala for building pipelines.
- Strong SQL and NoSQL knowledge, data modeling, and schema design.
- Ability to evaluate and optimize complex data workflows and transformations.
- Excellent analytical, problem-solving, and communication skills.
- Experience with Git, CI/CD pipelines, and DevOps practices.
- Certifications in AWS, Databricks, or IDMC are an advantage.
Additional Preferred Skills- Strong PySpark experience on Databricks.
- Knowledge of data governance and cataloguing (Informatica IDMC).
- Experience with Tableau or other BI tools.
- Familiarity with Docker, Kubernetes, and orchestration tools.
- Background as a Tech Manager or Cloud Data Architect.
