We are seeking a highly skilled Data Engineer to join our team in Singapore. The ideal candidate will have strong expertise in AWS native data services and Databricks with proven experience in designing, building, and optimizing end to end data pipelines. Prior exposure to public sector projects will be considered a significant advantage.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and ETL processes.
- Implement data solutions leveraging AWS native services (e.g., S3, Glue, Redshift, Athena, EMR, Lambda).
- Build and optimize data workflows using Databricks for advanced analytics and machine learning workloads.
- Apply DevOps practices to automate infrastructure provisioning, CI/CD pipelines, and monitoring of data systems.
- Implement MLOps frameworks to operationalize machine learning models, ensuring reproducibility, scalability, and performance.
- Collaborate with data scientists, analysts, and business stakeholders to deliver reliable and secure data solutions.
- Ensure data quality, governance, and compliance with organizational and regulatory standards.
- Document technical designs, processes, and best practices.
Requirements
- Bachelor's or Master's degree in Computer Science, Information Systems, or related field.
- Strong hands-on experience with AWS Data Services (S3, Glue, Redshift, Athena, EMR, Lambda).
- Proficiency in Databricks (PySpark, Delta Lake, MLflow).
- Solid understanding of data modeling, ETL/ELT, and data warehousing concepts.
- Experience with programming languages such as Python, SQL, and Scala.
- Proven experience with DevOps tools (e.g., Docker, Kubernetes, Terraform, Jenkins, GitHub Actions).
- Knowledge of MLOps practices (model deployment, monitoring, retraining pipelines, MLflow, Kubeflow).
- Familiarity with CI/CD pipelines, version control (Git), and cloud-native architectures.
- Strong problem-solving skills and ability to work in cross-functional teams.
- Excellent communication and stakeholder management skills.
