- Audit and validate AI/LLM model outputs to ensure accuracy, consistency, and relevance
- Design and implement data quality metrics, sampling strategies, and audit frameworks
- Perform statistical analysis and anomaly detection to identify risks and improve output quality
We are looking for a talented
Data Scientist to join a growing team within a global technology organisation, helping shape the future of
AI quality operations. This role sits at the intersection of
LLM / AI, data science, and quality engineering, ensuring that AI systems deliver reliable, accurate, and high-quality outputs at scale.
This is a high-impact role supporting systems used by
millions of users globally, where data integrity and model reliability are critical.
About The Role
You will be responsible for
validating and improving outputs from LLM and machine learning models, while building scalable systems to ensure data quality and model reliability. Working closely with cross-functional teams, you will apply
statistical methods, automation, and engineering best practices to strengthen AI quality processes.
Key Responsibilities
- Audit and validate AI/LLM model outputs to ensure accuracy, consistency, and relevance
- Design and implement data quality metrics, sampling strategies, and audit frameworks
- Perform statistical analysis and anomaly detection to identify risks and improve output quality
- Build and enhance automation tools, dashboards, and self-serve platforms for quality monitoring
- Develop Python-based solutions to process, evaluate, and improve data reliability
- Collaborate with data engineering, product, and operations teams to ensure high-quality datasets and workflows
- Identify risks, propose action plans, and continuously improve quality control processes
- Contribute to building scalable systems for large-scale AI validation and monitoring
Key Requirements
- Proven experience as a Data Scientist with strong foundations in statistics and data analysis
- Hands-on experience working with or validating LLM / ML models in production environments
- Strong proficiency in Python (e.g. Pandas, NumPy, Scikit-learn) and SQL
- Experience applying statistical methods for validation, experimentation, or anomaly detection
- Demonstrated ability to build tools, automation workflows, or data pipelines
- Solid understanding of data quality, risk identification, and validation methodologies
Preferred Skills
- Experience with LLM / agentic AI frameworks (e.g. RAG pipelines, embeddings, evaluation techniques)
- Experience building self-serve tools, dashboards, or internal platforms (e.g. Flask, Streamlit, Tableau, Power BI)
- Exposure to advanced statistical models or machine learning techniques
- Experience working within large-scale, service-oriented or high-volume data environments
- Strong problem-solving skills with a focus on scalability and risk mitigation
Why Join Us
- Work at the forefront of AI validation, LLM quality, and data reliability
- Build airtight, scalable systems that directly impact products used by millions
- Join a fast-moving, high-impact team spanning data science, AI, and engineering
- Collaborate with global stakeholders across data, search, and AI domains
We regret to inform that only shortlisted candidates will be notified.
EA Registration No.: R2414837, Garey Gan Chia Wen
Allegis Group Singapore Pte Ltd, Company Reg No. 200909448N, EA Licence No. 10C4544
