about company
I am currently working with a company focusing on AI, LLM and Computer Vision.
Hybrid working arrangement- 2 days office, 3 days WFH. Office location at near Buona Vista. 4 rounds of interview to offer stage.
about job
- Design and implement robust frameworks to evaluate the performance of generative AI systems, including text and multi-modal models for Large Language Models (LLMs), including but not limited to GPT-based models, BERT, T5, and other state-of-the-art architectures
- Perform technical AI evaluations on LLM including assessing them for robustness in performance, embedded biases, vulnerability to jailbreaks and prompt injection attacks
- Work with stakeholders to design strong LLM models, custom evaluation approaches and a suite of technical and analytical AI evaluation frameworks and tools
- Define and refine metrics for evaluating model performance, such as perplexity, BLEU, ROUGE, accuracy, coherence, factual consistency, and bias detection
- Lead efforts in curating and managing large, high-quality datasets for evaluating LLMs
skills and requirements
- Min 2 years for junior, 5 years for senior
- Experience with Agentic AI or Agentic LLM
- Strong experience in evaluating LLMs using metrics such as perplexity, BLEU, ROUGE, and human-centered evaluation techniques
- Proven track record of managing and analyzing large, complex language datasets, including text preprocessing and tokenization
- Solid programming skills in Python and experience building automated pipelines for continuous model evaluation
To apply online please use the apply function, alternatively you may contact Stella at 96554170 (EA: 94C3609 /R1875382)
