Senior Software Engineer

This is a 6-month contract role under Optimum Solutions.

This role supports our client, a leading Southeast Asian superapp that offers a wide range of services, including mobility, food delivery, and digital payments.

Work Schedule and Hours: Monday - Friday, 10am - 7pm (inclusive of a 1-hour lunch break)

Key Responsibilities:

. Take ownership of the ACS embedded-evaluation workstream.

. Establish reliable regression evaluation for at least one additional agent team.

. Make its distributed traces consistently usable for debugging and automated evaluation.

. Ship at least one reusable improvement to EvalsHub or its SDK/tooling.

. Reduce manual evaluation effort through automated datasets, evaluators, and CI integration.

. Document ownership, operational procedures, and onboarding guidance.

. Embed with AI product teams and own measurable agent-quality outcomes.

. Build and maintain golden datasets, regression suites, evaluation pipelines, and quality dashboards.

. Convert product rules and human review procedures into deterministic, numeric, LLM-as-judge, trajectory, tool-selection, SOP-adherence, and multi-turn evaluators.

. Diagnose evaluation and production-trace failures, then convert findings into dataset improvements, evaluator changes, or fixes to prompts, tools, skills, and agent workflows.

. Instrument distributed agent systems using OpenTelemetry, including Temporal workflows and Go/Python services.

. Ensure traces contain consistent inputs, outputs, tool calls, metadata, feedback, and root-agent results.

. Develop reusable capabilities across EvalsHub backend, SDK, CLI/plugins, and supporting frontend interfaces.

. Integrate evaluations into CI/CD and model-release workflows.

. Run model, retrieval, embedding, and agent-architecture experiments balance accuracy, latency, and cost.

. Drive migrations from legacy evaluation and observability systems.

. Write technical designs and documentation, run demonstrations, and enable engineering teams to use evaluation tooling independently.

. Participate in the LLMOps operational rotation and support production-quality integrations.

Qualifications:

. Minimum 2 years of relevant experience in one or more of the following areas:

EvalsHub, LangSmith, LangGraph/LangChain, FastAPI, Temporal, Grafana, Redis, or similar technologies.

React/TypeScript and internal developer-tooling interfaces.

LLM-as-judge, multi-turn evaluation, tool/MCP evaluation, and agent-trajectory analysis.

Text-to-SQL/Text-to-DSL, RAG, semantic retrieval, embedding evaluation, or model benchmarking.

Building platform capabilities that can be reused across several AI products.

. Strong analytical and problem-solving skills related to AI systems, evaluation frameworks, and data-driven quality improvements.

. Excellent project management and cross-department communication abilities.

. A detail-oriented and metrics-driven approach to continuous improvement