V
Senior Software Engineer
Job Description
'Job Description
What We're Looking For
○ Build evaluation pipelines using golden datasets, regression tests,
deterministic evaluators, and LLM-as-a-judge.
○ Implement observability, monitoring, guardrails, fallbacks, and validation for
reliable AI systems.
○ Balance model quality, latency, reliability, maintainability, and cost.
testing (unit, integration, E2E) and test automation frameworks.
○ Solid grasp of CI/CD, infrastructure concepts, and agile engineering practices.
What We're Looking For
- Proven experience in designing and building scalable, reliable, and maintainable
- Strong proficiency in modern software development (e.g., React.js, TypeScript,
- Build LLM workflows using RAG, tool calling, structured outputs, agents, and
○ Build evaluation pipelines using golden datasets, regression tests,
deterministic evaluators, and LLM-as-a-judge.
○ Implement observability, monitoring, guardrails, fallbacks, and validation for
reliable AI systems.
○ Balance model quality, latency, reliability, maintainability, and cost.
- Maintain automated tests, CI/CD pipelines, and technical documentation.
testing (unit, integration, E2E) and test automation frameworks.
○ Solid grasp of CI/CD, infrastructure concepts, and agile engineering practices.
- Strong troubleshooting and debugging skills, with an ability to analyze complex
- Excellent communication skills to explain technical trade-offs and quality standards
More Info
Key Skills
test automation frameworks
human-in-the-loop patterns
fallbacks
LLM-as-a-judge
agile engineering
CI/CD pipelines
deterministic evaluators
golden datasets
RAG tool
evaluation pipelines
LLM workflows
guardrails
structured outputs agents
regression tests
automated tests
observability

