Staff AI Engineer Agentic AI

About Gnani.ai

Gnani.ai builds voice-first AI for enterprises. Our products include an agentic AI platform, a voice API platform (speech APIs), and a conversation analytics platform. We build our own ASR, TTS, and small language models for 22+ Indian languages, serving BFSI, insurance, healthcare, and telecom customers at scale.

About the Role:

Our agentic AI platform lets enterprises build and run autonomous voice and chat agents. As Staff AI Engineer — Agentic AI, you will own the intelligence layer of this platform: how agents think, plan, retrieve knowledge, use tools, and work together. You will set the technical direction for LLM orchestration, RAG, and multi-agent systems, and lead a team of agentic AI engineers to ship it.

This is a hands-on leadership role. You will write code, review designs, and mentor engineers — not just manage.

What You Will Do:

LLM Orchestration

• Design and own the orchestration layer that routes requests across LLMs (hosted and self-hosted SLMs), with fallbacks, caching, and cost/latency controls.

• Build prompt management, structured output handling, and tool-calling pipelines that hold up in real-time voice conversations (strict latency budgets).

RAG Pipelines

• Own the end-to-end RAG stack: ingestion, chunking, embedding, retrieval, re-ranking, and grounding for enterprise knowledge bases.

• Improve answer accuracy and reduce hallucination for domain-heavy verticals (BFSI, insurance, healthcare), including code-mixed and multilingual content.

• Build freshness, versioning, and access control into retrieval so each tenant only sees its own data.

Multi-Agent Orchestration

• Design the multi-agent architecture: planner/worker patterns, agent hand-offs, shared memory, and inter-agent communication.

• Own agent memory design (contact, campaign, and agent-level memory) and how agents learn from production feedback.

Tool Use & Enterprise Integrations

• Build the tool-calling and integration framework that lets agents take real actions: CRM updates, ticket creation, payment flows, and API calls into customer systems.

• Make tool execution safe and auditable: schemas, validation, retries, and human-in-the-loop approval where needed.

Evaluation & Observability

• Define evaluation gates: offline evals, golden test sets, persona simulators, and LLM-as-judge pipelines before changes ship.

• Build observability for every agent decision: traces, decision logs, and quality dashboards so failures can be found and fixed fast.

Guardrails, Safety & Compliance

• Design guardrails against prompt injection, hallucinated actions, and off-policy behavior, with deterministic fallbacks and state recovery.

• Ensure agent behavior meets enterprise compliance needs (data privacy, consent, and disclosure rules) in partnership with product and legal teams.

Cost & Performance Engineering

• Own inference cost and latency: model selection and routing, caching, batching, and KV-cache reuse, so agents stay fast and affordable at scale.

Team Leadership

• Lead and mentor a team of agentic AI engineers (roughly 4–8). Set direction, review designs and code, and raise the quality bar.

• Plan the agentic AI roadmap with product and platform teams. Break big goals into sprint-sized work.

• Hire and grow the team as the platform scales.

What You Bring:

Must have

• 8+ years in software or ML engineering, with 2+ years building LLM-based or agentic systems in production.

• Deep, hands-on experience with LLM orchestration frameworks and patterns (function calling, tool use, structured outputs, streaming) — and knowing when to skip the framework and build it yourself.

• Production RAG experience: vector databases, retrieval quality tuning, re-ranking, and eval-driven iteration.

• Experience designing multi-agent systems: task decomposition, agent coordination, memory, and failure handling.

• Strong Python; comfort with Go is a plus. Solid grasp of distributed systems (queues/messaging, Redis, Kubernetes).

• Track record of leading engineers as a tech lead or staff engineer: design reviews, mentorship, delivery ownership.

Nice to have

• Real-time or voice AI experience (latency-sensitive pipelines, streaming ASR/TTS integration).

• Fine-tuning or serving SLMs (vLLM, TensorRT-LLM, or similar).

• Experience with Indic languages or code-mixed text.

• Familiarity with enterprise compliance needs (data residency, DPDP, RBI guidelines).

• Hands on experience with Livekit and Pipecat frameworks

Why This Role

• Own a core layer of a fast-growing agentic platform used by large enterprises, end to end.

• Work on hard, real problems: agents that talk on live phone calls with sub-second latency budgets.

• Build on proprietary models (ASR, TTS, SLM) — not just API wrappers.

• Small, senior team. High trust, high ownership, direct access to leadership.

How We Work

Bengaluru-based, in-office collaboration. Sprint-based delivery with a clear roadmap. Design docs and evals before big changes. We value engineers who ship, measure, and improve.