AI Engineer - Agentic AI and Voice Agents

About the Role

We are seeking AI Engineers to build advanced agentic AI systems. In this role, you will design and implement LLM agents capable of planning, utilizing tools, and completing complex business tasks end-to-end. A key focus will be on building agents that conduct live phone and web conversations, integrating real-time speech recognition, reasoning, tool calls, and speech synthesis. You will also be responsible for developing and maintaining robust evaluation systems to ensure agent reliability before releases, transitioning use cases from demonstrations to trusted, observable client services.

Key Responsibilities

Agentic Systems

• Design and build LLM agents that plan, call tools, and complete multi-step business tasks across voice, chat, and back-office channels.
• Implement task and conversation state, memory and context management, retrieval (RAG), and model routing between various models.
• Define tool schemas and permissions, validate every tool call, and integrate human approval points for high-impact actions.
• Build durable workflows with checkpoints, retries, idempotency keys, timeouts, and compensation for failed actions.
• Employ multi-agent designs (e.g., router with specialist agents) only when they measurably outperform a single-agent approach.
• Integrate with CRMs, ticketing systems, and databases for actions such as complaint registration, classification, routing, status updates, and notifications, including warm human handoff.

Voice Agents

• Build streaming pipelines connecting telephony (SIP, Twilio, Exotel) and WebRTC audio to ASR, LLMs, and TTS over WebSockets.
• Implement voice activity detection, turn detection, barge-in, and cancellation to ensure the agent stops speaking when a caller interjects.
• Set and maintain latency budgets per hop (ASR, LLM, TTS, network), targeting under 1 second from end of caller speech to first agent audio.
• Adapt agent behavior for speech interactions: utilizing short turns, confirmations before actions, and recovery mechanisms for mishearings.
• Support multilingual and code-switched calls, including language detection and matching TTS voices.

Evaluation

• Build and maintain golden test sets that define user goals, personas, expected tool calls, and pass criteria for each scenario.
• Conduct trajectory evaluations to verify each step's tool choice, arguments, and order, beyond just the final answer.
• Perform simulated-caller evaluations for voice agents: using an LLM to play the caller, TTS to voice with accents, noise, and interruptions, testing the agent end-to-end over real audio.
• Score task success, tool-call correctness, policy compliance, hallucination, word error rate, and latency, using rule checks and LLM judges calibrated against human ratings.
• Gate releases in CI based on evaluation results, block regressions, and convert every confirmed production failure into a new test case.

Backend and Operations

• Write asynchronous Python services (FastAPI, asyncio) with structured logging, tracing, and metrics for every agent run and conversation.
• Load-test concurrent sessions, identify bottlenecks, and maintain p95 latency and per-task model cost within budget.
• Implement robust data protection: authentication and authorization, audit trails for agent actions, PII redaction, and tenant isolation.
• Deploy with Docker and CI using staged releases and rollback, and troubleshoot live agents in production.
• Collaborate with product and delivery teams to translate client workflows into functional agent integrations.

Required Qualifications and Skills

• Strong proficiency in Python and backend development: APIs, asyncio, WebSockets, structured error handling, and automated tests.
• Hands-on experience deploying LLM agents beyond a prompt demo, including tool calling, state management, RAG, and failure handling in real-world workflows.
• Sound judgment in agent design: understanding when fixed workflows are superior to autonomous agents, single versus multi-agent approaches, and how to implement permissions and guardrails for agent safety.
• Working knowledge of real-time voice stacks: streaming ASR and TTS, VAD, turn-taking, and the trade-offs between latency and accuracy.
• Experience designing evaluations for LLM agents: developing test sets, trajectory and outcome metrics, leveraging LLM-as-judge, and regression tracking—along with an understanding of misleading metrics.
• Ability to debug across services using logs, traces, and recorded sessions to pinpoint agent failures or performance degradation.
• Proficiency with SQL or NoSQL databases, Git, Docker, and CI pipelines.
• Bachelor’s degree in a technical field or equivalent practical experience, with clear examples of delivered work.

Preferred Experience

• Experience with agent frameworks and orchestration tools such as LangGraph, OpenAI Agents SDK, Temporal, or MCP tool servers.
• Production experience with voice agents on telephony (SIP, Twilio, Exotel) or WebRTC, utilizing frameworks like LiveKit Agents or Pipecat.
• Familiarity with speech providers such as Deepgram, Whisper, Google STT, ElevenLabs, Azure TTS, or Cartesia, and experience in measuring their WER and latency on custom data.
• Experience with Indian-language and code-switched speech (Hindi, Telugu, Tamil, and English).
• Knowledge of queues, caching, and event-driven systems (Redis, Kafka, RabbitMQ), and load-testing tools (Locust, k6).
• Experience with access controls, PII redaction, and on-premises or restricted-network deployments.
• Exposure to other modalities like vision or document understanding (images, screenshots, scanned forms), and combining them with voice or text in a single agent.

Success in the Role

Success is defined by the reliable completion of real business tasks by AI agents (voice, chat, or back-office), supported by verifiable evidence of their functionality and a maintainable codebase. In your first six months, this entails:

• Business Outcomes: Agents measurably resolve tasks without human intervention, reduce handling time, and enhance customer satisfaction for client use cases.
• Reliability: Workflows effectively recover from timeouts, tool failures, and invalid input; write actions are never duplicated; every incident is traced to its root cause and resolved.
• Evaluation Culture: Every release is gated by comprehensive evaluations; metrics accurately reflect user experience; and production failures are systematically integrated into test sets.
• Performance and Cost: Latency and per-task model cost remain within agreed budgets as traffic scales.
• Delivery: Client requirements are translated into working integrations on schedule, with clear handover to product and delivery teams.
• Engineering Quality: Features are thoroughly tested, observable, well-documented, and easily maintainable; active participation in code reviews and shared tooling is expected.