• Architect & Scale AI Evaluation Systems: Design and oversee the end-to-end framework for testing AI agent systems at scale. You will leverage LLM-driven methodologies—including automated simulations, "LLM-on-LLM" rubrics, and adversarial red-teaming—to ensure reliability, policy adherence, and logic across complex, multi-turn conversational flows. • Drive Deployment Excellence: Partner with Forward Deployed Engineers and PMs to triage issues, identify bottlenecks, and create new test cases on the fly to address real-world deployment challenges. • Be the Customer’s Voice: Conduct manual UAT and voice-call testing to represent the end-customer experience. You take it personally when an agent lacks empathy or clarity, and you excel at articulating these nuances to the engineering team and clients. • Lead and Scale the Team: lead a pod of QA analysts and partners. You will define the best practices, communication loops, and shared knowledge base that allow the QA function to scale alongside our rapidly growing product line.