• Partner closely with Engineers, Product, and Design to productionize early-stage AI features • Focus on quality and safety testing: devising rigorous evaluation frameworks, refining prompts and pipelines, and optimizing model choices • Design and own comprehensive evaluations that measure accuracy, completeness, style, hallucination rate, bias, and safety • Tune and iterate on RAG pipelines, prompt chains, conversation loops • Build reusable data and evaluation pipelines, a shared semantic layer, and monitoring dashboards • Optimize for cost and latency, continuously benchmarking models and negotiating trade-offs • Implement robust data governance and lineage practices • Document best practices and share knowledge