Sully.ai

AI Medical Employees for Healthcare

Applied Research Scientist

Research ScientistResearch ScientistFull TimeRemoteTeam 11-50Since 2023H1B No SponsorCompany SiteLinkedIn

Location

United States

Posted

14 days ago

Salary

Not specified

EnglishPythonPy TorchTensorflow

Job Description

• Build and scale automated evaluation pipelines (LLM-as-judge + human review) with clinical-grade benchmarks. • Audit existing evaluation approaches for clinical and agentic tasks. • Define initial benchmarks and build early automated pipelines. • Partner with engineering to land first set of CI gates for accuracy, factuality, and safety. • Deliver a repeatable evaluation framework with automated pipelines in production. • Demonstrate measurable improvements in robustness, hallucination reduction, or safety. • Publish or present internal research findings that directly shape product reliability.

Job Requirements

  • Proven experience designing agentic processes and LLM evaluation/benchmarking frameworks.
  • Strong Python and ML background (PyTorch/TensorFlow, Hugging Face, LangChain/LlamaIndex).
  • Demonstrated ability to design rigorous experiments and translate findings into production.
  • Track record of published research or deep applied work in LLMs and agent evaluation.
  • Strong communication and technical writing skills to articulate complex findings clearly.

Benefits

  • Speed matters - we operate with urgency, autonomy, and ownership
  • You’ll work on real, first-of-their-kind problems at the edge of AI and medicine
  • Your work helps doctors reclaim their time - and patients get better, faster care

Related Categories

Related Job Pages

More Research Scientist Jobs

Full TimeRemoteTeam 1-10Since 1870H1B No Sponsor

Senior Research Scientist conducting research on microwave radiometric measurements for ice sheets and sea ice

United States
Remote

Led by Michael Antonov, a co-founder of Oculus, and well-funded by Formic Ventures, Deep Origin is poised to reinvent the way scientists work and life science innovations come to life. We see a future largely free of diseases, with a 150-year lifespan ...

Oregon
Remote

Led by Michael Antonov, a co-founder of Oculus, and well-funded by Formic Ventures, Deep Origin is poised to reinvent the way scientists work and life science innovations come to life. We see a future largely free of diseases, with a 150-year lifespan ...

District of Columbia
Remote

Led by Michael Antonov, a co-founder of Oculus, and well-funded by Formic Ventures, Deep Origin is poised to reinvent the way scientists work and life science innovations come to life. We see a future largely free of diseases, with a 150-year lifespan ...

United States