Jump - Advisor AI

Jump uses AI to help financial managers automatically take notes, stay compliant, update their CRM, and serve clients.

Applied AI Evaluation Scientist

Artificial IntelligenceArtificial IntelligenceFull TimeRemoteTeam 51-200Since 2023Company SiteLinkedIn

Location

United States

Posted

15 days ago

Salary

$180K - $270K / year

Bachelor Degree5 yrs expEnglishElixirPythonSQL

Job Description

• Design and curate evaluation datasets for retrieval quality • Measure retrieval quality using metrics like Recall@k, Precision@k, MRR, and NDCG@k • Recommend data cleaning/normalization strategies • Evaluate and optimize chunking strategies • Assess embedding and re-ranking strategies • Evaluate generation quality in context • Attribute failures across the pipeline • Conduct systematic error analysis on AI/ML system outputs • Design and validate LLM-as-Judge evaluators • Build and maintain golden datasets for CI regression testing of AI pipelines • Lead or facilitate annotation workflows

Job Requirements

  • BS in Computer Science, Statistics, Data Science, Information Science, Mathematics, Engineering, or a related quantitative field (MS or PhD preferred)
  • 5+ years of experience in data science, applied ML, information retrieval, or AI evaluation roles
  • Strong product sense
  • Statistical rigor
  • Information retrieval fundamentals
  • Proficiency with Python and SQL (Experience with Elixir is a plus)
  • Hands-on experience with LLMs
  • Data labeling and annotation experience
  • Strong written and verbal communication

Benefits

  • Competitive salary
  • Equity
  • Health insurance
  • Dental insurance
  • Vision insurance

Related Job Pages

More Artificial Intelligence Jobs

Vibe Coding Web Scraping Expert (Freelance)

Mindrift

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.

Artificial Intelligence15 days ago
Part TimeRemote

This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English.Mindrift is looking for highly skilled Vibecode...

New York

Vibe Coding Web Scraping Expert (Freelance)

Mindrift

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.

Artificial Intelligence15 days ago
Part TimeRemote

This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English.Mindrift is looking for highly skilled Vibecode...

New York

Vibe Coding Web Scraping Expert (Freelance)

Mindrift

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.

Artificial Intelligence15 days ago
Part TimeRemote

This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English.Mindrift is looking for highly skilled Vibecode...

Wisconsin

Vibe Coding Web Scraping Expert (Freelance)

Mindrift

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.

Artificial Intelligence15 days ago
Part TimeRemote

This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English.Mindrift is looking for highly skilled Vibecode...

South Carolina