Weekday (YC W21)

We are a Y-Combinator-backed startup building your AI-powered Recruiter Agent

Engineering Expert – AI Systems Evaluation

Software EngineerSoftware EngineerContractRemoteTeam 11-50Since 2021H1B No SponsorCompany SiteLinkedIn

Location

United States

Posted

16 days ago

Salary

$73 / hour

Postgraduate DegreeEnglish

Job Description

• Develop and refine prompts to guide AI behavior in engineering-specific scenarios • Evaluate model-generated responses for technical correctness, applied reasoning, completeness, and practical relevance • Fact-check technical claims using authoritative public sources and domain expertise • Annotate outputs by identifying conceptual gaps, flawed assumptions, and factual inaccuracies • Assess clarity, structure, and appropriateness of explanations for various audiences • Ensure responses align with expected conversational standards and system-level guidelines • Apply structured evaluation frameworks, taxonomies, and benchmarking standards consistently

Job Requirements

  • PhD in Engineering or a closely related field
  • Deep expertise in one or more of the following domains:
  • Mechanical & Physical Systems Engineering
  • Electrical, Electronic & Computer Engineering
  • Chemical, Materials & Process Engineering
  • Civil, Environmental & Infrastructure Engineering
  • Strong familiarity with large language models (LLMs) and their practical applications
  • Excellent written communication skills with the ability to clearly explain complex technical concepts
  • High attention to detail and ability to detect subtle technical inaccuracies
  • Experience reviewing, editing, or critiquing technical or academic writing
  • Applied research, industry engineering workflows, or systems design (preferred)
  • Experience with reinforcement learning from human feedback (RLHF), model evaluation, or structured data annotation (preferred)
  • Teaching, mentoring, or explaining engineering concepts to non-expert audiences (preferred)
  • Familiarity with structured evaluation rubrics, benchmarks, or quality assurance frameworks (preferred)

Related Job Pages

More Software Engineer Jobs

Software Engineer - Solana

Rain

The Corporate Card for Web3

Software Engineer16 days ago
Full TimeRemoteTeam 1-10Since 2021H1B Sponsor

Design, build, and maintain Solana programs and backend services that integrate with Solana, EVM, and Stellar. Scale on-chain infrastructure for high-volume transactions and data ingestion, drive architecture and product roadmap decisions, and ship secure, reliable capabilities to customers.

EvmNode.jsRustSolanaStellar
New York
$140K - $240K / year

Software Engineer - Stellar

Rain

The Corporate Card for Web3

Software Engineer16 days ago
Full TimeRemoteTeam 1-10Since 2021H1B Sponsor

Design, build, and maintain Soroban smart contracts and backend services integrating Stellar, EVM, and Solana. Scale on-chain infrastructure for high-volume transactions and real-time processing, drive architecture and roadmap, and ship features iteratively.

EvmNode.jsRustSmart ContractsSolanaSorobanStellar
New York
$140K - $240K / year

Founding Engineer

Clera

Your talent agent for your entire career. Connecting you to startups backed by a16z, Index, YC & more.

Software Engineer16 days ago
Full TimeRemoteTeam 1-10Since 2025

Founding Engineer at AI-driven recruiting startup

ReactTypeScript
California

Senior Director of Engineering

ClickUp

The world's most productive AI Workspace for projects, tasks, chat, docs, and more. All software and humans - converged.

Software Engineer16 days ago
Full TimeRemoteTeam 1,001-5,000Since 2017

Senior Director of Engineering leading multiple teams at ClickUp, a productivity software company

United States
$325K - $375K / year