We are a Y-Combinator-backed startup building your AI-powered Recruiter Agent
Generalist Evaluator Expert
Location
United States
Posted
20 days ago
Salary
Not specified
Job Description
This role is for one of our clients
Compensation: $35-$40 per hour
We are seeking detail-oriented writing professionals to contribute to a high-impact AI research initiative in collaboration with a leading research lab. In this role, you will develop high-quality prompt–golden answer pairs used to train and evaluate advanced language models.
This is a short-term, flexible opportunity ideal for individuals with strong academic foundations and exceptional clarity in written communication. The role is well-suited for professionals who enjoy translating complex ideas into structured, precise, and easy-to-understand content.
Job Requirements
- Key Responsibilities
- Design and Optimize Prompts: Develop detailed, constraint-rich prompts with clear instructions and multiple requirements
- Define Evaluation Standards: Establish expectations for high-quality responses in general consumer contexts and create comprehensive grading rubrics
- Model Testing and Assessment: Execute prompts using AI systems and evaluate outputs against defined standards
- Benchmarking & Quality Assurance: Collaborate in QA processes to ensure prompt tasks and rubrics meet high standards of rigor, clarity, and consistency before inclusion in benchmarking workflows
- Maintain structured documentation and adhere to project guidelines
- Minimum Qualifications
- Bachelor’s degree (BS or BA) from a reputable institution (completed or in progress)
- Strong writing, analytical, and critical thinking skills
- Ability to work independently and meet structured deadlines
- Meaningful familiarity with ChatGPT or similar AI tools for personal, academic, or professional use
- Must be based in the United States or Canada
- Preferred Qualifications
- Experience in teaching, curriculum design, academic research, or structured evaluation
- Experience developing grading rubrics or assessment frameworks
- Project Details
- Start: Immediate
- Duration: Approximately 2 months
- Commitment: Minimum 20 hours per week
- Fully remote with flexible scheduling
- Structured project environment with defined goals, workflows, and tools
- Application & Onboarding Process
- Complete a short AI-led interview (approximately 15 minutes)
- Complete a 45-minute written assessment focused on rubric development
- Selected candidates will receive project onboarding instructions
- Contract & Payment Terms
- Engagement will be structured as an independent contractor agreement
- Work can be completed remotely on your own schedule
- Projects may be extended, shortened, or concluded early based on performance and evolving project needs
- Assignments will not require access to confidential or proprietary information from any employer, client, or institution
- Payments are processed weekly via Stripe or Wise based on services rendered
- Visa sponsorship is not available; H1-B and STEM OPT candidates cannot be supported at this time
Related Guides
Related Job Pages
More Full-stack Engineer Jobs
Software Engineering Expert
Weekday (YC W21)We are a Y-Combinator-backed startup building your AI-powered Recruiter Agent
This role is for one of our clientsCompensation: $50-$150 per hourWe are seeking experienced Software Engineering professionals to contribute to high-impact research collaborations with leading AI laboratories. In this role, you will help enhance AI sy...
Special Projects Software Engineers
Weekday (YC W21)We are a Y-Combinator-backed startup building your AI-powered Recruiter Agent
This role is for one of our clientsCompensation: $100-$200 per hourWe are inviting select, highly capable software engineers to participate in specialized project-based engagements. This opportunity is designed for engineers who enjoy tackling unique, ...
Software Developer
WixWix is the comprehensive platform that gives you total creative freedom online.
Software Developer training the team on Splunk applications and solutions
Lead development of illumination and solar panel/array products: translate requirements into designs, perform mechanical/electrical/reliability/EMC analyses, generate CAD and manufacturing documentation, qualify products, support suppliers and manufacturing, and drive corrective actions and continuous improvement in a fast-paced aerospace manufacturing environment.
