Brainscape

Learning & productivity hacks for the world’s smartest humans.

AI Prompt Engineer

ContractRemoteTeam 11-50H1B No SponsorCompany SiteLinkedIn

Location

New York

Posted

14 days ago

Salary

$40 - $100 / hour

Bachelor Degree1 yr expExperience acceptedEnglishPython

Job Description

• Migrate and test existing bulk flashcard creation prompts in an updated AI environment with newer GPT models - and plan future migrations as OpenAI retires older models • Run test suites and manually review AI outputs for quality and correctness (fine-tune prompts) • Analyze real user data to identify failure patterns and inform prompt improvements • Streamline testing and evaluation workflows to make QA faster and more repeatable • Monitor production quality post-launch and detect regressions as underlying models shift • Build and maintain model evaluation datasets from real user inputs across all AI features • Write new test cases for edge cases, multilingual content, and messy real-world inputs • Document prompt changes, test results, and lessons learned • Work with the Content Team to apply flashcard authoring quality standards

Job Requirements

  • 1+ years hands-on prompt engineering experience with LLMs / OpenAI API (systematic testing and iteration, not just casual ChatGPT usage)
  • Familiarity with Cursor IDE or similar AI-assisted development tools (our work is primarily Python - Cursor experience is more important than raw Python skill)
  • Some experience with Git version control and collaborating via shared repositories (we use GitLab)
  • A habit of documenting what you tried, what worked, and why - you don't need a formal QA background, but you naturally keep track of your process
  • Clear written communication skills
  • Proactive attitude; ability to work independently and manage your own time
  • BONUS: Experience building prompt evals, AI quality assurance, or using GPT to grade GPT outputs
  • BONUS: Experience with regression testing for AI systems or detecting model drift
  • BONUS: Background in education technology (EdTech) or content creation - especially microlearning, flashcards, or other concise Q&A formats
  • BONUS: A degree in Computer Science, Information Science, or a similar field

Related Job Pages