AI Evaluation Rubric Auditor
Audit reviewers' work on AI agent shopping and delivery tasks, checking that rubric failures are fair and giving feedback to reviewers. Remote, $60-$85/hr.
Evaluator and trainer roles score model outputs against rubrics, compare responses and write the feedback that shapes how AI behaves.
67 open roles · Mercor, Micro1, Cincinnatus LLC (via Mercor). Listed pay runs from $10 to $285 per hour. Current openings include AI Evaluation Rubric Auditor, Visual Content Evaluator, AI Evaluation Rubric Reviewer, Enterprise Sales Manager – AI Training Specialist.
Each listing below links to a full description with requirements, pay and how to apply. New AI Evaluator & Trainer roles are added as they open, so check back regularly.
Audit reviewers' work on AI agent shopping and delivery tasks, checking that rubric failures are fair and giving feedback to reviewers. Remote, $60-$85/hr.
Upload original images and compare them side by side with AI-generated versions, documenting every visual flaw. Remote (US-based), $25-$45/hr.
Review AI-drafted shopping and delivery tasks, their grading rubrics and model attempts, fixing unfair criteria for an AI agent evaluation project. Remote, $40-$75/hr.
Mercor is hiring recent frontline sales managers to evaluate deals and write coaching analyses that train sales AI models. Remote (US only), part-time, $100-$125/hr.
micro1 needs everyday ChatGPT users to chat with different AI assistant versions and rate them over a short Nov 15-18 project. Remote worldwide, $40-$70/hr.
micro1 is hiring heart failure and transplant cardiologists to evaluate AI-generated clinical content on advanced cardiovascular care. Remote, $80-$285/hr.
micro1 is hiring US-based video professionals to compare AI-generated videos and write reasoning on editing, pacing and storytelling. Remote, $20-$40/hr.
Freelancers and solopreneurs critique AI-generated proposals, reports and client communications and share realistic case studies. Remote (US & Canada), $40-$120/hr.
Independent sales, marketing and strategy consultants review AI-generated business deliverables and design realistic case studies. Remote, $40-$120/hr.
Computational life scientists build hard AI-training tasks and grade model-written code and analyses, paid per accepted task. Remote worldwide, ~$60-$120/hr equivalent.
Computational physical scientists create tasks and grade AI calculations, code and simulations. Global remote, about $60-$120/hr, paid per accepted task.
Independent finance, data and engineering consultants evaluate AI-generated models, analyses and code. Remote, $60-$120/hr.
Computational earth and environmental scientists build tasks and grade AI code and analyses. Global remote, about $60-$120/hr, paid per accepted task.
Former JAG and operational law attorneys write and judge law-of-armed-conflict edge cases to train AI models. Remote, $95-$170/hr.
Micro1 seeks mathematicians, statisticians and optimization experts with AI-training experience to build and grade hard math tasks. Remote worldwide, $60-$120/hr equivalent.
Micro1 seeks machine learning engineers with prior AI-training experience to create and evaluate hard ML tasks for advanced models. Remote worldwide, $60-$120/hr equivalent.
Micro1 seeks computational engineers with prior AI-training experience to create and grade hard engineering tasks for AI models. Remote worldwide, $60-$120/hr equivalent.
Customer support and success experts audit and design realistic support workflows for AI training at a leading lab, Remote (US only), $55-$80/hr, full-time W-2.
Senior enterprise marketers design, review and grade realistic marketing tasks to train a leading AI lab's models, Remote (US only), $60-$90/hr, full-time W-2.
Licensed US clinicians write clinical cases and safety rubrics to improve a leading AI lab's models, Remote (US only), $90-$150/hr, W-2.
Finance professionals build financial models, valuations and grading rubrics to train a leading AI lab's models, Remote (US only), $80-$120/hr, W-2.
Micro1 six-week part-time contract for US-based privacy professionals to test AI agents for privacy violations. Remote, United States, $30-$60/hr (pending confirmation).
Contract role for certified softwood lumber graders to give rule-based grading feedback that trains AI models. Remote, $20-$60/hr.
Micro1 is hiring US-based Visual Content Evaluators to compare original images with AI-generated ones and document flaws. Remote, $25-$35/hr.
We'll email you when new roles match AI Evaluator & Trainer.
Free · 1 email per week · Unsubscribe anytime