Expert Talent Pool, Feedback and Alignment (Senior, 5+ Years)

About the Role

Join a remote expert network as a senior rater and reviewer, evaluating challenging AI model outputs and helping calibrate other graders. Your work supports the development of reliable, high-quality AI systems.

What You'll Do

• Rank and rate expert-level model answers in your domain using detailed rubrics and written rationales.

• Evaluate model reasoning step by step in specialized fields such as mathematics, code, law, medicine, or finance.

• Fact-check detailed claims against primary sources and verify accuracy.

• Assess answers against content policies, style guides, and edge cases.

• Review other raters' work, calibrate standards, and develop or refine grading guidelines.

• Assess search relevance and share feedback in live sessions.

What We're Looking For

• At least 5 years of professional experience in rating, review, editing, fact-checking, trust and safety, or deep expertise in a relevant field.

• Native or near-native written English proficiency.

• Experience grading or reviewing expert-level answers against rubrics, policies, or style guides.

• Experience leading, reviewing, or mentoring other raters or reviewers.

• An advanced degree, professional license, or deep working expertise in the domain you evaluate.

• Hands-on experience with AI data annotation, preference ranking, rubric grading, process labeling, or search-quality rating is a plus.

Compensation & Benefits

Compensation is $50 to $120 per hour, depending on project, field, and expertise. Payment is for hours worked.

Location

Fully remote. Projects typically last from one week to several months.