Search Evaluation Specialist - Expert /hr Hourly

CAPosted 11 weeks ago

Role Overview

Mercor is hiring a mid-level Search Evaluation Specialist - Expert /hr Hourly. This is a contract role in CA. Full responsibilities, required qualifications, and the apply link are listed in the description below.

Resume Keywords to Include

Make sure these keywords appear in your resume to improve ATS scoring

ORCompensationMercorAIHeadquarteredSanFranciscoBenchmark

Job Description

About The Job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Position: Search Generalist Expert

Type: Contract

Compensation: $10–$30/hour

Location: Remote

Role Responsibilities

Evaluate AI-generated search responses for factual accuracy, helpfulness, clarity, completeness, and overall quality.
Assess whether models use search appropriately and whether search queries are well-formed and effective.
Compare model responses side by side and provide concise, defensible rationales.
Write and refine prompts, golden answers, rubric criteria, and edge cases for search-related evaluations.
Apply project guidelines consistently across ambiguous, multi-step, and real-world search tasks.
Identify recurring failure modes and escalate unclear cases or rubric gaps to project leads.

Qualifications

Must-Have

Excellent written English and strong online research skills.
Strong judgment when synthesizing information from multiple sources.
Ability to distinguish factual accuracy from fluency, confidence, or style.
High attention to detail and comfort following structured guidelines.
Reliable, self-directed, and responsive in an asynchronous remote environment.

Preferred

Experience in search quality, fact-checking, content evaluation, trust and safety, annotation, QA, or prompt/rubric writing.
Familiarity with search evaluation concepts such as factuality, helpfulness, severity, side-by-side comparisons, or tool-use assessment.
Experience working with LLM evaluation workflows or human data projects.
Multilingual skills are a plus.
Bachelor’s degree preferred; advanced degree or strong professional background is a plus.

Application Process (Takes 20–30 mins to complete)

Upload resume
AI interview based on your resume
Submit form

Resources & Support

For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
For any help or support, reach out to: support@mercor.com

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

About Mercor

Mercor

mercor.com

On-site

163 other open roles at Mercor on TryApplyNow.

View all 164 roles at Mercor Visit website

Frequently Asked Questions

How do I apply for the Search Evaluation Specialist - Expert /hr Hourly position at Mercor?

Use the Apply button above to submit your application directly to Mercor. Most applications take less than 5 minutes if your resume and contact details are ready, and you'll be routed to the employer's official application system to finish.

Where is the Search Evaluation Specialist - Expert /hr Hourly position at Mercor located?

This position is based in CA. Mercor has not indicated remote or hybrid options for this role, so candidates should plan for on-site work.

What does a Search Evaluation Specialist - Expert /hr Hourly at Mercor earn?

Mercor has not disclosed a salary range in this posting. Many employers share specifics later in the interview process; you can also ask during a recruiter screen if compensation transparency is important to you.

When was the Search Evaluation Specialist - Expert /hr Hourly role at Mercor posted?

This role was posted on April 5, 2026 (83 days ago). It's still listed as actively hiring; we re-confirm openings against the source system multiple times per day and remove closed roles.