Legal Rubric & Ground Truth Auditor (Licensed Attorney)

Mercor is seeking experienced licensed attorneys to audit rubrics and ground-truth answers for a legal research dataset used to evaluate advanced AI systems. You will review existing legal research questions, verify the accuracy of model answers, and refine grading rubrics so the dataset reflects current law and sound legal reasoning. This is a short-term, fully remote engagement.

Responsibilities

• Audit grading rubrics for accuracy, completeness, consistency, and clarity.

• Verify ground-truth answers against controlling statutes, regulations, and case law.

• Identify legal errors, outdated authority, missing issues, and ambiguous grading criteria.

• Rewrite or correct ground-truth answers and rubric items where needed.

• Provide concise, structured written feedback on each item reviewed.

Practice Areas

The dataset spans eight practice areas. Experience in one or more is required, and breadth across several is a plus:

• Administrative / Regulatory

• Business & Commercial

• Civil Litigation

• Constitutional / Civil Rights

• Criminal

• Family

• Health

• Immigration

Required Qualifications

• Juris Doctor (JD) and active U.S. bar admission in good standing.

• 5+ years of legal practice.

• Substantive experience in at least one of the practice areas above.

• Strong legal research, analysis, and writing skills.

• High attention to detail and the ability to apply consistent judgment.

Preferred Qualifications

• Experience across multiple listed practice areas.

• Judicial clerkship, appellate, government, or law firm research experience.

• Prior experience reviewing or grading legal work product, or with AI / legal-tech evaluation projects.

Why Join?

• Apply your legal expertise to shape how AI systems are evaluated for legal research.

• Collaborate with a leading AI research organisation.