Knowledge Expert — Human Communication & Context Elicitation

In this role, you'll review conversations between participants and AI models, evaluate whether sufficient context was gathered before recommendations were made, identify missed opportunities for clarification, and provide structured feedback that helps improve the quality, reasoning, and conversational abilities of next-generation AI systems.

COMPENSATION

• Up to $28 per completed task (average task time: ~87.5 minutes)

• Pay-per-task compensation model

• Up to 40 hours per week

• At least 4 hours of overlap with Pacific Time (PST)

• Fully remote

• 4-week contractor engagement

• Immediate start available

WHAT WE'RE LOOKING FOR

We're seeking subject matter experts with one of the following qualifications:

• MSW with LCSW (or equivalent social work licensure)

• MA/MS with LMFT, LPC, LCPC, or equivalent counseling licensure

• PhD or PsyD in Clinical, Counseling, Social, or Industrial-Organizational Psychology

• MA or PhD in Human-Computer Interaction (HCI), Communication, Anthropology, Sociology, Conflict Resolution, Qualitative Research, or a closely related field

• A relevant professional credential combined with 5+ years of direct advisory, coaching, consulting, mediation, or client-facing experience

Successful candidates will also demonstrate:

• Strong analytical thinking and qualitative evaluation skills

• Excellent attention to detail and the ability to consistently apply detailed evaluation guidelines

• Comfort working with spreadsheets, annotation platforms, or data labeling tools

• Strong written communication and documentation skills

• The ability to evaluate nuanced human conversations objectively and consistently

• Experience making evidence-based judgments across ambiguous or complex scenarios

PREFERRED QUALIFICATIONS

While not required, experience in any of the following is a plus:

• Conversation analysis or discourse analysis

• AI evaluation, RLHF, or human preference labeling

• Qualitative research coding or thematic analysis

• Coaching, mediation, or conflict resolution frameworks

• Human-computer interaction (HCI) or conversational UX research

• Developing evaluation rubrics, annotation guidelines, or quality assurance processes

KEY RESPONSIBILITIES

• Review AI conversations involving real-world interpersonal and everyday decision-making scenarios

• Identify missing context and recommend relevant follow-up questions

• Evaluate whether additional information would have changed the AI's recommendations

• Distinguish between critical ("load-bearing") and non-essential information

• Conduct qualitative reviews and document findings using structured evaluation guidelines

• Collaborate with research teams to ensure consistent, high-quality assessments

SELECTION CRITERIA

Candidates whose profiles are selected will complete:

• Interest Check Form

• Delivery review

CONTRACT DETAILS

• Positions available: 10

• Employment type: Contractor assignment (no medical/paid leave)

• Duration: Short-term contract (4 weeks)

• Start Date: August 12th

• Commitment: 40 hours per week, with at least 4 hours overlap with PST

• Location: Remote — open to candidates based in Bangladesh, Brazil, Colombia, Egypt, Ghana, India, Pakistan, Indonesia, Turkey, and Vietnam

Please Note

• A MacBook is ideal for completing this work.

• Candidates should be comfortable working with spreadsheets, annotation tools, or labeling platforms as part of daily task delivery.