• Create and execute role-play–based evaluation scenarios that simulate realistic customer service interactions across multiple domains, including:
  • Flight bookings and travel support
  • Financial services
  • Telecommunications and technical support
• Contribute to the development of diverse and representative datasets used to assess conversational audio agents.
• Evaluate model performance across a standardized set of qualitative and quantitative metrics.
• Ensure evaluations reflect real customer expectations for clarity, efficiency, and natural conversational flow.

• Task completion accuracy and efficiency
• Conversational naturalness (tone, flow, and coherence)
• Audio comprehension and response quality
• Instruction adherence and contextual understanding
• Basic computer programming literacy, including:
  • Understanding of JSON structures
  • Familiarity with functions and methods
  • Ability to reason about structured data and simple logic
• Technical communication clarity when handling support-style problem-solving

• Strong verbal communication skills in a simulated customer support context
• English proficiency including fluency across all language skills: reading, listening, writing, and speaking.
• Access to a high-quality microphone to ensure clean, reliable audio input during evaluations
• Comfort working with structured prompts, evaluation rubrics, and technical guidelines
• Device capable of running audio recording software and opening large technical documentation

We offer a pay range of $11-to- $30.65 per hour, with the exact rate determined after evaluating your experience, expertise, and geographic location. Final offer amounts may vary from the pay range listed above. As a contractor you'll supply a secure computer and high‑speed internet; company‑sponsored benefits such as health insurance and PTO do not apply.

Employment type: Contract
Workplace type: Remote
Seniority level: Mid‑Senior Level