A project dedicated to assessing and benchmarking advanced agentic audio models against leading systems. The program’s mission is to evaluate and optimize model performance for real-world customer support use cases.
Responsibilities

Create and execute role-play–based evaluation scenarios that simulate realistic customer service interactions across multiple domains, including:
- Flight bookings and travel support
- Financial services
- Telecommunications and technical support
Contribute to the development of diverse and representative datasets used to assess conversational audio agents.
Evaluate model performance across a standardized set of qualitative and quantitative metrics.
Ensure evaluations reflect real customer expectations for clarity, efficiency, and natural conversational flow.

Evaluation Metrics
Model performance is assessed using a combination of conversational, technical, and audio-specific criteria, including but not limited to:

Task completion accuracy and efficiency
Conversational naturalness (tone, flow, and coherence)
Audio comprehension and response quality
Instruction adherence and contextual understanding
Basic computer programming literacy, including:
- Understanding of JSON structures
- Familiarity with functions and methods
- Ability to reason about structured data and simple logic
Technical communication clarity when handling support-style problem-solving

Technical & Equipment Requirements

Strong verbal communication skills in a simulated customer support context
Access to a high-quality microphone to ensure clean, reliable audio input during evaluations
Comfort working with structured prompts, evaluation rubrics, and technical guidelines

Job title: Spanish Audio Specialist – AI Trainer
Employment type: Contract
Workplace type: Remote
Seniority level: Mid‑Senior Level

Audio Generalist (Entry-Level) - Freelance AI Trainer Project

Job Description

More Jobs at Agency

Want AI-powered job matching?