AI Quality Analyst - Personalization (Hindi, Remote)

Role Overview

We are seeking an AI Quality Analyst specializing in Personalization to remotely evaluate an advanced AI model's ability to generate personalized and contextually relevant responses using personal data. The analyst will focus on how the AI integrates information from previous conversations, email, search, and video activity to produce accurate outputs.

Key Responsibilities

• Create and refine multi-turn prompts rooted in personal experiences to rigorously test the AI model's personalization skills.
• Assess AI-generated replies for grounding accuracy, integration quality, helpfulness, and consistency.
• Detect incorrect personalization, faulty inferences, or forced narrative links within model outputs.
• Perform comparative analyses of side-by-side model responses to identify subtle differences and ensure response reliability.
• Deliver comprehensive feedback on the AI's ability to align its outputs with user-specific context and intentions.

Required Qualifications

• Fluency in Hindi, with strong reading and writing capabilities.
• Experience evaluating AI-generated content, especially in personalization scenarios.
• Skill in creating innovative and contextually meaningful prompts for AI testing.
• High attention to detail to analyze nuanced or ambiguous AI behaviors.
• Willingness to use a personal Google account with enabled data sources for authentic assessment.
• Understanding of AI personalization, grounding concepts, and user intent alignment.
• Familiarity with analyzing multi-turn conversations and evaluating AI responses.
• Awareness of data privacy considerations related to handling personal user information.
• Capability to work full-time with flexible hours to support a 24-hour global operation.

Additional Information

This role provides the opportunity to contribute significantly to a pioneering AI system employed by major enterprises, aiming to ensure dependable and impactful personalization. The employer promotes equal opportunity hiring and welcomes all qualified applicants regardless of background, focusing solely on skills and qualifications.