Data Analyst, AI Agent Analytics — New Graduate

About the Role

Jobright.ai offers an innovative AI job search agent designed to transform the traditional job-hunting process into a streamlined, expert-supported experience. As a new graduate Data Analyst specializing in AI Agent Analytics, you will evaluate agent performance metrics, analyze user behavior, and convert complicated interaction data into useful insights that enhance AI functionalities.

Why Work Here

• Contribute to live AI agents actively used by end-users
• Hold significant responsibility with meaningful impact
• Engage at the crossroads of artificial intelligence, intelligent agents, and product development
• Influence how users experience AI-powered job search

Key Responsibilities

• Develop and define key performance indicators for agent task success, quality, response time, consistency, and cost efficiency
• Investigate agent dialogues, tool usage patterns, error occurrences, and feedback from users
• Create and maintain SQL queries, dashboards, and automated reporting systems
• Spot patterns in system failures and propose solutions for improvement
• Assist in experimental trials involving prompt design, model adjustments, tool integration, and workflow optimization
• Communicate data-driven recommendations to artificial intelligence and product development teams

Required Qualifications

• Recent graduate degree in Data Science, Statistics, Computer Science, Mathematics, Economics, or equivalent disciplines
• Proficient in SQL with working knowledge of Python
• Fundamental understanding of statistical methods, experimental design, and data visualization
• Strong analytical reasoning coupled with effective communication abilities
• Capable of performing efficiently in a remote work setup

Preferred Skills

• Experience analyzing large language models (LLMs), chatbots, or intelligent agent interactions
• Familiarity with product analytics platforms and event tracking systems
• Practical knowledge of pandas, Tableau, Looker, or related analytical tools
• Interest in AI quality assessment, prompt engineering, or human-AI interaction research