Senior Master Annotator

About Skild AI

Skild AI focuses on developing a pioneering robotics intelligence system that is versatile and reliable in adapting to new scenarios. By leveraging large-scale, data-driven machine learning, Skild AI aims to enable broad deployment of robots across society. The team has a diverse range of members from recent graduates to domain specialists, valuing skills and mindset over experience.

Position Summary

The Senior Master Annotator will take the lead in defining, upholding, and improving annotation criteria for robotic datasets involving various data types including image, video, 3D, and audio. This role serves as the authoritative source for labeling standards and is responsible for Inter-Annotator Agreement (IAA) implementation, annotator training and calibration, ensuring high annotation quality necessary for training robotic models.

Key Responsibilities

• Establish and maintain high-quality gold-standard annotated datasets for both new and ongoing projects.
• Set clear criteria for acceptance, quality thresholds, and tolerance specific to different annotation tasks.
• Create, update, and manage comprehensive annotation guidelines and SOPs, incorporating findings from disagreement analysis and quality reviews.
• Implement and monitor Inter-Annotator Agreement methodologies such as Cohen’s and Fleiss’ kappa, generating periodic quality reports.
• Investigate patterns of annotation disagreements, diagnose root causes, and recommend improvements based on data insights.
• Lead the onboarding process and continuous training sessions including calibration for annotation staff.
• Develop resources like edge-case repositories and example libraries to enhance annotation uniformity.
• Provide constructive feedback and retrain annotators to align with gold standards.
• Perform precision-driven quality control audits aligned with task requirements and approve pilot annotation batches before production rollout.
• Handle escalated complex annotation cases and collaborate with QA and Operations teams to refine sampling and quality assurance techniques.

Preferred Experience and Skills

• Between 4 to 7+ years in data annotation, quality control, or video labeling roles.
• Proficient with temporal video annotation and experience with datasets from robotics, computer vision, or autonomous systems is advantageous.
• In-depth knowledge of IAA techniques, quality metrics, sampling tactics, and gold set generation processes.
• Experience in training and mentoring annotators to enhance consistency and precision.
• Strong analytical capabilities with proficiency in tools like Excel or Google Sheets; familiarity with Python is a plus but not mandatory.
• Capability to convert vague SOPs into clear, executable annotation instructions.
• Exceptional detail orientation and decisive judgment in handling ambiguous visual data.
• Possession of a methodical and accountable approach driven by data and process improvement.
• Excellent communication skills both written and oral.