Remote Data Labeling Specialist

Overview

Join Rex.zone's team supporting AI and machine learning development by providing data labeling, evaluation, and quality assurance services. This full-time remote role based in Canada focuses on annotating datasets for large language models and computer vision applications, ensuring high standards for training data quality through strict guideline adherence.

Key Responsibilities

• Generate precise annotations for natural language processing (NLP) and computer vision datasets
• Conduct ranking and pairwise preference judgments as part of Reinforcement Learning from Human Feedback (RLHF) tasks for large language model (LLM) training
• Evaluate prompts and score model outputs using rubrics
• Tag entities and apply taxonomy labels in text data
• Perform content safety labeling with explanations
• Execute quality assurance evaluations, including audits, cross-verifications, and error analyses
• Monitor compliance with annotation guidelines and suggest improvements
• Report unclear cases and participate in calibration meetings to enhance agreement among annotators

Required Qualifications

• Professional background in data annotation, labeling, quality assurance evaluation, or trust and safety roles
• Exceptional attention to detail and capability to follow established annotation protocols
• Experience working with web-based annotation platforms and spreadsheets
• Ability to clearly justify labeling decisions using rubrics and rationales
• Familiarity with NLP techniques including named entity recognition and text classification
• Availability to work full-time remotely with stable internet access

Preferred Experience

• Involvement in RLHF workflows, large language model evaluations, and prompt assessment
• Knowledge of computer vision annotation methods such as bounding boxes, polygons, segmentation masks, and keypoints
• Understanding of metrics related to training data quality including accuracy, consistency, coverage, and bias
• Prior experience in content safety labeling and interpreting policies
• Collaboration with AI research labs, tech startups, business process outsourcers, or annotation service providers

Tools and Workflows

Work will involve using labeling platforms, internal quality dashboards, and documentation of guidelines. Tasks include gold-standard calibration, blind reviews, agreement checks among annotators, and structured error analyses aimed at enhancing model performance for deployed AI systems.

Additional Information

This opportunity is remote, full-time, and targeted at Canada-based candidates. Candidates should prepare to provide examples of past annotation work, QA experience, and any exposure to RLHF or large language model evaluations during the screening process.