Senior Data Labeling Specialist

About the Role

Rex.zone is recruiting a Mid-Senior Data Labeling Specialist for a fully remote, full-time position focused on enhancing the quality of training datasets and the evaluation of large language models (LLMs). While the job listing notes Hamburg as a location, the role is explicitly remote, supporting candidates seeking advanced data labeling opportunities.

Key Responsibilities

• Perform and critically review labeling tasks covering natural language processing (NLP), computer vision, and content safety domains.
• Conduct quality assurance by evaluating labels, mediating discrepancies, and improving agreement among annotators.
• Carry out Reinforcement Learning from Human Feedback (RLHF) preference ranking and labeling, including writing rationale aligned with established rubrics.
• Assess prompts for their helpfulness, harmlessness, and adherence to instructions.
• Check training data quality through sampling methods, error classification, and root cause analysis.
• Ensure adherence to annotation guidelines and suggest guideline improvements for unusual cases.
• Work closely with engineering and data operations teams to enhance annotation tools, workflow throughput, and audit processes.
• Keep detailed records of decision-making to facilitate reproducible LLM training procedures.

Required Qualifications

• Proven mid to senior-level experience in data labeling, annotation, or AI evaluation roles.
• Deep understanding of annotation standards compliance and methods for assuring training data quality.
• Direct experience operating RLHF processes such as preference ranking and rubric-based scoring.
• Knowledge of NLP tasks including named entity recognition and taxonomy development.
• Experience working with computer vision annotation formats and performing quality assessments.
• Skill in handling content safety labeling, interpreting related policies, and managing escalation processes.
• Strong written communication skills to clearly document edge cases and logical rationales.
• Ability to work effectively in a remote environment alongside cross-functional teams, maintaining full-time productivity.

Compensation

The base hourly pay ranges from $30 to $50 USD.