Senior Data Labeling Specialist

Position Overview

Rex.zone is inviting applications for a Senior Data Labeling Specialist to lead intricate annotation and evaluation projects. The aim is to enhance the quality of training datasets and improve large language model assessment. Although based in Hamburg, Germany, this full-time role is fully remote.

Core Duties

• Perform and review annotations in natural language processing, computer vision, and content safety areas.
• Conduct quality assurance evaluations, resolve annotation disagreements, and boost inter-annotator consistency.
• Engage in reinforcement learning from human feedback (RLHF) tasks, including preference ranking with accompanying rationale aligned to specific rubrics.
• Evaluate prompts on criteria such as helpfulness, safety, and adherence to instructions.
• Ensure training data quality by applying sampling methodologies, error classification, and root-cause analyses.
• Adhere to and suggest improvements for annotation guidelines, especially for uncommon or complex cases.
• Work closely with engineering and data operations teams to optimize annotation tools, throughput, and traceability.
• Keep detailed documentation to maintain reproducibility in large language model training workflows.

Required Experience and Skills

• Substantial mid-senior level background in data annotation, labeling, or AI evaluation roles.
• Expertise in upholding annotation guideline compliance and best practices in training data quality.
• Direct experience with RLHF processes such as preference ranking and rubric-based scoring systems.
• Solid understanding of NLP techniques including named entity recognition and taxonomy structuring.
• Knowledge of computer vision annotation standards and related quality assurance.
• Practical experience in content safety labeling, interpreting policies, and managing escalation workflows.
• Strong communication skills to clarify complex edge cases and write coherent evaluation rationales.
• Able to perform efficiently in a fully remote setup, collaborating with diverse cross-functional teams and achieving full-time workload requirements.

Compensation

The base hourly rate ranges between 30 to 50 US dollars.