- Experience
- Any
- Salary
- USD 30 – USD 50 / hour
- Openings
- 1
- Posted
- 4 days ago
- Work mode
- Work from home
- Resume
- Required to apply
Sign in to tell us what does and doesn't work for you here — it sharpens every match we show you.
Job description
About the Role
Rex.zone seeks a Senior Data Annotator to enhance AI and ML training workflows through the creation and evaluation of precisely labeled datasets. This full-time remote role involves applying labeling guidelines and quality control to aid model training and assessment in natural language processing, computer vision, and content safety domains.
Key Responsibilities
- Perform advanced data labeling and review of text, images, and multimodal datasets.
- Carry out preference ranking and evaluation of large language models using preset rubrics.
- Conduct quality assurance assessments, audits, and calibration cycles to maintain data integrity.
- Detect shortcomings in annotation guidelines, record edge cases, and suggest improvements to rubrics.
- Oversee inter-annotator agreement, facilitating resolution of discrepancies through calibration.
- Assist in evaluating prompts for helpfulness, harmlessness, and adherence to policy.
- Engage in content safety labeling and manage sensitive-topic annotations.
- Collaborate with engineering and data operations to enhance annotation tools, increase processing efficiency, and detect errors.
Required Qualifications
- Proven experience in data annotation, labeling, or dataset quality assurance for machine learning.
- Ability to apply detailed rubrics and ensure consistency in annotation guidelines.
- Knowledge of Reinforcement Learning with Human Feedback (RLHF), preference data handling, and large language model evaluation methods.
- Strong written communication skills for reasoning on judgment tasks and documenting complex cases.
- Hands-on experience in natural language processing annotation techniques (such as named entity recognition) and/or computer vision annotation (including bounding boxes and segmentation).
Quality Assurance Methods
Data quality is maintained through a combination of audits, inter-annotator agreement evaluations, rubric adherence checks, calibration rounds, and monitoring of error rates to safeguard downstream model accuracy.
Compensation
This position offers an hourly wage ranging from $30 to $50 USD.
Level
Senior