Machine Learning Engineer - Pre-Training Data

About Cohere

Cohere is a pioneering enterprise AI company focused on security-first principles. We create advanced foundational AI models and comprehensive products designed to solve practical business challenges. Our mission is to accelerate AI adoption by providing powerful language models to enterprises worldwide. Our team consists of dedicated researchers, engineers, and designers committed to enhancing model capabilities and delivering value to customers. Headquartered in Toronto, we operate globally with offices in major cities including Montreal, London, New York, San Francisco, Paris, Berlin, and Seoul.

Role Overview

As a Machine Learning Engineer specializing in pre-training data, you will develop and refine data pipelines that fuel Cohere's sophisticated language models. Your responsibilities include performing data ablation studies to gauge data quality and creating optimized pre-training data blends to boost model efficiency. By integrating research with engineering practices, you will contribute directly to enhancing key training factors such as throughput and accelerator usage. This position supports our objective to deliver powerful, efficient natural language understanding and generation technologies.

Key Duties

• Analyze and perform data ablations to evaluate and improve data quality.
• Design and implement advanced data modeling methods to prepare datasets that maximize training performance.
• Innovate and apply novel data curation strategies utilizing Cohere's infrastructure to advance NLP capabilities.
• Collaborate with multidisciplinary teams including researchers and engineers to maintain high-quality, scalable data pipelines that align with modern language models' needs.

Qualifications

• Proficient software engineering expertise, especially in Python and data pipeline development.
• Understanding of curriculum learning, data mixture techniques, and data attribution methodologies.
• Experience with large-scale data processing frameworks like Apache Spark, Apache Beam, Pandas, or equivalents.
• Familiarity with handling extensive datasets comprising web content, code repositories, and multilingual corpora.
• Capability to assess data quality metrics and experiment with various data combinations.
• Strong enthusiasm for merging research insights with engineering solutions to address complex AI training data challenges.
• Desirable: Published research in top-tier conferences or journals such as NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, or EMNLP.

Work Location & Environment

This role offers flexibility with remote work options or presence at any of Cohere’s offices in Toronto, London, Paris, San Francisco, or New York. We prioritize hiring top talent regardless of location but recommend alignment within EST to EU time zones for collaboration.

Compensation and Benefits

• Competitive salary range of CA$250,000 to CA$535,000.
• Weekly lunch stipend equivalent to $75 (or local currency).
• Comprehensive health and dental coverage, including dedicated mental health support budget.
• Retirement plans such as RRSP matching, 401K, or equivalent pension schemes.
• Full parental leave top-up for up to six months for any parent.
• Annual enrichment allocations covering arts, fitness, wellness, quality time, workspace enhancement, and continuous education, including conferences and coaching.
• Six weeks (30 working days) of paid vacation annually.
• Travel allowances for visiting other offices and participation in company-wide offsite events.
• Remote employees receive a $500 home office setup stipend.

Culture & Inclusion

Cohere fosters an inclusive and supportive environment, welcoming individuals from diverse backgrounds. We offer reasonable accommodations throughout the recruitment process to ensure equal opportunity. We utilize AI tools for applicant screening to identify suitable candidates but maintain comprehensive human review. Beware of fraudulent communications pretending to represent Cohere.