Staff Research Engineer - Large Language Model Pre-Training

About JetBrains and the Role

At JetBrains, our mission revolves around crafting some of the world's most robust and efficient developer tools. Since our inception in 2000, we focus on automating common development tasks to accelerate software production, enabling developers to innovate and create freely.

We are currently developing an advanced AI platform that integrates seamlessly with all JetBrains products. This platform leverages proprietary models specialized in coding and text assistance, combined with partnerships to enhance capabilities.

Position Overview

We seek a Staff Research Engineer to contribute to the foundational training of large language models (LLMs) tailored to coding tasks. The successful candidate will engage in building LLMs from their ground up and deploying them into live environments to serve a global user base.

Key Responsibilities

• Collaborate with stakeholders to translate business needs into detailed technical requirements.
• Manage the training of LLMs on large-scale GPU clusters from scratch.
• Assemble and curate datasets for pre-training and fine-tuning phases.
• Maintain, support, and enhance existing subsystems involved in the AI platform.

Desired Skills and Experience

• Hands-on experience designing, deploying, and supporting machine learning systems in production environments.
• Strong foundational knowledge in natural language processing and transformer architectures.
• Proficiency in deep learning frameworks such as PyTorch and usage of prominent NLP libraries.
• Expertise in distributed training for models with billions of parameters.
• Detail-oriented approach and excellent communication skills.

Preferred Qualifications

• Experience with LLM inference tools like vLLM, DeepSpeed, and TensorRT.
• Knowledge of LLM alignment techniques such as reinforcement learning with human feedback (RLHF) or human AI feedback (RLAIF).
• Familiarity with machine learning operations (MLOps) including CI/CD pipelines for ML workflows.
• Working knowledge of Kubernetes (K8s) and Kubeflow platforms.
• Contributions to scientific research publications in the NLP domain.

Development Environment at JetBrains AI

• Utilization of a multi-hundred NVIDIA GPU cluster for training workloads.
• Source control managed with Git.
• ML stack consisting of Python, PyTorch, and HuggingFace libraries.
• Experiment tracking with Kubeflow and Weights & Biases tools.
• Continuous integration automated through TeamCity.

Our Values and Equal Opportunities

We embrace independent decision-making and project ownership, aiming for simplicity in solutions while progressively tackling complexity. We recognize the importance of continuous learning, especially in a fast-evolving field like LLMs.

JetBrains is proud to be an equal opportunity employer committed to fostering an inclusive workplace welcoming individuals from all backgrounds, identities, and abilities.

Data Privacy Note

Your application data will be handled according to our Recruitment Privacy Policy.