Large Model Algorithm Researcher (Multimodal & Code AI) - Soaring Star Talent Program

Overview

The AI Innovation Center is dedicated to advancing AI infrastructure and pioneering cutting-edge research in artificial intelligence. Our focus areas include large language models (LLMs) and multimodal large models that can interpret multilingual content and extensive video data, enhancing user content consumption experiences. Through the Code AI sector, we utilize the robust code understanding and reasoning capabilities of LLMs to improve software performance and research & development efficiency.

Project Details

Multimodal foundational large models (VLMs) are a key area of industry research and crucial for practical business applications. In 2024, the Innovation Center launched VFM V1, a multimodal large model optimized for TikTok’s business needs. This model matches the performance of the leading open-source model Qwen VL on public datasets and surpasses other foundational models on internal business evaluation sets. Our future objectives include continuous development of foundational models with efficient perception and reasoning abilities, adept at multilingual and large-scale video content understanding, to improve end-user content experiences.

Key Challenges

• Optimizing the multimodal perception encoder: current encoders operate at a fixed frame rate; exploring adaptive, more effective frame rates and integrating additional modalities such as audio and user behavior is critical.
• Developing stronger combined perception and cognitive function by fusing multimodal sensing with advanced reasoning capacities.

Required Qualifications

• PhD degree with strong preference for candidates with published research in machine learning, computer vision, or natural language processing.
• Outstanding programming skills in C/C++ or Python, with strong knowledge of data structures and algorithms; achievements in competitions like ACM/ICPC, NOI/IOI, TopCoder, or Kaggle are advantageous.
• Research experience in machine learning emphasizing large-scale language modeling and generative AI.

Preferred Attributes

• A strong passion for technology and problem-solving, excellent analytical and communication skills, and a collaborative team spirit.

About ByteDance

Founded in 2012, ByteDance aims to inspire creativity and enrich lives globally. With a diversified portfolio of more than a dozen products including TikTok, Lemon8, CapCut, and Pico, along with region-specific platforms like Toutiao and Douyin, ByteDance facilitates creative expression, content discovery, and social connectivity.

Our Culture

ByteDance fosters an environment of continuous innovation driven by curiosity, humility, and a commitment to impactful work. We embrace an "Always Day 1" mindset, promoting growth and breakthroughs together as a community.

Diversity and Inclusion

We are committed to fostering an inclusive workplace that celebrates diverse perspectives and values individual skills and experiences. Our goal is to support a culture reflective of the global communities we serve while inspiring creativity and enriching life.