VLM Engineer

About Technology Innovation Institute

Technology Innovation Institute (TII), located in Abu Dhabi, UAE, is a publicly funded research organization composed of scientists, engineers, mathematicians, and researchers internationally. TII focuses on advancing society through pioneering research and innovative technology prototypes.

Artificial Intelligence and Digital Research Centre

Within TII's AI Research Center, the Extreme-Scale Language Model team works on state-of-the-art deep learning techniques spanning Natural Language Processing to Perception and Vision. Having developed the Falcon models, the team is progressing further in advanced applied large language model research.

Role Overview and Key Responsibilities

• Perform thorough vision model ablation studies, evaluating the effects of diverse components and configurations. Collaborate closely with researchers to interpret and report on architecture and setting impacts.
• Conduct data ablation analyses in partnership with team members to identify optimal types and structures of training data for vision-language models, emphasizing vision-language alignment.
• Design and apply stringent evaluation methods for vision-language models, measuring performance across multiple benchmarks and real-world use cases.
• Participate actively in model training and fine-tuning, focusing on integrating large language models with vision models such as CLIP.

Required Skills and Expertise

• Advanced knowledge in machine learning, especially in vision-language models and large language models.
• Deep understanding of model architectures like CLIP relevant to vision-language tasks.
• Proficiency with distributed training methods and optimization techniques for multi-GPU environments.
• Experience using deep learning frameworks such as PyTorch.
• Strong analytical abilities for conducting ablation studies and assessing model outcomes.
• Familiarity with curating and preprocessing datasets for combined vision and language tasks.

Qualifications

• PhD in deep learning or related fields.
• Demonstrated research and development experience in vision-language modeling.
• A strong publication record at leading conferences is highly preferred.

About TII’s Mission

TII aims to tackle complex societal challenges through rigorous scientific discovery, utilizing cutting-edge infrastructure and collaboration with global institutions. It strives for disruptive innovations in AI and advanced technology domains including robotics, cryptography, quantum computing, and secure systems.