Staff Software Engineer - Machine Learning TPU Workload Optimization

Overview

Join Google as a Staff Software Engineer focusing on Machine Learning workload optimization for TPUs at our Singapore location. This role prioritizes candidates with valid work authorization in Singapore and no visa sponsorship requirement.

Role and Responsibilities

• Lead the onboarding of training and serving workloads for the latest TPU NPIs, ensuring a stable software stack for customers.
• Define the technical strategy and architect frameworks to optimize ML training and inference solutions tailored to customer requirements.
• Collaborate extensively with ML research teams, performance specialists, and model optimization tooling groups to enhance system capabilities.
• Deliver scalable, high-performance solutions enabling large-scale reference models and advanced single-host and multi-host inference setups.

Required Qualifications

• Bachelor’s degree or equivalent experience in a relevant field.
• At least 8 years of software engineering experience.
• Minimum 4 years leading machine learning design and infrastructure optimization including deployment, evaluation, data processing, debugging, and fine tuning.
• At least 2 years hands-on experience with advanced training tools such as Megatron-LM and DeepSpeed, and inference technologies including TensorRT-LLM, vLLM, and SGLang.

Preferred Qualifications

• Master’s degree or PhD in Engineering, Computer Science, or related technical discipline.
• Proven expertise optimizing ML models specifically for large-scale training and inference workloads.
• Experience applying diverse ML optimization methods to improve latency and throughput metrics.
• Familiarity with hardware accelerators like TPUs or GPUs, or experience with high-performance computing environments.

About the Team and Impact

You will be part of Google’s AI Engine, focusing on building infrastructure and frameworks supporting machine learning models on Cloud GPUs and TPUs. This dynamic and innovative team helps external customers onboard and optimize their ML workloads, influencing industry-leading cloud AI capabilities.

The AI and Infrastructure group is central to powering Google and its customers with cutting-edge AI services and scalable infrastructure, contributing to products like Vertex AI, Google Global Networking, and advanced TPU development.

Equal Employment Opportunity

Google maintains a firm commitment to diversity, equity, and inclusion. We provide equal employment opportunities regardless of race, color, ancestry, religion, gender, national origin, sexual orientation, age, citizenship, disability, veteran status, or other protected classifications. Accommodation requests relating to disabilities can be submitted via a support form.