Site Reliability Engineer, AI Inference

About F5 and the Role

F5 is dedicated to enhancing the digital world by helping organizations create, secure, and operate applications. Committed to cybersecurity and innovation, F5 values a diverse community where individuals thrive. The AI Inference Engineer position is crucial in the AI lifecycle, focused on optimizing Large Language Models (LLMs) for inference across varied environments—from GPU-intensive data centers to constrained edge devices—aiming to maximize throughput, minimize latency, and preserve accuracy.

Responsibilities

• Develop and sustain reliable inference engines utilizing technologies such as vLLM, Text Generation Inference (TGI), and NVIDIA Triton to deliver scalable, high-performance AI serving.
• Implement deployment strategies to ensure AI solutions exhibit low latency for diverse business needs.
• Profile and fine-tune models for advanced hardware platforms, including NVIDIA GPUs using CUDA and TensorRT, Apple Silicon with CoreML, and AI accelerators such as TPUs and LPUs.
• Collaborate with hardware teams to enhance computational resource utilization and performance across multiple systems.
• Architect and deploy auto-scaling infrastructures for both real-time and batch inference pipelines, leveraging Kubernetes for routing and orchestration.
• Optimize software to maintain peak performance and reliability during periods of traffic surges.
• Develop comprehensive observability frameworks monitoring metrics like Time to First Token (TTFT), token throughput, and memory bandwidth against SLAs.
• Design and run performance and load testing procedures to detect bottlenecks and uphold system resilience at scale.

Technical Expectations

• Proficiency in programming languages such as Python, C++, Rust, or Golang to support high-performance AI workflows.
• Hands-on experience with inference tools including vLLM, TensorRT, Llama.cpp, and Ollama for the development and tuning of inference engines.
• Strong knowledge of infrastructure technologies such as Docker, Kubernetes, and cloud services like AWS, Google Cloud Platform, and Microsoft Azure.
• Expertise in hardware optimization techniques for GPUs and AI accelerators, including profiling and performance enhancement on NVIDIA GPUs and TPUs.

Preferred Qualifications

• Experience deploying Large Language Models using advanced methods such as Speculative Decoding and PagedAttention.
• Involvement in open-source projects related to inference libraries or low-level kernel development, including CUDA or Triton kernels.
• Background in MLOps or Site Reliability Engineering focusing on high-performance AI deployments and managing reliability during demand fluctuations.
• Skill in designing scalable, high-throughput inference solutions tailored for handling traffic spikes.

Key Performance Indicators

• Consistently reduce inference latency, focusing on minimizing Time to First Token and maximizing token processing speed.
• Enhance cost efficiency by optimizing resource usage and hardware performance to lower token processing expenses.
• Maintain system resilience and performance stability during peak traffic loads and across diverse environments.
• Maximize throughput by deploying models with optimized hardware utilization and process efficiency.

Why Join F5?

• Opportunity to engage with state-of-the-art AI technologies and hardware supporting real-time applications.
• Work in a dynamic, collaborative environment that fosters innovation, scalability, and complex problem-solving.
• Contribute to transformative AI projects ensuring reliability and efficiency for global customers.
• Advance your career through involvement with sophisticated MLOps solutions that enable enterprise AI scalability.

Success Criteria

• Apply technical knowledge and problem-solving abilities to build low-latency, scalable AI prediction systems.
• Collaborate effectively with interdisciplinary teams, sharing insights and refining systems.
• Drive continuous optimization across hardware, tooling, and orchestration, balancing urgent needs with long-term planning.
• Translate complex AI and inference processes into effective solutions aligned with company goals.

Additional Information

The job description provided outlines general duties and requirements; these may evolve over time. F5 strictly communicates only via official F5 email addresses or authorized Workday notifications. The company is an equal opportunity employer committed to fair and inclusive hiring practices, providing accommodations on request during the application process.