ML Systems Software Engineer

Role Overview

Join as an ML Systems Software Engineer working at the intersection where software models meet silicon hardware—optimizing scheduler decisions, memory management, and kernel execution paths crucial for efficient silicon computing.

Key Responsibilities

• Develop and enhance serving pipelines focusing on batching, memory handling, caching strategies, and scheduling to efficiently manage concurrent operations.
• Perform comprehensive profiling to accurately identify performance bottlenecks instead of relying on assumptions.
• Optimize resource sharing of accelerators among multiple models, accommodating hardware variations across different silicon generations and capacities.
• Engage closely with runtime components including drivers, kernels, memory allocation, and telemetry systems to improve system visibility and behavior.
• Create and maintain hardware-based measurement frameworks to ensure results are grounded in actual device performance.
• Transform conceptual hypotheses into benchmarks and advance them to production-ready implementations.

Candidate Requirements

• A minimum of four years’ professional experience in systems or machine learning infrastructure engineering.
• Proficiency in Python and one systems programming language like C++ or Rust.
• Demonstrable experience optimizing ML inference or training throughput with a clear understanding of performance gains.
• Strong grasp of accelerator memory architectures, kernel execution patterns, and time distribution within processes.
• Comfort working on bare-metal systems without reliance on managed services.

Additional Desirable Skills

• Experience with CUDA, ROCm, Triton, or similar technologies involving kernel-level programming.
• Contributions to projects such as vLLM, TensorRT-LLM, SGLang, or equivalents.
• Background in distributed serving systems and multi-accelerator workload partitioning.
• Published work in benchmarking or systems engineering.

What We Provide

• Opportunity to be part of one of the pioneering deep tech firms in the region, innovating foundational technology internally.
• Meaningful ownership and influence in a startup environment.
• Competitive remuneration suited for early-stage market entrants.
• Close collaboration within a small, senior expert team.
• Engagement with complex problems combining hardware, systems software, and AI on a large scale.