Software Engineer - AI Platform / LLM Gateway

About the Role

We are seeking a Software Engineer to contribute to our AI gateway product, Routero.ai, aimed at enabling enterprise clients to deploy AI workloads with security and efficiency. This product emphasizes large language model (LLM) API integration, comprehensive audit trails, private and VPC deployments, cost management, high reliability, and overall readiness for enterprise use. As part of a small, experienced engineering team, you will engage directly in building and scaling the platform primarily using Python, TypeScript, and React, considering Go for certain scalability components.

Key Responsibilities

• Develop backend and platform capabilities for the AI gateway and LLM routing solution.
• Implement API services, manage access controls, maintain audit logging, and develop deployment and monitoring tools.
• Ensure support for private cloud, VPC, and potential on-premises deployment configurations.
• Enhance the platform's scalability, reliability, and readiness via load testing and performance optimization.
• Take full ownership of technical features within a lean, senior engineering group.

Required Qualifications

• Proven experience in backend or platform software engineering roles.
• Proficient in Python and/or TypeScript, with experience or interest in React being advantageous.
• Familiarity with cloud infrastructure environments, API development, distributed systems, and tools such as Kubernetes, Helm, or Terraform.
• Strong grasp of system reliability, scalability, testing strategies, and performance optimization considerations.
• Interest or background in AI infrastructure, large language model applications, API gateways, or developer platform technologies.
• Ability to operate independently and coordinate with a geographically distributed team primarily aligned with GMT+8 time zone.

Additional Qualifications (Preferred)

• Experience working with LLM APIs, AI gateways, intelligent agents, retrieval-augmented generation (RAG), model routing, or AI infrastructure components.
• Background in supporting enterprise clients with private deployment solutions or compliance and security-related technical requirements.