Senior Software Engineer, AI Operations, GPS

Role Overview

As a Senior Software Engineer in AI Operations at Scale AI, you will be responsible for maintaining the performance, stability, and long-term technical health of AI systems implemented across critical public sector partnerships. While the Delivery Teams are focused on launching new AI applications, you will be the technical guardian ensuring operational excellence in these deployments. This role requires bridging software engineering, machine learning operations (MLOps), and client governance by managing tiered service-level agreements (SLAs), monitoring model drift, and executing maintenance procedures that safeguard both system stability and operational efficiency.

Key Responsibilities

• Serve as the technical gatekeeper during the handover process from Delivery to Maintenance, performing comprehensive reviews to validate baseline code quality, prompt accuracy, and architecture documentation before formal sign-off.
• Manage incident response and uphold resolution targets across multi-tiered SLA models—ranging from essential business-hours support to 24/7 mission-critical services. Lead incident governance initiatives, conduct root cause analyses, and oversee priority mitigations within defined support windows.
• Oversee AI lifecycle operations including monitoring production model latency, performance metrics, and data drift. Maintain prompt configuration repositories to ensure consistent model behavior and carry out regression testing when large language model (LLM) providers update their endpoints.
• Define operational boundaries by classifying incoming client requests, distinguishing between routine maintenance tasks and out-of-scope system evolution projects. Benchmark new AI models comparatively as part of this assessment.
• Reduce operational workload by designing automated solutions such as self-healing data pipelines, automated retrieval-augmented generation (RAG) index synchronization, and telemetry tools. Advocate for upstream Delivery teams to adopt architecture practices that simplify long-term maintenance.
• Act as the senior technical liaison for government and enterprise IT stakeholders, translating complex AI concepts like data drift, prompt versioning, and API deprecation into clear business implications for non-technical audiences.

Required Qualifications and Experience

• Minimum 5 years of experience in Software Engineering, MLOps, Site Reliability Engineering (SRE), or Forward Deployed Engineering focused on heavy data or production AI environments.
• Proficient in Python, SQL, REST and gRPC API integration, and cloud platforms such as AWS, Azure, or Google Cloud Platform (GCP).
• Hands-on experience with MLOps tools, vector databases, and orchestration frameworks for LLMs (e.g., LangChain, LlamaIndex).
• Strong expertise in AI governance practices including prompt version control, model benchmarking using evaluation datasets, RAG pipeline mechanics, and data drift monitoring.
• Engineering-oriented approach emphasizing systematic automated solutions over temporary fixes, with a solid understanding of continuous integration and continuous deployment (CI/CD) for machine learning pipelines.
• Excellent technical communication skills, adept at managing client expectations, enforcing operational boundaries between maintenance and system evolution, and guiding long-term system development strategies.

Additional Information

Applicants based in Qatar must possess applicable residency and work permits issued by Qatari authorities. Personal data including residency status and nationality will be collected solely for visa processing and immigration compliance and will not influence hiring decisions unfairly. Visa issuance is subject to government discretion, with Scale AI assisting successful candidates in the process. Candidates must observe a 90-day waiting period before reapplying to the same position to ensure fair evaluation.

About Scale AI

Scale AI develops dependable artificial intelligence solutions to support critical decisions worldwide. Our technologies deliver high-quality data and full-stack AI toolkits that empower top organizations—including global enterprises and government agencies—to build, deploy, and manage impactful AI applications. We champion workplace inclusion and equal opportunity, providing reasonable accommodations to applicants with disabilities. Our data privacy policies align with applicable laws, and we adhere to all transparency and labor regulations.