Artificial Intelligence Specialist

About NCS and Role Overview

NCS is a prominent AI technology services firm with a workforce of 15,000 across the Asia Pacific. The company partners globally to deliver agile and expert AI solutions across multiple sectors, blending AI with digital resilience for impactful business results. As a subsidiary of the Singtel Group, NCS focuses on innovative AI applications.

This position seeks an AI/LLM Specialist dedicated to prompt engineering, model integration, evaluation, and upholding production standards within the AI Central's Forward Deployed Engineering framework. The goal is to convert generative AI concepts from proof of concept to production by developing effective prompts, embedding foundational models, creating automated benchmarking tools, and ensuring continuous monitoring of quality, safety, hallucination, bias, drift, latency, and cost factors.

Key Duties

• Architect, test, and refine production prompts and prompt sequences to optimize accuracy, performance, and cost-efficiency.
• Embed foundation AI models into applications using APIs and gateways, contributing advice on model and version selections alongside AI Architects.
• Assist with light model fine-tuning and instruction-based tuning techniques (including LoRA and PEFT) in partnership with AI Engineers.
• Develop and maintain evaluation frameworks featuring benchmark datasets that measure accuracy, consistency, and safety of large language and agentic systems.
• Conduct comparative benchmarking on aspects like accuracy, latency, and cost per query to support informed model selection.
• Build automated regression test suites integrated with continuous integration workflows to ensure prompt and model quality stability.
• Engage in adversarial testing and red-teaming efforts to identify vulnerabilities and failure modes proactively.
• Quantify and report on hallucination, bias, and model drift using defensible metrics rather than subjective assessment.
• Facilitate governance and compliance by providing evaluation data for production readiness reviews and maintaining documentation aligned with client audit requirements.
• In fast-paced Forward Deployed Engineering projects, rapidly prototype prompts, integrate models, and stand up evaluation systems to deliver prompt, evidence-driven decisions.
• Oversee ongoing prompt tuning and monitoring of deployed systems to detect quality decline and update internal reusable assets like prompt libraries and evaluation templates.
• Collaborate closely with AI Architects, Solution Architects, AI Engineers, and Testers to provide technical evidence, recommendations, and to ensure smooth transition of tuned models and prompts into production environments.

Candidate Profile

• Minimum of 3 years hands-on experience with large language models encompassing prompt engineering, model integration, and evaluation.
• Proven skill in designing and optimizing prompts and chains for production contexts, including API and gateway model integrations.
• Strong understanding of evaluation techniques such as accuracy metrics, hallucination and toxicity detection, human-in-the-loop processes, and A/B testing methodologies.
• Proficient in scripting, particularly Python, to create automated evaluation harnesses and integrate testing pipelines.
• Statistically literate with the ability to design representative tests and interpret results rigorously.
• Excellent written communication skills to generate clear, structured evaluation reports suitable for technical and client audiences.
• Knowledgeable of Chinese AI models and technology stacks such as DeepSeek, Qwen, GLM, Kimi, and MiniMax, including deployment and licensing considerations.

Desirable Skills

• Experience in fine-tuning/instruction-tuning with techniques like LoRA/PEFT on open-weight models.
• Familiarity with evaluation tooling such as RAGAS, DeepEval, TruLens, promptfoo, or building bespoke evaluation frameworks.
• Competence in red-teaming and adversarial testing approaches for generative AI.
• Understanding of AI governance requirements within regulated sectors (Healthcare, Government, Finance).
• Background supporting presales or solution discussions using technical evidence (without direct proposal ownership).
• Practical experience benchmarking or integrating Chinese open-weight models alongside Western counterparts.

Technical Environment

• Programming languages primarily Python and SQL.
• Tools and frameworks: LangChain, LlamaIndex, model gateways like LiteLLM, Bedrock, Azure OpenAI, and prompt-versioning utilities.
• Fine-tuning platforms: LoRA/PEFT methods and Hugging Face Transformers.
• Evaluation tooling including RAGAS, DeepEval, TruLens, promptfoo, and custom harnesses.
• LLM runtime environments: APIs from OpenAI, Azure OpenAI, Bedrock, Vertex, plus China-based models DeepSeek, Qwen, GLM.
• Data handling with Pandas, Jupyter, and BI/reporting tools for dashboard creation.
• Continuous integration via GitHub Actions or GitLab CI for automated regression evaluations.

Why Work at NCS?

• Engage with innovative AI products shaping future technology landscapes.
• Collaborate with accomplished interdisciplinary teams including research, engineering, and design experts.
• Benefit from ongoing training and career advancement opportunities.
• Translate advanced AI research into practical solutions that impact clients and end-users.
• Drive innovation within a prominent technology services organization with regional influence.
• Contribute to socially responsible, human-centric AI initiatives.
• Experience a culture emphasizing human relationships, collaboration, and inclusivity.
• Join a united team guided by values of Adventure, Excellence, Integrity, Ownership, and Unity.