CyberSecurity Benchmark Engineer

About Lookup IT Solutions

Lookup IT Solutions Pvt. Ltd. is a premier technology and consulting firm specializing in cutting-edge solutions across Artificial Intelligence, Cloud Computing, Software Engineering, Data Analytics, and Digital Transformation. We are dedicated to assisting businesses in developing scalable, secure, and forward-thinking technology solutions, all while cultivating an environment of innovation, excellence, and continuous professional development.

Role Overview

We are seeking experienced cybersecurity professionals to contribute to the development of a benchmark designed to assess the cybersecurity capabilities of advanced AI systems. The ideal candidate will be adept at working with Large Language Models (LLMs) to conceptualize and validate complex security challenges that current AI systems find difficult to overcome. A strong foundation in cybersecurity, coupled with regular use of AI tools in your professional capacity, will make you a prime candidate for this role.

Key Responsibilities

• Conceptualize and develop cybersecurity tasks and challenges that push the boundaries of frontier LLMs and AI agents.
• Investigate real-world security vulnerabilities and translate these findings into precisely defined AI evaluation tasks.
• Employ LLM agents to rigorously test designed tasks, meticulously documenting failure points and their underlying causes.
• Articulate security scenarios clearly and concisely, suitable for use as prompts for AI systems.
• Ensure that tasks possess correct and reproducible solutions that can be reliably verified.
• Collaborate with the research team to iteratively refine tasks and enhance the overall quality of the benchmark.

Required Skills

Cybersecurity

• A minimum of 3 years of experience in a cybersecurity domain, such as application security, network security, penetration testing, vulnerability research, Capture The Flag (CTF) competitions, or equivalent.
• Demonstrated ability to comprehend and articulate the mechanics of security vulnerabilities and attack vectors.
• Proficiency in reviewing code and identifying security flaws across prevalent programming languages.

Working with AI / LLMs

• Substantial hands-on experience utilizing LLMs (e.g., ChatGPT, Claude, Gemini) for technical applications.
• A keen ability to accurately assess whether an AI system has successfully resolved a given problem.
• Foundational familiarity with LLM APIs or AI-assisted workflow methodologies.