Google has created a new framework called Project Naptime, which allows a large language model (LLM) to conduct vulnerability research and enhance automated discovery methods.

According to Google Project Zero researchers Sergei Glazunov and Mark Brand, the Naptime architecture revolves around the interaction between an AI agent and a target codebase. The agent is equipped with a collection of specialized tools specifically designed to replicate the workflow of a human security researcher.

You might be interested in: Google Launches AI-Enhanced Security for Android Phones

The initiative is titled “Naptime” because it enables humans to engage in regular periods of rest while it aids in vulnerability research and automates variant analysis.

The fundamental objective of this technique is to leverage the advancements made in code understanding and the general reasoning capabilities of LLMs. This enables them to imitate human behavior in terms of detecting and showcasing security vulnerabilities.

The system includes various elements, such as a Code Browser tool that allows the AI agent to navigate through the target codebase, a Python tool for running Python scripts in a controlled environment for fuzzing, a Debugger tool for observing program behavior with different inputs, and a Reporter tool for monitoring task progress.

According to CYBERSECEVAL 2 benchmarks, Google stated that Naptime is both model-agnostic and backend-agnostic. Additionally, it is more effective at identifying buffer overflow and advanced memory corruption issues. CYBERSECEVAL 2, which was launched in April by researchers from Meta, is a suite designed to assess and measure the security threats associated with LLM (Low-Level Machine) systems.

During the tests conducted by the search giant to replicate and take advantage of the weaknesses, the two categories of vulnerabilities received higher scores, reaching 1.00 and 0.76, compared to their previous ratings of 0.05 and 0.24, respectively, for OpenAI GPT-4 Turbo.

The researchers stated that Naptime allows an LLM to conduct vulnerability research in a manner that closely emulates the iterative and hypothesis-driven process of human security experts. This architectural design not only improves the agent’s capacity to detect and analyze vulnerabilities but also ensures that the outcomes are precise and can be replicated.