Refactor prompt generation and evaluation loop.

[oliverchang]

Currently our prompt generation is tied to our template format here: https://github.com/google/oss-fuzz-gen/tree/main/prompts/template_xml

We should make it easier and more flexible for others to test different prompt generation strategies, by allowing these custom prompts to be python modules instead that look something like the following:

def generate(benchmark:  Benchmark) -> str:
  ...

i.e. the module would be expected to define a generate function which produces a full prompt to pass to the LLM.

Similarly, we should also make our generation/evaluation loop more configurable, e.g. extract the logic here:

oss-fuzz-gen/run_one_experiment.py

Line 247 in 51a636b

model.prompt_path = model.prepare_generate_prompt(

into a driver.py that can be similarly replaced:

def evaluate(model: models.LLM,  benchmark: Benchmark, prompt_generator: Module):
  prompt = generator.generate(benchmark)
  targets = generate_targets(model, prompt)
  results = evaluate(targets) 
  ...

And tying this all together, the resulting invocations would look something like:

./run_all_experiments --driver /path/to/driver.py --prompt_generator prompts/custom_generator.py

[oliverchang]

@DavidKorczynski @DonggeLiu WDYT?

[DonggeLiu]

Am I correct that the driver.py is effectively orchestrating core operations like prompt_generation, target_generator, and target_evalutor, while run_all_experiments.py becomes a wrapper that invokes it?

[oliverchang]

Am I correct that the driver.py is effectively orchestrating core operations like prompt_generation, target_generator, and target_evalutor, while run_all_experiments.py becomes a wrapper that invokes it?

That's right!

[DavidKorczynski]

I think this is nice, and I think it would be super useful for researchers to focus on developing interfaces for interacting with iLLMs while OSS-Fuzz-gen provides all the facilities for e.g. doing it at scale with convenient infra, and perhaps also providing data sources.

function which produces a full prompt to pass to the LLM.

The above sentence confused me slightly, and this may be a similar question to @DonggeLiu's -- How about a scenario where the custom prompt generation want to have some logic the depends on the interaction with the LLM? Is the idea custom driver.py enables this (the concept of full prompt made me unsure if that means no custom interaction)?

[oliverchang]

The above sentence confused me slightly, and this may be a similar question to @DonggeLiu's -- How about a scenario where the custom prompt generation want to have some logic the depends on the interaction with the LLM? Is the idea custom driver.py enables this (the concept of full prompt made me unsure if that means no custom interaction)?

The intention is that can be done in custom_generator.py. It's an escape hatch where anything goes, as long as it outputs an initial prompt to feed to the llm. You could in theory bypass that as well, if you wanted to do something different in driver.py, e.g. a chat session.

Thinking more, maybe we don't even need a custom_generator.py, that can juste go into driver.py, with a default implementation available for our prompt generation method.

[vidixha]

Hi @oliverchang , building on this, I've done some research and I think DSPy would work very well here. We can make each prompt generation strategy a Module.

dspy supports caching so it would become easy to benchmark different prompt generation strategies.

Here is what that would look like:

1. generate function would be refactored to look like:

# prompts/custom_generator.py
import dspy

class CustomPromptGenerator(dspy.Module):
    def forward(self, benchmark) -> str:
        # build the prompt using benchmark
        return f"Describe vulnerabilities in the following code:\n{benchmark.code}"

2. driver.py would look like:

# driver.py
import dspy

class EvaluationDriver(dspy.Module):
    def __init__(self, model, prompt_generator):
        super().__init__()
        self.model = model
        self.prompt_generator = prompt_generator

    def forward(self, benchmark):
        prompt = self.prompt_generator(benchmark)
        output = self.model(prompt)
        return evaluate_output(output)

3. The execution script would call the Modules as following:

model = dspy.OpenAI(model='gpt-4')  # LLM
PromptGen = load_module(prompt_path, "CustomPromptGenerator")()
Driver = load_module(driver_path, "EvaluationDriver")(model, PromptGen)

[vidixha]

@oliverchang Please let me know what you think of this. I would like to contribute to this issue. Thanks!