TypeError when running benchmark_code_python.py with Llama-3.1-8B-Instruct and aux-model Qwen2.5-0.5B-Instruct

[gauravjain14]

Traceback (most recent call last):
  File "/disk1/universal_assisted_generation/huggingface-demos/experiments/faster_generation/benchmark_code_python.py", line 64, in <module>
    new_outputs = run_model_with_assistant(args, AutoTokenizer, AutoModelForCausalLM, run_prediction_loop)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/disk1/universal_assisted_generation/huggingface-demos/experiments/faster_generation/utils.py", line 139, in run_model_with_assistant
    new_outputs = run_prediction_loop(
                  ^^^^^^^^^^^^^^^^^^^^
  File "/disk1/universal_assisted_generation/huggingface-demos/experiments/faster_generation/benchmark_code_python.py", line 48, in run_prediction_loop
    gen_out = model.generate(**inputs, **generate_kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/disk1/universal_assisted_generation/transformers/src/transformers/generation/utils.py", line 2027, in generate
    generation_config, model_kwargs = self._prepare_generation_config(generation_config, **kwargs)
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/disk1/universal_assisted_generation/transformers/src/transformers/generation/utils.py", line 1563, in _prepare_generation_config
    model_kwargs = generation_config.update(**kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/disk1/universal_assisted_generation/transformers/src/transformers/generation/configuration_utils.py", line 1325, in update
    self.validate()
  File "/disk1/universal_assisted_generation/transformers/src/transformers/generation/configuration_utils.py", line 605, in validate
    if self.pad_token_id is not None and self.pad_token_id < 0:
                                         ^^^^^^^^^^^^^^^^^^^^^
TypeError: '<' not supported between instances of 'list' and 'int'

I have slightly modified the benchmark_code_python.py to run the tau/scrolls dataset instead of bigcode/the-stack

    # ds = load_dataset("bigcode/the-stack", data_dir="data/python", split="train", streaming=True)
    ds = load_dataset("tau/scrolls", 'qasper', split="test")
    ds_iterator = iter(ds.take(num_samples))

    desc = "ORIGINAL model" if assistant_model is None else f"ASSISTED model"
    pbar = tqdm(range(num_samples), desc)
    for i in pbar:
        # next_data = next(ds_iterator)["content"]
        # Commented `content` for `tau/scrolls` dataset
        next_data = next(ds_iterator)["input"]
        inputs = tokenizer([next_data], return_tensors="pt", max_length=INPUT_LEN, truncation=True)
        inputs = inputs.to(TORCH_DEVICE)

Transformers version - Version: 4.48.0.dev0
Repo/Branch - https://github.com/keyboardAnt/transformers/tree/usd
Command to repro -

python benchmark_code_python.py meta-llama/Llama-3.1-8B-Instruct --aux-model Qwen/Qwen2.5-0.5B-Instruct --dtype fp16```