[Bug]: GitHub resolver fails with 'The model produced invalid content. Consider modifying your prompt if you are seeing this error persistently.'

[erv4gen]

Is there an existing issue for the same bug?

• I have checked the existing issues.

Describe the bug and reproduction steps

GitHub resolver fails with The model produced invalid content. Consider modifying your prompt if you are seeing this error persistently. when use OpenAI o1-preview mode in both github actions and docker environments.

Variable used in the action:
LLM_MODEL=openai/o1-preview

Workflow file:

name: Resolve Issue with OpenHands

on:
  issues:
    types: [labeled]
  pull_request:
    types: [labeled]
  issue_comment:
    types: [created]
  pull_request_review_comment:
    types: [created]
  pull_request_review:
    types: [submitted]

permissions:
  contents: write
  pull-requests: write
  issues: write

jobs:
  call-openhands-resolver:
    uses: All-Hands-AI/OpenHands/.github/workflows/openhands-resolver.yml@main
    with:
      macro: ${{ vars.OPENHANDS_MACRO || '@openhands-agent' }}
      max_iterations: ${{ fromJson(vars.OPENHANDS_MAX_ITER || 50) }}
      base_container_image: ${{ vars.OPENHANDS_BASE_CONTAINER_IMAGE || '' }}
      LLM_MODEL: ${{ vars.LLM_MODEL || 'anthropic/claude-3-5-sonnet-20241022' }}
    secrets:
      PAT_TOKEN: ${{ secrets.PAT_TOKEN }}
      PAT_USERNAME: ${{ secrets.PAT_USERNAME }}
      LLM_API_KEY: ${{ secrets.LLM_API_KEY }}
      LLM_BASE_URL: ${{ secrets.LLM_BASE_URL }}

The same error occurs when I call o1 model from Docker env.

OpenHands Installation

Development workflow

OpenHands Version

0.21

Operating System

Linux

Logs, Errors, Screenshots, and Additional Context

Run cd /tmp && python -m openhands.resolver.resolve_issue \
/opt/hostedtoolcache/Python/3.12.8/x64/lib/python3.12/site-packages/pydantic/_internal/_config.py:345: UserWarning: Valid config keys have changed in V2:
* 'fields' has been removed
  warnings.warn(message, UserWarning)
20:35:06 - openhands:INFO: issue_definitions.py:175 - Limiting resolving to issues [1].
20:35:07 - openhands:INFO: resolve_issue.py:372 - Using output directory: output
Cloning into 'output/repo'...
20:[35](https://github.com/Symbolic-Link/Ticket-Planner/actions/runs/12978118512/job/36192319463#step:10:36):07 - openhands:INFO: resolve_issue.py:394 - Base commit: 965e9816deddcd3cebecfd89f[36](https://github.com/Symbolic-Link/Ticket-Planner/actions/runs/12978118512/job/36192319463#step:10:37)dd8563b9f51f4
20:35:07 - openhands:INFO: resolve_issue.py:405 - Writing output to output/output.jsonl
20:35:07 - openhands:INFO: resolve_issue.py:420 - Resolving issue 1 with Agent CodeActAgent, model o1-preview, max iterations 50.
20:35:07 - openhands:INFO: resolve_issue.py:162 - Starting fixing issue 1.
20:35:07 - openhands:INFO: docker_runtime.py:142 - [runtime 885f9b6e-6a70-4dd0-a1ea-dc46c28a8461-7d3d9cc2c51bebc3] Starting runtime with image: ghcr.io/all-hands-ai/runtime:0.21.0-nikolaik
20:36:20 - openhands:INFO: docker_runtime.py:146 - [runtime 885f9b6e-6a70-4dd0-a1ea-dc46c28a8461-7d3d9cc2c51bebc3] Container started: openhands-runtime-885f9b6e-6a70-4dd0-a1ea-dc46c28a8461-7d3d9cc2c51bebc3. VSCode URL: None
20:36:20 - openhands:INFO: docker_runtime.py:157 - [runtime 885f9b6e-6a70-4dd0-a1ea-dc46c28a8461-7d3d9cc2c51bebc3] Waiting for client to become ready at http://localhost:35458...
20:36:41 - openhands:INFO: docker_runtime.py:163 - [runtime 885f9b6e-6a70-4dd0-a1ea-dc46c28a8461-7d3d9cc2c51bebc3] Runtime is ready.
20:36:41 - openhands:INFO: resolve_issue.py:52 - ------------------------------
20:36:41 - openhands:INFO: resolve_issue.py:53 - BEGIN Runtime Completion Fn
20:36:41 - openhands:INFO: resolve_issue.py:54 - ------------------------------
20:36:41 - ACTION
**CmdRunAction (source=None, is_input=False)**
COMMAND:
cd /workspace
20:36:42 - OBSERVATION
**CmdOutputObservation (source=None, exit code=0, metadata={
  "exit_code": 0,
  "pid": -1,
  "username": "openhands",
  "hostname": "6ee5854bd99b",
  "working_dir": "/workspace",
  "py_interpreter_path": "/openhands/poetry/openhands-ai-5O4_aCHf-py3.12/bin/python",
  "prefix": "",
  "suffix": "\n[The command completed with exit code 0.]"
})**
--BEGIN AGENT OBSERVATION--

[The command completed with exit code 0.]
[Current working directory: /workspace]
[Python interpreter: /openhands/poetry/openhands-ai-5O4_aCHf-py3.12/bin/python]
[Command finished with exit code 0]
--END AGENT OBSERVATION--
20:36:42 - ACTION
**CmdRunAction (source=None, is_input=False)**
COMMAND:
git config --global core.pager ""
20:36:42 - OBSERVATION
**CmdOutputObservation (source=None, exit code=0, metadata={
  "exit_code": 0,
  "pid": -1,
  "username": "openhands",
  "hostname": "6ee5854bd99b",
  "working_dir": "/workspace",
  "py_interpreter_path": "/openhands/poetry/openhands-ai-5O4_aCHf-py3.12/bin/python",
  "prefix": "",
  "suffix": "\n[The command completed with exit code 0.]"
})**
--BEGIN AGENT OBSERVATION--

[The command completed with exit code 0.]
[Current working directory: /workspace]
[Python interpreter: /openhands/poetry/openhands-ai-5O4_aCHf-py3.12/bin/python]
[Command finished with exit code 0]
--END AGENT OBSERVATION--
20:36:42 - openhands:INFO: codeact_agent.py:94 - Function calling not enabled for model openai/o1-preview. Mocking function calling via prompting.
20:36:42 - openhands:INFO: base.py:249 - [runtime 885f9b6e-6a70-4dd0-a1ea-dc46c28a8461-7d3d9cc2c51bebc3] Selected repo: None, loading microagents from /workspace/.openhands/microagents (inside runtime)
20:36:42 - openhands:INFO: resolve_issue.py:198 - **MessageAction** (source=EventSource.USER)
CONTENT: Please fix the following issue for the repository in /workspace.
An environment has been set up for you to start working. You may assume all necessary tools are installed.

# Problem Statement
test issue

print hello world

Issue Thread Comments:
[OpenHands](https://github.com/All-Hands-AI/OpenHands) started fixing the issue! You can monitor the progress [here](https://github.com/Symbolic-Link/Ticket-Planner/actions/runs/12978118512).

IMPORTANT: You should ONLY interact with the environment provided to you AND NEVER ASK FOR HUMAN HELP.
You SHOULD INCLUDE PROPER INDENTATION in your edit commands.

For all changes to actual application code (e.g. in Python or Javascript), add an appropriate test to the testing directory to make sure that the issue has been fixed.
Run the tests, and if they pass you are done!
You do NOT need to write new tests if there are only changes to documentation or configuration files.

When you think you have fixed the issue through code changes, please call the finish action to end the interaction.
20:36:42 - openhands:INFO: resolve_issue.py:198 - NullObservation(content='', observation='null')
20:36:42 - openhands:INFO: agent_controller.py:447 - [Agent Controller default] Setting agent(CodeActAgent) state from AgentState.LOADING to AgentState.RUNNING
[Agent Controller default] LEVEL 0 LOCAL STEP 0 GLOBAL STEP 0
20:36:42 - openhands:INFO: resolve_issue.py:198 - AgentStateChangedObservation(content='', agent_state=<AgentState.RUNNING: 'running'>, observation='agent_state_changed')
20:[37](https://github.com/Symbolic-Link/Ticket-Planner/actions/runs/12978118512/job/36192319463#step:10:38):29 - openhands:ERROR: retry_mixin.py:47 - litellm.APIError: APIError: OpenAIException - Error code: 500 - {'error': {'message': 'The model produced invalid content. Consider modifying your prompt if you are seeing this error persistently.', 'type': 'model_error', 'param': None, 'code': None}}. Attempt #1 | You can customize retry values in the configuration.
20:[38](https://github.com/Symbolic-Link/Ticket-Planner/actions/runs/12978118512/job/36192319463#step:10:39):44 - openhands:ERROR: retry_mixin.py:47 - litellm.APIError: APIError: OpenAIException - Error code: 500 - {'error': {'message': 'The model produced invalid content. Consider modifying your prompt if you are seeing this error persistently.', 'type': 'model_error', 'param': None, 'code': None}}. Attempt #2 | You can customize retry values in the configuration.
20:[39](https://github.com/Symbolic-Link/Ticket-Planner/actions/runs/12978118512/job/36192319463#step:10:40):42 - openhands:ERROR: retry_mixin.py:47 - litellm.APIError: APIError: OpenAIException - Error code: 500 - {'error': {'message': 'The model produced invalid content. Consider modifying your prompt if you are seeing this error persistently.', 'type': 'model_error', 'param': None, 'code': None}}. Attempt #3 | You can customize retry values in the configuration.
20:[40](https://github.com/Symbolic-Link/Ticket-Planner/actions/runs/12978118512/job/36192319463#step:10:41):41 - openhands:ERROR: retry_mixin.py:47 - litellm.APIError: APIError: OpenAIException - Error code: 500 - {'error': {'message': 'The model produced invalid content. Consider modifying your prompt if you are seeing this error persistently.', 'type': 'model_error', 'param': None, 'code': None}}. Attempt #4 | You can customize retry values in the configuration.
20:[41](https://github.com/Symbolic-Link/Ticket-Planner/actions/runs/12978118512/job/36192319463#step:10:42):38 - openhands:ERROR: retry_mixin.py:47 - litellm.APIError: APIError: OpenAIException - Error code: 500 - {'error': {'message': 'The model produced invalid content. Consider modifying your prompt if you are seeing this error persistently.', 'type': 'model_error', 'param': None, 'code': None}}. Attempt #5 | You can customize retry values in the configuration.
20:[42](https://github.com/Symbolic-Link/Ticket-Planner/actions/runs/12978118512/job/36192319463#step:10:43):53 - openhands:ERROR: retry_mixin.py:47 - litellm.APIError: APIError: OpenAIException - Error code: 500 - {'error': {'message': 'The model produced invalid content. Consider modifying your prompt if you are seeing this error persistently.', 'type': 'model_error', 'param': None, 'code': None}}. Attempt #6 | You can customize retry values in the configuration.
20:[44](https://github.com/Symbolic-Link/Ticket-Planner/actions/runs/12978118512/job/36192319463#step:10:45):46 - openhands:ERROR: retry_mixin.py:47 - litellm.APIError: APIError: OpenAIException - Error code: 500 - {'error': {'message': 'The model produced invalid content. Consider modifying your prompt if you are seeing this error persistently.', 'type': 'model_error', 'param': None, 'code': None}}. Attempt #7 | You can customize retry values in the configuration.

[huangmiao0725]

@openhands-agent can you fix this bug?