[Question] Relationships between instructions, prompt, handoff, and input_type/output_type in openai-agent-sdk

[hkjoker23]

Please read this first

• Have you read the docs?Agents SDK docs
• Have you searched for related issues? Others may have had similar requests

Question

Describe your question. Provide details if available.

Hi, I have a few questions regarding how openai-agent-sdk handles instructions, prompts, handoff, and input/output types for Agents.

1. Relationship between instructions and prompt in Agent
   Suppose I have code like the following:

agent_config = {
    "instructions": "You are a helpful assistant that can answer questions about computers.",
    "model": "gpt-4o-mini",
    "prompt": {
        "id": ${openai_platform_dashboard_prompt_id}
    }
}

test_agent = Agent(
    name="test_agent",
    instructions=agent_config["instructions"],
    model=agent_config["model"],
    prompt=agent_config["prompt"], # type: ignore
)

When both instructions and a prompt_id (from the OpenAI platform dashboard) are provided, which one actually applies to the Agent? If both are applied, which takes precedence in the behavior of the agent?

2. Handoff Mechanism
   I am also curious about how handoff works.
   Suppose I have the following scenario:

A_agent = Agent(
    instructions="You are a helpful assistant that can answer questions about computer hardware.",
    model="gpt-4o-mini",
)

B_agent = Agent(
    instructions="You are a helpful assistant that can answer questions about computer programming languages.",
    model="gpt-4o-mini",
)

C_agent = Agent(
    instructions="You are a helpful assistant that can answer questions about computers.",
    model="gpt-4o-mini",
    handoff=[A_agent, B_agent]
)

In this situation, how does C_agent delegate incoming inputs to A_agent or B_agent?
How do agents “know” their specialty or usage after being created, and how does the workflow determine (based on a user’s natural language input) which agent should handle the request?
I’d like to understand the detailed workflow for agent handoff and assignment.

3. About input_type and output_type
   I understand that you can specify an output_type for structured outputs, as in this example:

class CalendarEvent(BaseModel):
    name: str
    date: str
    participants: list[str]

agent = Agent(
    name="Calendar extractor",
    instructions="Extract calendar events from text",
    output_type=CalendarEvent,
)

Is it also possible to specify an input_type so that I can define a structured input schema for the agent?
I would like the agent to analyze the input according to this schema and produce a corresponding structured output.
However, I noticed that when using handoff together, the structured output sometimes does not work as expected.
Are there any known limitations or recommended practices regarding input_type with handoff?

I would appreciate any clarifications or explanations regarding these three topics.
Thank you for your help!

[seratch]

Thanks for asking the questions.

1. When both instructions and a prompt_id (from the OpenAI platform dashboard) are provided, which one actually applies to the Agent? If both are applied, which takes precedence in the behavior of the agent?

If you pass both, the instructions (aka system prompt) part would be overwritten by the passed instructions. If you pass only prompt, the system prompt provided by the saved prompt data would be used.

2. In this situation, how does C_agent delegate incoming inputs to A_agent or B_agent? How do agents “know” their specialty or usage after being created, and how does the workflow determine (based on a user’s natural language input) which agent should handle the request?

Reading this document page is helpful. Setting clear metadata (name, description for the handoff tools) plus using the recommended prompt prefix should ensure the stable handoff behavior.

3. Is it also possible to specify an input_type so that I can define a structured input schema for the agent? I would like the agent to analyze the input according to this schema and produce a corresponding structured output. However, I noticed that when using handoff together, the structured output sometimes does not work as expected. Are there any known limitations or recommended practices regarding input_type with handoff?

Agents are supposed to be accept OpenAI input type data only, so you cannot define other types of inputs for them. For handling a specific type of inputs, handoff chain of agents may not be the best solution not only for the input type restrictions but also in terms of predictable request handling.

I don't have any elegant suggestions here, but one possible approach could be dispatching requests in a different way than handoffs, and directly calling the agent with a text message input, which has JSON data. Then, your agent can have system prompt like "You will receive the following JSON data format in the input, and you're tasked to do ..." or something like that. This is not the same with the input type definition for tools, but it could be one possible way of implementing such.

I hope this helps.