Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix brainstorming tasksolving critic #51

Closed

Conversation

kierangilliam
Copy link
Contributor

For the brainstorming task, N-1 reviews are shown to the user when N reviews should be given.

For example, if the user asks for ideas on "building a compressed hydrogen storage station in Ohio", four critics would be assigned:

A mechanical engineer specialized in hydrogen storage and compression systems
A renewable energy specialist with expertise in integrating renewable energy sources into infrastructure projects
A safety engineer with knowledge of hydrogen safety and environmental impact assessments
An expert in regulatory frameworks and public-private partnerships to navigate the regulatory landscape and promote collaboration with industry stakeholders  

But only three reviews would be shown to the user:

Reviews:  
[A renewable energy specialist with expertise in integrating renewable energy sources into infrastructure projects]: While building a compressed hydrogen...
[A safety engineer with knowledge of hydrogen safety and environmental impact assessments]: While building a compressed ...
[An expert in regulatory frameworks and public-private partnerships to navigate the regulatory landscape and promote collaboration with industry stakeholders]: While building a compressed hydrogen storage station in Ohio is a step in the right direction...

This is because during initialization, only N - 1 agents are initialized but all N are expected:

    if i == 2 and agent_config.get("agent_type", "") == "critic":
                agent = load_agent(agent_config)
                agents[agent_type] = [
                    copy.deepcopy(agent)
                    for _ in range(task_config.get("cnt_agents", 1) - 1)
                ]

Summarizer's role and name would be overwritten by the first critic agent during DescriptionAssigner.

Output after this fix:

A civil engineer with expertise in infrastructure development and construction
A mechanical engineer specialized in hydrogen storage technologies and systems
An environmental scientist with knowledge of hydrogen safety and environmental impact assessments
An energy policy analyst familiar with regulatory frameworks and incentives for promoting hydrogen infrastructure development  
...
[Reviewer: A mechanical engineer specialized in hydrogen storage systems: This expert will have in-depth knowledge of the technical aspects of building a compressed hydrogen storage station, including the design, construction, and maintenance of the storage infrastructure]: While building a ...
[Reviewer: An environmental scientist with expertise in renewable energy: This expert will provide insights into the environmental impact of the project and help identify sustainable practices for the station's operation. They can also suggest ways to integrate renewable energy sources into the station's power supply]: While a compressed ...
[Reviewer: An urban planner with experience in infrastructure development: This expert will contribute to the planning and design of the storage station, ensuring it aligns with the existing urban infrastructure and meets the needs of the local community. They can also provide insights into zoning regulations and land use considerations]: While a compressed hydrogen storage station...
[Reviewer: A business strategist with knowledge of the hydrogen market: This expert will analyze the economic viability of the project, including market demand, potential revenue streams, and investment opportunities. They can also help develop a business plan and identify potential partnerships or funding sources]: While building a compressed hydrogen storage station in Ohio is a step in the right direction...

@chenweize1998
Copy link
Collaborator

The inconsistency between cnt_agents and the actual number of agents participating in the brainstorming is indeed a problem. While the changes in this PR address the issue, they also impact situations where the solver needs to join the discussion during the decision-making stage. We recognize that categorizing agents in the decision-making stage as one solver and multiple critics may lack generality. At the moment, we haven't thought of a better design 😕. I'm inclined to not accept this PR unless a more comprehensive and general design is suggested.

@kierangilliam
Copy link
Contributor Author

Got it - thanks for the detailed response explanation. I'll close this for now and reopen if I can meet your other concerns.

Do you all have a community space where I can ask follow up questions (Discord, Slack, etc)?

@chenweize1998
Copy link
Collaborator

Sure! We've just created a discord server https://discord.gg/cnutfCtC.

@archenovalis
Copy link

Sure! We've just created a discord server https://discord.gg/cnutfCtC.

Hi @chenweize1998 ! thank you for the link. However, the link here and in README.md has expired. Would you please update the invite code to a never expiring code?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants