-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Sweep: Auto Summarize Working Context #23
Comments
🚀 Here's the PR! #24See Sweep's progress at the progress dashboard! ⚡ Sweep Basic Tier: I'm using GPT-4. You have 5 GPT-4 tickets left for the month and 3 for the day. (tracking ID:
1ba266c898 )For more GPT-4 tickets, visit our payment portal. For a one week free trial, try Sweep Pro (unlimited GPT-4 tickets). Tip I can email you next time I complete a pull request if you set up your email here! Actions (click)
Sandbox Execution ✓Here are the sandbox execution logs prior to making any changes: Sandbox logs for
|
async def update_context(self): | |
ctx = self.working_context.get_context() | |
print("Working Context: ", ctx) | |
prompt = f""" | |
You are monitoring a conversation between an engineer and their AI Assistant. | |
Your mission is to manage the working memory for the AI Assistant. | |
You do this by adding information to the working context (short-term memory) based on the conversation history. | |
## Guidelines | |
- Your insertions should be short, concise, and relevant to the future of the conversation. | |
- Keep track of facts, ideas, and concepts that are important to the conversation. | |
- Monitor the personality of the person you're speaking with and adjust your responses accordingly. | |
- Keep track of things that the user appeared to like or dislike. | |
- In your thoughts, justify why you are adding or removing information from the working context. | |
You can see the current working context below. | |
Working Context: | |
{ctx} | |
Please make any updates accordingly. Be sure the think step by step as you work. | |
""" | |
messages = [ | |
{"role": item["role"], "content": item["content"]} | |
for item in self.get_messages() | |
] | |
for message in messages: | |
if message["role"] == "system": | |
message["content"] = prompt | |
print(messages) | |
update = await self.working_context.client.chat.completions.create( | |
model="gpt-4-1106-preview", | |
response_model=ContextUpdate, | |
messages=messages, | |
) | |
print(update) | |
self.working_context = update.execute(self.working_context) | |
self.prompt_handler.set_system() |
GPT-CodeApp/backend/memory/memory_manager.py
Lines 15 to 54 in 08b3592
class WorkingContext: | |
def __init__(self, db_connection, project_directory) -> None: | |
"""Initializes the WorkingContext class. | |
Args: | |
db_connection: The database connection object. | |
project_directory: The path to the project directory. | |
Attributes: | |
context: The working context string. | |
conn: The database connection. | |
cur: The database cursor. | |
client: The OpenAI API client. | |
project_directory: The project directory path. | |
""" | |
self.context = "The user is named Joe" | |
self.conn = db_connection | |
self.cur = self.conn.cursor() | |
self.client = CLIENT | |
self.project_directory = project_directory | |
self.create_tables() | |
def create_tables(self) -> None: | |
try: | |
self.cur.execute( | |
""" | |
CREATE TABLE IF NOT EXISTS working_context | |
( | |
context TEXT, | |
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, | |
project_directory TEXT | |
); | |
""" | |
) | |
except Exception as e: | |
print("Failed to create tables: ", str(e)) | |
return | |
GPT-CodeApp/backend/tests/test_memory_manager.py
Lines 85 to 118 in 08b3592
def test_get_context(self): | |
# Arrange: Prepare the context to be added | |
messages = [ | |
{ | |
"role": "system", | |
"content": "You are an AI Pair Programmer and a world class python developer helping the Human work on a project.", | |
}, | |
{ | |
"role": "user", | |
"content": "Hello! My name is John. I am a software engineer working at Google. I am working on a project to build a new search engine.", | |
}, | |
{ | |
"role": "assistant", | |
"content": "Hello John! I am an AI Pair Programmer and a world class python developer helping you work on your project.", | |
}, | |
] | |
self.memory_manager.get_messages = Mock(return_value=messages) | |
self.memory_manager.working_context.get_context = Mock( | |
return_value="test_context" | |
) | |
# Act: Call the get_context method | |
context = self.memory_manager.working_context.get_context() | |
# Assert: Verify that a database insert command was executed | |
self.memory_manager.working_context.get_context.assert_called() | |
self.assertIn("test_context", context) | |
# def test_update_context(self): | |
# messages = [ | |
# { | |
# "role": "system", | |
# "content": "You are an AI Pair Programmer and a world class python developer helping the Human work on a project.", | |
# }, | |
# { |
GPT-CodeApp/backend/agent/coding_agent.py
Lines 59 to 120 in 08b3592
class CodingAgent: | |
""" | |
A class to represent a coding agent that uses OpenAI's GPT-3 model to generate code. | |
Attributes: | |
memory_manager (MemoryManager): Manages the memory of the agent. | |
functions (Optional[List[dict]]): A list of functions that the agent can call. | |
callables (Optional[List[Callable]]): A list of callable functions. | |
GPT_MODEL (str): The GPT-3 model used by the agent. | |
function_map (dict): A dictionary mapping function names to their callable objects. | |
""" | |
def __init__( | |
self, | |
memory_manager, | |
function_map: Optional[dict] = None, | |
codebase: Optional[MyCodebase] = None, | |
): | |
""" | |
Constructs all the necessary attributes for the CodingAgent object. | |
Args: | |
memory_manager (MemoryManager): Manages the memory of the agent. | |
functions (Optional[List[dict]]): A list of functions that the agent can call. | |
callables (Optional[List[Callable]]): A list of callable functions. | |
""" | |
self.memory_manager = memory_manager | |
self.function_map = function_map | |
self.GPT_MODEL = None | |
self.codebase = codebase | |
self.max_tokens = 4000 | |
self.temperature = 0.75 | |
self.tool_choice = "auto" | |
self.function_to_call = None | |
self.ops_to_execute = [] | |
self.client = instructor.patch(OpenAI()) | |
if function_map: | |
self.tools = [ | |
{"type": "function", "function": op.openai_schema} | |
for op in self.function_map[0].values() | |
] | |
else: | |
self.tools = None | |
def query(self, input: str, command: Optional[str] = None) -> List[str]: | |
""" | |
Queries the GPT-3 model with the given input and command. | |
Args: | |
input (str): The input text to be processed by the GPT-3 model. | |
command (Optional[str]): The command to be executed by the agent. | |
Returns: | |
List[str]: The output generated by the GPT-3 model. | |
""" | |
print(f"Input Text: {input}\nCommand: {command}") | |
self.memory_manager.add_message("user", input) | |
message_history = [ | |
{"role": i["role"], "content": i["content"]} | |
for i in self.memory_manager.get_messages() |
Step 2: ⌨️ Coding
Modify backend/memory/memory_manager.py with contents:
• Import the summarize method from gensim at the top of the file: `from gensim.summarize import summarize`.
• Add a new attribute to the WorkingContext class to keep track of the turns. Initialize it to 0 in the __init__ method: `self.turn_counter = 0`.
• Add a new method to the WorkingContext class called summarize_context. This method will call the gensim summarize method on the context attribute and return the result: ``` def summarize_context(self): return summarize(self.context) ```
• In the update_context method, increment the turn_counter by 1 each time the method is called: `self.turn_counter += 1`.
• Still in the update_context method, check if the turn_counter is equal to 5. If it is, call the summarize_context method, assign the result to the context attribute, and reset the turn_counter to 0: ``` if self.turn_counter == 5: self.context = self.summarize_context() self.turn_counter = 0 ```--- +++ @@ -9,6 +9,7 @@ from instructor import OpenAISchema from openai import OpenAI, AsyncOpenAI import logging +from gensim.summarize import summarize CLIENT = instructor.patch(AsyncOpenAI()) @@ -34,6 +35,7 @@ self.cur = self.conn.cursor() self.client = CLIENT self.project_directory = project_directory + self.turn_counter = 0 self.create_tables() def create_tables(self) -> None: @@ -92,6 +94,9 @@ (context, self.project_directory), ) self.conn.commit() + + def summarize_context(self): + return summarize(self.context) def __str__(self) -> str: return self.context @@ -321,6 +326,10 @@ async def update_context(self): ctx = self.working_context.get_context() + self.turn_counter += 1 + if self.turn_counter == 5: + self.context = self.summarize_context() + self.turn_counter = 0 print("Working Context: ", ctx) prompt = f""" You are monitoring a conversation between an engineer and their AI Assistant.
- Running GitHub Actions for
backend/memory/memory_manager.py
✓ Edit
Check backend/memory/memory_manager.py with contents:Ran GitHub Actions for b11cd11459272b993c369412c13f17cf0877806a:
Modify backend/tests/test_memory_manager.py with contents:
• Add a new test method to test the summarize_context method. This method will create a long context, call the summarize_context method, and assert that the returned summary is shorter than the original context.
• Add a new test method to test the turn_counter attribute. This method will call the update_context method 5 times and assert that the turn_counter is reset to 0 after the 5th call.--- +++ @@ -68,6 +68,24 @@ self.cursor.fetchall.return_value = [("test_context",)] self.memory_manager = MemoryManager(db_connection=self.conn) + def test_summarize_context_shortens_long_context(self): + # Arrange: Create a context longer than what summarize would return + long_context = 'A very long repetitive context ' * 30 # assuming summarization would shorten this + self.memory_manager.working_context.context = long_context + # Act: Call summarize_context method + summary = self.memory_manager.working_context.summarize_context() + # Assert: Check that the summary is shorter than the original context + assert len(summary) < len(long_context), 'Summarize did not shorten the context' + + def test_turn_counter_reset_after_5th_update(self): + # Arrange: Reset the turn counter to 0 + self.memory_manager.working_context.turn_counter = 0 + # Act: Call the update_context method 5 times + for _ in range(5): + asyncio.run(self.memory_manager.update_context()) + # Assert: Check whether the turn_counter is reset to 0 + assert self.memory_manager.working_context.turn_counter == 0, 'Turn counter was not reset after 5th update' + def test_add_message(self): # Arrange: Prepare the message to be added role = "user"
- Running GitHub Actions for
backend/tests/test_memory_manager.py
✓ Edit
Check backend/tests/test_memory_manager.py with contents:Ran GitHub Actions for 459d4cf4b6896fb17bc275efaa5e919603af55ad:
Modify backend/agent/coding_agent.py with contents:
• In the query method, after the line where the add_message method is called, check if the turn_counter of the memory_manager's working_context is 0. If it is, print the summarized context: ``` if self.memory_manager.working_context.turn_counter == 0: print("Summarized Context: ", self.memory_manager.working_context.context) ```--- +++ @@ -114,6 +114,9 @@ """ print(f"Input Text: {input}\nCommand: {command}") self.memory_manager.add_message("user", input) + + if self.memory_manager.working_context.turn_counter == 0: + print("Summarized Context: ", self.memory_manager.working_context.context) message_history = [ {"role": i["role"], "content": i["content"]}
- Running GitHub Actions for
backend/agent/coding_agent.py
✓ Edit
Check backend/agent/coding_agent.py with contents:Ran GitHub Actions for 1e0a34a20233a2c8b5293f950db716e9dea2a755:
Step 3: 🔁 Code Review
I have finished reviewing the code for completeness. I did not find errors for sweep/auto_summarize_working_context
.
🎉 Latest improvements to Sweep:
- We just released a dashboard to track Sweep's progress on your issue in real-time, showing every stage of the process – from search to planning and coding.
- Sweep uses OpenAI's latest Assistant API to plan code changes and modify code! This is 3x faster and significantly more reliable as it allows Sweep to edit code and validate the changes in tight iterations, the same way as a human would.
- Try using the GitHub issues extension to create Sweep issues directly from your editor! GitHub Issues and Pull Requests.
💡 To recreate the pull request edit the issue title or description. To tweak the pull request, leave a comment on the pull request.
Join Our Discord
Details
Add a feature to the working context that automatically summarizes the working context after every 5 turns of the conversation.
Checklist
backend/memory/memory_manager.py
✓ b11cd11 Editbackend/memory/memory_manager.py
✓ Editbackend/tests/test_memory_manager.py
✓ 459d4cf Editbackend/tests/test_memory_manager.py
✓ Editbackend/agent/coding_agent.py
✓ 1e0a34a Editbackend/agent/coding_agent.py
✓ EditThe text was updated successfully, but these errors were encountered: