Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory manager #99

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from
Draft

Memory manager #99

wants to merge 10 commits into from

Conversation

chefron
Copy link
Contributor

@chefron chefron commented Jan 24, 2025

Draft: Add document memory system

This PR implements a memory system that allows agents to retain and reference information from uploaded documents. It's currently limited to the CLI chat feature for testing.

Core Components

  • MemoryManager: Manages document ingestion, chunking, and retrieval
  • VectorStore: Handles vector storage and similarity search using ChromaDB
  • CLI commands for memory management

Key Features

  • Agent-specific persistent memory
  • Category-based organization with semantic search
  • PDF text extraction and support for markdown/text files
  • Memory management through CLI:
    • Upload documents with automatic chunking
    • Search across stored memories
    • List/browse memory contents
    • Remove (“wipe”) memories

Implementation Notes

Similarity scores may appear low when using small test datasets due to sparse embedding space (I think), but the relative ranking works correctly -- more relevant memories consistently receive higher scores than less relevant ones. As more documents are added, similarity scores would (hopefully) increase due to greater semantic overlap, while maintaining the same relevance ranking. At least that's the way it's worked so far in testing.

Limitations/Future Work

  • Currently no conversational memory (only document-based)
  • Basic chunking strategy could be improved

chefron and others added 10 commits January 19, 2025 12:06
- Implement basic memory manager with PDF ingestion
- Add memory-related CLI commands
- Integrate memory system with agent
Move memory management operations from cli.py to proper modules. Clean up unused methods. Improve error handling and logging.
Deleted blank file
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant