Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Graph Store Drivers #1006

Closed
wants to merge 21 commits into from
Closed

Conversation

Kornspan
Copy link

@Kornspan Kornspan commented Jul 22, 2024

Describe your changes

Additional Context:
This implementation allows future integrations with other graph databases.

Changes Made:

  • BaseGraphStoreDriver: Abstract class for graph store operations.
  • FalkorDBGraphStoreDriver: Implements the base class for FalkorDB.
  • Unit Tests: Comprehensive tests for FalkorDBGraphStoreDriver.
  • Documentation: Updated with details and usage examples.
  • Changelog: Added entries for the new features.
  • Dependencies: Added FalkorDB to pyproject.toml.

Issue ticket number and link

Closes #915


📚 Documentation preview 📚: https://griptape--1006.org.readthedocs.build//1006/

@collindutter
Copy link
Member

collindutter commented Jul 22, 2024

@Kornspan from the read the docs build:

Installing dependencies from lock file

pyproject.toml changed significantly since poetry.lock was last generated. Run `poetry lock [--no-update]` to fix the lock file.

Seems to be a source of issues on the unit tests as well.

Copy link

codecov bot commented Jul 23, 2024

Codecov Report

Attention: Patch coverage is 72.28916% with 46 lines in your changes missing coverage. Please review.

Files Patch % Lines
...ptape/drivers/graph/falkordb_graph_store_driver.py 77.50% 20 Missing and 7 partials ⚠️
griptape/drivers/graph/base_graph_store_driver.py 58.69% 18 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@galshubeli
Copy link

Hi can you comment what error occurred in the mkdocs? @collindutter @vachillo

@collindutter
Copy link
Member

@galshubeli here's the output:

INFO    -  Cleaning site directory
INFO    -  Building documentation to directory: /home/docs/checkouts/readthedocs.org/user_builds/griptape/checkouts/1006/_readthedocs/html
INFO    -  The following pages exist in the docs directory, but are not included in the "nav" configuration:
  - griptape-cloud/index.md
ERROR   -  Error reading page 'reference/griptape/artifacts/index.md': No module named 'griffe.collections'
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocs/__main__.py", line 366, in <module>
    cli()
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocs/__main__.py", line 284, in build_command
    build.build(cfg, dirty=not clean)
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocs/commands/build.py", line 310, in build
    _populate_page(file.page, config, files, dirty)
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocs/commands/build.py", line 167, in _populate_page
    page.render(config, files)
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocs/structure/pages.py", line 285, in render
    self.content = md.convert(self.markdown)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/markdown/core.py", line 357, in convert
    root = self.parser.parseDocument(self.lines).getroot()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/markdown/blockparser.py", line 117, in parseDocument
    self.parseChunk(self.root, '\n'.join(lines))
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/markdown/blockparser.py", line 136, in parseChunk
    self.parseBlocks(parent, text.split('\n\n'))
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/markdown/blockparser.py", line 158, in parseBlocks
    if processor.run(parent, blocks) is not False:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocstrings/extension.py", line 125, in run
    html, handler, data = self._process_block(identifier, block, heading_level)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocstrings/extension.py", line 196, in _process_block
    handler = self._handlers.get_handler(handler_name, handler_config)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocstrings/handlers/base.py", line 458, in get_handler
    module = importlib.import_module(f"mkdocstrings_handlers.{name}")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/.asdf/installs/python/3.11.9/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocstrings_handlers/python/__init__.py", line 3, in <module>
    from mkdocstrings_handlers.python.handler import get_handler
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocstrings_handlers/python/handler.py", line 14, in <module>
    from griffe.collections import LinesCollection, ModulesCollection
ModuleNotFoundError: No module named 'griffe.collections'

Note that you can run the build process yourself with make docs.

@galshubeli
Copy link

@collindutter I encountered this error while running it on the dev and main branch as well. Do you know what the possible cause might be?

@collindutter
Copy link
Member

@galshubeli Are your dependencies up to date? Can you try:

make install/all
make docs

I do get quite a few DeprecationWarnings but you can ignore those.

Separately, I want to apologize for the lack of attention our team has given to this PR. We're definitely interested in incorporating Graph Store Drivers into Griptape and have already started some initial work ourselves. However, we haven't yet fully figured out how they fit into the bigger picture. They will likely play a key role in RagEngine, but we need more time to work through the details.

Given this uncertainty, it's hard for us to provide meaningful feedback on the PR right now. For the time being, it might make sense to close the PR and revisit it once we've solidified our approach. When we're ready, we'd love to get your input on it!

CC @Kornspan

@collindutter
Copy link
Member

Hey all, I'm going to close this PR for the reasons mentioned above. Once we're ready to support Graph Drivers we'll reach back out!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Support for Graph Store Drivers in Griptape
4 participants