SDLReader for Graphql

llama_hub/file/sdl/README.md

@@ -0,0 +1,18 @@
+# SDL Loader
+
+This loader extracts definitions from a Schema Definition Language file, used to specify the data for a GraphQL endpoint
+
+## Usage
+
+To use this loader, pass in the filename for the SDL file.
+
+This tool has a more extensive example usage documented in a Jupyter notebook [here](https://github.com/emptycrown/llama-hub/tree/main/llama_hub/tools/notebooks/shopify.ipynb) and [here](https://github.com/emptycrown/llama-hub/tree/main/llama_hub/tools/notebooks/shopify.ipynb)
+
+```python
+from llama_hub.file.sdl.base import SDLReader
+
+loader = SDLReader()
+documents = loader.load_data('./data/shopify.graphql')
+```
+
+This loader is designed to be used as a way to load data into [LlamaIndex](https://github.com/jerryjliu/gpt_index/tree/main/gpt_index) and/or subsequently used as a Tool in a [LangChain](https://github.com/hwchase17/langchain) Agent. See [here](https://github.com/emptycrown/llama-hub/tree/main) for examples.

llama_hub/file/sdl/__init__.py

@@ -0,0 +1 @@
+## init

llama_hub/file/sdl/base.py

@@ -0,0 +1,28 @@
+from typing import List
+
+from llama_index.readers.base import BaseReader
+from llama_index.readers.schema.base import Document
+
+class SDLReader(BaseReader):
+    """Schema definition langauge reader
+
+    Reads GraphQL Schema files
+
+    """
+
+    def load_data(
+        self, filename: str
+    ) -> List[Document]:
+        """Parse file."""
+        try:
+            import graphql
+        except ImportError:
+            raise ImportError("Please install graphql 'pip install graphql-core' ")
+        with open(filename, 'r') as f:
+            txt = f.read()
+
+        ast = graphql.parse(txt)
+        chunks = []
+        for definition in ast.definitions:
+            chunks.append(txt[definition.loc.start:definition.loc.end])
+        return [Document(text=chunk) for chunk in chunks]

llama_hub/file/sdl/requirements.txt

@@ -0,0 +1 @@
+graphql-core

llama_hub/library.json

@@ -87,6 +87,11 @@
     "author": "ephe-meral",
     "keywords": ["spreadsheet"]
   },
+  "SDLReader": {
+    "id": "file/sdl",
+    "author": "ajhofmann",
+    "keywords": ["graphql", "schema"]
+  },
   "SimpleWebPageReader": {
     "id": "web/simple_web",
     "author": "thejessezhang"