Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial cloud docs for data sources, knowledge bases, and structures #1110

Merged
merged 13 commits into from
Aug 29, 2024
21 changes: 21 additions & 0 deletions docs/griptape-cloud/data-sources/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Data Sources

Use [Data Sources](https://cloud.griptape.ai/data-sources) to connect and ingest your own data for use with your AI code.
emjay07 marked this conversation as resolved.
Show resolved Hide resolved

## Create a Data Source

You can [Create a Data Source](https://cloud.griptape.ai/data-sources/create) by specifying the required configuration for your chosen data source.

If you do not see a data source coniguration you'd wish to use, you can submit a request via Discord or `[email protected]`.
emjay07 marked this conversation as resolved.
Show resolved Hide resolved
emjay07 marked this conversation as resolved.
Show resolved Hide resolved

## Refresh a Data Source

By default your Data Source will not refresh automatically. You can enable automatic refresh and specify a CRON expression when creating a Data Source.
emjay07 marked this conversation as resolved.
Show resolved Hide resolved

For example, if you wish your Data Source to refresh every day at midnight you can use the following expression: `0 0 * * *`.

If you wish to manually refresh a data source you can do so either via the `Refresh` button on the UI or by API using the `Data Source ID` on the `Config` tab. You will need a [Griptape Cloud API Key](https://cloud.griptape.ai/configuration/api-keys) to make API calls.
emjay07 marked this conversation as resolved.
Show resolved Hide resolved

```shell
curl -H "Authorization: Bearer ${GT_CLOUD_API_KEY}" --json '{}' https://cloud.griptape.ai/api/data-connectors/${data_source_id}/data-jobs
emjay07 marked this conversation as resolved.
Show resolved Hide resolved
```
emjay07 marked this conversation as resolved.
Show resolved Hide resolved
11 changes: 9 additions & 2 deletions docs/griptape-cloud/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# Griptape Cloud

Griptape Cloud provides managed services for your AI app stack. Deploy and scale end-to-end solutions, from LLM-powered data prep and retrieval to AI agents, pipelines and workflows.
[Griptape Cloud](https://cloud.griptape.ai/) provides managed services for your AI app stack. Deploy and scale end-to-end solutions, from LLM-powered data prep and retrieval to AI Agents, Pipelines and Workflows.

To get started with AI Structures in the Cloud, check out the [managed-structure-template](https://github.com/griptape-ai/managed-structure-template) or deploy one of the [griptape-sample-structures](https://github.com/griptape-ai/griptape-sample-structures/tree/main).
## Build Your Own RAG Pipeline
Connect to your data with our [Data Sources](data-sources/index.md) and prepare them for retrieval with [Knowledge Bases](knowledge-bases/index.md).

## Host and Run Your Code
emjay07 marked this conversation as resolved.
Show resolved Hide resolved
Have custom AI code? You can host your python code using [Structures](structures/index.md) whether it uses the Griptape Framework or not.

## APIs
All of our features can be called via API. See the [API Reference](api/api-reference.md) for detailed information.
35 changes: 35 additions & 0 deletions docs/griptape-cloud/knowledge-bases/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Knowledge Bases

Use [Knowledge Bases](https://cloud.griptape.ai/knowledge-bases) to prep your data ingested from [Data Sources](../data-sources/index.md) for retrieval.

## Create a Knowledge Base

You can [create a Knowledge Base](https://cloud.griptape.ai/knowledge-bases/create) by specifying which Data Sources you wish to include.

## Search or Query a Knowledge Base

Once created you can `Search` or `Query` the Knowledge Base for information contained in your Data Sources. `Search` will return a natural language response while `Query` will return the individual entries. Use whichever one best fits your use case.

### From the UI

You can try out both actions from the UI on the `Test` tab of your Knowledge Base.

### From the API

You can enact both `Query` and `Search` via the API by hitting their respective endpoints using a [Griptape Cloud API Key](https://cloud.griptape.ai/configuration/api-keys) and the Knowledge Base ID found on the `Config` tab of your Knowledge Base.

#### Query

```shell
curl -H "Authorization: Bearer ${GT_CLOUD_API_KEY}" --json '{"query": "test query string"}' https://cloud.griptape.ai/api/knowledge-bases/${knowledge-base-id}/query
emjay07 marked this conversation as resolved.
Show resolved Hide resolved
```

#### Search

```shell
curl -H "Authorization: Bearer ${GT_CLOUD_API_KEY}" --json '{"query": "test query string"}' https://cloud.griptape.ai/api/knowledge-bases/${knowledge-base-id}/search
emjay07 marked this conversation as resolved.
Show resolved Hide resolved
```

### Using the Griptape Framework

You can use [VectorStoreDrivers](../../griptape-framework/drivers/vector-store-drivers/#griptape-cloud-knowledge-base) in the Griptape Framework to run your code.
emjay07 marked this conversation as resolved.
Show resolved Hide resolved
44 changes: 44 additions & 0 deletions docs/griptape-cloud/structures/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Structures

Use [Structures](https://cloud.griptape.ai/structures) to host any python code in the cloud in seconds.
emjay07 marked this conversation as resolved.
Show resolved Hide resolved

## Create a Structure

1. First [Connect Your GitHub Account](https://cloud.griptape.ai/account)
emjay07 marked this conversation as resolved.
Show resolved Hide resolved
1. Install the [Griptape Cloud GitHub app](https://github.com/apps/griptape-cloud/installations/new/) to any account or organization you'd like to pull code in from
collindutter marked this conversation as resolved.
Show resolved Hide resolved
1. Be sure to allow the app access to `All Repositories` or select the specific repositories you need
emjay07 marked this conversation as resolved.
Show resolved Hide resolved
collindutter marked this conversation as resolved.
Show resolved Hide resolved

You can then [create a Structure](https://cloud.griptape.ai/structures/create) by providing your GitHub repository information. Make sure that your repository has a [Structure Config YAML](structure-config.md).
emjay07 marked this conversation as resolved.
Show resolved Hide resolved

### Quickstart With Samples and Templates

To get started with Structures in the Cloud, check out the [managed-structure-template](https://github.com/griptape-ai/managed-structure-template) or deploy one of the [griptape-sample-structures](https://github.com/griptape-ai/griptape-sample-structures/tree/main).

## Run a Structure

Once your Structure is created and deployed, you can run your Structure one of three ways outlined below. In order to view the output of any of your runs, no matter how you created them, you can view them in the `Runs` tab of your Structure.

### From the UI

In the UI, click on the name of the Structure you wish to run and then go to the `Test` tab. Here you can specify arguments to pass to your Structure run and any run-specific environment variables you need.

When passing arguments through the UI, pass each new argument on a new line. For example if your local code could be ran with the inputs `-i input_file.txt` then the arguments you would pass in the cloud would be:

```
-i
input_file.txt
```

### From the API

You can run your Structure via the API using CURL or any other code that can make HTTP requests. You will need a [Griptape Cloud API Key](https://cloud.griptape.ai/configuration/api-keys) and the `Structure Invocation URL` which is located on the `Config` tab of your Structure.

```shell
curl -H "Authorization: Bearer ${GT_CLOUD_API_KEY}" --json '{"args": ["arg1"], "env": {"var1":"value1"}}' ${INVOCATION_URL}
emjay07 marked this conversation as resolved.
Show resolved Hide resolved
```

For more information on other Structure run APIs, check out the [StructureRuns API docs](api/api-reference/#/StructureRuns).

### Using the Griptape Framework

You can use [StructureRunDrivers](../../griptape-framework/drivers/structure-run-drivers/#griptape-cloud) in the Griptape Framework to run your code.
emjay07 marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Griptape Topic Guides discuss key topics at a high level and provide useful back

### Griptape Cloud

[Griptape Cloud](griptape-cloud/api/api-reference.md) provides an overview of the APIs available in the managed cloud service.
[Griptape Cloud](griptape-cloud/index.md) provides an overview of the features in Griptape's cloud offering.

### Griptape Framework

Expand Down
9 changes: 7 additions & 2 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,10 +77,15 @@ nav:
- Contributing: "contributing.md"
- Cloud:
- Overview: "griptape-cloud/index.md"
- Data Sources:
- Overview: "griptape-cloud/data-sources/index.md"
- Knowledge Bases:
- Overview: "griptape-cloud/knowledge-bases/index.md"
- Structures:
- Overview: "griptape-cloud/structures/index.md"
- Structure Config YAML: "griptape-cloud/structures/structure-config.md"
- Cloud API:
- API Reference: "griptape-cloud/api/api-reference.md"
- Structures:
- Structure Config: "griptape-cloud/structures/structure-config.md"
- Framework:
- Overview: "griptape-framework/index.md"
- Structures:
Expand Down
Loading