Skip to content

Commit

Permalink
Artifacts are just data
Browse files Browse the repository at this point in the history
  • Loading branch information
collindutter committed Sep 5, 2024
1 parent 346c8ec commit 31ba036
Show file tree
Hide file tree
Showing 18 changed files with 69 additions and 152 deletions.
5 changes: 2 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed
- **BREAKING**: Removed `MediaArtifact`, use `ImageArtifact` or `AudioArtifact` instead.
- **BREAKING**: Removed `BooleanArtifact`, use `JsonArtifact` instead.
- **BREAKING**: Removed `CsvRowArtifact`.
- **BREAKING**: `CsvLoader`, `DataframeLoader`, and `SqlLoader` now return `list[TextArtifact]`.
- **BREAKING**: Removed `ImageArtifact.media_type`.
Expand All @@ -20,8 +19,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **BREAKING**: Moved `ImageArtifact.prompt` and `ImageArtifact.model` into `ImageArtifact.meta`.
- **BREAKING**: `ImageArtifact.to_text()` now returns the base64 encoded image.
- Updated `JsonArtifact` value converter to properly handle more types.
- `AudioArtifact` now subclasses `BaseArtifact` instead of `MediaArtifact`.
- `ImageArtifact` now subclasses `BaseArtifact` instead of `MediaArtifact`.
- `AudioArtifact` now subclasses `BlobArtifact` instead of `MediaArtifact`.
- `ImageArtifact` now subclasses `BlobArtifact` instead of `MediaArtifact`.
- Passing a dictionary as the value to `TextArtifact` will convert to a key-value formatted string.
- Removed `__add__` method from `BaseArtifact`, implemented it where necessary.

Expand Down
19 changes: 0 additions & 19 deletions MIGRATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,25 +36,6 @@ audio_artifact = AudioArtifact(
)
```

### Removed `BooleanArtifact`

`BooleanArtifact` has been removed. Use `JsonArtifact` instead.

#### Before

```python
boolean_artifact = BooleanArtifact("true")

print(boolean_artifact.value) # Value is True
```

#### After
```python
json_artifact = JsonArtifact("true")

print(json_artifact.value) # Value is True
```

### Removed `CsvRowArtifact`

`CsvRowArtifact` has been removed. Use `TextArtifact` instead.
Expand Down
50 changes: 23 additions & 27 deletions docs/griptape-framework/data/artifacts.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,54 +5,50 @@ search:

## Overview


**[Artifacts](../../reference/griptape/artifacts/base_artifact.md)** are used to store data that can be provided as input to or received as output from a Language Learning Model (LLM).
**[Artifacts](../../reference/griptape/artifacts/base_artifact.md)** are the core data structure in Griptape. They are used to encapsulate data and enhance it with metadata.

## Text

[TextArtifact](../../reference/griptape/artifacts/text_artifact.md)s store textual data. They can be used to count tokens using the [token_count()](../../reference/griptape/artifacts/text_artifact.md#griptape.artifacts.text_artifact.TextArtifact.token_count) method with a tokenizer, generate a text embedding through the [generate_embedding()](../../reference/griptape/artifacts/text_artifact.md#griptape.artifacts.text_artifact.TextArtifact.generate_embedding) method, and access the embedding with the [embedding](../../reference/griptape/artifacts/text_artifact.md#griptape.artifacts.text_artifact.TextArtifact.embedding) property.
[TextArtifact](../../reference/griptape/artifacts/text_artifact.md)s store textual data. They offer methods such as [token_count()](../../reference/griptape/artifacts/text_artifact.md#griptape.artifacts.text_artifact.TextArtifact.token_count) for counting tokens with a tokenizer, and [generate_embedding()](../../reference/griptape/artifacts/text_artifact.md#griptape.artifacts.text_artifact.TextArtifact.generate_embedding) for creating text embeddings. You can also access the embedding via the [embedding](../../reference/griptape/artifacts/text_artifact.md#griptape.artifacts.text_artifact.TextArtifact.embedding) property.

[TaskMemory](../../reference/griptape/memory/task/task_memory.md) automatically stores `TextArtifacts` returned by tool activities and provides their IDs back to the LLM.
When `TextArtifact`s are returned from Tools, they will be stored in [Task Memory](../../griptape-framework/structures/task-memory.md) if the Tool has set `off_prompt=True`.

## Image
## Blob

[ImageArtifact](../../reference/griptape/artifacts/image_artifact.md)s store image data. They include binary image data and metadata such as MIME type, dimensions, and prompt and model information for images returned by [image generation drivers](../drivers/image-generation-drivers.md). They inherit functionality from [BlobArtifacts](#blob).
[BlobArtifact](../../reference/griptape/artifacts/blob_artifact.md)s store binary large objects (blobs).

## Audio
When `BlobArtifact`s are returned from Tools, they will be stored in [Task Memory](../../griptape-framework/structures/task-memory.md) if the Tool has set `off_prompt=True`.

[AudioArtifact](../../reference/griptape/artifacts/audio_artifact.md)s store audio content, including binary audio data and metadata such as format, duration, and prompt and model information for audio returned by generative models. They inherit from [BlobArtifacts](#blob).
### Image

## Action
[ImageArtifact](../../reference/griptape/artifacts/image_artifact.md)s store image data. This includes binary image data along with metadata such as MIME type and dimensions. They are a subclass of [BlobArtifacts](#blob).

[ActionArtifact](../../reference/griptape/artifacts/action_artifact.md)s represent actions taken by the LLM. Currently, the only supported action is [ToolAction](../../reference/griptape/common/actions/tool_action.md), which is used to execute a [Tool](../../griptape-framework/tools/index.md).
### Audio

## JSON
[AudioArtifact](../../reference/griptape/artifacts/audio_artifact.md)s store audio content. This includes binary audio data and metadata such as format, and duration. They are a subclass of [BlobArtifacts](#blob).

[JsonArtifact](../../reference/griptape/artifacts/json_artifact.md)s store JSON-serializable data. Any data assigned to the `value` property is converted using `json.dumps(json.loads(value))`.

## Generic
## List

[GenericArtifact](../../reference/griptape/artifacts/generic_artifact.md)s act as an escape hatch for passing any type of data that does not fit into any other artifact type. While generally not recommended, they are suitable for specific scenarios. For example, see [talking to a video](../../examples/talk-to-a-video.md), which demonstrates using a `GenericArtifact` to pass a Gemini-specific video file.
[ListArtifact](../../reference/griptape/artifacts/list_artifact.md)s store lists of Artifacts.

## System Artifacts
When `ListArtifact`s are returned from Tools, their elements will be stored in [Task Memory](../../griptape-framework/structures/task-memory.md) if the element is either a `TextArtifact` or a `BlobArtifact` and the Tool has set `off_prompt=True`.

These Artifacts don't map to an LLM modality. They must be transformed in some way before they can be used as LLM input.
## Info

### Blob
[InfoArtifact](../../reference/griptape/artifacts/info_artifact.md)s store small pieces of textual information. These are useful for conveying messages about the execution or results of an operation, such as "No results found" or "Operation completed successfully."

[BlobArtifact](../../reference/griptape/artifacts/blob_artifact.md)s store binary large objects (blobs) and are used to pass unstructured data back to the LLM via [InfoArtifact](#info).

`TaskMemory` automatically stores `BlobArtifacts` returned by tool activities, allowing them to be reused by other tools.
## JSON

### Info
[JsonArtifact](../../reference/griptape/artifacts/json_artifact.md)s store JSON-serializable data. Any data assigned to the `value` property is processed using `json.dumps(json.loads(value))`.

[InfoArtifact](../../reference/griptape/artifacts/info_artifact.md)s store short notifications that are passed back to the LLM without being stored in Task Memory.
## Error

### Error
[ErrorArtifact](../../reference/griptape/artifacts/error_artifact.md)s store exception information, providing a structured way to convey errors.

[ErrorArtifact](../../reference/griptape/artifacts/error_artifact.md)s store errors that are passed back to the LLM without being stored in Task Memory.
## Action

### List
[ActionArtifact](../../reference/griptape/artifacts/action_artifact.md)s represent actions taken by an LLM. Currently, the only supported action type is [ToolAction](../../reference/griptape/common/actions/tool_action.md), which is used to execute a [Tool](../../griptape-framework/tools/index.md).

[ListArtifact](../../reference/griptape/artifacts/list_artifact.md)s store lists of Artifacts that can be passed to the LLM.
## Generic

[GenericArtifact](../../reference/griptape/artifacts/generic_artifact.md)s provide a flexible way to pass data that does not fit into any other artifact category. While not generally recommended, they can be useful for specific use cases. For instance, see [talking to a video](../../examples/talk-to-a-video.md), which demonstrates using a `GenericArtifact` to pass a Gemini-specific video file.
14 changes: 6 additions & 8 deletions griptape/artifacts/__init__.py
Original file line number Diff line number Diff line change
@@ -1,27 +1,25 @@
from .base_artifact import BaseArtifact
from .base_system_artifact import BaseSystemArtifact

from .error_artifact import ErrorArtifact
from .info_artifact import InfoArtifact
from .text_artifact import TextArtifact
from .json_artifact import JsonArtifact
from .blob_artifact import BlobArtifact
from .boolean_artifact import BooleanArtifact
from .list_artifact import ListArtifact
from .image_artifact import ImageArtifact
from .audio_artifact import AudioArtifact
from .json_artifact import JsonArtifact
from .action_artifact import ActionArtifact
from .generic_artifact import GenericArtifact

from .error_artifact import ErrorArtifact
from .info_artifact import InfoArtifact
from .list_artifact import ListArtifact


__all__ = [
"BaseArtifact",
"BaseSystemArtifact",
"ErrorArtifact",
"InfoArtifact",
"TextArtifact",
"JsonArtifact",
"BlobArtifact",
"BooleanArtifact",
"ListArtifact",
"ImageArtifact",
"AudioArtifact",
Expand Down
10 changes: 0 additions & 10 deletions griptape/artifacts/base_system_artifact.py

This file was deleted.

4 changes: 4 additions & 0 deletions griptape/artifacts/blob_artifact.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@ class BlobArtifact(BaseArtifact):
encoding: str = field(default="utf-8", kw_only=True)
encoding_error_handler: str = field(default="strict", kw_only=True)

@property
def mime_type(self) -> str:
return "application/octet-stream"

def to_bytes(self) -> bytes:
return self.value

Expand Down
17 changes: 13 additions & 4 deletions griptape/artifacts/boolean_artifact.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,17 +9,23 @@

@define
class BooleanArtifact(BaseArtifact):
"""Stores a boolean value.
Attributes:
value: The boolean value.
"""

value: bool = field(converter=bool, metadata={"serializable": True})

@classmethod
def parse_bool(cls, value: Union[str, bool]) -> BooleanArtifact: # noqa: FBT001
"""Convert a string literal or bool to a BooleanArtifact. The string must be either "true" or "false" with any casing."""
def parse_bool(cls, value: Union[str, bool]) -> BooleanArtifact:
"""Convert a string literal or bool to a BooleanArtifact. The string must be either "true" or "false"."""
if value is not None:
if isinstance(value, str):
if value.lower() == "true":
return BooleanArtifact(True) # noqa: FBT003
return BooleanArtifact(value=True)
elif value.lower() == "false":
return BooleanArtifact(False) # noqa: FBT003
return BooleanArtifact(value=False)
elif isinstance(value, bool):
return BooleanArtifact(value)
raise ValueError(f"Cannot convert '{value}' to BooleanArtifact")
Expand All @@ -29,3 +35,6 @@ def __add__(self, other: BaseArtifact) -> BooleanArtifact:

def __eq__(self, value: object) -> bool:
return self.value is value

def to_text(self) -> str:
return str(self.value).lower()
34 changes: 0 additions & 34 deletions griptape/artifacts/csv_row_artifact.py

This file was deleted.

7 changes: 5 additions & 2 deletions griptape/artifacts/error_artifact.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@

from attrs import define, field

from griptape.artifacts import BaseSystemArtifact
from griptape.artifacts import BaseArtifact


@define
class ErrorArtifact(BaseSystemArtifact):
class ErrorArtifact(BaseArtifact):
"""Represents an error that may want to be conveyed to the LLM.
Attributes:
Expand All @@ -18,3 +18,6 @@ class ErrorArtifact(BaseSystemArtifact):

value: str = field(converter=str, metadata={"serializable": True})
exception: Optional[Exception] = field(default=None, kw_only=True, metadata={"serializable": False})

def to_text(self) -> str:
return self.value
4 changes: 2 additions & 2 deletions griptape/artifacts/image_artifact.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@

from attrs import define, field

from griptape.artifacts import BaseArtifact
from griptape.artifacts import BlobArtifact


@define
class ImageArtifact(BaseArtifact):
class ImageArtifact(BlobArtifact):
"""Stores image data.
Attributes:
Expand Down
7 changes: 5 additions & 2 deletions griptape/artifacts/info_artifact.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@

from attrs import define, field

from griptape.artifacts import BaseSystemArtifact
from griptape.artifacts import BaseArtifact


@define
class InfoArtifact(BaseSystemArtifact):
class InfoArtifact(BaseArtifact):
"""Represents helpful info that can be conveyed to the LLM.
For example, "No results found" or "Please try again.".
Expand All @@ -16,3 +16,6 @@ class InfoArtifact(BaseSystemArtifact):
"""

value: str = field(converter=str, metadata={"serializable": True})

def to_text(self) -> str:
return self.value
4 changes: 2 additions & 2 deletions griptape/artifacts/list_artifact.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@

from attrs import Attribute, define, field

from griptape.artifacts import BaseArtifact, BaseSystemArtifact
from griptape.artifacts import BaseArtifact

if TYPE_CHECKING:
from collections.abc import Sequence


@define
class ListArtifact(BaseSystemArtifact):
class ListArtifact(BaseArtifact):
value: Sequence[BaseArtifact] = field(factory=list, metadata={"serializable": True})
item_separator: str = field(default="\n\n", kw_only=True, metadata={"serializable": True})
validate_uniform_types: bool = field(default=False, kw_only=True, metadata={"serializable": True})
Expand Down
4 changes: 2 additions & 2 deletions griptape/artifacts/text_artifact.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,11 +41,11 @@ def to_text(self) -> str:
def to_bytes(self) -> bytes:
return str(self.value).encode(encoding=self.encoding, errors=self.encoding_error_handler)

def generate_embedding(self, driver: BaseEmbeddingDriver) -> Optional[list[float]]:
def generate_embedding(self, driver: BaseEmbeddingDriver) -> list[float]:
self._embedding.clear()
self._embedding.extend(driver.embed_string(str(self.value)))

return self.embedding
return self._embedding

def token_count(self, tokenizer: BaseTokenizer) -> int:
return tokenizer.count_tokens(str(self.value))
4 changes: 2 additions & 2 deletions griptape/common/prompt_stack/contents/text_message_content.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

from attrs import define, field

from griptape.artifacts import TextArtifact
from griptape.artifacts import BaseArtifact, TextArtifact
from griptape.common import BaseDeltaMessageContent, BaseMessageContent, TextDeltaMessageContent

if TYPE_CHECKING:
Expand All @@ -13,7 +13,7 @@

@define
class TextMessageContent(BaseMessageContent):
artifact: TextArtifact = field(metadata={"serializable": True})
artifact: BaseArtifact = field(metadata={"serializable": True})

@classmethod
def from_deltas(cls, deltas: Sequence[BaseDeltaMessageContent]) -> TextMessageContent:
Expand Down
6 changes: 2 additions & 4 deletions griptape/common/prompt_stack/prompt_stack.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
from griptape.artifacts import (
ActionArtifact,
BaseArtifact,
ErrorArtifact,
GenericArtifact,
ImageArtifact,
ListArtifact,
Expand Down Expand Up @@ -70,8 +69,6 @@ def __to_message_content(self, artifact: str | BaseArtifact) -> list[BaseMessage
return [ImageMessageContent(artifact)]
elif isinstance(artifact, GenericArtifact):
return [GenericMessageContent(artifact)]
elif isinstance(artifact, ErrorArtifact):
return [TextMessageContent(TextArtifact(artifact.to_text()))]
elif isinstance(artifact, ActionArtifact):
action = artifact.value
output = action.output
Expand All @@ -81,6 +78,7 @@ def __to_message_content(self, artifact: str | BaseArtifact) -> list[BaseMessage
return [ActionResultMessageContent(output, action=action)]
elif isinstance(artifact, ListArtifact):
processed_contents = [self.__to_message_content(artifact) for artifact in artifact.value]

return [sub_content for processed_content in processed_contents for sub_content in processed_content]
else:
raise ValueError(f"Unsupported artifact type: {type(artifact)}")
return [TextMessageContent(TextArtifact(artifact.to_text()))]
2 changes: 1 addition & 1 deletion griptape/mixins/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
from .actions_subtask_origin_mixin import ActionsSubtaskOriginMixin
from .rule_mixin import RuleMixin
from .serializable_mixin import SerializableMixin
from .media_artifact_file_output_mixin import ArtifactFileOutputMixin
from .artifact_file_output_mixin import ArtifactFileOutputMixin
from .futures_executor_mixin import FuturesExecutorMixin
from .singleton_mixin import SingletonMixin

Expand Down
Loading

0 comments on commit 31ba036

Please sign in to comment.