Skip to content

Commit

Permalink
Updated Docs for Zetaforge WebPage (#264)
Browse files Browse the repository at this point in the history
* Updated Docs for Zetaforge WebPage

* grammatical fixes to Deploy-a-Pipeline

* Update docs/deploy-a-pipeline.md

---------

Co-authored-by: Jon Magoon <[email protected]>
  • Loading branch information
OGHO-SAYE and jmagoon authored Dec 16, 2024
1 parent 815292a commit 5f0daed
Show file tree
Hide file tree
Showing 29 changed files with 152 additions and 81 deletions.
4 changes: 2 additions & 2 deletions docs/assemble-a-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

A Pipeline is composed of Blocks. To construct a Pipeline, drag and drop Blocks into your workspace and connect the Blocks by adding links between the output of one Block and the input of the next Block. ZetaForge supports directed acyclic connections between the Blocks.

![drag-drop-blocks-add-links.gif](assets%2Fdrag-drop-blocks-add-links.gif)
![assemble-pipeline.gif](assets%2Fassemble-pipeline.gif)

The changes made to the workspace are saved in `frontend/.cache`. When the first Block is added to an empty workspace, a new directory (`pipeline-${pipeline_id}`) will be created under `frontend/.cache` for the Pipeline representing the workspace. The information about the pipeline, such as the Blocks making the pipeline and the connections between them, is stored in a JSON file with the same name as the pipeline, i.e., `pipeline-${pipeline_id}.json`. Note that the `${pipeline_id}` is displayed at the top of the window in the ZetaForge app.

Expand Down Expand Up @@ -33,4 +33,4 @@ Finally, we add the Save Text Block to store the story GPT generated to a file a

We run the Pipeline by clicking on the Run button at the top of the window. Here is a screenshot of the full Pipeline:

![GPT Story Teller Pipeline](assets/gpt-storyteller-pipeline.png)
![Super Resolution Pipeline](assets/Super-resolution.png)
Binary file added docs/assets/Super-resolution.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/assemble-pipeline.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/block-editor.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/block-example.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/canny-edge-mona-result.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/canny-edge-specs.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/compile-run-block-editor.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/deploy-to-payload.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/deployed-payload.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/drop-a-block.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/executed-pipeline-button.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/executed-pipelines.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/import-files-folder.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/past-executions.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/remove-a-link.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/remove-block.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/rerun-pipeline.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/run-a-core-pipeline-example.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/run-test-example.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/selected-executed-pipeline.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/user-interface-preview.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/view-result-button.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
41 changes: 22 additions & 19 deletions docs/create-blocks.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Parameter Blocks do not have any input nodes; instead, they have a UI element th
the other hand, Compute Blocks have both input and output nodes. View Blocks have a small box on top that will visualize
its input based on the implemented logic in that View Block.

Each Block, regardless of its type, is saved in a directory named after the Block in `frontend/core/blocks`. However,
Each Block, regardless of its type, is saved in a directory named after the Block in `frontend/core/blocks`. However,
the content of the directory representing a Block varies depending on its type.

### Components of Parameter Blocks
Expand All @@ -36,9 +36,9 @@ the structural information about a Block and is stored in `frontend/core/blocks/

A directory representing a Compute Block should contain five files:

1. `computations.py`: Includes (at least) `compute(args)` and `test(args)` functions. The `compute` function implements
the logic of the Block, while the `test` function is designed to test the compute function. The `compute` function
returns a dictionary with keys that match the output node names from the specs file.
1. `computations.py`: Includes (at least) `compute(args)` and `test(args)` functions. The `compute` function implements
the logic of the Block, while the `test` function is designed to test the compute function. The `compute` function
returns a dictionary with keys that match the output node names from the specs file.

2. `Dockerfile`: Describes the instructions to build a Docker image. This file should always include `COPY computations.py .`.

Expand All @@ -49,28 +49,30 @@ returns a dictionary with keys that match the output node names from the specs f
5. `specs.json`: Includes the structural information about the Block.

### Components of View Blocks

The role of View Blocks is to create and display visualizations for the computed Block outputs.
View Blocks have the same five components as the Compute Blocks; i.e., `computations.py`, `Dockerfile`, `requirements.txt`,
`run_test.py`, and `specs.json`. Using View Blocks, we create HTML-formatted visualizations that will be visualized
on top of the View Block as the Pipeline runs.

on top of the View Block as the Pipeline runs.

## Creating Blocks

To create a Block, use a template Block listed on the sidebar. You can drag and drop a template Block, such as
`New Python`, and modify its components to build your own Block.
`Denoising`, and modify its components to build your own Block.

![drop-a-block.gif](assets%2Fdrop-a-block.gif)

## Modifying Blocks with the Block Editor

To make any modifications, click on the `</ >` icon to open the Block Editor sidebar on the right.

To modify the compute or view logic, simply edit the `compute` function. You can add as many lines of code as you like to the
`computations.py` file.
To modify the compute or view logic, simply edit the `compute` function. You can add as many lines of code as you like to the
`computations.py` file.

[//]: # "You can also write a specific prompt to modify the code with the help of an AI agent, such as GPT-4."

[//]: # (You can also write a specific prompt to modify the code with the help of an AI agent, such as GPT-4.)
### Block Templates

The `computations.py` file should follow this template:

```python
Expand Down Expand Up @@ -101,15 +103,15 @@ def compute(in1,...):
"""
Description
"""

html_template = """
<!DOCTYPE html>
<html>
...
...
<!-- Some html code... -->
</html>
"""

unique_id = str(uuid.uuid4())
html_path = f"viz_{unique_id}.html"
in1_str = json.dumps(in1)
Expand All @@ -131,13 +133,14 @@ compute function should always be a dictionary with keys and values correspondin

### Saving Changes to Blocks

You can save the changes you made to a Block by compiling that Block inside the Block Editor. Just look for the "compile
Block files" button as shown below:
You can save the changes you made to a Block by compiling that Block inside the Block Editor. Just follow instructions as shown

![compile-run-block-editor.png](assets%2Fcompile-run-block-editor.png)
![compile-run-block-editor.gif](assets%2Fcompile-run-block-editor.gif)

Within the files section you can modify the code, and navigate to the `Test Block` section to compile and `run test` will run the `test` function inside the `computations.py` file

The "compile files and run Block" button will compile the changes and run the test functions specified by the `test`
function inside the `computations.py` file.
<!--
The "compile files and run test" button will compile the changes and run the test functions specified by the `test`
function inside the `computations.py` file. -->

To learn more about the Block Editor, please refer to our [user interface guide](user-interface.md/#block-editor).
To learn more about the Block Editor, please refer to our [user interface guide](user-interface.md/#block-editor).
24 changes: 23 additions & 1 deletion docs/deploy-a-pipeline.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,25 @@
# Deploy a Pipeline

Coming soon, if you can't wait [get in touch with us](https://zetane.com/contact-us)!
After the creation or modification of a pipeline, our system stores the information of every previously executed pipelines. After some pipelines have been executed you will be able to navigate through all past pipelines executions, deploy an existing pipeline, and retrieve code examples to call the API for the deployed pipeline.

# Steps to Deploy a Pipeline

1. **Access Past Executions**
To begin, you would need to navigate to the past executions section in the interface. This section will display a lis of all the previously executed pipeline, both succesful and unsuccesful executions. To navigate to past executions click on the button as shown below:

![executed-pipeline-button.png](assets%2Fexecuted-pipeline-button.png)

After clicking the button you should be presented with the list of previously executed pipelines as shown below:
![executed-pipelines.png](assets%2Fexecuted-pipelines.png)

2. **Locate your desired pipeline**
After the table has been displayed you can now browse through all the previously executed pipelines, and find the one you wish to deploy. Each row would contain information such as the `pipeline ID`, `Date Created`, `Last Execution`, `Host`, `Deployment status`, and `# of executions`. and upon selecting a desired pipeline, you can have access to more information such as the `execution status` refer to the image shown below :

![selected-executed-pipeline.png](assets%2Fselected-executed-pipeline.png)

3. **Deploy desired pipeline**
Once you have selected your desired pipeline, you can deploy it by simply clicking on the deploy button corresponding to it. The system would process the request, and update the deploy button to **Payload**
![deploy-to-payload.gif](assets%2Fdeploy-to-payload.gif)
4. **Retrieve Code Examples**
After succesfully deploying your pipeline, you can click the **Payload** button, and this will display code examples demonstrating how to call the API for your pipeline. You can utilize these examples to have them integrated into your applications.
![deployed-payload.png](assets%2Fdeployed-payload.png)
14 changes: 7 additions & 7 deletions docs/run-a-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,21 @@

Prior to running a Pipeline, you need to ensure that all the necessary Blocks are in the workspace and the links between
Blocks have been correctly established. Also, when applicable, inputs should be provided. If you have made any changes
to a Block, make sure to save and compile those changes before running the Pipeline.
to a Block, make sure to save and compile those changes before running the Pipeline.
When everything is in place, run the Pipeline by
clicking the `Run` button.

![run-a-pipeline-example.gif](assets%2Frun-a-pipeline-example.gif)
![run-a-core-pipeline-example.gif](assets%2Frun-a-core-pipeline-example.gif)

Once the Pipeline begins running, you can review the logs by clicking on the `Log` button. When you run a Pipeline for
the first time, Docker images are built. After that, the containers are executed without rebuilding if
Once the Pipeline begins running, you can review the logs by clicking on the `Log` button. When you run a Pipeline for
the first time, Docker images are built. After that, the containers are executed without rebuilding if
no changes have been made. Therefore, the first time you build the Docker images usually takes more time.

Upon running a Pipeline, a new directory named `history` is formed within the Pipeline directory under
`frontend/.cache/pipeline-${pipeline_id}`. Within this directory, there is a unique subdirectory generated
`frontend/.cache/pipeline-${pipeline_id}`. Within this directory, there is a unique subdirectory generated
for each run, identified by a distinct UUID. Each of these subdirectories contains folders for
`files` and `logs`, where output files and log files are stored, respectively.

Moreover, there's a `pipeline` folder including a `pipeline.json` file that outlines the Pipeline's structure
Moreover, there's a `pipeline` folder including a `pipeline.json` file that outlines the Pipeline's structure
and connections between the Blocks. If the run is successful, a `results.json` file is created in the `results`
directory, capturing detailed information about the Pipeline execution, including Block inputs and outputs.
directory, capturing detailed information about the Pipeline execution, including Block inputs and outputs.
16 changes: 8 additions & 8 deletions docs/run-your-first-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ To experience ZetaForge in action, start by dragging and dropping a core pipelin

Follow these step-by-step screenshots to help you run your first pipeline:

1) Load a Pipeline:
![canny-edge-pipeline.png](assets%2Fquick-start.gif)
2) Click Run
![click-run.png](assets%2Fclick-run.png)
3) View the logs:
![canny-edge-logs.png](assets%2Fcanny-edge-logs.png)
4) See the results:
![canny-edge-results.png](assets%2Fcanny-edge-results.png)
1. Load and run Pipeline:
![run-a-core-pipeline-example.gif](assets%2Frun-a-core-pipeline-example.gif)

2. View the results:
![view-result-button.png](assets%2Fview-result-button.png)

3. Display results:
![canny-edge-mona-result.png](assets%2Fcanny-edge-mona-result.png)
Loading

0 comments on commit 5f0daed

Please sign in to comment.