Skip to content

Commit

Permalink
Merge pull request #4 from una-auxme/documentation
Browse files Browse the repository at this point in the history 
Documentation
  • Loading branch information
@JulianTrommer
JulianTrommer authored Dec 6, 2023
2 parents cb2f0a0 + 5e1e10b commit 353ef2f
Show file tree
Hide file tree
Showing 10 changed files with 62 additions and 49 deletions.
4 changes: 2 additions & 2 deletions Project.toml
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name = "MeshGraphNets"
uuid = "f7b4726d-4195-44ab-b39c-37bbdadae004"
authors = ["JT <[email protected]>"]
version = "0.1.0"
version = "0.1.1"

[deps]
CUDA = "052768ef-5323-5732-b1bb-66c8b64840ba"
Expand Down Expand Up @@ -33,7 +33,7 @@ ChainRulesCore = "1.16.0 - 1.18"
DataFrames = "1.6"
DifferentialEquations = "7.11"
Distributions = "0.25"
GraphNetCore = "0.1.0"
GraphNetCore = "0.1"
HDF5 = "0.17"
JLD2 = "0.4"
JSON = "0.21"
Expand Down
4 changes: 3 additions & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,13 @@ You can find the original implementation of MeshGraphNets in their GitHub reposi

[*MeshGraphNets.jl*](https://github.com/una-auxme/MeshGraphNets.jl) is designed to be part of the [SciML](https://sciml.ai/) ecosystem. The original framework was remodeled into a NeuralODE so that solvers from the [DifferentialEquations.jl](https://docs.sciml.ai/DiffEqDocs/stable/) can be used to evaluate the system.

Base functionality for the `Encode-Process-Decode` architecture of `DeepMind` that *MeshGraphNets* is based on is provided in the core package [*GraphNetCore.jl*](https://github.com/una-auxme/GraphNetCore.jl).

## How to use MeshGraphNets.jl

Examples from the original paper are implemented in the [examples folder](https://github.com/una-auxme/MeshGraphNets.jl/tree/main/examples). You can also refer to the [documentation](https://una-auxme.github.io/MeshGraphNets.jl/dev/overview) if you want to model your own system.

## Currently supported
## Currently Supported

- Customizable input & output quantities
- 1D & 3D meshes
Expand Down
3 changes: 2 additions & 1 deletion docs/Project.toml
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
[deps]
Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
GraphNetCore = "7809f980-de1b-4f9a-8451-85f041491431"
MeshGraphNets = "f7b4726d-4195-44ab-b39c-37bbdadae004"

[compat]
julia = "1.9"
julia = "1.9"
18 changes: 15 additions & 3 deletions docs/make.jl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,17 @@
import Pkg; Pkg.develop(path = joinpath(@__DIR__, "../../MeshGraphNets.jl"))
using Documenter, MeshGraphNets, GraphNetCore

import Documenter: GitHubActions

makedocs(sitename = "MeshGraphNets.jl",
format = Documenter.HTML(
sidebar_sitename = false,
edit_link = nothing
),
authors = "Julian Trommer, and contributors.",
modules = [GraphNetCore],
modules = [MeshGraphNets, GraphNetCore],
checkdocs = :exports,
linkcheck = true,
linkcheck = false,
pages = [
"Home" => "index.md",
"Overview" => "overview.md",
Expand All @@ -32,4 +34,14 @@ makedocs(sitename = "MeshGraphNets.jl",
]
)

deploydocs(repo = "github.com/una-auxme/MeshGraphNets.jl.git", devbranch = "main")
function deployConfig()
github_repository = get(ENV, "GITHUB_REPOSITORY", "")
github_event_name = get(ENV, "GITHUB_EVENT_NAME", "")
if github_event_name == "workflow_run"
github_event_name = "push"
end
github_ref = get(ENV, "GITHUB_REF", "")
return GitHubActions(github_repository, github_event_name, github_ref)
end

deploydocs(repo = "github.com/una-auxme/MeshGraphNets.jl.git", devbranch = "main", deploy_config = deployConfig())
30 changes: 15 additions & 15 deletions docs/src/cylinder_flow.md
View file
Original file line number Diff line number Diff line change
@@ -1,29 +1,29 @@
# CylinderFlow by Google DeepMind

This examples provides information on preparing the data for the CylinderFlow example provided by [Google DeepMind](https://deepmind.google/) in their corresponding [repository](https://github.com/google-deepmind/deepmind-research/tree/master/meshgraphnets) an how you can train and evaluate the resulting network.
This examples provides information on preparing the data for the `CylinderFlow` example provided by [Google DeepMind](https://deepmind.google/) in their corresponding [repository](https://github.com/google-deepmind/deepmind-research/tree/master/meshgraphnets) an how you can train and evaluate the resulting network.

## Data preparation
## Data Preparation

First you need to download the provided datasets for training, evaluation and testing. An explanation on how you can download the files is provided in the repository:

> https://github.com/google-deepmind/deepmind-research/tree/master/meshgraphnets#datasets
> [https://github.com/google-deepmind/deepmind-research/tree/master/meshgraphnets#datasets](https://github.com/google-deepmind/deepmind-research/tree/master/meshgraphnets#datasets)
If you execute the file [download_dataset.sh](https://github.com/google-deepmind/deepmind-research/blob/master/meshgraphnets/download_dataset.sh) with the argument `cylinder_flow` the following files should download:

- meta.json
- train.tfrecord
- valid.tfrecord
- test.tfrecord
- `meta.json`
- `train.tfrecord`
- `valid.tfrecord`
- `test.tfrecord`

You can keep the `.tfrecord` files as is. You only need to change the `meta.json` file to be compatible with [*MeshGraphNets.jl*](https://github.com/una-auxme/MeshGraphNets.jl). The correct file is provided in the [examples folder](https://github.com/una-auxme/MeshGraphNets.jl/tree/main/examples/cylinder_flow) and you only need to copy it to the same folder as the `.tfrecord` files.
You can keep the `.tfrecord` files as is. You only need to change the `meta.json` file to be compatible with *MeshGraphNets.jl*. The correct file is provided in the [examples folder](https://github.com/una-auxme/MeshGraphNets.jl/tree/main/examples/cylinder_flow) and you only need to copy it to the same folder as the `.tfrecord` files.

> If you want to understand the structure of the `meta.json` file take a look at the section [Training Data](https://una-auxme.github.io/MeshGraphNets.jl/dev/training_data).
The default path for the data folder that is specified in the [example script]((https://github.com/una-auxme/MeshGraphNets.jl/tree/main/examples/cylinder_flow/cylinder_flow.jl)) is:
The default path for the data folder that is specified in the [example script](https://github.com/una-auxme/MeshGraphNets.jl/blob/main/examples/cylinder_flow/cylinder_flow.jl) is:

`{path_to_cylinder_flow.jl}/data/datasets/`
{path_to_cylinder_flow.jl}/data/datasets/

## Training the network
## Training the Network

In order to train the system you can simply comment in/out the lines of code provided in the script:

Expand All @@ -49,7 +49,7 @@ train_network(
)
```

## Evaluating the system
## Evaluating the Trained Network

The same applies to evaluating the system. Simply comment in/out the desired lines of code:

Expand All @@ -74,8 +74,8 @@ eval_network(

```

## Addition: Arguments for training & evaluation
## Addition: Arguments for Training & Evaluation

The arguments provided at the top of the example script correspond to the default values that were used by DeepMind. You can change them to see how that affects runtime and accuracy of the network.
The arguments provided at the top of the example script and in the function call correspond to the default values that were used by DeepMind. You can change them to see how that affects runtime and accuracy of the network.

The arguments inside the function calls can also be modified. An explanation can be found in the [documentation](https://una-auxme.github.io/MeshGraphNets.jl/dev/train_eval).
The arguments inside the function calls can also be modified. An explanation can be found in section [Training & Evaluation](https://una-auxme.github.io/MeshGraphNets.jl/dev/train_eval).
1 change: 0 additions & 1 deletion docs/src/graph_net_core.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,5 +32,4 @@ parse_edges
one_hot
minmaxnorm
mse_reduce
inverse_data
```
4 changes: 2 additions & 2 deletions docs/src/overview.md
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Overview

This page provides links for the different sections depending on your use case. You can start with one of the examples if you want to try out the package or create your own network of MeshGraphNets for your use case.
This page provides links for the different sections of this documentation. You can start with one of the examples if you want to try out the package or create your own network via [*MeshGraphNets.jl*](https://github.com/una-auxme/MeshGraphNets.jl) for your use case.

References for training strategies and the core package [GraphNetCore.jl](https://github.com/una-auxme/GraphNetCore.jl) are also listed.
References for training strategies and the core package [*GraphNetCore.jl*](https://github.com/una-auxme/GraphNetCore.jl) are also listed.

**Getting Started**

Expand Down
41 changes: 20 additions & 21 deletions docs/src/training_data.md
View file
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Training data
# Training Data

If you want to train your own network via *MeshGraphNets.jl* you have to provide your data files and a correspond metafile in a specific manner. This section describes the structure of those files.
If you want to train your own network via *MeshGraphNets.jl* you have to provide your data files and a corresponding metadata file in a specific manner. This section describes the structure of those files.

## Folder structure
## Folder Structure

Your files should be placed inside the same folder so that *MeshGraphNets.jl* can find them. The structure of your files should for example look like this:
Your files have to be placed inside the same folder so that *MeshGraphNets.jl* can find them. The structure of your files should for example look like this:

- data
- datasets
Expand All @@ -15,11 +15,11 @@ Your files should be placed inside the same folder so that *MeshGraphNets.jl* ca

## Files for Training, Evaluation and Testing

For each of the steps a separate file has to be provided (see [Folder Structure](#folder-structure)). It is recommended to use HDF5 files since they are easier to work with in Julia. You can also use TFRecord files, however an implementation is only provided for handling files that are exactly like the ones from the [CylinderFlow](https://una-auxme.github.io/MeshGraphNets.jl/dev/cylinder_flow) example.
For each of the steps a separate file has to be provided (see [Folder Structure](#folder-structure)). It is recommended to use `HDF5` files since they are easier to work with in Julia. You can also use `TFRecord` files, however an implementation is only provided for handling files that are exactly like the ones from the [CylinderFlow](https://una-auxme.github.io/MeshGraphNets.jl/dev/cylinder_flow) example.

> ⚠️ The following sections only contain explanations for using HDF5 files.
> ⚠️ The following sections only contain explanations for using `HDF5` files.
### Data file structure (`train.h5`, `valid.h5`, `test.h5`)
### Data File Structure (`train.h5`, `valid.h5`, `test.h5`)

Your data files should have the following structure:

Expand All @@ -35,20 +35,19 @@ Your data files should have the following structure:

## File for Metadata (`meta.json`)

Your metadata file also has to follow a defined structure. Since the metadata file for CylinderFlow is handled differently, two files are explaned in the following.
Your metadata file also has to follow a defined structure. Since the metadata file for the [CylinderFlow](https://una-auxme.github.io/MeshGraphNets.jl/dev/cylinder_flow) example is handled differently, two files are explained in the following.

### Default metadata
### Default Metadata

The default structure that you should use for your metadata is the following (example derived from the CylinderFlow example, not the actual metadata file):
The default structure that you should use for your metadata is the following (example derived from the [CylinderFlow metadata](#cylinderflow-metadata), not an actual metadata file):

```json
{
"dt": "time", # key inside the HDF5 file for timesteps
"trajectory_length": 600, # length of trajectories (i.e. number of steps) inside data files
"dims": [ # dimensions of the mesh (here a mesh of dimensions (5, 3, 5))
"dims": [ # dimensions of the mesh (here a mesh of dimensions (5, 3))
5,
3,
5
3
],
"feature_names": [ # names of all features, mesh_pos and node_type are required
"mesh_pos",
Expand Down Expand Up @@ -77,7 +76,7 @@ The default structure that you should use for your metadata is the following (ex
},
"velocity": {
"key": "cl_mesh[%d,%d].velocity",
"dim": 1,
"dim": 2,
"type": "dynamic",
"dtype": "float32"
}
Expand All @@ -93,32 +92,32 @@ Here is a detailed description of each possible metadata:
| `"trajectory_length"` | Integer | each trajectory needs to have the same length i.e. the same amount of steps |
| `"dims"` | Vector{Integer} | dimensions can be 1-, 2- or 3-dimensional |
| `"feature_names"` | Vector{String} | list all features that are also used as an input of the network |
| `"target_features"` | Vector{String} | list all features that the network should predict, they have to be part of `"feature names"` |
| `"target_features"` | Vector{String} | list all features that the network should predict, they have to be part of `"feature_names"` |

Each feature has its own metadata:

| Feature Metadata | Data Type | Description |
|------------------|-----------|------------------------------------------------------------------------------------------------------|
| `"key"` | String | further description of HDF5 key structure are below |
| `"split"` | Bool | keys are split at the end (e.g. `"cl_mesh[%d].pos[1]"` and `"cl_mesh[%d].pos[2]"`) |
| `"key"` | String | further description of `HDF5` key structure are below |
| `"split"` | Bool | keys are split at the end (e.g. `"cl_mesh[%d,%d].pos[1]"` and `"cl_mesh[%d,%d].pos[2]"`) |
| `"dim"` | Integer | dimension of the feature |
| `"type"` | String | `"static"` if the feature does not change over time, `"dynamic"` otherwise |
| `"dtype"` | String | possible datatypes: `"int32"`, `"float32"`, `"Bool"` |
| `"onehot"` | Bool | can be used if for example you want to convert types represented as Integer to a onehot vector |
| `"onehot"` | Bool | can be used if you want to convert types represented as Integer to a onehot vector |
| `"data_min"` | Float | if you specify `"data_min"` and `"data_max"`, offline normalization is used, online otherwise |
| `"data_max"` | Float | see `"data_min"` |
| `"target_min"` | Float | equivalent to `"data_min"` and `"data_max"`, specifies interval for normalization target |
| `"target_max"` | Float | see `"target_min"` |

The structure for the HDF5 key has to follow one rule:
The structure for the `HDF5` key has to follow one rule:

> ⚠️ Square brackets are exlusively used
> - once for the index of the mesh point (e.g. `"cl_mesh[%d,%d].cellType"`) and
> - once at the end of the key if the feature `"split"` is set to `true`.
### CylinderFlow metadata
### CylinderFlow Metadata

The [metadata file](https://github.com/una-auxme/MeshGraphNets.jl/tree/main/examples/cylinder_flow/meta.json) (taken from the from [CylinderFlow](https://una-auxme.github.io/MeshGraphNets.jl/dev/cylinder_flow) example) has the following structure:
The [metadata file](https://github.com/una-auxme/MeshGraphNets.jl/blob/main/examples/cylinder_flow/meta.json) (taken from the from [CylinderFlow](https://una-auxme.github.io/MeshGraphNets.jl/dev/cylinder_flow) example) has the following structure:

```json
{
Expand Down
2 changes: 1 addition & 1 deletion src/MeshGraphNets.jl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ Starts the training process with the given configuration.
- `SingleShooting`
- `MultipleShooting`
See [strategies.jl](@ref) for reference.
See [documentation](https://una-auxme.github.io/MeshGraphNets.jl/dev/cylinder_flow) for reference.
# Returns
- `mgn`: The trained network as a [`GraphNetwork`](@ref) struct.
Expand Down
4 changes: 2 additions & 2 deletions src/strategies.jl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -273,7 +273,7 @@ Useful for initial training of the system since it it faster than training with
# Keyword Arguments
- `window_size`: Number of steps from each trajectory (starting at the beginning) that are used for training. If the number is zero then the whole trajectory is used.
- `eigeninformed`: Whether eigeninformed training is used at each training step or not. See [util.jl](@ref) for reference.
- `eigeninformed`: Whether eigeninformed training is used at each training step or not. See [utils.jl](https://github.com/una-auxme/MeshGraphNets.jl/blob/main/src/utils.jl) for reference.
- `plot_progress`: Whether the training progress is plotted or not.
"""
Expand All @@ -294,7 +294,7 @@ Similar to Collocation, but timesteps are sampled randomly from the trajectory i
# Keyword Arguments
- `window_size`: Number of steps from each trajectory (starting at the beginning) that are used for training. If the number is zero then the whole trajectory is used.
- `eigeninformed`: Whether eigeninformed training is used at each training step or not. See [util.jl](@ref) for reference.
- `eigeninformed`: Whether eigeninformed training is used at each training step or not. See [utils.jl](https://github.com/una-auxme/MeshGraphNets.jl/blob/main/src/utils.jl) for reference.
- `plot_progress`: Whether the training progress is plotted or not.
"""
Expand Down

2 comments on commit 353ef2f

@JulianTrommer
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JuliaRegistrator register

@JuliaRegistrator
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Registration pull request created: JuliaRegistries/General/96613

Tip: Release Notes

Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.

@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.

Tagging

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a v0.1.1 -m "<description of version>" 353ef2f80ab1c54bb0f7774e9662b07f523717c5
git push origin v0.1.1

Please sign in to comment.