Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding docs to README #85

Merged
merged 79 commits into from
Oct 18, 2023
Merged
Show file tree
Hide file tree
Changes from 62 commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
ae1c89f
remove gh_pages template files
ruaridhg Oct 16, 2023
95b8a73
Added /doc folder and updated README.md
ruaridhg Oct 16, 2023
1d76f1f
added separate, detailed mat_geom.md and updated README
ruaridhg Oct 16, 2023
10dd12e
Added details about how to avoid errors for UD props
ruaridhg Oct 16, 2023
edd00ca
define_prop init added
ruaridhg Jun 6, 2023
f63e25c
Properties.py imports collection of socket properties (geom-like) and…
ruaridhg Jun 6, 2023
07002e0
Combined string list operators with geom operators
ruaridhg Jun 6, 2023
22149aa
Added custom props as example of UI list - remove after integrated
ruaridhg Jun 6, 2023
fd824b6
Skeleton of collection_UD, need to sort sockets and remove input nodes
ruaridhg Jun 6, 2023
deff6fa
Geom skeleton copied, could use subpanel for UIlist
ruaridhg Jun 6, 2023
c44ec8c
minor refactoring with combination of UIlist and geom code loading wi…
ruaridhg Jun 7, 2023
2901c03
Added UIlist as subpanel in UD panel
ruaridhg Jun 7, 2023
b30f8df
UIlist print clear list add and remove works
ruaridhg Jun 7, 2023
f847286
Introducing collection gngs as UD props equivalent
ruaridhg Jun 7, 2023
449d7de
Removed collection gngs - created branch off this one to include alte…
ruaridhg Jun 8, 2023
66ecab8
Added collection_gngs.py
ruaridhg Jun 8, 2023
72287f9
UIlist working and UD-sockets code in progress
ruaridhg Jun 8, 2023
9c69e6f
UIlist not disappearing and adds subpanels
ruaridhg Jun 8, 2023
b45165d
removed input and outputs from nodes code
ruaridhg Jun 8, 2023
ed97d72
Removed old custom props folder
ruaridhg Jun 8, 2023
76612e1
custom + custom_idx in properties.py works but needs moving into othe…
ruaridhg Jun 8, 2023
217c7ec
changed UD props collection from UD socket properties to materials.so…
ruaridhg Jun 9, 2023
16b11c1
UD property can appear in subplanel when hardcoded camera.location
ruaridhg Jun 9, 2023
8ca6b57
Map props to attr for vector float3d working with attr_get_type
ruaridhg Jun 9, 2023
500a1f2
able to split and pass in attribute_only_str
ruaridhg Jun 9, 2023
1ca8858
Tidy up before checking string is valid property
ruaridhg Jun 9, 2023
2658421
fake property ranch does not appear in subpanel - list_UD_props modified
ruaridhg Jun 12, 2023
933b5c9
Multiple Vector3 properties can be added
ruaridhg Jun 12, 2023
9ada8c8
Added float 1d to attr_type, need to fix subpanels per attr_type
ruaridhg Jun 12, 2023
a0afb85
Added bool and int data types UD prop - int min-max needs fixing
ruaridhg Sep 12, 2023
9a823c0
Fixed duplicated subpanels min-max vals for UD props - only names dup…
ruaridhg Sep 14, 2023
b0568b8
obj_path_resolve_test.py added and if-else in ui.py
ruaridhg Sep 15, 2023
c25b090
Fixed utils error
ruaridhg Sep 25, 2023
dd73e9d
Testing bug fix for int min-max custom prop
ruaridhg Sep 27, 2023
e58e078
Set min-max for int in config.py
ruaridhg Sep 27, 2023
07fe4a5
branch off rmg/define_prop - commented out update_sockets_collection …
ruaridhg Sep 27, 2023
01522f3
Added functionality to define_prop randomise button for int
ruaridhg Sep 28, 2023
195e1a7
Commented out print statements in ui.py
ruaridhg Sep 28, 2023
9248f48
Fixed naming of each subpanel
ruaridhg Sep 28, 2023
3f1b3fe
Commented out attempt at default value not found fix
ruaridhg Sep 28, 2023
49d7c63
bpy.context.scene.objects['Cube'].collision.absorption case subpanel …
ruaridhg Sep 29, 2023
74eddfa
mod used in get_attr_only_str
ruaridhg Oct 2, 2023
037193e
obj_path_resolve_test.py now includes randomising
ruaridhg Oct 3, 2023
ceaf62a
Can randomise all UD_props and hardcoded Cube str replaced with idx
ruaridhg Oct 4, 2023
e35b76f
Added previous seed code to operators execute similar to other panels
ruaridhg Oct 4, 2023
2f70b85
Define_props bool randomise working
ruaridhg Oct 4, 2023
f0bb857
Added UD examples that don't work to test file
ruaridhg Oct 9, 2023
6aae5db
Removed redundant up/down arrows from UD props panel
ruaridhg Oct 9, 2023
b8c4e80
Fixed bpy.data.cameras[Camera].dof.aperture_fstop example
ruaridhg Oct 12, 2023
bceeacc
bpy.data.objects[Cube].rotation_euler now working
ruaridhg Oct 13, 2023
e14cdf6
euler property added to UD sckt props
ruaridhg Oct 13, 2023
f26e1cd
test file updated
ruaridhg Oct 13, 2023
d2e944f
Added check_idx_test file
ruaridhg Oct 13, 2023
4c47ca2
Tidied up attr_set_val print statements
ruaridhg Oct 16, 2023
0a1e08f
Tidy up code/remove print statements
ruaridhg Oct 16, 2023
1a13210
Register all panels
ruaridhg Oct 16, 2023
80f9542
Removed redundant tests - kept per_frame placeholders
ruaridhg Oct 16, 2023
ebe5627
Register all panels
ruaridhg Oct 16, 2023
2abc7c3
Changed example for UD prop in README
ruaridhg Oct 16, 2023
1dcb01d
Remove placeholder doc
ruaridhg Oct 16, 2023
47793a6
Removed original Dev notes.md - now in doc folder
ruaridhg Oct 16, 2023
9cea040
Added input_output doc
ruaridhg Oct 16, 2023
8b5b6b8
Apply suggestions from code review re docs
ruaridhg Oct 17, 2023
cdca5f2
Modified README with boxes and added UD_panel docs file
ruaridhg Oct 17, 2023
66a7462
Modified .md docs mentioned in README.md
ruaridhg Oct 17, 2023
70772b1
Delete test in Blender files for public release
ruaridhg Oct 17, 2023
ef119c3
Updated docs for MacOS pytest steps and deleted unused test folder
ruaridhg Oct 17, 2023
48015dc
Tidied up define_prop and config
ruaridhg Oct 17, 2023
e145726
Tidied up geometry and materials
ruaridhg Oct 17, 2023
11509d5
Tidied up utils
ruaridhg Oct 17, 2023
23b9312
Tidied up install and enable addons
ruaridhg Oct 17, 2023
bfab4a4
Fixed geometry testing
ruaridhg Oct 17, 2023
a4f31c6
Added UD props rand per frame
ruaridhg Oct 17, 2023
f20abd4
Randomise all and save button integrated with new ui layout
ruaridhg Oct 17, 2023
bd0b76e
Tidy up random_all
ruaridhg Oct 17, 2023
147477d
UD panel works
ruaridhg Oct 18, 2023
1c1b5e8
merge conflict testing resolved
ruaridhg Oct 18, 2023
f765763
Merge branch 'main' into rmg/docs_on_gh
ruaridhg Oct 18, 2023
d76630a
Print statement for outputs saved
ruaridhg Oct 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
119 changes: 96 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,42 @@
# Blender_Randomiser

## Installation

Blender Randomiser is a Blender add-on that allows different Blender properties to be randomised between minimum and maximum bounds. This can be achieved either by:
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved
- Manually setting the bounds in the UI Randomiser panel located in the Geometry Nodes space
- Using an input `.json` [file](/input_bounds.json) to [set the bounds from the command line](/doc/input_output.md)

## Purpose

The original and main use case of this add-on is for rendering synthetic (near) photo-realstic data for laparoscopic surgery, with a view to generating large scale labelled data sets for ML applications in surgery. With that context to replicate the different camera positions used in surgery as well as the shape and appearance of the tissues involved with surgery, the three main components (contained with separate UI panels) to be randomised are:
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved
- **Camera transforms** (location and Euler rotation with toggle for randomising in absolute or relative i.e. delta terms)
- [**Geometry**](/doc/Materials_geometry_panel.md)
- [**Materials**](/doc/Materials_geometry_panel.md)

ruaridhg marked this conversation as resolved.
Show resolved Hide resolved
Additionally, there is a **user defined properties** panel where the user can specify the full path of a property they want to randomised and it will create a subpanel for each of these properties with min-max bounds. Examples include:
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved
- bpy.data.objects["Cube"].location (Float 3D)
- bpy.context.scene.frame_current (int 1D)
- bpy.data.objects["Sphere"].scale (Float 3D)
- bpy.data.objects["Cube"].collision.absorption (Float 1D)
- bpy.data.cameras["Camera"].dof.aperture_fstop (Float 1D)
- bpy.data.objects["Cube"].rotation_euler (Euler)

It should be noted that bpy.data.objects["Cube"].location[0] or bpy.data.cameras["Camera"].location[0] will throw an error if trying to add to the UD panel. However, bpy.data.objects["Cube"].location will work fine and if you only want to change the x-location then you can leave the y-/z-location bounds as 0 (min) to 0 (max) and the Cube won't be moved in those directions.

Another example that will not work is if the object has a '.' in the middle of the name i.e. "Cube.001" so please remove this if you want to have multiple cubes in your scene.
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved

Other functionality includes:
- Toggle on/off button for including/excluding each property in the randomisation
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved
- Seed panel to set the random seed
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved
- Randomise properties per frame
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved
- [Save Parameter panel](/doc/input_output.md) with outputs saved to `.json` [file](/output_randomisations_per_frame1697116725.310647.json) with a timestamp


## Installation
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved
First, clone the repository in your desired directory
```
git clone https://github.com/UCL/Blender_Randomiser.git
```

### To install via Blender settings
1. Zip the following directory:
`Blender_Randomiser/randomiser/`
2. Launch Blender
3. Navigate to Edit > Preferences
4. On the left-hand side panel, select 'Add-ons'
5. On the top-right select 'Install an add-on'
6. Browse to select the randomiser.zip file.
- In the background, Blender will copy the file to the add-ons folder and unzip it
- In MacOS, this add-on directory is usually located at:
`/Users/user/Library/Application Support/Blender/3.4/scripts/addons`
(see [Blender directory layout](https://docs.blender.org/manual/en/latest/advanced/blender_directory_layout.html) for further details)

7. Click on the checkbox next to the installed add-on to enable it.
- The add-on can be found by searching for 'randomisation' in the top right search bar.
- Before searching, note that the checkbox to show enabled add-ons only ('Enabled Add-ons Only') may be selected!

Note that:
- if you install an add-on, it will overwrite any pre-existing add-on with the same name.
- Blender will only have access to the add-ons copied to any of the add-ons directories (see [Blender directory layout](https://docs.blender.org/manual/en/latest/advanced/blender_directory_layout.html)).
- Remember that if you modify the code, you will need to create a new zip file with the latest version and repeat steps 1-7

## To install via command line
1. Launch a terminal at the cloned directory
- You should be at the `Blender_Randomiser` directory
Expand All @@ -37,3 +46,67 @@ Note that:
```
- This will zip the `randomiser` subdirectory, open the `sample.blend` file with Blender, and use Blender's Python interpreter to execute the `install_and_enable_addons.py` script.
- The `install_and_enable_addons.py` script installs and enables any add-ons that are passed as command line arguments (add-ons can be one Python file or a zip file)
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved

Alternatively, install [manually](/doc/Install_addon_manually.md) via Blender settings

## To install via command line with additional inputs
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved

For step 2 above, run the following bash script instead:
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved

sh install_randomiser.sh
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved

This will follow the same steps as above, but there are optional inputs as well:
- `--seed 32` which is an input to Blender
- `--input ./input_bounds.json` (input to `install_and_enable_addons.py`)
- `--output ./output_randomisations_per_frame1697116725.310647.json` (input to `install_and_enable_addons.py`)

## Learning and tutorials

[Add-on tutorial from docs](https://docs.blender.org/manual/en/3.4/advanced/scripting/addon_tutorial.html)

[Developer documentation: Python](https://wiki.blender.org/wiki/Python)
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved


## License and copyright

[BSD 3-Clause License](/LICENSE)

## Testing

[Testing script](/tests/test_integration/test_installing_and_enabling.py) which used [pytest-blender plugin](https://github.com/mondeja/pytest-blender#pytest-blender)

ruaridhg marked this conversation as resolved.
Show resolved Hide resolved
To use the plugin, you need to install `pytest` and all other dependencies used in testing (`pytest-cov`) in the site-packages of Blender's Python. The repo provides some guidance for this [here](https://github.com/mondeja/pytest-blender#usage). It is important to make sure you use the correct Python interpreter and pip (Blender's ones) when installing `pytest` and `pytest-cov`.
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved

#### Linux:
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved
Running the following code in a script within Blender:

`import pip `

`pip.main(["install", "pytest", "--user"])`

`pip.main(["install", "pytest-cov", "--user"])`

`pip.main(["install", "pytest-blender", "--user"])`


#### MacOS:

The following steps were need on MacOS:
- `get-pip.py` downloaded (this step may not be needed since newer versions of Blender have pip installed already in the Blender python)
- Changing Mac permissions to grant full disk access from where you're running pytest i.e. VS code or terminal
- `/Applications/Blender.app/Contents/Resources/3.4/python/bin/python3.10 -m pip install pytest -t "/Applications/Blender.app/Contents/Resources/3.4/python/lib/python3.10/site-packages"` the target flag -t makes sure that the installation ends up in the correct place


### Running Tests

Run `pytest` on the CLI within the project folder to test the camera transforms, materials and geometry panels as well as testing the `.blend` file loads, the add-on is installed and enabled and the randomising works per frame and with the seed set.

N.B. Alternative approach would be to use [Blender as a python module](https://wiki.blender.org/wiki/Building_Blender/Other/BlenderAsPyModule), but the only pytest-blender plugin approach is implemented in this repo.
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved


## Dev notes
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved

Please see [Dev Notes](./doc/Dev_notes.md)


## Contributions
11 changes: 0 additions & 11 deletions Dev notes.md → doc/Dev_notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,10 +122,6 @@ with open(filepath, 'rb') as file:
## Add-ons


[Add-on tutorial from docs](https://docs.blender.org/manual/en/3.4/advanced/scripting/addon_tutorial.html)

[Developer documentation: Python](https://wiki.blender.org/wiki/Python)

*An add-on is simply a Python module with some additional requirements so Blender can display it in a list with useful information.*


Expand All @@ -149,14 +145,7 @@ To enable an already installed add-on from the CLI:

[Accepted space-regions combinations](https://blender.stackexchange.com/a/97268)

## Testing
[PR 62](https://github.com/UCL/Blender_Randomiser/pull/62) is a first attempt at testing with Blender's Python interpreter.

I used this [pytest-blender plugin](https://github.com/mondeja/pytest-blender#pytest-blender), which seems to have very useful fixtures and also provides some steps towards CI.

To use the plugin, we need to install `pytest` and all other dependencies used in testing (`pytest-cov` mainly) in the site-packages of Blender's Python. The repo provides some guidance for this [here](https://github.com/mondeja/pytest-blender#usage). It is important to make sure you use the correct Python interpreter and pip (Blender's ones) when installing `pytest` and `pytest-cov`.

I think an alternative approach would be to use [Blender as a python module](https://wiki.blender.org/wiki/Building_Blender/Other/BlenderAsPyModule), but the pytest-blender plugin approach seems easier.


## Handover notes 6/6/2023
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved
Expand Down
21 changes: 21 additions & 0 deletions doc/Install_addon_manually.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
### To install via Blender settings
1. Zip the following directory:
`Blender_Randomiser/randomiser/`
2. Launch Blender
3. Navigate to Edit > Preferences
4. On the left-hand side panel, select 'Add-ons'
5. On the top-right select 'Install an add-on'
6. Browse to select the randomiser.zip file.
- In the background, Blender will copy the file to the add-ons folder and unzip it
- In MacOS, this add-on directory is usually located at:
`/Users/user/Library/Application Support/Blender/3.4/scripts/addons`
(see [Blender directory layout](https://docs.blender.org/manual/en/latest/advanced/blender_directory_layout.html) for further details)

7. Click on the checkbox next to the installed add-on to enable it.
- The add-on can be found by searching for 'randomisation' in the top right search bar.
- Before searching, note that the checkbox to show enabled add-ons only ('Enabled Add-ons Only') may be selected!

Note that:
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved
- if you install an add-on, it will overwrite any pre-existing add-on with the same name.
- Blender will only have access to the add-ons copied to any of the add-ons directories (see [Blender directory layout](https://docs.blender.org/manual/en/latest/advanced/blender_directory_layout.html)).
- Remember that if you modify the code, you will need to create a new zip file with the latest version and repeat steps 1-7
21 changes: 21 additions & 0 deletions doc/Materials_geometry_panel.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
## Materials
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved
- A panel to randomise properties relative to the material nodes:
- nodes and node groups are aggregated based on the material they belong to.
- only materials with use_nodes=True are added to the panel. By default, use_nodes is set to True, but this is a convenient way to add/remove materials from the panel.
- If new nodes that are marked for randomisation are added or deleted, these appear automatically in the randomisation panel.
- Recursive node groups are accepted.
- Convenience functions were added to visualise the node graph per material, constrain the min and max values for the randomisation and unselect nodes that are candidates for randomisation but are unlinked.

### When is a new material slot automatically added?
Clicking the name of the material in a subpanel header shows its node graph. If a material is clicked and it has no slot assigned, a new slot will be created for it



## Geometry
- A panel to randomise properties relative to the geometry nodes:
- nodes are aggregated based on the node group they belong to.
- Same functionalities as in material nodes panel: new or deleted nodes are automatically added, recursive node groups are accepted,etc.


### When is a new modifier automatically added to an object’s geometry?
If a geometry node group is not inside another node group, and is not linked to a modifier of the currently active object, a new modifier is created and the geometry node group is linked to it.
13 changes: 13 additions & 0 deletions doc/input_output.md
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
## Min-max bounds via input file
[Example input .json file](/input_bounds.json) for setting bounds. Currently can be used for setting camera transforms, materials and geometry min-max boundaries. Could be extended in the future to use parameter names from the [output .json file](/output_randomisations_per_frame1697116725.310647.json).
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved

The input file is read in as a dictionary with the key names following the pattern of "Values" + "material or geometry name" + "socket name":
- "Values Geometry Nodes.001RandomSize.001"
- "Values MaterialRandomMetallic"


## Save parameter outputs button

In this panel, you can set how many frames you want to randomise before clicking the Save Parameter Outputs button. This will randomise all the variables that are toggled on for randomisation and then a dictionary of parameter values are saved to an output .json file.

Due to the layout of geometry/materials panels, they are saved as a dictionary within the overall dictionary.
ruaridhg marked this conversation as resolved.
Show resolved Hide resolved
174 changes: 0 additions & 174 deletions docs/README.md

This file was deleted.

Loading
Loading