= read_nb('../../tests/image.ipynb')
+ test_nb assert test_nb.cells[0].outputs[0].data['image/png'][-1] == "\n" # Make sure it was not converted by acccident
+
+ clean_nb(test_nb)assert test_nb.cells[0].outputs[0].data['image/png'][-1] != "\n"
clean
+To avoid pointless conflicts while working with jupyter notebooks (with different execution counts or cell metadata), it is recommended to clean the notebooks before committing anything (done automatically if you install the git hooks with nbdev_install_hooks
). The following functions are used to do that.
Trust
++ +
nbdev_trust
++++nbdev_trust (fname:str=None, force_all:bool=False)
Trust notebooks matching fname
+ | Type | +Default | +Details | +
---|---|---|---|
fname | +str | +None | +A notebook name or glob to trust | +
force_all | +bool | +False | +Also trust notebooks that haven’t changed | +
Clean
++ +
clean_nb
++++clean_nb (nb, clear_all=False, allowed_metadata_keys:list=None, + allowed_cell_metadata_keys:list=None, clean_ids=True)
Clean nb
from superfluous metadata
+ | Type | +Default | +Details | +
---|---|---|---|
nb | ++ | + | The notebook to clean | +
clear_all | +bool | +False | +Remove all cell metadata and cell outputs? | +
allowed_metadata_keys | +list | +None | +Preserve the list of keys in the main notebook metadata | +
allowed_cell_metadata_keys | +list | +None | +Preserve the list of keys in cell level metadata | +
clean_ids | +bool | +True | +Remove ids from plaintext reprs? | +
Jupyter adds a trailing to images in cell outputs. Vscode-jupyter does not.
+Notebooks should be brought to a common style to avoid unnecessary diffs:
The test notebook has metadata in both the main metadata section and contains cell level metadata in the second cell:
+= read_nb('../../tests/metadata.ipynb')
+ test_nb
+assert {'meta', 'jekyll', 'my_extra_key', 'my_removed_key'} <= test_nb.metadata.keys()
+assert {'meta', 'hide_input', 'my_extra_cell_key', 'my_removed_cell_key'} == test_nb.cells[1].metadata.keys()
After cleaning the notebook, all extra metadata is removed, only some keys are allowed by default:
+
+ clean_nb(test_nb)
+assert {'jekyll', 'kernelspec'} == test_nb.metadata.keys()
+assert {'hide_input'} == test_nb.cells[1].metadata.keys()
We can preserve some additional keys at the notebook or cell levels:
+= read_nb('../../tests/metadata.ipynb')
+ test_nb ={'my_extra_key'}, allowed_cell_metadata_keys={'my_extra_cell_key'})
+ clean_nb(test_nb, allowed_metadata_keys
+assert {'jekyll', 'kernelspec', 'my_extra_key'} == test_nb.metadata.keys()
+assert {'hide_input', 'my_extra_cell_key'} == test_nb.cells[1].metadata.keys()
Passing clear_all=True
removes everything from the cell metadata:
= read_nb('../../tests/metadata.ipynb')
+ test_nb =True)
+ clean_nb(test_nb, clear_all
+assert {'jekyll', 'kernelspec'} == test_nb.metadata.keys()
+1].metadata, {}) test_eq(test_nb.cells[
Passing clean_ids=True
removes id
s from plaintext repr outputs, to avoid notebooks whose contents change on each run since they often lead to git merge conflicts. For example:
<PIL.PngImagePlugin.PngImageFile image mode=L size=28x28 at 0x7FB4F8979690>
+becomes:
+<PIL.PngImagePlugin.PngImageFile image mode=L size=28x28>
++ +
process_write
++++process_write (warn_msg, proc_nb, f_in, f_out=None, disp=False)
+ +
nbdev_clean
++++nbdev_clean (fname:str=None, clear_all:bool=False, disp:bool=False, + stdin:bool=False)
Clean all notebooks in fname
to avoid merge conflicts
+ | Type | +Default | +Details | +
---|---|---|---|
fname | +str | +None | +A notebook name or glob to clean | +
clear_all | +bool | +False | +Remove all cell metadata and cell outputs? | +
disp | +bool | +False | +Print the cleaned outputs | +
stdin | +bool | +False | +Read notebook from input stream | +
By default (fname
left to None
), all the notebooks in config.nbs_path
are cleaned. You can opt in to fully clean the notebook by removing every bit of metadata and the cell outputs by passing clear_all=True
.
If you want to keep some keys in the main notebook metadata you can set allowed_metadata_keys
in settings.ini
. Similarly for cell level metadata use: allowed_cell_metadata_keys
. For example, to preserve both k1
and k2
at both the notebook and cell level adding the following in settings.ini
:
...
+allowed_metadata_keys = k1 k2
+allowed_cell_metadata_keys = k1 k2
+...
++ +
clean_jupyter
++++clean_jupyter (path, model, **kwargs)
Clean Jupyter model
pre save to path
This cleans notebooks on-save to avoid unnecessary merge conflicts. The easiest way to install it for both Jupyter Notebook and Lab is by running nbdev_install_hooks
. It works by implementing a pre_save_hook
from Jupyter’s file save hook API.
Hooks
++ +
nbdev_install_hooks
++++nbdev_install_hooks ()
Install Jupyter and git hooks to automatically clean, trust, and fix merge conflicts in notebooks
+See clean_jupyter
and nbdev_merge
for more about how each hook works.