BioSegment is a software stack to enable segmentation of microscopy data using machine learning models.
- Ubuntu 18.04, 20.04
- Windows 10
Install:
- Docker
- Conda
- docker-compose
- at least
version 1.26.2, build eefe0d31
- at least
Note that the Docker images take up ~8GB of disk space.
Create a data folder according to the specification or use the provided script to create one at data/. Edit data/setup.json to add more datasets.
bash scripts/create-data-folder.sh
Create and run the BioSegment stack using docker-compose. The first time could take ~10 minutes.
docker-compose up -d --build- BioSegment is now running in dev mode at localhost
- Go to
http://localhost/dash/viewerto see the Dash viewer on the default dataset. - For more information on running a GPU worker to create a segmentation, see the documentation.
dash_frontend/for dashboard and segmentation viewer, using Python Dash: http://localhost/dashfrontend/for account managment, built with Javascript Vue, with routes handled based on the path: http://localhostbackend/, JSON based web API using Python FastAPI based on OpenAPI: http://localhost/api/- Automatic interactive documentation with Swagger UI (from the OpenAPI backend): http://localhost/docs
- Alternative automatic documentation with ReDoc (from the OpenAPI backend): http://localhost/redoc
gpu_worker/Python Celery task runner to run PyTorch models (e.g. neuralnets) directly on the host GPUdiagrams/with system diagrams using mermaid- extra
- PGAdmin, PostgreSQL web administration: http://localhost:5050
- Flower, administration of Celery tasks: http://localhost:5555
- Traefik UI, to see how the routes are being handled by the proxy: http://localhost:8090
.envcontains all login credentials and configurations
See Developer Guide.
In .env a ROOT_DATA_FOLDER is defined with the default value of ./data, relative to this project folder. The structure of the folder is documented in the User Guide.
Overwriting ROOT_DATA_FOLDER can be done using an environment variable:
# On Linux:
ROOT_DATA_FOLDER=/personal/data/folder/location docker-compose up -d --build
# On Windows:
set ROOT_DATA_FOLDER=X:/biosegment/data
docker-compose up -d --buildWe can use TensorBoard to visualize model (re)training.
Install Tensorboard with
pip install tensorboard
Then start tensorboard in the terminal with logdir pointing to the directory where biosegment stores the model training files, for example
tensorboard --logdir "/home/johndoe/code/biosegment/data/models/EMBL/my retrained model 1"
The tensorboard can be viewed in the browser at http://localhost:6006.
The .env file is version controlled, so it should not hold production secrets. Create from it a seperate .env.prod file with different secrets and credentials. This new file can be used instead of the default with an env-file option parameter:
docker-compose --env-file .env.prod up