Caution
This is a FastAPI (with uvicorn and pydantic V2) based server component for the new ZoneMinder ML add-on. Please see the workflow docs for information on how the server processes inference requests. There is also rudimentary support for color detection of cropped bounding boxes.
Note
At the moment, only pretrained models are supported. Custom models are planned for the future (or you can open PR's!).
The server currently has basic support for several ML backends and hardware accelerators.
- OpenCV (DarkNet - YOLO version 3, 4, 7 - [5, 6] WIP) [CUDA support]
- PyTorch -
torchvision(CUDA supported, ROCm WIP) - ONNXRuntime (YOLO v8, YOLO-NAS, [YOLO v10 WIP]) [CUDA supported, ROCm WIP]
- TensorRT (YOLO v8, YOLO-NAS, [YOLO v10 WIP]) [compile trtexec locally]
- pycoral (coral.ai Edge TPU)
- openalpr (compile local binary)
- face-recognition based on D-Lib (CUDA support)
- HTTP
- Open an issue or pull request to get a backend/api supported
- CPU
- OpenVINO is planned
- Nvidia GPU (CUDA / cuDNN / Tensor RT)
- AMD GPU (Pytorch/onnxruntime ROCm) WIP / Untested
- Coral.ai Edge TPU
See the docs/hardware directory for more information on hardware support.
Please see the installation docs for more information.
Important
The server defines models in its configuration file models: section. The name: of the model is how it will
be called in inference requests. The client will use the name: to send requests to the server.
The name: of each of the models: defined is the link between zomi-server and zomi-client.
- A client sends an inference request to the server with at least 1 image and 1 model name;
yolo v10. - The server will look in its internal state to see if there is a model named
yolo v10. - If the model is found, enabled and no issues loading into memory, the server will run the image through the model and return the results.
- if the model is not found/enabled/loaded, the server will return an error message. WIP
Tip
Swagger UI is available at the server root: http://<server>:<port>/
The server uses FastAPIs built-in Swagger UI which shows available endpoints, response/request schema and serves as self-explanatory documentation.
Warning
Make sure to authorize first! All requests require a valid JWT token.
If you haven't enabled auth in the server.yml config file, any username:password combo will work.
Caution
You can enable and disable authentication, but all requests must have a valid JWT token. When authentication is disabled, the login endpoint will accept any username:password combo and supply a valid usable token.
The default user is imoz with the password zomi.
Tip
After installation, there should be a system-wide mlapi script command available in /usr/local/bin
The server can be started with the mlapi script.
mlapi -C /path/to/config/file.yml #--debug
# The server can be started in debug mode with the `--debug` or `-D` flag.
mlapi -C /path/to/config/file.yml --debugUser management is done using the mlapi script and the user sub-command.
For more information, please see the User Management docs.
Important
You must edit the following example files first before installing them!
A SystemD service file example is provided in the configs/systemd directory.
# Edit the service file first!
sudo cp ./configs/systemd/mlapi.service /etc/systemd/system
sudo chmod 644 /etc/systemd/system/mlapi.service
sudo systemctl daemon-reload
# --now also starts the service while enabling it to run on boot
sudo systemctl enable mlapi.service --nowImportant
You must edit the following example files first before installing them!
A Fail2Ban filter and jail configuration example is provided in the configs/fail2ban directory.
Important
You must edit the following example files first before installing them!
A logrotate configuration file example is provided in the configs/logrotate directory.
Caution
Work is being done to create Docker images for the server.