diff --git a/.nojekyll b/.nojekyll new file mode 100644 index 000000000..e69de29bb diff --git a/404.html b/404.html new file mode 100644 index 000000000..e88d6739d --- /dev/null +++ b/404.html @@ -0,0 +1,1438 @@ + + + +
+ + + + + + + + + + + + + + + + + + +Before following the quickstart, it could be helpful to skim the User Interface Guide
+How to quickly create multiple detections on a single image frame.
+The demo below shows how to use Detection mode to quickly create numerous detections of the same type.
+ +How to quickly create track annotations for a video or image sequence.
+Linear interpolation is a kind of spatio-temporal annotation that allows the inference of bounding boxes between keyframes. Interpolation mode is the fastest and easiest way to generate track annotations.
+Interpolation editing for existing tracks will only be enabled on tracks that span more than one frame. It is enabled on new tracks by default.
+Click Events in the Timeline controls to see where interpolation occurs and where the keyframes are located.
+This mode keeps you editing the same track while automatically advancing the frame each time a detection is drawn. In most cases interpolation mode will be easier.
+The demo below shows how to use AdvanceFrame mode to travel through the video while creating annotations.
+ +You can create a track by starting with a head/tail annotation or just a single point.
+The head point is denoted by a filled circle, while the tail point is denoted by a hollow circle.
+ +You don't have to place both markers. Press Esc on your keyboard at anytime to exit out of the line creation mode.
+Every track is required to have a bounding box, but a polygon region may be added. When a polygon is created or edited it will generate or adjust the bounding box to fit the size of the polygon.
+This documentation section provides a reference guide to the annotation interface organized by screen region.
+ +Note
+This page is not related to the VIAME command line (i.e. kwiver
, viame_train_detector
)
Some of the DIVE data conversion features are exposed through dive
.
Follow the docs in the Debug Utils and Command Line Tools section of server/README.md
.
1 +2 +3 |
|
1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 |
|
DIVE Desktop and Web support a number of annotation and configuration formats. The following formats can be uploaded or imported alongside your media and will be automatically parsed.
+Info
+The current DIVE schema version is v2. Version 2 was introduced in DIVE version 1.9.0. It is backward-compatible with v1.
+Files are typically named result_{dataset-name}.json
. Their schema is described as follows.
1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14 +15 +16 +17 +18 +19 +20 +21 +22 +23 +24 +25 +26 +27 +28 +29 +30 +31 +32 +33 +34 +35 +36 +37 +38 +39 +40 +41 +42 +43 +44 +45 |
|
The full source TrackData definition can be found here as a TypeScript interface.
+This is a relatively simple example, and many optional fields are not included.
+1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14 +15 +16 +17 +18 +19 +20 +21 +22 +23 +24 +25 +26 +27 +28 +29 +30 +31 +32 +33 +34 +35 +36 +37 +38 +39 +40 +41 +42 +43 +44 +45 +46 +47 +48 |
|
This information provides the specification for an individual dataset. It consists of the following.
+customTypeStyling
and customGroupStyling
.confidenceFilters
attributes
The full DatasetMetaMutable definition can be found here.
+1 +2 +3 +4 +5 +6 +7 |
|
Read the VIAME CSV Specification.
+Warning
+VIAME CSV is the format that DIVE exports to. It doesn't support all features of the annotator (like groups) so you may need to use the DIVE Json format. It's easier to work with.
+DIVE supports MEVA KPF
+Info
+KPF is typically broken into 3 files, but DIVE only supports annotations being loaded as a single file. However, the 3-file breakdown is just convention and KPF can be loaded from a single combined file.
+1 +2 |
|
Only supported on web.
+Start here once you have SSH access and sudo
privileges for a server or VM.
Note
+Docker server installation is only supported on Linux distributions
+A DIVE Web deployment consists of 2 main services.
+In addition, a database (MongoDB) and a queue service (RabbitMQ) are required.
+ +SSH into the target server and install these system dependencies.
+Tip
+You can skip this section if you used Ansible to configure your server, as it already installed all necessary dependencies.
+sudo ubuntu-drivers install
usually works.docker
version 19.03+ guidedocker-compose
version 1.28.0+ guideClone this repository and configure options in .env
.
1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14 +15 +16 +17 +18 +19 |
|
VIAME server will be running at http://localhost:8010. You should see a page that looks like this. The default username and password is admin:letmein
.
If you have a server with a public-facing IP address and a domain name that points to it, you should be able to use our production deployment configuration. This is the way we deploy viame.kitware.com.
+containrrr/watchtower
updates the running containers on a schedule using automated image builds from docker hub (above).linuxserver/duplicati
is included to schedule nightly backups, but must be manually configured.You should scale the girder web server up to an appropriate number. This stack will automatically load-balance across however many instances you bring up.
+1 +2 +3 +4 +5 +6 +7 +8 |
|
It's possible to split your web server and task runner between multiple nodes. This may be useful if you want to run DIVE Web without a GPU or if you want to save money by keeping your GPU instance stopped when not in use. You could also increase parallel task capacity by running task runners on multiple nodes.
+.env
on both with the same configuration.WORKER_API_URL
and CELERY_BROKER_URL
in particular are uncommented and set to the IP or domain name of your web server. This is how the worker will talk to the web server, so the web server must be network accessible from the worker.1 +2 +3 +4 +5 |
|
In order to run any jobs (video transcoding, pipelines, training, addon upgrades) the GPU server will need to be running.
+After initial deployment, DIVE Server will require an addon upgrade in order to download and scan for VIAME addons. Run this by issuing a POST /dive_configuration/upgrade_pipelines
request from the swagger UI at http://{server_url}:{server_port}/api/v1
.
force
or not, only those pipelines from addons from the exact urls passed will be enabled on the server.force
should be used to force re-download of all URLs in the payload even if their zipfiles have been cached.force
.You can configure the brand and messaging that appears in various places in the DIVE Web UI using the config API.
+PUT /dive_configuration/brand_data
where the body is a JSON object from the template below. If you do not want to set a value and use the default, omit the key and value from the config body.1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14 +15 +16 +17 +18 +19 +20 |
|
This image contains both the backend and client.
+Variable | +Default | +Description | +
---|---|---|
GIRDER_MONGO_URI | +mongodb://mongo:27017/girder |
+a mongodb connection string | +
GIRDER_ADMIN_USER | +admin |
+admin username | +
GIRDER_ADMIN_PASS | +letmein |
+admin password | +
CELERY_BROKER_URL | +amqp://guest:guest@default/ |
+rabbitmq connection string | +
WORKER_API_URL | +http://girder:8080/api/v1 |
+Address for workers to reach web server | +
There is additional configuration for the RabbitMQ Management plugin. It only matters if you intend to allow individual users to configure private job runners in standalone mode, and can otherwise be ignored.
+Variable | +Default | +Description | +
---|---|---|
RABBITMQ_MANAGEMENT_USERNAME | +guest |
+Management API username | +
RABBITMQ_MANAGEMENT_PASSWORD | +guest |
+Management API password | +
RABBITMQ_MANAGEMENT_VHOST | +default |
+Virtual host should match CELERY_BROKER_URL |
+
RABBITMQ_MANAGEMENT_URL | +http://rabbit:15672/ |
+Management API Url | +
You can also pass girder configuration and celery configuration.
+This image contains a celery worker to run VIAME pipelines and transcoding jobs.
+++Note: Either a broker url or DIVE credentials must be supplied.
+
Variable | +Default | +Description | +
---|---|---|
WORKER_WATCHING_QUEUES | +null | +one of celery , pipelines , training . Ignored in standalone mode. |
+
WORKER_CONCURRENCY | +# of CPU cores |
+max concurrnet jobs. Lower this if you run training | +
WORKER_GPU_UUID | +null | +leave empty to use all GPUs. Specify UUID to use specific device | +
CELERY_BROKER_URL | +amqp://guest:guest@default/ |
+rabbitmq connection string. Ignored in standalone mode. | +
KWIVER_DEFAULT_LOG_LEVEL | +warn |
+kwiver log level | +
DIVE_USERNAME | +null | +Username to start private queue processor. Providing this enables standalone mode. | +
DIVE_PASSWORD | +null | +Password for private queue processor. Providing this enables standalone mode. | +
DIVE_API_URL | +https://viame.kitware.com/api/v1 |
+Remote URL to authenticate against | +
You can also pass regular celery configuration variables.
+Linux Only.
+Individual users can run a standalone worker to process private jobs from VIAME Web.
+/opt/noaa/viame
.source setup_viame.sh
.cd bin && download_viame_addons.sh
from the VIAME install directory.++Note: The
+--volume
mount maps to the host installation. You may need to change the source from/opt/noaa/viame
depending on your install location, but you should not change the destination from/tmp/addons/extracted
.
1 +2 +3 +4 +5 +6 +7 +8 +9 |
|
The goal of this page is to provide an overview of the ways to run VIAME or VIAME Web in various types of compute environments.
+Using our server | +Running your own | +
---|---|
Free to use; no maintenance costs | +You pay hosting and maintenance costs | +
Always up to date | +Possible to configure automated updates | +
One shared environment for everyone | +Your organization has full control over access | +
Our team monitors this service for errors and can respond to issues proactively | +Support requires logs, screenshots, and other error information if applicable | +
Our team can provide guidance on annotation and training because we have direct access to your data | +Support usually requires example data and annotations | +
Having user data in our environment helps us understand user needs and improve the product | +Feedback is always appreciated. | +
Limited shared compute resources (2 GPUs) available to process jobs. Can be mitigated by hybrid compute options | +As much compute as you pay for | +
For deploying an instance in NOAA Fisheries GCP environment please use the following repository for reference:
+ +The easiest option to get started using VIAME is to try our public server.
+You may wish to run your own deployment of VIAME Web in your lab or a cloud environment. Deploying VIAME Web is relatively straightforward with docker-compose
.
Environment | +Instructions | +
---|---|
Local server | +If you already have SSH access to an existing server and sudo permissions, proceed to the docker compose guide. |
+
Google Cloud | +Continue to the Provisioning Google Cloud page for Scenario 1 |
+
AWS / Azure | +Create a server on your own through the cloud management console, then proceed to the docker compose guide. | +
You may not want to use the web annotator and job orchestration at all, and instead run VIAME using the command line in a cloud environment with GPU.
+Environment | +Instructions | +
---|---|
Local server | +This is a standard VIAME install. See the VIAME documentation install instructions. | +
Google Cloud | +Continue to the Provisioning Google Cloud page for Scenario 2 |
+
AWS / Azure | +Create a server through the cloud management console, then proceed to the VIAME documentation install instructions. | +
Instead of running the whole web stack, it's possible to deploy a worker by itself to process compute-intensive jobs. This is referred to in the docs as standalone mode. For example, you could:
+How it works
+To set up a private worker, continue to the Provisioning Google Cloud page for Scenario 3
.
Any instance of VIAME Web, including our public server, can connect to S3-compatible storage. This means your lab or group could make your existing data available at viame.kitware.com, either privately or publicly.
+Storage Product | +Support level | +
---|---|
Google Cloud Buckets | +Use as backing storage, import existing data, monitor for changes and automatically discover new uploads | +
AWS S3 | +Use as backing storage, import existing data | +
MinIO | +Use as backing storage, import existing data | +
Azure Blob Storage | +Limited import support using MinIO Azure Gateway | +
Contact us for support with any of these topics.
+ + + + + + + + + + + + + + + +Note
+Be sure to read the Deployment Overview first.
+The terraform section is the same for all scenarios. The Ansible section will have differences.
+You'll need a GCP Virtual Machine (VM) with the features listed below. This section will guide you through creating one and deploying VIAME using Terraform and Ansible.
+Together, these tools allow you to quickly create a reproducible environment. If you do not want to use these tools, you can create your own VM manually through the management console and skip to the docker documentation instead.
+Feature | +Recommended value | +
---|---|
Operating system | +Ubuntu 20.04 | +
Instance Type | +n1-standard-4 or larger |
+
GPU Type | +nvidia-tesla-t4 , nvidia-tesla-p4 , or similar |
+
Disk Type | +SSD, 128GB or more depending on your needs | +
To run the provisioning tools below, you need the following installed on your own workstation.
+Note
+Google Cloud worker provisioning can only be done from an Ubuntu Linux 18.04+ host. Ansible and terraform should work on Windows Subsystem for Linux (WSL) if you only have a windows host. You could also use a cheap CPU-only cloud instance to run these tools.
+project-name-123456
.Tip
+Google Cloud imposes GPU Quotas. You may need to request a quota increase. Anecdotally, request increases of 1 unit are approved automatically, but more are rejected.
+1 +2 +3 +4 +5 +6 |
|
Warning
+GPU-accelerated VMs are significantly more expensive than typical VMs. Make sure you are familiar with the cost of the machine and GPU you choose. See main.tf for default values.
+See devops/main.tf
for a complete list of variables. The default machine_type
, gpu_type
, and gpu_count
can be overridden.
1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14 +15 +16 |
|
Later, when you are done with the server and have backed up your data, use terraform to destroy your resources.
+1 |
|
This step will prepare the new host to run a VIAME worker by installing nvidia drivers, docker, and downloading VIAME and all optional addons.
+Warning
+The playbook may take 30 minutes or more to run because it must install nvidia drivers and download several GB of software packages.
+These are all the variables that can be provided with --extra-vars
.
Variable | +Default | +Description | +
---|---|---|
run_server | +no |
+Set run_server=yes for scenario 1 (Web Instance) (Fastest option) |
+
run_viame_cli | +no |
+Set run_viame_cli=yes for scenario 2 (VIAME CLI) |
+
run_worker_container | +no |
+Set run_worker_container=yes for scenario 3 (Standalone Worker) |
+
viame_bundle_url | +latest bundle url | +Optional for scenario 2 & 3. Change to install a different version of VIAME. This should be a link to the latest Ubuntu Desktop (18/20) binaries from viame.kitware.com (Mirror 1) | +
DIVE_USERNAME | +null | +Required for scenario 3. Username to start private queue processor | +
DIVE_PASSWORD | +null | +Required for scenario 3. Password for private queue processor | +
WORKER_CONCURRENCY | +2 |
+Optional for scenario 3. Max concurrnet jobs. Change this to 1 if you run training | +
DIVE_API_URL | +https://viame.kitware.com/api/v1 |
+Optional for scenario 3. Remote URL to authenticate against. | +
KWIVER_DEFAULT_LOG_LEVEL | +warn |
+Optional for scenario 3. kwiver log level | +
The examples below assumes the inventory
file was created by Terraform above.
1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 |
|
Once provisioning is complete, jobs should begin processing from the job queue. You can check viame.kitware.com/#/jobs to see queue progress and logs.
+Tip
+This Ansible playbook is runnable from any Ubuntu 18.04+ host to any Ubuntu 18.04+ target. To run it locally, use the inventory.local
file instead. If you already have nvidia or docker installed, you can comment out these lines in the playbook.
If you run locally you'll need to restart the machine and run the playbook a second time. The playbook will do this automatically for remote provisioning, but cannot restart if you're provisioning localhost.
+1 |
|
Tip
+You may need to run through the docker post-install guide if you have permissions errors when trying to run docker
.
1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14 +15 +16 +17 |
|
You can enable your private queue on the jobs page and begin running jobs.
+GPUS_ALL_REGIONS
quota in IAM -> Quotasubuntu-drivers
This page is intended for storage administrators who would like to make their existing data available through VIAME Web.
+Tip
+This guide assumes you are working with viame.kitware.com. If you are using a different deployment, be sure to change the appropriate fields.
+Tip
+Regarding data transfer costs, if you choose to keep both your data storage and job runners in Google Cloud (or AWS), you will avoid paying a data egress fee for transferring data between storage and the processing node.
+DIVE Web can mirror your data from Google Cloud storage buckets such that your team fully controls upload and data organization, but is able to view, annotate, and run analysis within VIAME Web.
+Access Key
, Secret Key
, Storage URI
, and Bucket Name
.You'll also need to configure CORS headers for any buckets where media will be served.
+bucket-cors-config.json
."origin"
should be whatever you type into your browser to get to the web application.1 +2 +3 +4 +5 +6 +7 +8 |
|
Then use gsutils
to configure each bucket.
1 |
|
Choose a folder as a mount-point inside DIVE Web. This folder should ideally be dedicated to mapping from your GCS buckets.
+We recommend creating a Google Cloud Storage
folder with sub-folders named for each bucket you mount. You can do this using the New Folder
button in DIVE Web's File Browser. You can get the folder ID from your browser's URL bar.
If you want to use your bucket with viame.kitware.com, send us an email with the following details to viame-web@kitware.com
.
1 +2 +3 +4 +5 +6 +7 +8 |
|
If you have data in S3 or MinIO, you can mirror it in DIVE for annotation.
+Creating pub/sub notifications is optional, but will keep your mount point up-to-date automatically with new data added to the bucket. In order to make use of this feature, your DIVE server must have a public static IP address or domain name.
+https://viame.kitware.com/api/v1/bucket_notifications/gcs
Our server will process events from this subscription to keep your data current.
+If you have your own dive deployment, you can create a bucket mirror yourself through the Girder admin console.
+/girder#assetstores
in your browser.us-east-1
.The import may take several minutes. You should begin to see datasets appear inside the mount point folder you chose.
+ + + + + + + + + + + + + + + +DIVE is available as an electron based desktop application with VIAME integration. It has most of the same UI and features web. You may want to use desktop if...
+DIVE Desktop is fully supported on Windows and Linux. MacOS users can use it as an annotator, but without NVIDIA Driver support, the machine learning features from VIAME are unavailable.
+ +Download the latest DIVE Desktop from GitHub
+Choose an asset from the list matching your operating system:
+OS | +Extension | +Description | +
---|---|---|
Windows | +.exe | +Portable executable (recommended) | +
Windows | +.msi | +Installer file | +
MacOS | +.dmg | +MacOS DiskImage (Intel only, M1 not supported) | +
Linux | +.AppImage | +Portable executable for all Linux platforms (recommended) | +
Linux | +.snap | +Ubuntu SnapCraft package | +
This is the installation guide for DIVE. If you want the full VIAME toolkit, you can get it from github.com/viame/viame. The full toolkit installation includes DIVE.
+DIVE Desktop supports single- and multi-camera datasets.
+.npz
file (numpy transformation matrix).Click either Open Image Sequence or Open Video to begin a single camera default import. Click the dropdown button to show additional import options.
+.txt
file that contains an image name or full path on each line.The import routine will look for .csv
and .json
files in the same directory as the source media, and you will be prompted to manually select an annotation file and a configuration file. Neither is required.
DIVE Desktop is an Electron application built on web technologies. Certain video codecs require automatic transcoding to be usable. Video will be transcoded unless all the following conditions are met.
+codec
= h264
sample_aspect_ratio (SAR)
= 1:1
Transcoding is done with ffmpeg, which comes bundled with the DIVE Desktop executable.
+DIVE Desktop requires a local installation of the VIAME toolkit to run ML pipelines and training.
+launch_dive_interface.[bat|sh]
script from a VIAME install. Otherwise, you may need to change this yourself.The data storage path is not at all related to "project folders" in VIAME. It's just a place for DIVE Desktop to keep and structure all the data it needs to run.
+A typical data storage directory has 3 subfolders:
+DIVE_Jobs
- Each job run has a working directory, kept here.DIVE_Projects
- Each dataset you import into desktop has metadata and annotation data (with revision history) kept here.DIVE_Pipelines
- Training runs produce models that get copied into here.Here's an example of structure you might find in the storage path.
+1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14 +15 +16 +17 +18 +19 +20 +21 +22 +23 +24 +25 +26 +27 +28 +29 +30 |
|
DIVE Desktop looks for the these environment variables on launch.
+Name | +Default | +Description | +
---|---|---|
DIVE_VIAME_INSTALL_PATH | +/opt/noaa/viame (Linux/macOS) C:\Program Files\VIAME (Windows) | +Overrides the location of the VIAME installation. Users may not change this value in the settings pane if provided. | +
DIVE_READONLY_MODE | +None | +Overrides read only mode to true or false. Users may still change this value in the settings pane if provided. | +
Trained models are kept in ${Project Data Storage Path}/DIVE_Pipelines
as described above. Each model file consists of exactly 1 pipe file and some number of other model files.
detector.pipe
, tracker.pipe
, or generate.pipe
..zip
, .svm
, .lbl
, or .cfg
.You can use externally trained models in DIVE by creating a folder containing these files. The name of the configuration or pipeline in dive will be the folder name you create.
+++I imported some data, but I don't see my annotations
+
See Importing images and video above. You most likely need to specify your annotation file in the import dialog.
+++Some VIAME canned pipelines are missing, or there are no training configuration files.
+
You may need to install VIAME Toolkit, or correct your VIAME Install Base Path setting.
+If you don't see some pipelines you expect, you may not have installed the addons (also called Optional Patches) yet. Download and install these based on the VIAME installation docs.
+++Advanced troubleshooting
+
If you experience problems or have questions about DIVE Desktop, contact us and include the content from the settings page such as Build Version
as well as your currently installed VIAME version.
It's also helpful to look in the debug console. Press Ctrl+Shift+I to launch the Dev Tools and look under the console tab. Errors and warnings will appear in red and yellow. You can right-click in the console area and click "Save As" to save the log file and open a support ticket
+ + + + + + + + + + + + + + + + +The Training Data Collection is organized roughly by domain and collection method.
+This use case is covered on the sharing page.
+If you want to publish your data so that other groups can use it, please contact us.
+In DIVE, these are called pipelines. You'll need to see what sorts of analysis workflows are currently available on the pipeline page.
+These sorts of AI workflows are the final goal for most users. They allow the user to quickly perform quantitative analysis to answer questions like how many individuals of each type appear on each image or video frame?
+If no suitable existing analysis exists for your use case or you aren't sure how to proceed, you're welcome to contact our team and ask for help.
+You want to perform analysis (detection, tracking, measurement, etc) on object types not yet covered by the community data and pre-trained analysis pipelines available. This will involve training new models based on ground-truth annotations.
+Training configurations are listed on the pipeline page.
+If you have data in lots of places or it arrives at different times, it's probably best to break these batches or groups into individual datasets and annotate each individually. Using the checkboxes in web, you can use multiple datasets to generate a trained model.
+Breaking large amounts of data up into manageable groups is generally a good idea.
+DIVE Web doesn't currently support notifications through email for job status changes. You can check your job status through the web interface.
+No. VIAME Web and DIVE Desktop perform automatic transcoding if it is necessary.
+When you annotate a video in DIVE, the true video is played in the browser using a native HTML5 video player.
+Web browsers report and control time in floating point seconds rather than using frame numbers, but annotations are created using frame numbers as their time indicators, so it's important to make sure these line up.
+Most of the time, videos are downsampled for annotation, meaning that the true video framerate (30hz, for example) is annotated at a lower rate, such as 5hz or 10hz. Kwiver (the computer vision tool behind VIAME) uses a downsampling approach that sorts actual frames into downsampled buckets based on the start time of the frame.
+An implementation of this approach is described here.
+1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 |
|
There are caveats with this approach.
+control | +description | +
---|---|
Left Click | +select track/detection | +
Right Click | +toggle edit mode | +
Middle Click | +pan camera | +
Scroll Wheel | +zoom | +
Mouse Drag | +pan | +
Shift + mouse drag | +select area to zoom | +
Up | +select previous track in list | +
Down | +select next track in list | +
Esc | +unselect, exit edit mode | +
A | +toggle attribute / merge pane | +
control | +description | +
---|---|
Left or D | +previous frame | +
Right or F | +next frame | +
Space | +play/pause | +
Most editing controls are available when a track or detection is selected.
+control | +description | +
---|---|
Del | +delete entire track or detection | +
N | +create new track or detection | +
Home | +go to first frame of selected track | +
End | +go to the last frame of selected track | +
1 | +Enter bounding-box edit mode on selection | +
2 | +Enter polygon edit mode on selection | +
3 | +Enter head/tail/line edit mode on selection | +
H or G | +while in line mode, place head point next | +
T or Y | +while in line mode, place tail point next | +
Esc | +unselect, exit edit mode, exit merge mode | +
K | +toggle keyframe for the current frame and selected track | +
I | +toggle interpolation for the current range of the selected track | +
M | +enter merge mode for on selection | +
Shift+M | +commit (finalize) merge for selected tracks. | +
G | +create new group including the selected track | +
Shift+Enter | +focus class select/text box on selected track in track list. Press Down to open all options. Pres Enter twice to accept an option. Press Esc to unfocus. | +
If you think a new shortcut or hotkey would be useful, please send us feedback.
+ + + + + + + + + + + + + + + +A beta release of the Desktop Only software supports both multicamera and stereo datasets for viewing and editing.
+Dive Desktop has documenation under "Supported Dataset Types" which describes how to import multiCamera or stereoscopic data. Reminder that stereoscopic data requies a calibration file where multicamera data doesn't.
+Data is loaded amongst multiple folders to creae a mutlicamera dataset. In these cases trackIds will be linked if they are the same across the cameras. Selection of a trackId that exists across multiple cameras will be linked together in the Track List.
+++Example If Camera 1 and Camera 2 both have annotation files with TrackId 1 they will be automatically be linked together and selecting one will select them in both cameras.>
+
Editing and interacting with a camera requires that you select the camera first. There is a dropdown in the upper right of the screen which contains the name of the currently selected camera. Also the currently selected camera will contain a dashed light blue outline. Left or Right clicking within a camera will cause that camera to be selected.
+Track creation for a single camera works much in the same way it does for single camera datasets. Using the New Track button or N key to create a new track and draw. To quickly create a track on another camera and have it link to the current cameras can be done using the "MultiCamera Tools" or by selecting the the desired base track and right clicking on the new camera to add the track. This will put the annotation tool into creation mode for the current TrackId on a new camera. Alternatively the MultiCamera Tools panel can simplify this be clicking on the Edit button.
+Next to the dropdown for the camera selection is a camera settings icon. Clicking on that will open the context menu. Within this menu is a dropdown for selecting MultiCamTools. These tools provide a quick view of the selected track across all cameras. +When a track is selected it will easily show the existing detections and tracks across multiple cameras.
+ +Importing and exporting of data works similarily to a single dataset except that it will occur on the current selected camera. Selecting "Starboard" camera and clicking export will only export the annotations for the "Starboard" camera. Similarily importing annotations will only occur on the selected camera as well.
+Single camera pipelines can be used by selecting the camera and then running the pipeline from the pipeline menu.
+++Note it is suggested that single camera pipelines only be run on empty datasets that don't have annotations already. When the pipeline finishes it will create tracks with TrackIds that may conflict with the other cameras. So it is reccomended that all tracks be removed before running single camera pipelines.
+
There are specific pipelines that can be used on multi-camera or stereo datasets. These pipelines are related to the type of dataset (mutlicamera vs stereo) and the number of cameras that exist.
+Pipeline | +Category | +Type | +Cameras | +Description | +
---|---|---|---|---|
gmm |
+measurment | +stereoscopic | +2 | +Stereo pipeline used to compute fish length measurement | +
X-cam |
+X-cam | +multicamera | +2 or 3 | +Multiple pipelines that can act on either 2 or 3 camera datasets. | +
Both web and desktop versions are capable of running canned pipelines and model training on your ground truth data. This document is to help you decide which pipeline to run.
+Contact our team if you need help choosing the right data analysis strategy. Please upload some sample data to viame.kitware.com to allow us to better assist you.
+Best for a series of images that have no temporal relationship, such as arial photography of multiple scenes. Also preferred if you only care about aggregate data for the dataset, such as max occurrences of an object per scene.
+Pipeline | +Use case | +
---|---|
arctic seal eo yolo |
+detector for color imagery | +
arctic seal ir yolo |
+detector for infrared | +
em tuna |
+detector for identifying individual features of tuna | +
fish without motion |
+simple single-class fish detector | +
generic proposals |
+generic object detector | +
motion |
+detect regions of motion in video or time-series images | +
pengcam kw |
+Penguin cam full-frame classifier | +
pengcam swfsc |
+Penguin cam full-frame classifier | +
scallop and flatfish |
+detector for benthic images | +
scallop and flatfish left |
+detector for benthic images, process left half of each frame only | +
scallop netharn |
+deep learning detector for benthic images | +
scallop netharn left |
+deep learning detector for benthic images, process left half of each frame only | +
sea lion multi class |
+detects bulls, cows, pups, etc | +
sea lion single class |
+detector | +
sefsc bw group |
+black-and-white fish detector (18 class, lower granularity) (oldest, v1) | +
sefsc bw species v2 |
+black-and-white fish species detector (updated, v2) | +
Run full tracking pipelines on your data. Appropriate for videos and image sequences that derive from a video. Tracking involves first running a detection pipeline then performing detection linking to form connected object tracks.
+++Note some trackers can perform differently on time-series data depending on the annotation framerate selected when you upload or import your dataset. Higher framerates take longer to process, but may produce better results.
+
Pipeline | +Use case | +
---|---|
em tuna |
+tracker | +
fish |
+simple fish tracker | +
fish.sfd |
+tracker | +
generic |
+generic object tracker puts generic boxes around arbitrary objects | +
motion |
+identifies moving object tracks | +
mouss |
+tracker, trained with data from MOUSS (Modular Optical Underwater Survey System) | +
sefsc bw * |
+same as above, but with tracking | +
An assortment of other types of utility pipelines. Utility pipelines are named utility_<name>.pipe
and are unique in that they may take detections as inputs (but are not required to).
Pipeline | +Use case | +
---|---|
add segmentations watershed |
+Transform existing bounding boxes into polygons | +
empty frame lbls {N}fr |
+Add an empty bounding box covering the whole media element for the purpose of adding full-frame classifier attributes. Unique tracks are created every N frames. | +
track user selections |
+Create tracks from user-initialized detection bounding boxes. Draw a box on the first frame of a track, and the pipeline will continue tracking the selected object(s) | +
Run model training on ground truth annotations. Currently, training configurations are available to do object detection, object classification, and full-frame classification. Tracker training will be added in a future update.
+empty frame lbls
utility pipe and add type annotations to each generated frame.A recognizable name for the pipeline that results from the training run.
+One of the configuration options in the table below.
+This optional file controls the output classes that a newly trained model will generate.
+The following example labels.txt
shows how to train a FISH
classifier by combining redfish
and bluefish
, preserve the ROCK
label, and omit every other label.
1 +2 |
|
By default, all classes from all input datasets are preserved in the output model.
+By default, training runs include all frames from the chosen input datasets, and frames without annotations are considered negative examples. If you choose to use annotated frames only, frames or images with zero annotations will be discarded. This option is useful for trying to train on datasets that are only partially annotated.
+Configuration | +Availability | +Use Case | +
---|---|---|
detector_default | +both | +alias: train detector netharn cfrnn | +
detector_netharn_cfrnn | +both | ++ |
detector_netharn_mask_rcnn | +both | ++ |
detector_svm_over_generic_detections | +both | +general purpose svm | +
detector_svm_over_fish_detections | +both | +fish svm | +
frame_classifier_default | +both | +alias: frame classifier netharn resnet | +
frame_classifier_netharn_resnet | +both | +full-frame | +
frame_classifier_svm_overn_resnet | +both | +full-frame | +
object_classifier_default | +both | +alias: netharn resnet object classifier | +
object_classifier_netharn_resnet | +both | ++ |
yolo | +desktop only | +can train, but resulting model is not runnable with desktop yet | +
Pipelines created outside of VIAME Web can be upload and shared with other users. See Pipeline Import and Export for details.
+ + + + + + + + + + + + + + + + + + + + +You can download your trained models through the administrative interface.
+Warning
+Use caution when modifying data through the admin interface
+myserver.com/girder
if you host your own instance)Navigate to your personal workspace by clicking My Folders under your user dropdown in the top right corner.
+ +Navigate to the VIAME/VIAME Training Results
folder and into the folder you wish to download
Select all items and download using the menu
+ +It's possible to upload custom pipes to DIVE Web through the girder interface.
+Warning
+This feature is not yet standardized, and the instructions below may change.
+/girder
and create a new private folder called MyPipelines
.pipe
file extension.zip
files are required by the pipe, named exactly as they appear in your .pipe
file above.trained_pipeline
with value true
.Run Pipeline -> Trained
menu from the DIVE web app.If your pipe must accept input, set the pipeline folder metadata property requires_input
to true
.
User-uploaded pipelines may depend on any pipe already installed from the base image or an addon using include <pipename>.pipe
. Depending on other user-uploaded pipes is not supported.
Tip
+KWIVER pipe files can be exported for use with DIVE using kwiver pipe-config
+This page provides a general overview of the differences between desktop and web through screenshots.
+Web
+ +Desktop
+ +Web
+ +Desktop
+ +Web
+ +Desktop
+ +Web
+ +Desktop
+ +Desktop
+ + + + + + + + + + + + + + + + +DIVE is free and open-source software published under an Apache 2.0 License in accordance with Kitware's Open Philosophy.
+For feedback, problems, questions, or feature requests, please reach out on Discourse. Our team would be happy to hear from you!
+ +If you or your employer have an active support contract with Kitware and you need individualized support, please email us directly at viame-web@kitware.com
.
DIVE is built and maintained by the Data & Analytics group at Kitware with financial support from contracts with government and commercial customers. If you would like to schedule a demo, sponsor custom software development, or purchase support related to DIVE, please email us or reach out through our contact page.
+ + + + + + + + + + + + + + + + + + + + +The annotation window will look different based on the current mode and what visibility toggles are enabled.
+Default Mode - In the default mode the annotation will have bounds associated with it as well as a text name for the type and an associated confidence level. The color and styling will match what is specified in the Type List Style Settings. There are additional modes which can be toggled on and off in the Edit Bar.
+Selected Annotation - selected annotations are cyan in color
+ +Editing Annotation - Editing annotations are cyan in color and provide handles to resize the annotation as well as a central handle to move the annotation to different spot.
+ +Creating Annotation - Creating an annotation requires clicking and dragging the mouse. Creating in the annotation window is indicated by a cursor crosshair and an icon that shows the type being drawn.
+ +Interpolated Annotation - If a track has an interpolated box on the current frame it will appear slightly faded.
+ +Attributes can be configured through their main editing window. This provides several sections will allow for the customization of values.
+The Main section is where the attribute Name, Datatype, any predefined values and the rendering color are specified
+Datatype - Can either be Text, Number or Boolean data types.
+If the track is Text type a list of predefined values can be used in dropdowns to make it eaiser to set attribute values.
+Color - The color is used to determine the display color for the attribute within the track details and the default color utilized in any graphs or rendering of the attribute values.
+Attributes can be displayed within the annotation area next to the tracks they are associated with. +Within the Attribute Editor there is a tab for Rendering and when turned on there are settings which can specify how the attribute is displayed and what tracks it is displayed. +By default if no settings are touched the attributes are rendered below the bottom right of the track.
++In the above demo the Detection attributes are rendered to the side of the track with custom text and colors for displaying each.
+Under the Rendering Tab for the Attribute Editor if you turn on Render there will be numerous settings which determine how the attribute is displayed.
+Attributes can specify specific render colors based on their values. This means that attributes with text values could be set so that specific values will have specific colors. I.E You may have an attribute for weather classification and it could be set so that Sunny is Yellow, Cloudy is Gray and Rainy is Blue.
+On loading this page any existing attribute text values or predefined values will be populated and assigned a random color. These colors can be clicked on and modified to suit needs. If a text value doesn't exist, add it to the predefined values and it will show up when you return to this tab.
+When rendering attribute Values if the color is left to Auto it will utilize the colors specified here to render the text value for the attribute.
+ +If the Attribute Datatype is 'Number' that allows for the creation of color gradients to be assigned to the numerical values. +Numbers can be added by clicking on "Add Color". Each Number in the range can then be specified to be a specific color. Using D3.js it then creates a gradient between the colors in the list. To help with visualization a display of the gradient is located at the bottom of the editor next to the Add Colors button.
+Like the Text Values if attribute rendering value is set to auto color it will use the gradient to determine the color of the attribute value.
+ + + + + + + + + + + + + + + + +The Attribute Details Panel is a side panel that provides additional tools for Filtering and Viewing Attributes.
+ +Filters the values of the selected type to reduce the number of results displayed. The filters can be applied to +"Track" or "Detection" attributes and are applied in order.
+Attribute filters have an "Applies To" list which is a list of the Attribute key names that the filter will apply to. +There is a special name called 'all' which will cause the filter to apply to all items. +The settings for individual filters can be accessed by clicking on the
+only operates on the attributes that have numeric values.
+only operates on the attributes that have text values
+Filter which will only show the selected Attribute names regardless of their current value.
+The special name 'all' will show all of the attributes.
+The timeline visualization only applies to numeric and detection attributes currently. It will graph the selected attributes in a chart at the bottom of the screen when a Track is selected.
+A Key Filter is used to determine which attributes to graph.
+ + + + + + + + + + + + + + + + +Attribute Track Filtering is used to apply a filter to the currently visible tracks based on the values defined in the Track or Detection Attributes
+To add attribute filters open up the right side panel and go to the drop down for Attribute Track Filters
+ +A new Atribute Track Filter can be added by clicking on the new button
+ ++Atribute Track Filter with Range Settings
+After adding multiple Track Attribute Filters they are listed in the side panel:
+ +All filters that have the Primary Display switched to on will be displayed in left hand side of the main viewer:
+ +String
, Number
, or Boolean
. Definitions must be created before attribute values can be assigned. Tracks and detections each have their own set of definitions.Boolean
number (cm)
Boolean
Boolean
{ "FishLength": 20 }
{ "Eating": True }
{ "Swimming": False, "Eating": True }
Info
+All Attribute definitions do not need to be assigned to values. CompleteTrack (Track Attribute) and Swimming for Frame 1 (Detection Attribute) weren't assigned in this example.
+Info
+Attributes found during import in a VIAME CSV will automatically show up in the list. The data type of the attribute is guessed by examining values and may need to be manually corrected.
+By default, all attributes associated with the dataset are visible and editable. You can hide unused attributes by clicking the toggle next to Attribute.
+Show Unused | +Hide Unused | +
---|---|
+ | + |
Number
Boolean
(True/False)Text
Click the button next to an existing attribute to edit its definition.
+ +Warning
+Editing or deleting an attribute definition doesn’t change any existing attribute values.
+Attributes are part of the dataset configuration that can be imported and exported.
+The group manager is one pane of the context sidebar.
+DIVE supports complex group annotation.
+Person
type tracks participate in a Conversation
activity.[track.begin, track.end]
. A single track can belong to a group for multiple sub-ranges, typically interpreted as the tracked object leaving and re-joining a group.See the data format documentation for the complete capabilities of group annotations in the DIVE json schema.
+The group type summary list allows for enabling and disabling annotations based on their group and bulk-editing group type characteristics.
+Each group instance includes the following.
+To enter group edit mode, click a group's ID number in the group list.
+Some notes about group editing behavior.
+The group feature was initially developed for compatibility with the The Multiview Extended Video with Activities (MEVA) dataset. Find example data at mevadata.org.
+ + + + + + + + + + + + + + + +The navigation bar is the row of controls at the very top of the window.
+.json
or .csv
annotation file..json
file.The editing bar is the second row below navigation.
+On the far left, the editing mode status indicator shows you what mode you're in, what input is expected, and usually reminds you to press Esc to cancel.
+Editing mode toggles control the type of geometry being created or edited during annotation. See the Annotation Quickstart for an in-depth guide to annotation.
+The visibility section contains toggle buttons that control the different types of annotation data can be hidden or shown.
+The timeline provides a control bar and a few different temporal visualizations. All timeline visualizations are updated live by type confidence slider(s), type checkboxes, and track checkboxes.
+ + +This is the default visualization. It is a stacked histogram of track/detection types over the duration of the sequence.
+The event viewer shows the start/stop frames for all tracks. It is a kind of Gantt chart, also similar to a swimlane chart but with more compact packing.
+The group viewer is just like the event viewer, but shows the start and end times of track groups, colored by group types. Switching to the group view changes the coloring scheme of annotations in the annoation window.
+The track list allows for selecting and editing tracks. A selected track will look different depending on whether it's a single detection or a multi-frame track.
+A track that spans a single frame.
+Warning
+The button will remove the whole track if it's longer than a single detection. To remove individual keyframes, use (the keyframe toggle button).
+A track that spans multiple frames and has more options
+Each dataset maintains its own list of types, and types can be defined on-the-fly.
+The Type List is used to control visual styles of the different types as well as filter out types that don't need to be displayed.
+The type style editor controls the visual appearance of annotations in all other areas of the application. Launch the editor by hovering over a type row in the list and clicking (the edit pencil).
+Click the button in the type list heading to open type settings.
+In ad-hoc mode, new object classes are added as you annotate. The type list updates automatically when new classes are added or the last member of a class is deleted.
+In locked mode, only a specified list of classes can be used, and must be selected or autocompleted from the list for each object.
+Info
+VIAME Web is automatically updated at 2AM EST/EDT every Thursday. If you are running a pipeline or training workflow during update, it will be interrupted and restarted.
+Also note that pipelines and training jobs on our public server are limited to 3 days of execution time on 1 GPU. If you have a job that needs more time, please run it with the Desktop version, your own cloud environment like GCP, or contact us for support.
+Use our public server +Deploy your own
+A user account is required to store data and run pipelines on viame.kitware.com.
+annotations.csv
or an annotation or configuration JSON select that too.Info
+All video uploaded to the web server will be transcoded as mp4/h264
.
A zip import can have one of the following file combinations:
+Zip import also accepts zip archive files that were generated by the Download Everything export button.
+Data can be downloaded from the FileBrowser by clicking the checkmark to the left of a dataset name. This allows you to download the source images/video, the current detection file converted to .csv
or everything including all backups of the detection files.
.zip
.csv
of annotations.json
This information will be relevant to teams where several people need to work on the same data.
+By default, data uploaded to your personal user space follows these conventions.
+A common scenario is for a group to have a lot of shared data that several members should be able to view and annotate.
+For most teams, we recommend keeping data consolidated under a single account then following the sharing instructions below to make sure all team members have appropriate access.
+It's easiest to create a single parent folder to share and then put all individual datasets inside that parent.
+Warning
+You should note that 2 people cannot work on the same video at the same time. Your team should coordinate on who will work on each dataset.
+DIVE uses Girder's Permissions Model.
+There are four levels of permission a User can have on a resource.
+Right click a dataset or a folder of datasets and choose Access Control
+ +Search for and select users you want to grant access to.
+Select the correct permissions in the drop-down next to each user.
+ +If this is a folder of datasets, enable the Include Subfolders switch.
+You can view data shared with you by selecting the Shared With Me tab above the data browser.
+ +You can copy and paste any URL from the address bar and share with collaborators. This includes folders in the data browser as well as direct links to the annotation editor.
+A clone is a shallow copy of a dataset.
+Warning
+Be careful when deleting data that has been cloned. Clones "point to" their source dataset for loading media, so if the source is deleted, all of its clones will fail to load.
+Warning
+Merging cloned data back to the source is not currently supported. To collaborate with others on annotations, the sharing use case above is preferred.
+Revision history is accessible through the annotation UI in the Web version. Each time you press Save, a new revision of your annotation state is created. It is possible to inspect (or "check out") past revisions. The viewer will be in read-only mode when past revisions are checked out because only the most recent revision can be modified.
+Info
+Revision roll-back is not yet supported, but will be added in a future update. If you need to roll-back to a previous version of your anotation state
+{"use strict";/*!
+ * escape-html
+ * Copyright(c) 2012-2013 TJ Holowaychuk
+ * Copyright(c) 2015 Andreas Lubbe
+ * Copyright(c) 2015 Tiancheng "Timothy" Gu
+ * MIT Licensed
+ */var Va=/["'&<>]/;qn.exports=za;function za(e){var t=""+e,r=Va.exec(t);if(!r)return t;var o,n="",i=0,a=0;for(i=r.index;i