Update docs prepare for a release

Makefile

@@ -58,7 +58,7 @@ run-docker: debian-docker
 	cp server/src/tests/assets/60fps-example.mp4 test/VOLUME/data/incoming/
 	docker run --rm -it -p 0.0.0.0:8080:80 --mount type=bind,source="$$(pwd)"/test/VOLUME,target=/mnt/clapshot-data  clapshot-comb
 
-build-docker-demo: #debian-docker
+build-docker-demo: debian-docker
 	@which jq || (echo "ERROR: Please install jq first." && exit 1)
 	$(eval PVER=$(shell jq -r '.version' client/package.json))
 	docker build -t clapshot:${PVER}-demo --build-arg UID=1002 --build-arg GID=1002 -f Dockerfile.demo .

README.md

@@ -66,6 +66,8 @@ See the [Sysadmin Guide](doc/sysadmin-guide.md) for information on
 - implementing advanced authentication methods
 - building manually and running unit tests
 
+See [Upgrading Guid](doc/upgrading.md) for instructions when installing a new release over an old one.
+
 ## Architecture Overview
 
 Main components:

client/debian/changelog

@@ -1,9 +1,12 @@
 clapshot-client (0.6.0) bookworm; urgency=low
 
-  * New communication API
   * Support for folders and custom UI views (requires an Organizer plugin in the backend)
+  * 'Download original video' button in the player
+  * New communication API, lots of changes under the hood
+  * Connection errors now shown in the "connecting..." page
+  * Updated dependencies, bump Svelte and Vite versions
 
- -- Jarno Elonen <[email protected]>  Mon, 6 May 2024 19:45:00 +0200
+ -- Jarno Elonen <[email protected]>  Sat, 18 May 2024 17:00:00 +0200
 
 clapshot-client (0.5.6) bullseye; urgency=low
 

doc/sysadmin-guide.md

@@ -21,7 +21,7 @@ Preferred deployment and upgrade method is to install server and client as Debia
 the Debian package contains a systemd service file that demonizes it, and config file `/etc/clapshot-server.conf` that is translated into the appropriate CLI options automatically.
 
 Server should be put behind a reverse proxy in production, but
-can be developed and tested without one. The server .deb package contains an example Nginx config file (`/usr/share/doc/clapshot-server/examples/`) that
+can be developed and tested without one. The client .deb package contains an example Nginx config file (`/usr/share/doc/clapshot-client/examples/`) that
 
  1. reverse proxies the server API (websocket),
  2. serves out frontend files (.html .js .css),
@@ -32,11 +32,8 @@ While the server uses mostly Websocket, there's a `/api/health` endpoint that ca
 
 ### Database upgrades
 
-Some releases require database migrations. If you're upgrading from a previous version, **make a backup of your database** (`clapshot.sqlite`) and then either add line `migrate = true` to `/etc/clapshot-server.conf` (Debian package) or use `--migrate` option when running the server manually.
-
-Once the server is started with migrate enabled, it will run database migrations on startup. After that, you can remove the `migrate` option and restart the server.
-
-Running the server without migrations enabled will detect that the database is out of date, log an error and exit.
+Running a new version of Clapshot Server (and/or Organizer) will often upgrade database schemas on first start.
+See [upgrading.md](Upgrading Guid) for details about upgrading.
 
 ### Advanced Authentication
 

doc/upgrading.md

@@ -0,0 +1,19 @@
+# Upgrading Clapshot to a new release
+
+These instruction are for basic .deb -based deployments, adapt as necessary for custom ones.
+
+1. Stop the server, `systemctl stop clapshot-server`
+2. Install the new packages: `dpkg -i clapshot-*.deb`
+3. Compare you current configuration (`/etc/clapshot-server.conf`) to the latest example at `/usr/share/doc/clapshot-server/examples/clapshot-server.conf`. Edit as necessary.
+4. Make sure `migration = true` in the config file.
+5. Start the server, `systemctl start clapshot-server`
+6. Check that it started, `systemctl status clapshot-server`
+7. Review server log at `/var/log/clapshot.log`. If the server or organizer applied any **database migrations**, they are mention in the log, and a backup `.tar.gz` will be present next to you database (usually, `/mnt/clapshot-data/data/clapshot.sqlite`).
+8. If the server didn't start properly, set `debug = true` in `/etc/clapshot-server.conf`, and start again. This will increase log verbosity level, to give you more clues on what went wrong.
+
+### Notes
+
+ - Make sure to fully reload the Client page on a browers if Client package was upgraded. You might otherwise see unexpected behavior.
+ - The basic_folders Organizer plugin doesn't have its own systemd entry (it's executed by the Server), so you don't need to `systemct stop/start` it. It also piggybacks the server when doing migrations.
+
+If you find this migration guide lacking, please contribute corrections and additions on the [Clapshot's GitHub page](https://github.com/elonen/clapshot).

organizer/basic_folders/debian/changelog

@@ -2,4 +2,4 @@ clapshot-organizer-basic-folders (0.6.0) bookworm; urgency=low
 
   * Initial debian packaging
 
- -- Jarno Elonen <[email protected]>  Mon, 06 May 2024 12:40:00 +0200
+ -- Jarno Elonen <[email protected]>  Sat, 18 May 2024 17:00:00 +0200

server/Cargo.toml

@@ -26,8 +26,11 @@ over websocket, processes video files and maintains a databases of them.
 
 assets = [
     ["target/release/clapshot-server", "usr/bin/", "755"],
+
     ["README.md", "usr/share/doc/clapshot-server/README", "644"],
     ["LICENSE", "usr/share/doc/clapshot-server/LICENSE.GPL2", "644"],
+    ["debian/additional_files/clapshot-server.conf", "usr/share/doc/clapshot-server/examples/", "644"],
+
     ["debian/additional_files/clapshot-server.conf", "etc/", "644"],
     ["debian/additional_files/run-with-conf.sh", "usr/share/clapshot-server/", "755"],
 ]

server/README.md

@@ -1,15 +1,53 @@
-# Clapshot server
+# Clapshot Server
 
-This is the server component for Clapshot - a self-hosted collaborative
-video review tool. See github page for an overview of the whole tool.
+This is the server component for Clapshot, an open-source, self-hosted tool designed for collaborative video review and annotation. For a comprehensive overview and detailed documentation, please visit the [GitHub project page](https://github.com/elonen/clapshot).
 
-The server is a HTTP daemon running behind a reverse proxy, and
-controlled by systemd or some other process manager.
+## Overview
 
-Frontend is provided in another package, and consists of static
-HTML, JS and CSS files.
+Clapshot Server is a Rust-based HTTP/Websocket daemon that manages the server-side logic for Clapshot. It is designed to run behind a reverse proxy and is typically managed using systemd or another process manager.
 
-If you have installed it in a Debian system, configure it
-by editing `/etc/clapshot-server.conf`.
-Otherwise, try `clapshot-server --help` for startup options.
-
+## Configuration and Operation
+
+After installation on a Debian-based system, configure Clapshot Server by editing the configuration file at `/etc/clapshot-server.conf`.
+
+For available startup options, use `clapshot-server --help`. All options listed by `--help` can be used in the config file. In fact, a script at `/usr/share/clapshot-server/run-with-conf.sh` will convert the config file into CLI options upon startup.
+
+### Running the Server
+
+The server runs as a systemd service by default. To start, stop, or check the status of the Clapshot Server, use the following commands:
+
+```bash
+sudo systemctl start clapshot-server
+sudo systemctl stop clapshot-server
+sudo systemctl status clapshot-server
+```
+
+After installation, enable auto start by `sudo systemctl enable clapshot-server`.
+
+### Data directories
+
+Video files and an Sqlite database are by default stored in `/mnt/clapshot-data/`, but the location can be changed in config file.
+
+### Log Files
+
+Log files for the Clapshot Server can be found at `/var/log/clapshot.log`.
+When debugging, also take a look at `sudo systemctl status clapshot-server` in case the startup failed before writing anything to the log file. Setting `debug = true` in the config file will increase log verbosity.
+
+## Database Upgrades
+
+When installing a new version over an existing system, make sure `migrate = true` is set in `/etc/clapshot-server.conf`.
+Keeping it there permanently should be safe in current versions, as the server will back up the database in a .tar.gz before applying migrations.
+
+## External dependencies
+
+Most deployments will also run an Nginx instance, that reverse proxies API calls to the Clapshot Server, and serves Clapshot Client .html and .js files to the web browsers.
+
+### Clapshot Client
+
+Clapshot Client is a Svelte-based Single Page Application (SPA) that runs in the browser and connects to the Clapshot Server via WebSocket. It provides the user interface for video review and annotation.
+
+### Clapshot Organizer
+
+Clapshot Organizer is a plugin system that enables custom video organization, access control, and workflow enhancements. The included `basic_folders` plugin organizes videos into a hierarchical folder structure.
+
+Organizer communicates with Clapshot Server using gRPC, over Unix Sockets by default. The server initiates a connection with the plugin, which remains active as long as the server is running.

server/debian/changelog

@@ -2,9 +2,12 @@ clapshot-server (0.6.0) bullseye; urgency=low
 
   * New communication API
   * Organizer plugin support through gRPC
-  * Update libraries, bump Rust version
+  * Update dependencies, bump Rust version
+  * Improved CORS config options, to avoid cross-origin API attacks
+  * Configuration file changes due to Organizer support
+  * Major codebase revamp, lots of bug fixes, and probably lots of new bugs
 
- -- Jarno Elonen <[email protected]>  Mon, 6 May 2024 19:45:00 +0200
+ -- Jarno Elonen <[email protected]>  Sat, 18 May 2024 17:00:00 +0200
 
 clapshot-server (0.5.6) bullseye; urgency=low