diff --git a/Cloud VPS/jdownloader/index.html b/Cloud VPS/jdownloader/index.html index 83ab36a..9b57298 100755 --- a/Cloud VPS/jdownloader/index.html +++ b/Cloud VPS/jdownloader/index.html @@ -1 +1 @@ - JDownloader - Documentation
Skip to content

JDownloader

After setting up JDownloader and it appears well in WebUI.

The section is useless now as UHDMV has shutdown and it’s pointless to setup multiple automated JDownloader server on VPS.

Settings for JDownloader

Debloat settings https://rentry.org/jdownloader2
Advanced Settings
GraphicalUserInterfaceSettings: Banner -> disable
GraphicalUserlnterfaceSettings: Premium Alert Task Column - > disable
GraphicalUserInterfaceSeftings: Premium Alert Speed Column -> disable
GraphicalUserInterfaceSettings: Premium Alert ETA Column -> disable
GraphicalUsserInterfaceSeftings: Special Deal Oboom Dialog Visible On Startup -> disable
GraphicalUsserInterfaceSeftings: Special Deals -> disable
GraphicalUsserInterfaceSeftings: Donate Button State -> Hidden (automode)

Theming

GraphicalUserInterfaceSettings: Look And Feel Theme - > BLACK_EYE
For Colors
LAFSettings: Color For

  • Panel background and header background and alternate row background- #ff222222
  • Selected Rows Background - #ff666666
  • Package Row Background - #ff333333
  • Mouse Over Row Background - #ff666666
  • Panel Header Foreground, Tooltip Foreground, Selected Rows Foreground, Package Row Foreground, Mouse Over Row Foreground, Alternate Row Foreground, Account Temp Error Row Foreground, Account Error Row Foreground- #ffffffff
    • basically, change all the black values to white when searching for color fore, change everything except blue colors and error color
  • Enabled Text Color, Speed Meter Text, Speed Meter Average Text, Config Panel Description Text, Config Header Text Color - #ffffffff
  • Disabled Text Color - #ff666666
    • basically, when searching for color text, change all to white except for disabled text
\ No newline at end of file + JDownloader - Documentation
Skip to content

JDownloader

After setting up JDownloader and it appears well in WebUI.

The section is useless now as UHDMV has shutdown and it’s pointless to setup multiple automated JDownloader server on VPS.

Settings for JDownloader

Debloat settings https://rentry.org/jdownloader2
Advanced Settings
GraphicalUserInterfaceSettings: Banner -> disable
GraphicalUserlnterfaceSettings: Premium Alert Task Column - > disable
GraphicalUserInterfaceSeftings: Premium Alert Speed Column -> disable
GraphicalUserInterfaceSettings: Premium Alert ETA Column -> disable
GraphicalUsserInterfaceSeftings: Special Deal Oboom Dialog Visible On Startup -> disable
GraphicalUsserInterfaceSeftings: Special Deals -> disable
GraphicalUsserInterfaceSeftings: Donate Button State -> Hidden (automode)

Theming

GraphicalUserInterfaceSettings: Look And Feel Theme - > BLACK_EYE
For Colors
LAFSettings: Color For

  • Panel background and header background and alternate row background- #ff222222
  • Selected Rows Background - #ff666666
  • Package Row Background - #ff333333
  • Mouse Over Row Background - #ff666666
  • Panel Header Foreground, Tooltip Foreground, Selected Rows Foreground, Package Row Foreground, Mouse Over Row Foreground, Alternate Row Foreground, Account Temp Error Row Foreground, Account Error Row Foreground- #ffffffff
    • basically, change all the black values to white when searching for color fore, change everything except blue colors and error color
  • Enabled Text Color, Speed Meter Text, Speed Meter Average Text, Config Panel Description Text, Config Header Text Color - #ffffffff
  • Disabled Text Color - #ff666666
    • basically, when searching for color text, change all to white except for disabled text
\ No newline at end of file diff --git a/Docker Apps/01 docker-infra/index.html b/Docker Apps/01 docker-infra/index.html index f63790d..a36bbd2 100755 --- a/Docker Apps/01 docker-infra/index.html +++ b/Docker Apps/01 docker-infra/index.html @@ -1,4 +1,4 @@ - 01 Docker Infrastructure - Documentation
Skip to content

01 Docker Infrastructure

Filesystem

Compose

All docker-compose.yml files are stored in ~/docker folder, which then by default is under the network docker_default.

  • by default for newly created apps, a new folder is created and docker-compose.yml is created for that app for testing
    • once app testing is complete, the compose file can be moved docker root folder if appropriate or remain
  • some apps can be grouped together and these compose files are in the root docker folder such as media.yml, network.yml, the grouping allows multiple services to be managed by a single compose. For grouping, some of the property can include
    • the apps share common properties such as arrs apps
    • it is preferable for apps to live in same network, eg. teslamate
    • a large app requiring multiple containers eg. frontend, mysql etc..
    • apps share similar/same category, such as qBittorrent and nzbget can be put together in downloader.yml even though they do not have common properties or require same networking
Storage

The storage used for all containers are bind mount.
- application configs are stored in ~/docker/[app]
- if an app has multiple components needing persistence (eg. app with database, helpers), a folder will be created as such ~/docker/[app]/postgres etc.
- apps that also store non-config data (such as music, documents etc.) and not using a lot of space can bind mount /mnt/nvme/share (a directory on local or another SSD) for fast data access and without spinning up HDD
- exceptions are home assistant or its related home automation containers and these are stored at /srv/homeassistant

Backup

The entire docker root folder is copied to a NFS share on another computer. With exception of minecraft and home assistant which a specialized method is used.

Network

With docker-compose, a new network is created with the name of folder the compose is located, while it’s possible to change network, it is not straightforward, therefore, there is no points in manually defining networks unless required.

Public 172.80.0.0/16 - bridge network for public facing applications with reverse proxy, this way when configuring Nginx Proxy Manager, all it need is to enter container_name:80 rather than IP address.

  • Nginx Proxy Manager - 172.80.44.3
  • Other containers will use docker DHCP to get address
  • Containers that need to public facing can attach to this network
    Media 172.96.0.0/16 - bridge network for arrs, downloader and management applications for easy interconnection when configuring
    Minecraft 172.255.255.0/24 - bridge network for Minecraft related networks
  • Minecraft server (mcserver) - 172.255.255.65

Categories

Media Apps - apps related to media acquisition, curation and other functions services for Jellyfin
Networking - reverse proxy, DNS, VPN and related services
Home Automation - home assistant and its associated functions
VNC - containers based on jlesage-vnc-apps or Linuxserver Kasm images, usually desktop apps run in a browser via noVNC
Management - tools for managing docker containers or entire server
Games - game servers and associated tools
Filesharing - apps that share files to other clients
Documentation - notes and operation procedures for server infrastructure
Authentication - services that handle single sign-on (SSO) with users

Comments

01 Docker Infrastructure

Filesystem

Compose

All docker-compose.yml files are stored in ~/docker folder, which then by default is under the network docker_default.

  • by default for newly created apps, a new folder is created and docker-compose.yml is created for that app for testing
    • once app testing is complete, the compose file can be moved docker root folder if appropriate or remain
  • some apps can be grouped together and these compose files are in the root docker folder such as media.yml, network.yml, the grouping allows multiple services to be managed by a single compose. For grouping, some of the property can include
    • the apps share common properties such as arrs apps
    • it is preferable for apps to live in same network, eg. teslamate
    • a large app requiring multiple containers eg. frontend, mysql etc..
    • apps share similar/same category, such as qBittorrent and nzbget can be put together in downloader.yml even though they do not have common properties or require same networking
Storage

The storage used for all containers are bind mount.

  • application configs are stored in ~/docker/[app]
    • if an app has multiple components needing persistence (eg. app with database, helpers), a folder will be created as such ~/docker/[app]/postgres etc.
  • apps that also store non-config data (such as music, documents etc.) and not using a lot of space can bind mount /mnt/nvme/share (a directory on local or another SSD) for fast data access and without spinning up HDD
  • exceptions are home assistant or its related home automation containers and these are stored at /srv/homeassistant
Backup

The entire docker root folder is copied to a NFS share on another computer. With exception of minecraft and home assistant which a specialized method is used.

Network

With docker-compose, a new network is created with the name of folder the compose is located, while it’s possible to change network, it is not straightforward, therefore, there is no points in manually defining networks unless required.

Public 172.80.0.0/16 - bridge network for public facing applications with reverse proxy, this way when configuring Nginx Proxy Manager, all it need is to enter container_name:80 rather than IP address.

  • Nginx Proxy Manager - 172.80.44.3
  • Other containers will use docker DHCP to get address
  • Containers that need to public facing can attach to this network
    Media 172.96.0.0/16 - bridge network for arrs, downloader and management applications for easy interconnection when configuring
    Minecraft 172.255.255.0/24 - bridge network for Minecraft related networks
  • Minecraft server (mcserver) - 172.255.255.65

Categories

Media Apps - apps related to media acquisition, curation and other functions services for Jellyfin
Networking - reverse proxy, DNS, VPN and related services
Home Automation - home assistant and its associated functions
VNC - containers based on jlesage-vnc-apps or Linuxserver Kasm images, usually desktop apps run in a browser via noVNC
Management - tools for managing docker containers or entire server
Games - game servers and associated tools
Filesharing - apps that share files to other clients
Documentation - notes and operation procedures for server infrastructure
Authentication - services that handle single sign-on (SSO) with users

Comments

\ No newline at end of file + Documentation
\ No newline at end of file diff --git a/search/search_index.json b/search/search_index.json index 10db849..04ff9b2 100755 --- a/search/search_index.json +++ b/search/search_index.json @@ -1 +1 @@ -{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Home","text":""},{"location":"#recent-updates","title":"Recent Updates","text":"
  • 01 Docker Infrastructure
  • JDownloader
  • OliveTin
  • Samba(SMB) Setup
  • Dynamic DNS Updater Docker
  • Filebrowser
  • Mkdocs
  • Debian-Based Server Setup
  • Tunneling Basic Services (Jellyfin, Web) with Caddy and Tailscale
  • Useful Plugins
"},{"location":"mkdocs/","title":"Mkdocs","text":""},{"location":"mkdocs/#mkdocs-gotchas","title":"Mkdocs Gotchas","text":"
  • yaml highlighting is broken with mdx-breakless-lists
  • when using heading #, if there are no line breaks between headings, any lists that is after content of the second heading will not be rendered properly, even with mdx-breakless-lists
  • when referencing a subheading in another file, mkdocs uses [](file.md#heading-with-space) while obsidian uses [](file.md#heading%20with%20space)
  • Before switching from lists to normal content, a line break is needed, otherwise the text below will be rendered with a indent
  • mkdocs subheadings [](#subheadings) must be in lower case
"},{"location":"mkdocs/#admonitioncallouts","title":"Admonition/Callouts","text":"Mkdocs native callout

callout content mkdocs

Nested

Nesting

  • ??? is also valid syntax for mkdocs
  • ???+ makes the callout collapsible and opens by default, while ???- makes it closed by default
    !!! notes \"Title\"\n    content\n
    Obsidian callouts requires the plugin mkdocs-callouts
Obsidian Native Callout

Callout content mkdocs

Nested callout

callout

> [!notes]+/- Callout title\n> Callout content\n
  • obsidian callout syntax also follows the same +,- for collapsing, it is to be inserted after the brackets

Available callouts include notes, info, warning, danger, success, failure, example, abstract, tip, question, bug.

"},{"location":"mkdocs/#keys-caret-mark-tilde","title":"Keys, Caret, Mark, Tilde","text":"

Keys ++ctrl+alt+plus++ Ctrl+Alt++ mark highlighting tilde strikethrough

"},{"location":"mkdocs/#tabbed-content","title":"Tabbed Content","text":"Tab 1Tab 2

Tab 1 content mkdocs Second line here.

Tab 2 content

=== \"Tab Name\"\n    Tab content\n

  • not supported in obsidian
"},{"location":"mkdocs/#attr_list","title":"attr_list","text":"

Fancy Buttons mkdocs [button text](link.md){ .md-button } Tooltip I\u2019m a tooltip that you can hover or click. [tooltip](https://link \"hover text\") Annotation I\u2019m an annotation, but you need to click the plus icon (1) to show. (2)

  1. annotation 1
  2. annotation 2
    Annotation location 1 (1), location (2)\n{ .annotate }\n1. annotation text to be shown\n

Footnote Insert footnote like [^1] 1

  • for inserting footnote [^1]
  • [^1]: at the end to explain the footnote; not supported in obsidian
"},{"location":"mkdocs/#code-highlighting","title":"Code Highlighting","text":"
from python import python\npython.run(arg1=123, arg2=\"mystr\")[2]\n
#!/bin/bash\nvar=\"myvar\"\necho $var+3\n
# yaml highlighting has to be `yaml` not `yml` and it's broken\n---\nversion: \"2.1\"\nservices:\n  clarkson:\n    image: lscr.io/linuxserver/clarkson\n    container_name: clarkson\n    environment:\n\n      - PUID=1000\n      - PGID=1000\n    ports:\n      - 3000:3000\n    restart: unless-stopped\n
  1. explaining the footnote.\u00a0\u21a9

"},{"location":"Cloud%20VPS/basic-server-setup-caddy-docker-jdownloader/","title":"Basic Server Setup, Caddy, Docker, JDownloader","text":"

Creating the VM in oracle cloud.

  1. Go to instances, new instance.
  2. Select the Always Free image, ARM or x86. 1 core only, recommended 4GB RAM, should be exceed 6 GB.
  3. Choose Ubuntu image.
  4. Download the SSH key and name it accordingly.

Key Pair

Using PuttyGen.

  • Place the key in ./ssh/openssh_keys
  • Open PuttyGen, conversion -> import keys
  • Save the key files as ppk file in root folder of ./ssh

Putty

  • Grab the IP address in the cloud console
  • Give a name in saved sessions
  • Go to behavior, choose these options
  • Under Data, make sure Terminal-type string is xterm-256color
  • Under Terminal -> Features, check \u201cdisable application keypad mode\u201d to fix issues with nano
  • The private key needs to be load in Connection -> SSH -> Auth -> Credentials

To get the IP address of the VPS at any time

curl ifconfig.me\n

Basic Setup + Docker

  1. Installing Caddy web server (simple to use reverse proxy), lightweight, easy and no need for docker. (Nginx is also a good candidate for reverse proxy as the command is easy to memorize and does not require consulting documentation sites. However, the syntax for nginx is extremely complex compared to caddy and might not be easily memorized.

https://caddyserver.com/docs/install#debian-ubuntu-raspbian

sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https\ncurl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg\ncurl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list\nsudo apt update\nsudo apt install caddy net-tools\n# net-tools is good utility, optionally can install firewall-cmd or nginx\n# sudo apt install firewalld nginx\n

2. Install Docker

https://docs.docker.com/engine/install/ubuntu/

sudo apt-get update\nsudo apt-get install \\\n    ca-certificates \\\n    curl \\\n    gnupg \\\n    lsb-release\n\nsudo mkdir -p /etc/apt/keyrings\ncurl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg\n\necho \\\n  \"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \\\n  $(lsb_release -cs) stable\" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null\n\nsudo apt-get update\nsudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin docker-compose\n# code modified to install docker-compose, each space in paragraph indicates a separate step in their official blog\n
sudo groupadd docker \\\nsudo usermod -aG docker ubuntu\nnewgrp docker # activate docker group immediately\n

The machine needs to be rebooted from Oracle Cloud console to finish installation.

JDownloader

https://hub.docker.com/r/jlesage/jdownloader-2

docker run -d \\\n    --name=jdownloader-2 \\\n    -p 5800:5800 \\\n    -v $HOME/appdata/jdownloader-2:/config:rw \\\n    -v $HOME/Downloads:/output:rw \\\n    --restart unless-stopped \\\n    jlesage/jdownloader-2\n

If port forwarding configured properly, entering ipaddress:5800 should work. If not open ports manually.

sudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 5800 -j ACCEPT\nsudo netfilter-persistent save\n

Other Useful Ports

sudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 443 -j ACCEPT\nsudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 80 -j ACCEPT\nsudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 25565 -j ACCEPT\nsudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 19132 -j ACCEPT\nsudo iptables -I INPUT 6 -m state --state NEW -p udp --dport 25565 -j ACCEPT\nsudo iptables -I INPUT 6 -m state --state NEW -p udp --dport 19132 -j ACCEPT\nsudo iptables -I INPUT 6 -m state --state NEW -p udp --dport 51820 -j ACCEPT\nsudo netfilter-persistent save\n

Alternative (firewall-cmd)

sudo apt install firewalld\n

Firewalld is a CentOS package, it may be unstable and crash, but command easy to memorize.

sudo firewall-cmd --zone=public --add-port 19132/tcp --permanent\nsudo firewall-cmd --zone=public --add-port 19132/udp --permanent\nsudo firewall-cmd --zone=public --add-port 25565/tcp --permanent\nsudo firewall-cmd --zone=public --add-port 25565/udp --permanent\nsudo firewall-cmd --zone=public --add-port 80/tcp --permanent\nsudo firewall-cmd --zone=public --add-port 443/tcp --permanent\nsudo firewall-cmd --zone=public --add-port 5800/tcp --permanent\nsudo firewall-cmd --reload\n

Troubleshooting network

For firewall-cmd, use this command to check all open ports.

sudo firewall-cmd --list-all\n

Using netstat, or pipe it to grep

netstat -tln\n# | grep 8080 etc...\n

Configuring JDownloader

  • Go to the JDownloader WebUI
  • Go to Settings
  • Under general, change the max number of downloads (2) and DL per hoster (1) to minimize issues

  • Go to MyJDownloader and configure MyJDownloader account

  • Go to extension modules, install and enable \u201cfolderwatch\u201d

The configuration for JDownloader is complete and should appear and be functional in WebUI. Advanced JDownloader documentation will be covered in detailed in another section. It is recommended to close port 5800 after configuring to prevent others accessing.

Basic Caddy Syntax (if applicable)

If the server that is being setup or restored needs functional service like bookstack or uptime-kuma, reverse proxy is needed.

sudo nano /etc/caddy/Caddyfile\n
{\n    email weebly2x10@gmail.com\n}\n\nyour-uptime-kuma.yoursubdomain.duckdns.org {\n        reverse_proxy http://127.0.0.1:3001\n}\n\nwiki.yoursubdomain.duckdns.org {\n        reverse_proxy http://127.0.0.1:6975\n}\n

Advanced

Tunneling Jellyfin and other web services with tailscale and caddy

Minecraft Server tunneling via Nginx (tcp only)

Minecraft Tunneling

"},{"location":"Cloud%20VPS/jdownloader/","title":"JDownloader","text":"

After setting up JDownloader and it appears well in WebUI.

The section is useless now as UHDMV has shutdown and it\u2019s pointless to setup multiple automated JDownloader server on VPS.

"},{"location":"Cloud%20VPS/jdownloader/#settings-for-jdownloader","title":"Settings for JDownloader","text":"

Debloat settings https://rentry.org/jdownloader2 Advanced Settings GraphicalUserInterfaceSettings: Banner -> disable GraphicalUserlnterfaceSettings: Premium Alert Task Column - > disable GraphicalUserInterfaceSeftings: Premium Alert Speed Column -> disable GraphicalUserInterfaceSettings: Premium Alert ETA Column -> disable GraphicalUsserInterfaceSeftings: Special Deal Oboom Dialog Visible On Startup -> disable GraphicalUsserInterfaceSeftings: Special Deals\u00a0-> disable GraphicalUsserInterfaceSeftings: Donate Button State\u00a0-> Hidden (automode)

"},{"location":"Cloud%20VPS/jdownloader/#theming","title":"Theming","text":"

GraphicalUserInterfaceSettings: Look And Feel Theme - > BLACK_EYE For Colors LAFSettings: Color For

  • Panel background and header background and alternate row background- #ff222222
  • Selected Rows Background - #ff666666
  • Package Row Background - #ff333333
  • Mouse Over Row Background - #ff666666
  • Panel Header Foreground, Tooltip Foreground, Selected Rows Foreground, Package Row Foreground, Mouse Over Row Foreground, Alternate Row Foreground, Account Temp Error Row Foreground, Account Error Row Foreground- #ffffffff
    • basically, change all the black values to white when searching for color fore, change everything except blue colors and error color
  • Enabled Text Color, Speed Meter Text, Speed Meter Average Text, Config Panel Description Text, Config Header Text Color - #ffffffff
  • Disabled Text Color - #ff666666
    • basically, when searching for color text, change all to white except for disabled text
"},{"location":"Cloud%20VPS/tunneling-basic-services-jellyfin-web-with-caddy-and-tailscale/","title":"Tunneling Basic Services (Jellyfin, Web) with Caddy and Tailscale","text":"

This procedure is not reproducible yet. Rigorous testing is still required before being documented. Here are the known procedures.

The purpose is to tunnel normal web or network intensive traffic such as Jellyfin when faced with CG-NAT or similar situations (in this case locked down dorm internet), also configure hardware transcoding (in this case NVENC, but Intel QSV for future) to mitigate limitations with Canadian ISP(s).

Jellyfin Install

https://jellyfin.org/downloads/server

Download and run the server installer.

Jellyfin Server Configuration

Tailscale (Windows Client)

https://tailscale.com/download/windows

Download, install and login.

Tailscale (Linux Server)

curl -fsSL https://tailscale.com/install.sh | sh\n
sudo tailscale up\n

All the tailscale management is done in the WebUI.

The Windows client is given a tailscale network IP address in 100 range. Check if Windows client is pingable on server.

ping 100.x.y.z\n

Check if Jellyfin is running and tunneled properly on Oracle cloud. It should get a webpage html rather than unable to resolve host etc.

curl http://100.x.y.z:8096\n

Reverse Proxy basic-server-setup-caddy-docker-jdownloader

Caddy installation and syntax is can be found on this page. Replace 127.0.0.1 with the tailscale IP address.

{\n    email weebly2x10@gmail.com\n}\n\nmovies.yoursubdomain.duckdns.org {\n        reverse_proxy http://100.x.y.z\n}\n

It is possible to set use the root domain (yoursub.duckdns.org) or a subfolder domain (movies.yousub.duckdns.org) for Jellyfin. After configuring the Caddyfile.

sudo systemctl reload caddy\n

Use netstat to check port 80, 443 is being listened. Make sure to port forward Oracle VPS.

Other Services

Follow the same syntax as the caddy file provided, if the root domain is used, then a subdomain must be used for other services.

Results

Inconclusive yet, more testing required.

"},{"location":"Cloud%20VPS/tunneling-minecraft-server-tcp-only-with-nginx/","title":"Tunneling Minecraft Server (tcp only) with Nginx","text":"

Procedure not reproducible yet, will be documented later.

"},{"location":"Computer%20Stuff/demucs-nvidia/","title":"Demucs Nvidia","text":"

Demucs is an music separation tool that has potential for a karaoke setup.

https://github.com/facebookresearch/demucs

https://www.youtube.com/watch?v=9QnFMKWEFcI&t=585s

https://docs.google.com/document/d/1XMmLrz-Tct1Hdb_PatcwEeBrV9Wrt15wHB1xhkB2oiY/edit

Installation on PC with Nvidia

  1. Firstly install Anaconda. Download Anaconda for Windows https://www.anaconda.com/products/distribution
  2. Install PyTorch. https://pytorch.org/get-started/locally/. Select the correct version of pytorch.
  3. Install ffmpeg. [https://www.gyan.dev/ffmpeg/builds/]](assets/gallery/2022-12/TwJimage.png)

Demucs

After installing the prerequesties.

Open \u201cAnaconda terminal\u201d and type

python.exe -m pip install -U demucs\n
pip install PySoundFile \n

Running Demucs

demucs \"C:\\path\\to\\music\\file.mp3\"\n

This will run demucs with CUDA GPU acceleration, make sure to put the path in double quote. The extracted file will be found in the directory where you run the command eg. the default Anaconda prompt starts in ~/separated

"},{"location":"Docker%20Apps/01%20docker-infra/","title":"01 Docker Infrastructure","text":""},{"location":"Docker%20Apps/01%20docker-infra/#filesystem","title":"Filesystem","text":""},{"location":"Docker%20Apps/01%20docker-infra/#compose","title":"Compose","text":"

All docker-compose.yml files are stored in ~/docker folder, which then by default is under the network docker_default.

  • by default for newly created apps, a new folder is created and docker-compose.yml is created for that app for testing
    • once app testing is complete, the compose file can be moved docker root folder if appropriate or remain
  • some apps can be grouped together and these compose files are in the root docker folder such as media.yml, network.yml, the grouping allows multiple services to be managed by a single compose. For grouping, some of the property can include
    • the apps share common properties such as arrs apps
    • it is preferable for apps to live in same network, eg. teslamate
    • a large app requiring multiple containers eg. frontend, mysql etc..
    • apps share similar/same category, such as qBittorrent and nzbget can be put together in downloader.yml even though they do not have common properties or require same networking
"},{"location":"Docker%20Apps/01%20docker-infra/#storage","title":"Storage","text":"

The storage used for all containers are bind mount. - application configs are stored in ~/docker/[app] - if an app has multiple components needing persistence (eg. app with database, helpers), a folder will be created as such ~/docker/[app]/postgres etc. - apps that also store non-config data (such as music, documents etc.) and not using a lot of space can bind mount /mnt/nvme/share (a directory on local or another SSD) for fast data access and without spinning up HDD - exceptions are home assistant or its related home automation containers and these are stored at /srv/homeassistant

"},{"location":"Docker%20Apps/01%20docker-infra/#backup","title":"Backup","text":"

The entire docker root folder is copied to a NFS share on another computer. With exception of minecraft and home assistant which a specialized method is used.

"},{"location":"Docker%20Apps/01%20docker-infra/#network","title":"Network","text":"

With docker-compose, a new network is created with the name of folder the compose is located, while it\u2019s possible to change network, it is not straightforward, therefore, there is no points in manually defining networks unless required.

Public 172.80.0.0/16 - bridge network for public facing applications with reverse proxy, this way when configuring Nginx Proxy Manager, all it need is to enter container_name:80 rather than IP address.

  • Nginx Proxy Manager - 172.80.44.3
  • Other containers will use docker DHCP to get address
  • Containers that need to public facing can attach to this network Media 172.96.0.0/16 - bridge network for arrs, downloader and management applications for easy interconnection when configuring Minecraft 172.255.255.0/24 - bridge network for Minecraft related networks
  • Minecraft server (mcserver) - 172.255.255.65
"},{"location":"Docker%20Apps/01%20docker-infra/#categories","title":"Categories","text":"

Media Apps - apps related to media acquisition, curation and other functions services for Jellyfin Networking - reverse proxy, DNS, VPN and related services Home Automation - home assistant and its associated functions VNC - containers based on jlesage-vnc-apps or Linuxserver Kasm images, usually desktop apps run in a browser via noVNC Management - tools for managing docker containers or entire server Games - game servers and associated tools Filesharing - apps that share files to other clients Documentation - notes and operation procedures for server infrastructure Authentication - services that handle single sign-on (SSO) with users

"},{"location":"Docker%20Apps/bookstack/","title":"Bookstack","text":""},{"location":"Docker%20Apps/bookstack/#installation","title":"Installation","text":"

Change port to 6975

Add in docker-compose: restart: unless-stopped

$docker directory = /home/docker .... etc

Docker-Compose file reference

https://github.com/solidnerd/docker-bookstack/blob/master/docker-compose.yml

version: '2'\nservices:\n  mysql:\n    image: mysql:8.0\n    environment:\n\n    - MYSQL_ROOT_PASSWORD=secret\n    - MYSQL_DATABASE=bookstack\n    - MYSQL_USER=bookstack\n    - MYSQL_PASSWORD=secret\n    volumes:\n    - mysql-data:/var/lib/mysql\n    restart: unless-stopped\n\n  bookstack:\n    image: solidnerd/bookstack:22.10.2\n    depends_on:\n\n    - mysql\n    environment:\n    - DB_HOST=mysql:3306\n    - DB_DATABASE=bookstack\n    - DB_USERNAME=bookstack\n    - DB_PASSWORD=secret\n    #set the APP_ to the URL of bookstack without without a trailing slash APP_URL=https://example.com\n    - APP_URL=http://xxx.xxxmydomainxxx.duckdns.org\n    volumes:\n    - $docker/public-uploads:/var/www/bookstack/public/uploads\n    - $docker/storage-uploads:/var/www/bookstack/storage/uploads\n    ports:\n    - \"6975:8080\"\n    restart: unless-stopped\n

Notice: The default password for bookstack is

admin@admin.com

password

Permissions: remember the set write permission on public-uploads folder so users can upload photos.

"},{"location":"Docker%20Apps/bookstack/#backup-and-restore","title":"Backup and Restore","text":"

Files Backup:

tar -czvf bookstack-files-backup.tar.gz public-uploads storage-uploads\n

Restore:

tar -xvzf bookstack-files-backup.tar.gz\n

Database backup:

sudo docker exec bookstack_mysql_1 /usr/bin/mysqldump -u root --password=secret bookstack > ./bookstack/bookstack_db.sql\n

Restore:

sudo docker exec -i bookstack_mysql_1 mysql -u root --password=secret bookstack < /$docker/bookstack/bookstack_db.sql\n
  • bookstack_mysql1 is the container name
  • password is secret or the database password
"},{"location":"Docker%20Apps/bookstack/#reverse-proxy","title":"Reverse Proxy","text":"

Use subdomain in proxy manager.

Backing Up and Restoring with LinuxServer.io container

Due to limits or Oracle Cloud free tier. The only arm image is from linuxserver io container, and it is different than solidnerd image.

Docker-Compose file

version: \"2\"\nservices:\n  bookstack:\n    image: lscr.io/linuxserver/bookstack\n    container_name: bookstack\n    environment:\n\n      - PUID=1001\n      - PGID=1001\n      - APP_URL=https://wiki.xxx.duckdns.org\n      - DB_HOST=bookstack_db\n      - DB_USER=bookstack\n      - DB_PASS=secret\n      - DB_DATABASE=bookstackapp\n    volumes:\n      - /home/ubuntu/bookstack:/config\n    ports:\n      - 6975:80\n    restart: unless-stopped\n    depends_on:\n      - bookstack_db\n\n  bookstack_db:\n    image: lscr.io/linuxserver/mariadb\n    container_name: bookstack_db\n    environment:\n\n      - PUID=1001\n      - PGID=1001\n      - MYSQL_ROOT_PASSWORD=secret\n      - TZ=Europe/London\n      - MYSQL_DATABASE=bookstackapp\n      - MYSQL_USER=bookstack\n      - MYSQL_PASSWORD=secret\n    volumes:\n      - /home/ubuntu/bookstack:/config\n    restart: unless-stopped\n

Notice: In Oracle cloud free tier, the default ubuntu user is 1001, not 1000. For database name, it it bookstackapp, keep in mind when executing restore command. The folder structure is also different. In the solidnerd container, the images are stored at /public-uploads while in LSIO container it is stored at /www/uploads

"},{"location":"Docker%20Apps/bookstack/#backing-up-from-home-pc","title":"Backing Up (from home PC)","text":"

Images

cd into /public-uploads and make a tar archive

tar -czvf images.tar.gz images\n

Backup the database

sudo docker exec bookstack_mysql_1 /usr/bin/mysqldump -u root --password=secret bookstack > ./bookstack_db.sql\n

Transfer to Oracle Cloud Server

scp -i oracle-arm-2.key images.tar.gz bookstack_db.sql ubuntu@$IPADDR:/home/ubuntu/bookstack/www/uploads\n

Take in consideration the location where LSIO image stores the images.

"},{"location":"Docker%20Apps/bookstack/#restore-into-oracle-cloud","title":"Restore (into Oracle Cloud)","text":"

Images (/home/ubuntu/bookstack/www/uploads)

tar -xvzf images.tar.gz\n

Database

The image url in the database still refers to old server url, it needs to be changed. The following command replace the subdomain in the sq1 dump.

sed -i 's/wiki.$home.duckdns.org/wiki.$oracle.duckdns.org/g' bookstack_db.sql\n

Restore the database.

sudo docker exec -i bookstack_db mysql -u root --password=secret bookstackapp < /home/ubuntu/bookstack/www/uploads/bookstack_db.sql\n
"},{"location":"Docker%20Apps/bookstack/#crontab","title":"Crontab","text":"

On Home PC

0 23 * * 2,5 /home/karis/bookstack.sh\n
#!/bin/bash\n\ncd ~/docker/bookstack/public-uploads #location of bookstack public uploads\ntar -czvf images.tar.gz images\nsudo docker exec bookstack_mysql_1 /usr/bin/mysqldump -u root --password=secret bookstack > ./bookstack_db.sql\nscp -i oracle-arm-2.key images.tar.gz bookstack_db.sql ubuntu@$ORACLEIP:/home/ubuntu/bookstack/www/uploads\n

Make sure to copy the oracle-arm-2.key to the appropriate location (~/docker/bookstack/public-uploads)

Also make sure the permission of oracle-arm-2.key is in correct permission (600). Especially changing the permission of public-uploads folder to allow write access.

Do a backup sequence in crontab at 11pm every Tuesday and Friday.

Oracle Cloud Server

0 8 * * 3,6 /home/ubuntu/bookstack.sh\n
#!/bin/bash\n\ncd ~/bookstack/www/uploads #directory where bookstack files scp from home are located\ntar -xvzf images.tar.gz\nsed -i 's/wiki.$homeip.duckdns.org/wiki.$oracle.duckdns.org/g' bookstack_db.sql\nsudo docker exec -i bookstack_db mysql -u root --password=secret bookstackapp < /home/ubuntu/bookstack/www/uploads/bookstack_db.sql\n

Restore the sequence after backup, every Wednesday and Saturday at 8am (need to consider the TZ between Vancouver, Edmonton and Toronto, or any the time zone of the remote server)

"},{"location":"Docker%20Apps/ddns-update/","title":"Dynamic DNS Updater Docker","text":"

Official Image: https://hub.docker.com/r/linuxserver/duckdns Custom Github Page: https://github.com/vttc08/docker-duckdns-dynu

This is a docker container that automatically updates the public IPv4 address of the server every 5 minutes to dynamic DNS services Dynu and DuckDNS. It is the fork of Linuxserver DuckDNS container.

"},{"location":"Docker%20Apps/ddns-update/#docker-compose","title":"Docker Compose","text":"
  services:\n      duckdns:\n        image: vttc08/docker-duckdns-dynu:latest\n        container_name: duckdns\n        env_file: ddns.env\n        environment:\n\n          - TZ=America/Vancouver\n          - PUID=1000\n          - PGID=1001\n        restart: unless-stopped\n

These need to be filled in the ddns.env

DYNU_HOST= # full name of dynu domains\nDYNU_PASS= # md5 hashed dynu login pass\nSUBDOMAINS= # DuckDNS domains without the duckdns.org part\nTOKEN= # DuckDNS token \n

  • token will be visible in DuckDNS dashboard
  • Dynu pass is the same as login; alternatively, it is possible to create a dedicated password just for IP update MD5 generator
    echo -n \"password\" | md5sum\n
  • when setting the IP to 10.0.0.0 in Dynu update API, dynu will automatically update the IP address to the IP address making that request
"},{"location":"Docker%20Apps/ddns-update/#other-usage","title":"Other Usage","text":"

docker restart duckdns will manually run IP update docker exec -it duckdns /app/debug.sh or other scripts, debug script will print out IP address of subdomains resolved by Cloudflare

"},{"location":"Docker%20Apps/epic-games-free-games/","title":"Epic Games Free Games","text":"

Buy Free Games from Epic Games

https://hub.docker.com/r/charlocharlie/epicgames-freegames

Config

NEED TO CHANGE

Email: email address

Password: password

Webhook URL: make a discord channel and click settings. Go to integrations, then webhook, copy webhook URL.

mentioned Users: right click your profile, and click Copy ID

TOTP

  1. Go here to login. https://www.epicgames.com/account/password Login with Epic Games account.
  2. Click \u201cenable authenticator app.\u201d
  3. In the section labeled \u201cmanual entry key,\u201d copy the key.
  4. Use your authenticator app to add scan the QR code.
  5. Activate 2FA by completing the form and clicking activate.
  6. Once 2FA is enabled, use the key you copied as the value for the TOTP parameter.

Docker

docker run -d -v /home/karis/docker/epicgames:/usr/app/config:rw -p 3000:3000 -m 2g --name epicgames --restart unless-stopped charlocharlie/epicgames-freegames:latest\n

Change the name of the container to a friendly name. Restart unless stopped so it restart automatically.

Copy and Paste

The default json configuration is located at /home/karis/docker/epicgames or $HOME/docker/epicgames.

Fix Login Issue Using Cookies

https://store.epicgames.com/en-US/

  1. Visit this site and make sure it\u2019s logged in.
  2. Install this extension EditThisCookie https://chrome.google.com/webstore/detail/editthiscookie/fngmhnnpilhplaeedifhccceomclgfbg/related
  3. Open the extension and change the url to epicgames.com/id as in screenshot below
  4. Export the cookie

  1. Go to $HOME/docker/epicgames and create a new file email@gmail.com-cookies.json
  2. If the json file is already there, truncate it with \u2013size 0
  3. Paste the cookie value to the json file
  4. Restart container.

Update

docker pull charlocharlie/epicgames-freegames:latest\ndocker rm -f epicgames\ndocker images | grep epicgames\n# use docker rmi to remote the corresponding image \n# re run the epicgames docker run command\n
"},{"location":"Docker%20Apps/filebrowser/","title":"Filebrowser","text":"

Filebrowser app on a webbrowser, port 4455. free-games-claimer

Docker-compose deployment

version: '3.9'\nservices:\n    filebrowser:\n        container_name: filebrowser\n        image: filebrowser/filebrowser\n        ports:\n\n            - '4455:80'\n        user: 1000:1000\n        restart: unless-stopped\n        volumes:\n            - '~/docker/filebrowser/.filebrowser.json:/.filebrowser.json'\n            - '~/docker/filebrowser/filebrowser.db:/database.db'\n            - '~/docker/filebrowser/branding:/branding'\n            - '~/docker:/srv/docker'\n            - '/mnt/data:/srv/data'\n            - '/mnt/nvme/share:/srv/nvme-share'\n

The first 3 bind mount are for configuration of filebrowser, eg. config, database and branding files. On first deployment, need to create an empty database.db file. The remaining bind mount are for the folders that need to be accessed, the folders should be bound under /srv.

This is the content of .filebrowser.json

{\n    \"port\": 80,\n    \"baseURL\": \"\",\n    \"address\": \"\",\n    \"log\": \"stdout\",\n    \"database\": \"/database.db\",\n    \"root\": \"/srv\"\n  }\n
"},{"location":"Docker%20Apps/filebrowser/#usershare","title":"User/Share","text":"

The user and share management in filebrowser is simple. The shares have a expiring time, and can optionally have a password. The recipient can view and download files in the share but cannot upload.

To create a new user, it\u2019s under settings -> User Management, and add a user and password accordingly, and give appropriate permission. The scope is where the root folder where the user have access to, since the docker data folder is bound at /srv/docker and /srv is defined as root folder in config, the folder name to put in scopes would be /docker. Only one scope is allowed.

It is also possible to add rules to prevent user access of files within a scope. Under rules, enter the path that is relative to the scope, for example /docker/minecraft/config would be /config

"},{"location":"Docker%20Apps/filebrowser/#personalization","title":"Personalization","text":"

Enable dark theme - Setting -> Global Settings -> Branding

  • also change the branding directory path to /branding which is bind mount in docker

Under the branding folder, create a file custom.csswhich is used for css customization. Then create a folder img and place logo.svg in it for custom icon. The icon is the same as egow entertainment and stored in OliveTin icon PSD file. Under the folder img, create a folder icons and use favicon generator site to create an icon archive and put all the content of that archive in the icons folder, the result should look like this.

Reverse Proxy/Homepage

Reverse proxy is normal procedure using NPM. To add bookmark to a file location, use browser/homepages bookmark function.

"},{"location":"Docker%20Apps/free-games-claimer/","title":"Free Games Claimer","text":"

https://github.com/vogler/free-games-claimer

This is the Github repo for the new and advanced free games claimer. This is implemented after Epicgames FreeGames keeps failing.

"},{"location":"Docker%20Apps/free-games-claimer/#configuration","title":"Configuration","text":"

Using Docker-Compose

In the folder structure

server: ~/docker/fgc$\ndocker-compose.yml\nfgc.env\n

fgc.env is the environment file for all the password/keys to login to different game services, fill it in manually or use a backup.

EG_OTPKEY=\nEG_EMAIL=\nEG_PASSWORD=\nNOTIFY=discord://123456/ABCD\nPG_EMAIL=\nPG_PASSWORD=\nGOG_EMAIL=\nGOG_PASSWORD=\nTIMEOUT=300\n

NOTIFY=discord://123456/ABCD if the webhook looks like this https://discord.com/api/webhooks/123456/ABCD

TIMEOUT=300 sets the timeout to 300s before the container skip and error out due to EpicGames captcha problems. However, the impact on prime gaming and GOG are not tested.

docker-compose.yml

services:\n  free-games-claimer:\n    container_name: FGC # is printed in front of every output line\n    image: ghcr.io/vogler/free-games-claimer # otherwise image name will be free-games-claimer-free-games-claimer\n    build: .\n    ports:\n\n      - \"5990:5900\" # VNC server\n      - \"5890:6080\" # noVNC (browser-based VNC client)\n    volumes:\n      - ~/docker/fgc:/fgc/data\n      - ~/docker/fgc/epic-games.js:/fgc/epic-games.js\n      - ~/docker/fgc/prime-gaming.js:/fgc/prime-gaming.js\n      - ~/docker/fgc/gog.js:/fgc/gog.js\n    command: bash -c \"node epic-games; node prime-gaming; node gog; echo sleeping; sleep 1d\"\n    env_file:\n      - fgc.env\n    restart: unless-stopped\n

This docker-compose file use the environment file fgc.env as indicated above and runs once every day. It also contains VNC server/web based client.

"},{"location":"Docker%20Apps/free-games-claimer/#missing-captcha-session","title":"Missing Captcha Session","text":"

This should no longer be needed. Edit the line to epicgames.js code and replace with the following message. When the captcha is missed, it will send a notification for manual claiming.

wait notify(`epic-games: got captcha challenge right before claim. Use VNC to solve it manually. Game link: \\n ${url}`)\n

EpicGames require a captcha to claim free games. If the 5 minute timeout window for EpicGames is missed, it is no longer possible to claim the games unless waiting for the next day, which due to the nature of discord notifications, there is a slim to none chance of catching the captcha at next day. To continuing claiming after acknowledging the missed session, use portainer, ConnectBot Android to temporarily restart the container to restore VNC session.

In order to restore the default time of claiming the games. Eg. waking up on Thurs or Fri and a predictable time and claim games, use the linux at command. Need to install at using apt.

at 9:20\n> docker restart FGC\n> <EOT>\n

This will run the command at 9:20 AM the next day. Ctrl-D to exit at prompt and verify the time is correct.

"},{"location":"Docker%20Apps/jlesage-vnc-apps/","title":"jlesage VNC Apps","text":"

VNC apps consists of desktop applications that have the GUI in a web browser, mostly from the creator jlesage.

"},{"location":"Docker%20Apps/jlesage-vnc-apps/#environments","title":"Environments","text":"

At least for apps from jlesage, it supports an environment variable. Create an environment file called vnc.env

The environment file can be reference in many docker images from jlesage using docker-compose. The current environment variable specify U/GID, time zone and make every app dark mode. It is also possible to set VNC passwords. This is the full list of environment variables.

USER_ID=1000\nGROUP_ID=1000\nTZ=America/Vancouver\nDARK_MODE=1\n

The jlesage apps have 2 ports, port 5800 for viewing the VNC app on a web browser on desktop; port 5900 is for VNC protocol that can be used in dedicated VNC viewer or mobile viewing.

"},{"location":"Docker%20Apps/jlesage-vnc-apps/#general-bind-mounts","title":"General Bind Mounts","text":"

The appdata bind mount is located in the ~/docker/vnc, as seen from the yml example, the vnc environment file vnc.env is placed in the appdata folder. For application requiring access to movie storage, the bind mount is on the corresponding hard drive or pool. As for applications requiring access to storage but not large media, it\u2019s best to put the files on a SSD.

This is an example of VNC container of MKVToolNix. The vnc.yml file is backed up elsewhere.

    mkvtoolnix:\n        image: jlesage/mkvtoolnix\n        env_file:\n\n            - ./vnc/vnc.env\n        volumes:\n            - '/mnt/data/nzbget:/storage:rw'\n            - '~/docker/vnc/mkvtoolnix:/config:rw'\n        ports:\n            - '5820:5800'\n            - '5920:5900'\n        container_name: mkvtoolnix\n
"},{"location":"Docker%20Apps/jlesage-vnc-apps/#ports","title":"Ports","text":"

The application port start from 5800/5900 for its corresponding access and add 10 for each application.

JDownloader: 5800

Firefox: 5810

MKVToolNix: 5820

MKVCleaver: 5840

MegaBasterd: 5860 (no VNC viewer 59xx port)

There are also some application specific setup. For applications accessing hard drive or intensive apps, it is best to stop when not used. Lazytainer and ContainerNursery and possibly using DNS server can help automate this process.

JDownloader

JDownloader Setup

"},{"location":"Docker%20Apps/minecraft/","title":"Minecraft","text":"

This chapter is about Minecraft setup in Docker server.

"},{"location":"Docker%20Apps/minecraft/#minecraft-prep-and-install","title":"Minecraft Prep and Install","text":"

Client Setup (Java + Online)

  1. Download Java
  2. Download OptiFine the latest version.
  3. On the official Minecraft client, go add a new installation and match the version with OptiFine.
  4. Download and try the official version, then install OptiFine with Java.
  5. Under Settings -> Keep the Launcher open while games are running

Client Setup (Java + Offline)

  1. Use the client PolyMC to enable offline play.
  2. Go to the right corner, manage accounts and create an offline account.
  3. Click on add an instance and follow the guide.
  4. To install OptiFine, need the official launcher first, then download OptiFine
  5. Extract OptiFine, the extracted file should be ending in _MOD.jar
  6. Open the jar file in WinRAR, then move the files from notch folder into the base folder. Save the jar archive.
  7. Go to PolyMC, right click on the instance, click Edit -> Versions -> Add to minecraft.jar and select the modified OptiFine.

Docker Server Setup

Docker-compose for minecraft server

version: \"3.9\"\nservices:\n  minecraft:\n    image: marctv/minecraft-papermc-server:latest\n    restart: unless-stopped\n    container_name: mcserver\n    environment:\n\n      - MEMORYSIZE=4G\n      - PAPERMC_FLAGS=\"\"\n      - PUID=1000\n      - PGID=1000\n    volumes:\n      - ~/docker/minecraft:/data:rw\n    ports:\n      - 25565:25565\n      - 19132:19132\n    stdin_open: true\n    tty: true\n

This downloads the latest version of Minecraft, to use another PaperMC version, need to build the image from scratch.

Warning: PaperMC cannot be downgraded, only newerversion of PaperMC can be installed after first run.

git clone https://github.com/mtoensing/Docker-Minecraft-PaperMC-Server\n# go edit the \"ARG version=1.xx.x\" to the correct version\ndocker build -t marctv/mcserver:1.xx.x\n

Folders and Plugins

Plugins are located in folder ./plugins some plugins have .yml files. To update or download plugins, use scp, wget on the server or VSCode.

The world folder consists of the save data. It is separated into world, nether, the_end.

Before starting the server, the eula.txt must have eula=true.

bukkit and spigot.yml in the root folder are configuration files for PaperMC.

Rcon Commands

To access the rcon-cli, use docker attach mcserver, to exit, use Ctrl-P and Q, if using VSCode may need to edit keyboard shortcut.

Editing VSCode Shortcut

Press Ctrl-Shift-P and search for keyboard shortcut json.

[\n    {\n        \"key\": \"ctrl+p\",\n        \"command\": \"ctrl+p\",\n        \"when\": \"terminalFocus\"\n    },\n\n    {\n        \"key\": \"ctrl+q\",\n        \"command\": \"ctrl+q\",\n        \"when\": \"terminalFocus\"\n    },\n\n    {\n        \"key\": \"ctrl+e\",\n        \"command\": \"ctrl+e\",\n        \"when\": \"terminalFocus\"\n    }\n\n]\n
"},{"location":"Docker%20Apps/minecraft/#useful-plugins","title":"Useful Plugins","text":"

WorldEdit

EssentialX

CoreProtect

ViaVersions - allow other similar version to join the server without conflict

Offline Mode/Mobile Bedrock

To allow offline play for PC version. Change server.properties and edit these lines

enforce-whitelist=false\nonline-mode=false\n

Refer to Minecraft Prep and Install to install offline client.

For bedrock compatibility, need the geyser plugin.

Geyser

To allows offline play for bedrock mobile version. Go to ./plugins/Geyser-Spigot/config.yml and change these lines. Do not install the plugin floodgate, if it\u2019s installed, removed the plugin. ViaVersions is also needed for mobile play.

auth-type: offline\nenable-proxy-connections: true\n

Now client can play without login to Xbox or Java.

WorldGuard

"},{"location":"Docker%20Apps/tesla-homepage/","title":"Tesla Homepage","text":"

This is a homepage that allows Tesla browser to enter full screen mode.

Docker-compose

services:\n  homepage-for-tesla:\n    image: jessewebdotcom/homepage-for-tesla:latest\n    container_name: homepage-for-tesla\n    environment:\n\n      - DEFAULT_THEME=13\n    volumes:\n      - ~/docker/tesla/public/bookmarks.json:/app/public/bookmarks.json\n      - ~/docker/tesla/public/images:/app/public/images\n    ports:\n      - \"3000:3000\"\n
"},{"location":"Docker%20Apps/Media%20Apps/audiobookshelf/","title":"Audiobookshelf","text":"

Audiobooks and podcasts.

Docker-compose, place it in the media apps compose media.yml

version: \"3.7\"\nservices:\n  audiobookshelf:\n    image: ghcr.io/advplyr/audiobookshelf:latest\n    environment:\n\n      - AUDIOBOOKSHELF_UID=99\n      - AUDIOBOOKSHELF_GID=100\n    ports:\n      - 13378:80\n    volumes:\n      - /mnt/m/Audios/audiobooks:/audiobooks # hard drive mount\n      - /mnt/m/Audios/podcasts:/podcasts # hard drive mount\n      - $HOME/audiobookshelf/config:/config\n      - $HOME/audiobookshelf/metadata:/metadata\n    restart: unless-stopped\n
"},{"location":"Docker%20Apps/Media%20Apps/audiobookshelf/#usage","title":"Usage","text":"

To add a library, go to settings, libraries and add the path as mounted in docker.

Go to Users, change the root password and create a new user. Note, the user cannot scan library, only the root can do that.

"},{"location":"Docker%20Apps/Media%20Apps/audiobookshelf/#adding-media","title":"Adding Media","text":"

Make sure the contents are in a separate folder. Follow naming like this. A cover image can also be created. The best bitrate should be under 128 kbps for smooth playback.

/audiobooks\n--- ./Author - Book\n---  --- ./cover.jpg\n---  --- ./book - 001 or book - chapter 1\n---  --- ./book - 002\n---  --- ./book - 003\n

In the WebUI, make sure logged in as root. Go to settings, library and scan. It will scan the newly added media.

If the media does not match or not have an image, go click the edit icon, go to Match, the best result is usually Audible.ca.

If the chapter does not match, chapters can be edited manually. Go to Chapter and Lookup.

"},{"location":"Docker%20Apps/Media%20Apps/audiobookshelf/#mobile-app","title":"Mobile App","text":"

https://play.google.com/store/apps/details?id=com.audiobookshelf.app

Mobile app also has download functionality, however, the directory cannot be changed, the default for download is /Internal Storage/Download/{Podcast or Audiobook}

The statistic of minutes listened is the actual minutes listened, not the minutes of audiobook progress listened (eg. playing at faster speed).

"},{"location":"Docker%20Apps/Media%20Apps/audiobookshelf/#scripting-windows","title":"Scripting (Windows)","text":"

ffmpeg detect audio silence (for splitting a large audio file into multiple chapters)

ffmpeg -i input.mp3 -af silencedetect=n=-50dB:d=1.5 -f null -\n
ffmpeg -i input.mp3 -af silencedetect=n=-50dB:d=1.5 -f null -loglevel debug 2>&1 - | findstr \"silence_duration\" | find /c /v \"\"\n

This will find silence parts below -50dB and duration threshold of 1.5s.

The second code (windows cmd only) for linux use grep -c, finds how many silence parts can be detected, this should correlate to number of chapters.

Once the optimal duration is set, use split.py.

ffmpeg that remove silence from audio

ffmpeg -i input.mp4 -af silenceremove=stop_periods=-1:stop_duration=4:stop_threshold=-50dB -b:a 96k output.mp3\n
  • stop_duration (threshold duration for removing silence part)
  • stop_periods = -1 (search for the entire audio track)

Use edge_reader.py to utilize Edge AI reader to read the audiobook if only the pdf book is provided.

After reading, put all the recorded files and pdf in the project folder and run processing.py twice.

"},{"location":"Docker%20Apps/Media%20Apps/rich-media/","title":"Rich Media","text":"

Hello Everyone

This is a demo consisting of medias.

Some Code

docker-compose up -d\n
import os\nimport time\n\nprint(\"hello world\")\nif a=b:\n  print(a)\nelif b=c:\n  try:\n    print(c)\n  except:\n    print(c+a)\nelse:\n  print(\"what is the meaning of life\")\n

More sample media

Portainer is a software for managing docker containers.

"},{"location":"Docker%20Apps/Minecraft/minecraft-prep-and-install/","title":"Minecraft Prep and Install","text":"

Client Setup (Java + Online)

  1. Download Java
  2. Download OptiFine the latest version.
  3. On the official Minecraft client, go add a new installation and match the version with OptiFine.
  4. Download and try the official version, then install OptiFine with Java.
  5. Under Settings -> Keep the Launcher open while games are running

Client Setup (Java + Offline)

  1. Use the client PolyMC to enable offline play.
  2. Go to the right corner, manage accounts and create an offline account.
  3. Click on add an instance and follow the guide.
  4. To install OptiFine, need the official launcher first, then download OptiFine
  5. Extract OptiFine, the extracted file should be ending in _MOD.jar
  6. Open the jar file in WinRAR, then move the files from notch folder into the base folder. Save the jar archive.
  7. Go to PolyMC, right click on the instance, click Edit -> Versions -> Add to minecraft.jar and select the modified OptiFine.

Docker Server Setup

Docker-compose for minecraft server

version: \"3.9\"\nservices:\n  minecraft:\n    image: marctv/minecraft-papermc-server:latest\n    restart: unless-stopped\n    container_name: mcserver\n    environment:\n\n      - MEMORYSIZE=4G\n      - PAPERMC_FLAGS=\"\"\n      - PUID=1000\n      - PGID=1000\n    volumes:\n      - ~/docker/minecraft:/data:rw\n    ports:\n      - 25565:25565\n      - 19132:19132\n    stdin_open: true\n    tty: true\n

This downloads the latest version of Minecraft, to use another PaperMC version, need to build the image from scratch.

Warning: PaperMC cannot be downgraded, only newer version of PaperMC can be installed after first run.

git clone https://github.com/mtoensing/Docker-Minecraft-PaperMC-Server\n# go edit the \"ARG version=1.xx.x\" to the correct version\ndocker build -t marctv/mcserver:1.xx.x\n

Folders and Plugins

Plugins are located in folder ./plugins some plugins have .yml files. To update or download plugins, use scp, wget on the server or VSCode.

The world folder consists of the save data. It is separated into world, nether, the_end.

Before starting the server, the eula.txt must have eula=true.

bukkit and spigot.yml in the root folder are configuration files for PaperMC.

Rcon Commands

To access the rcon-cli, use docker attach mcserver, to exit, use Ctrl-P and Q, if using VSCode may need to edit keyboard shortcut.

Editing VSCode Shortcut

Press Ctrl-Shift-P and search for keyboard shortcut json.

[\n    {\n        \"key\": \"ctrl+p\",\n        \"command\": \"ctrl+p\",\n        \"when\": \"terminalFocus\"\n    },\n\n    {\n        \"key\": \"ctrl+q\",\n        \"command\": \"ctrl+q\",\n        \"when\": \"terminalFocus\"\n    },\n\n    {\n        \"key\": \"ctrl+e\",\n        \"command\": \"ctrl+e\",\n        \"when\": \"terminalFocus\"\n    }\n\n]\n
"},{"location":"Docker%20Apps/Minecraft/useful-plugins/","title":"Useful Plugins","text":"

WorldEdit

EssentialX

CoreProtect

ViaVersions - allow other similar version to join the server without conflict

Offline Mode/Mobile Bedrock

To allow offline play for PC version. Change server.properties and edit these lines

enforce-whitelist=false\nonline-mode=false\n

Refer to Minecraft Prep and Install to install offline client.

For bedrock compatibility, need the geyser plugin.

Geyser

To allows offline play for bedrock mobile version. Go to ./plugins/Geyser-Spigot/config.yml and change these lines. Do not install the plugin floodgate, if it\u2019s installed, removed the plugin. ViaVersions is also needed for mobile play.

auth-type: offline\nenable-proxy-connections: true\n

Now client can play without login to Xbox or Java.

WorldGuard

"},{"location":"Linux%20Server/debian-based-server-setup/","title":"Debian-Based Server Setup","text":"

Debian 12 with Desktop

Run update and upgrade distro first. Install NTP package is there are errors with that. Reboot

Setup powertop and powersaving features

sudo apt install powertop\npowertop --auto-tune\n

Powersave governor and at reboot. Remember to run the command again

@reboot echo \"powersave\" | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor >/dev/null 2>&1\n

Ensure these packages are installedi

powertop htop iotop fio curl gnupg wget ntfs-3g neofetch ca-certificates lsb-release hdparm hd-idle openssh-server at\n

HDD

lsblk and blkid to get the ntfs hard drive /dev name and the /dev/by-uuid/\u2026

Edit the fstab to mount the drive, same entry for nvme drive

UUID=CC34294F34293E38 /mnt/data ntfs-3g 0 0\n

If the mounted device is HDD array, need to spindown disk with hdparm

hdparm -B 120 /dev/sdb # set the APM level\nhdparm -S 241 /dev/sdb\n

For the -S spindown, 0-240 is multiple of 5s, 241-255 is multiple of 30 min. The above command set spindown every 30min.

If hdparm does not work, hd-idle can be used. Edit the file in /etc/defaults/hd-idle

-i 60 -a disk/by-uuid/xxx -l /var/log/hd-idle.log\n

Sudo without password, go to visudo and add the lines to the bottom, replace $USER with the actual username.

$USER ALL=(ALL) NOPASSWD: ALL\n

Edit shortcuts in bashrc

source .bashrc\n

OpenSSH with Keys

Generate the key using the terminal

ssh-keygen\n
  • give a location to put the key pair
  • this generate a public (.pub) and private key pair
ssh-copy-id -i key.pub username@server\n
  • key.pub is the public key that was generated

The key is ready to use for authorization.

Generate keys using PuTTY software

  1. Copy the red part and use nano to add it in the server ~/.ssh/authorized_keys
  2. Make sure permissions are correct
  3. bash mkdir -p ~/.ssh chmod 700 ~/.ssh chmod 600 ~/.ssh/authorized_keys nano ~/.ssh/authorized_keys
  4. Save private key as ppk file on the root ssh folder.
  5. If the client with private key is Linux machine, need to change the permission of the private key.

    chmod 600 private.key\n
  6. Convert the private key Conversion > Export OpenSSH Keys and save the file to a folder OpenSSH Keys

Setting Up SMB

Refer to Samba(SMB) Setup to setup SMB server.

Desktop Environment Setup

Firefox

The location of firefox profile is at /home/$USER/.mozilla/firefox/xxxxx.default

Make a tarball and copy it and extract it in destination.

In the profile folder, look for compatibility.ini, go to a random profile in the dest machine and copy the compatibility.ini settings to the one that is copied over. This ensure compatibility so that the new profile works without warning.

Check the profile.ini with the name and the location of the new profile folder, firefox should be the same as before.

[Profile0]\nName=karis\nIsRelative=1\nPath=ims58kbd.default-esr-1\n

Themes

To backup/restore settings of cinnamon

Icons

The icons are located at these locations.

/usr/share/icons\n~/.icons\n

Scripts

Copy the scripts and put it into ~/script for organization and copy the old crontab for executing these scripts.

"},{"location":"Linux%20Server/olivetin/","title":"OliveTin","text":"

OliveTin exposes a webpage with buttons that execute shell command (eg. docker, scripts) on the server and allow others for easy access. It should be used internally only.

Installation

Download the correct file from this site. https://github.com/OliveTin/OliveTin/releases OliveTin_linux_amd64.deb

Go to the directory and install the package.

sudo dpkg -i OliveTin\u2026\u200bdeb\nsudo systemctl enable --now OliveTin\n
"},{"location":"Linux%20Server/olivetin/#configuration","title":"Configuration","text":"

The configuration file is located at <em>/etc/OliveTin/config.yaml\u00a0</em>

Example Configuration

listenAddressSingleHTTPFrontend: 0.0.0.0:1378 # set the port of OliveTin to 1378\n\n# Choose from INFO (default), WARN and DEBUG\nlogLevel: \"INFO\"\n\n# Actions (buttons) to show up on the WebUI:\nactions:\n  # This will run a simple script that you create.\n\n- title: Update Music\n  shell: /home/karis/scripts/script\n  icon: '&#127925'\n

More possible configurations (many are not possible on Docker)

Execute a shell command with textbox input.

- title: Restart a Docker CT\n  icon: '<img src = \"icons/restart.png\" width=\"48px\" />'\n  shell: docker restart {{ container }}\n  arguments:\n    - name: container\n      type: ascii\n
  • use {{ }} and give a variable
  • under arguments type, assign a type for it, ascii only allows letters and numbers

Execute a shell command with choices

- title: Manage Docker Stack Services\n  icon: \"&#128736;\"\n  shell: docker-compose -f /home/karis/docker/bookstack/docker-compose.yml {{ action }}\n  arguments:\n    - name: action\n      choices:\n        - title: Start Stack\n          value: up -d\n\n\n        - title: Stop Stack\n          value: down\n

This example give choices to start or stop a docker stack of a docker-compose file. If a argument is given the parameter choices, it will be in dropdown mode.

"},{"location":"Linux%20Server/olivetin/#icons-customization","title":"Icons Customization","text":"

The icons need to be placed in a folder in /var/www/[icon-folder]/icon.png. To use the icons, offline image or web address, it should be in HTML format. The size of 48px is the default size of OliveTin icons. Other CSS options such as <em>style=\"background-color: white;\"</em> also works.

icon: '<img src = \"icons/minecraft.png\" size=\"48px\" />'\n

Icon with emoji, to use emoji, need to use the html code. https://symbl.cc/en/emoji/

For example, &#9786; \ud83d\ude0a.

icon: \"&#9786;\"\n
"},{"location":"Linux%20Server/olivetin/#icon-management","title":"Icon Management","text":"

The default icon folder is /var/www/olivetin/icons

The icon folder of all homelab icons is in ~/icons/homelab

"},{"location":"Linux%20Server/olivetin/#api","title":"API","text":"

Simple action button.

curl -X POST \"http://mediaserver:1378/api/StartAction\" -d '{\"actionName\": \"Update Music\"}'\n

Action with Arguments.

curl -X POST 'http://mediaserver:1378/api/StartAction' -d '{\"actionName\": \"Rename Movies\", \"arguments\": [{\"name\": \"path\", \"value\": \"value\"}]}'\n
"},{"location":"Linux%20Server/sambasmb-setup/","title":"Samba(SMB) Setup","text":""},{"location":"Linux%20Server/sambasmb-setup/#setting-up-smb-server-on-linux","title":"Setting up SMB Server on Linux","text":"

Install the samba tool on Linux.

sudo apt update\nsudo apt install samba -y\n

Edit the /etc/samba/smb.conf

[nvme_share]\n   comment = NVMe Share\n   path = /mnt/nvme/share\n   browseable = yes\n   read only = no\n

nvme_share is the name of the Samba path which will appear in SMB clients and its path is accessed by \\\\192.168.0.1\\nvme_share

path is the location where the files are stored

browseable and read only are flags that are needed to make sure read/write access on the SMB share

Lastly, add the user and password for the SMB share

sudo smbpasswd -a $USER # enter the password twice\n

In the case when Windows fail to write files in the samba share for odd reason. Go to Manage Credentials -> Windows Credentials -> Add a Windows Credential and fill the necessary address, username and password.

"}]} \ No newline at end of file +{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Home","text":""},{"location":"#recent-updates","title":"Recent Updates","text":"
  • 01 Docker Infrastructure
  • JDownloader
  • OliveTin
  • Samba(SMB) Setup
  • Dynamic DNS Updater Docker
  • Filebrowser
  • Mkdocs
  • Debian-Based Server Setup
  • Tunneling Basic Services (Jellyfin, Web) with Caddy and Tailscale
  • Useful Plugins
"},{"location":"mkdocs/","title":"Mkdocs","text":""},{"location":"mkdocs/#mkdocs-gotchas","title":"Mkdocs Gotchas","text":"
  • yaml highlighting is broken with mdx-breakless-lists
  • when using heading #, if there are no line breaks between headings, any lists that is after content of the second heading will not be rendered properly, even with mdx-breakless-lists
  • when referencing a subheading in another file, mkdocs uses [](file.md#heading-with-space) while obsidian uses [](file.md#heading%20with%20space)
  • Before switching from lists to normal content, a line break is needed, otherwise the text below will be rendered with a indent
  • mkdocs subheadings [](#subheadings) must be in lower case
"},{"location":"mkdocs/#admonitioncallouts","title":"Admonition/Callouts","text":"Mkdocs native callout

callout content mkdocs

Nested

Nesting

  • ??? is also valid syntax for mkdocs
  • ???+ makes the callout collapsible and opens by default, while ???- makes it closed by default
    !!! notes \"Title\"\n    content\n
    Obsidian callouts requires the plugin mkdocs-callouts
Obsidian Native Callout

Callout content mkdocs

Nested callout

callout

> [!notes]+/- Callout title\n> Callout content\n
  • obsidian callout syntax also follows the same +,- for collapsing, it is to be inserted after the brackets

Available callouts include notes, info, warning, danger, success, failure, example, abstract, tip, question, bug.

"},{"location":"mkdocs/#keys-caret-mark-tilde","title":"Keys, Caret, Mark, Tilde","text":"

Keys ++ctrl+alt+plus++ Ctrl+Alt++ mark highlighting tilde strikethrough

"},{"location":"mkdocs/#tabbed-content","title":"Tabbed Content","text":"Tab 1Tab 2

Tab 1 content mkdocs Second line here.

Tab 2 content

=== \"Tab Name\"\n    Tab content\n

  • not supported in obsidian
"},{"location":"mkdocs/#attr_list","title":"attr_list","text":"

Fancy Buttons mkdocs [button text](link.md){ .md-button } Tooltip I\u2019m a tooltip that you can hover or click. [tooltip](https://link \"hover text\") Annotation I\u2019m an annotation, but you need to click the plus icon (1) to show. (2)

  1. annotation 1
  2. annotation 2
    Annotation location 1 (1), location (2)\n{ .annotate }\n1. annotation text to be shown\n

Footnote Insert footnote like [^1] 1

  • for inserting footnote [^1]
  • [^1]: at the end to explain the footnote; not supported in obsidian
"},{"location":"mkdocs/#code-highlighting","title":"Code Highlighting","text":"
from python import python\npython.run(arg1=123, arg2=\"mystr\")[2]\n
#!/bin/bash\nvar=\"myvar\"\necho $var+3\n
# yaml highlighting has to be `yaml` not `yml` and it's broken\n---\nversion: \"2.1\"\nservices:\n  clarkson:\n    image: lscr.io/linuxserver/clarkson\n    container_name: clarkson\n    environment:\n\n      - PUID=1000\n      - PGID=1000\n    ports:\n      - 3000:3000\n    restart: unless-stopped\n
  1. explaining the footnote.\u00a0\u21a9

"},{"location":"Cloud%20VPS/basic-server-setup-caddy-docker-jdownloader/","title":"Basic Server Setup, Caddy, Docker, JDownloader","text":"

Creating the VM in oracle cloud.

  1. Go to instances, new instance.
  2. Select the Always Free image, ARM or x86. 1 core only, recommended 4GB RAM, should be exceed 6 GB.
  3. Choose Ubuntu image.
  4. Download the SSH key and name it accordingly.

Key Pair

Using PuttyGen.

  • Place the key in ./ssh/openssh_keys
  • Open PuttyGen, conversion -> import keys
  • Save the key files as ppk file in root folder of ./ssh

Putty

  • Grab the IP address in the cloud console
  • Give a name in saved sessions
  • Go to behavior, choose these options
  • Under Data, make sure Terminal-type string is xterm-256color
  • Under Terminal -> Features, check \u201cdisable application keypad mode\u201d to fix issues with nano
  • The private key needs to be load in Connection -> SSH -> Auth -> Credentials

To get the IP address of the VPS at any time

curl ifconfig.me\n

Basic Setup + Docker

  1. Installing Caddy web server (simple to use reverse proxy), lightweight, easy and no need for docker. (Nginx is also a good candidate for reverse proxy as the command is easy to memorize and does not require consulting documentation sites. However, the syntax for nginx is extremely complex compared to caddy and might not be easily memorized.

https://caddyserver.com/docs/install#debian-ubuntu-raspbian

sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https\ncurl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg\ncurl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list\nsudo apt update\nsudo apt install caddy net-tools\n# net-tools is good utility, optionally can install firewall-cmd or nginx\n# sudo apt install firewalld nginx\n

2. Install Docker

https://docs.docker.com/engine/install/ubuntu/

sudo apt-get update\nsudo apt-get install \\\n    ca-certificates \\\n    curl \\\n    gnupg \\\n    lsb-release\n\nsudo mkdir -p /etc/apt/keyrings\ncurl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg\n\necho \\\n  \"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \\\n  $(lsb_release -cs) stable\" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null\n\nsudo apt-get update\nsudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin docker-compose\n# code modified to install docker-compose, each space in paragraph indicates a separate step in their official blog\n
sudo groupadd docker \\\nsudo usermod -aG docker ubuntu\nnewgrp docker # activate docker group immediately\n

The machine needs to be rebooted from Oracle Cloud console to finish installation.

JDownloader

https://hub.docker.com/r/jlesage/jdownloader-2

docker run -d \\\n    --name=jdownloader-2 \\\n    -p 5800:5800 \\\n    -v $HOME/appdata/jdownloader-2:/config:rw \\\n    -v $HOME/Downloads:/output:rw \\\n    --restart unless-stopped \\\n    jlesage/jdownloader-2\n

If port forwarding configured properly, entering ipaddress:5800 should work. If not open ports manually.

sudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 5800 -j ACCEPT\nsudo netfilter-persistent save\n

Other Useful Ports

sudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 443 -j ACCEPT\nsudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 80 -j ACCEPT\nsudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 25565 -j ACCEPT\nsudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 19132 -j ACCEPT\nsudo iptables -I INPUT 6 -m state --state NEW -p udp --dport 25565 -j ACCEPT\nsudo iptables -I INPUT 6 -m state --state NEW -p udp --dport 19132 -j ACCEPT\nsudo iptables -I INPUT 6 -m state --state NEW -p udp --dport 51820 -j ACCEPT\nsudo netfilter-persistent save\n

Alternative (firewall-cmd)

sudo apt install firewalld\n

Firewalld is a CentOS package, it may be unstable and crash, but command easy to memorize.

sudo firewall-cmd --zone=public --add-port 19132/tcp --permanent\nsudo firewall-cmd --zone=public --add-port 19132/udp --permanent\nsudo firewall-cmd --zone=public --add-port 25565/tcp --permanent\nsudo firewall-cmd --zone=public --add-port 25565/udp --permanent\nsudo firewall-cmd --zone=public --add-port 80/tcp --permanent\nsudo firewall-cmd --zone=public --add-port 443/tcp --permanent\nsudo firewall-cmd --zone=public --add-port 5800/tcp --permanent\nsudo firewall-cmd --reload\n

Troubleshooting network

For firewall-cmd, use this command to check all open ports.

sudo firewall-cmd --list-all\n

Using netstat, or pipe it to grep

netstat -tln\n# | grep 8080 etc...\n

Configuring JDownloader

  • Go to the JDownloader WebUI
  • Go to Settings
  • Under general, change the max number of downloads (2) and DL per hoster (1) to minimize issues

  • Go to MyJDownloader and configure MyJDownloader account

  • Go to extension modules, install and enable \u201cfolderwatch\u201d

The configuration for JDownloader is complete and should appear and be functional in WebUI. Advanced JDownloader documentation will be covered in detailed in another section. It is recommended to close port 5800 after configuring to prevent others accessing.

Basic Caddy Syntax (if applicable)

If the server that is being setup or restored needs functional service like bookstack or uptime-kuma, reverse proxy is needed.

sudo nano /etc/caddy/Caddyfile\n
{\n    email weebly2x10@gmail.com\n}\n\nyour-uptime-kuma.yoursubdomain.duckdns.org {\n        reverse_proxy http://127.0.0.1:3001\n}\n\nwiki.yoursubdomain.duckdns.org {\n        reverse_proxy http://127.0.0.1:6975\n}\n

Advanced

Tunneling Jellyfin and other web services with tailscale and caddy

Minecraft Server tunneling via Nginx (tcp only)

Minecraft Tunneling

"},{"location":"Cloud%20VPS/jdownloader/","title":"JDownloader","text":"

After setting up JDownloader and it appears well in WebUI.

The section is useless now as UHDMV has shutdown and it\u2019s pointless to setup multiple automated JDownloader server on VPS.

"},{"location":"Cloud%20VPS/jdownloader/#settings-for-jdownloader","title":"Settings for JDownloader","text":"

Debloat settings https://rentry.org/jdownloader2 Advanced Settings GraphicalUserInterfaceSettings: Banner -> disable GraphicalUserlnterfaceSettings: Premium Alert Task Column - > disable GraphicalUserInterfaceSeftings: Premium Alert Speed Column -> disable GraphicalUserInterfaceSettings: Premium Alert ETA Column -> disable GraphicalUsserInterfaceSeftings: Special Deal Oboom Dialog Visible On Startup -> disable GraphicalUsserInterfaceSeftings: Special Deals\u00a0-> disable GraphicalUsserInterfaceSeftings: Donate Button State\u00a0-> Hidden (automode)

"},{"location":"Cloud%20VPS/jdownloader/#theming","title":"Theming","text":"

GraphicalUserInterfaceSettings: Look And Feel Theme - > BLACK_EYE For Colors LAFSettings: Color For

  • Panel background and header background and alternate row background- #ff222222
  • Selected Rows Background - #ff666666
  • Package Row Background - #ff333333
  • Mouse Over Row Background - #ff666666
  • Panel Header Foreground, Tooltip Foreground, Selected Rows Foreground, Package Row Foreground, Mouse Over Row Foreground, Alternate Row Foreground, Account Temp Error Row Foreground, Account Error Row Foreground- #ffffffff
    • basically, change all the black values to white when searching for color fore, change everything except blue colors and error color
  • Enabled Text Color, Speed Meter Text, Speed Meter Average Text, Config Panel Description Text, Config Header Text Color - #ffffffff
  • Disabled Text Color - #ff666666
    • basically, when searching for color text, change all to white except for disabled text
"},{"location":"Cloud%20VPS/tunneling-basic-services-jellyfin-web-with-caddy-and-tailscale/","title":"Tunneling Basic Services (Jellyfin, Web) with Caddy and Tailscale","text":"

This procedure is not reproducible yet. Rigorous testing is still required before being documented. Here are the known procedures.

The purpose is to tunnel normal web or network intensive traffic such as Jellyfin when faced with CG-NAT or similar situations (in this case locked down dorm internet), also configure hardware transcoding (in this case NVENC, but Intel QSV for future) to mitigate limitations with Canadian ISP(s).

Jellyfin Install

https://jellyfin.org/downloads/server

Download and run the server installer.

Jellyfin Server Configuration

Tailscale (Windows Client)

https://tailscale.com/download/windows

Download, install and login.

Tailscale (Linux Server)

curl -fsSL https://tailscale.com/install.sh | sh\n
sudo tailscale up\n

All the tailscale management is done in the WebUI.

The Windows client is given a tailscale network IP address in 100 range. Check if Windows client is pingable on server.

ping 100.x.y.z\n

Check if Jellyfin is running and tunneled properly on Oracle cloud. It should get a webpage html rather than unable to resolve host etc.

curl http://100.x.y.z:8096\n

Reverse Proxy basic-server-setup-caddy-docker-jdownloader

Caddy installation and syntax is can be found on this page. Replace 127.0.0.1 with the tailscale IP address.

{\n    email weebly2x10@gmail.com\n}\n\nmovies.yoursubdomain.duckdns.org {\n        reverse_proxy http://100.x.y.z\n}\n

It is possible to set use the root domain (yoursub.duckdns.org) or a subfolder domain (movies.yousub.duckdns.org) for Jellyfin. After configuring the Caddyfile.

sudo systemctl reload caddy\n

Use netstat to check port 80, 443 is being listened. Make sure to port forward Oracle VPS.

Other Services

Follow the same syntax as the caddy file provided, if the root domain is used, then a subdomain must be used for other services.

Results

Inconclusive yet, more testing required.

"},{"location":"Cloud%20VPS/tunneling-minecraft-server-tcp-only-with-nginx/","title":"Tunneling Minecraft Server (tcp only) with Nginx","text":"

Procedure not reproducible yet, will be documented later.

"},{"location":"Computer%20Stuff/demucs-nvidia/","title":"Demucs Nvidia","text":"

Demucs is an music separation tool that has potential for a karaoke setup.

https://github.com/facebookresearch/demucs

https://www.youtube.com/watch?v=9QnFMKWEFcI&t=585s

https://docs.google.com/document/d/1XMmLrz-Tct1Hdb_PatcwEeBrV9Wrt15wHB1xhkB2oiY/edit

Installation on PC with Nvidia

  1. Firstly install Anaconda. Download Anaconda for Windows https://www.anaconda.com/products/distribution
  2. Install PyTorch. https://pytorch.org/get-started/locally/. Select the correct version of pytorch.
  3. Install ffmpeg. [https://www.gyan.dev/ffmpeg/builds/]](assets/gallery/2022-12/TwJimage.png)

Demucs

After installing the prerequesties.

Open \u201cAnaconda terminal\u201d and type

python.exe -m pip install -U demucs\n
pip install PySoundFile \n

Running Demucs

demucs \"C:\\path\\to\\music\\file.mp3\"\n

This will run demucs with CUDA GPU acceleration, make sure to put the path in double quote. The extracted file will be found in the directory where you run the command eg. the default Anaconda prompt starts in ~/separated

"},{"location":"Docker%20Apps/01%20docker-infra/","title":"01 Docker Infrastructure","text":""},{"location":"Docker%20Apps/01%20docker-infra/#filesystem","title":"Filesystem","text":""},{"location":"Docker%20Apps/01%20docker-infra/#compose","title":"Compose","text":"

All docker-compose.yml files are stored in ~/docker folder, which then by default is under the network docker_default.

  • by default for newly created apps, a new folder is created and docker-compose.yml is created for that app for testing
    • once app testing is complete, the compose file can be moved docker root folder if appropriate or remain
  • some apps can be grouped together and these compose files are in the root docker folder such as media.yml, network.yml, the grouping allows multiple services to be managed by a single compose. For grouping, some of the property can include
    • the apps share common properties such as arrs apps
    • it is preferable for apps to live in same network, eg. teslamate
    • a large app requiring multiple containers eg. frontend, mysql etc..
    • apps share similar/same category, such as qBittorrent and nzbget can be put together in downloader.yml even though they do not have common properties or require same networking
"},{"location":"Docker%20Apps/01%20docker-infra/#storage","title":"Storage","text":"

The storage used for all containers are bind mount.

  • application configs are stored in ~/docker/[app]
    • if an app has multiple components needing persistence (eg. app with database, helpers), a folder will be created as such ~/docker/[app]/postgres etc.
  • apps that also store non-config data (such as music, documents etc.) and not using a lot of space can bind mount /mnt/nvme/share (a directory on local or another SSD) for fast data access and without spinning up HDD
  • exceptions are home assistant or its related home automation containers and these are stored at /srv/homeassistant
"},{"location":"Docker%20Apps/01%20docker-infra/#backup","title":"Backup","text":"

The entire docker root folder is copied to a NFS share on another computer. With exception of minecraft and home assistant which a specialized method is used.

"},{"location":"Docker%20Apps/01%20docker-infra/#network","title":"Network","text":"

With docker-compose, a new network is created with the name of folder the compose is located, while it\u2019s possible to change network, it is not straightforward, therefore, there is no points in manually defining networks unless required.

Public 172.80.0.0/16 - bridge network for public facing applications with reverse proxy, this way when configuring Nginx Proxy Manager, all it need is to enter container_name:80 rather than IP address.

  • Nginx Proxy Manager - 172.80.44.3
  • Other containers will use docker DHCP to get address
  • Containers that need to public facing can attach to this network Media 172.96.0.0/16 - bridge network for arrs, downloader and management applications for easy interconnection when configuring Minecraft 172.255.255.0/24 - bridge network for Minecraft related networks
  • Minecraft server (mcserver) - 172.255.255.65
"},{"location":"Docker%20Apps/01%20docker-infra/#categories","title":"Categories","text":"

Media Apps - apps related to media acquisition, curation and other functions services for Jellyfin Networking - reverse proxy, DNS, VPN and related services Home Automation - home assistant and its associated functions VNC - containers based on jlesage-vnc-apps or Linuxserver Kasm images, usually desktop apps run in a browser via noVNC Management - tools for managing docker containers or entire server Games - game servers and associated tools Filesharing - apps that share files to other clients Documentation - notes and operation procedures for server infrastructure Authentication - services that handle single sign-on (SSO) with users

"},{"location":"Docker%20Apps/bookstack/","title":"Bookstack","text":""},{"location":"Docker%20Apps/bookstack/#installation","title":"Installation","text":"

Change port to 6975

Add in docker-compose: restart: unless-stopped

$docker directory = /home/docker .... etc

Docker-Compose file reference

https://github.com/solidnerd/docker-bookstack/blob/master/docker-compose.yml

version: '2'\nservices:\n  mysql:\n    image: mysql:8.0\n    environment:\n\n    - MYSQL_ROOT_PASSWORD=secret\n    - MYSQL_DATABASE=bookstack\n    - MYSQL_USER=bookstack\n    - MYSQL_PASSWORD=secret\n    volumes:\n    - mysql-data:/var/lib/mysql\n    restart: unless-stopped\n\n  bookstack:\n    image: solidnerd/bookstack:22.10.2\n    depends_on:\n\n    - mysql\n    environment:\n    - DB_HOST=mysql:3306\n    - DB_DATABASE=bookstack\n    - DB_USERNAME=bookstack\n    - DB_PASSWORD=secret\n    #set the APP_ to the URL of bookstack without without a trailing slash APP_URL=https://example.com\n    - APP_URL=http://xxx.xxxmydomainxxx.duckdns.org\n    volumes:\n    - $docker/public-uploads:/var/www/bookstack/public/uploads\n    - $docker/storage-uploads:/var/www/bookstack/storage/uploads\n    ports:\n    - \"6975:8080\"\n    restart: unless-stopped\n

Notice: The default password for bookstack is

admin@admin.com

password

Permissions: remember the set write permission on public-uploads folder so users can upload photos.

"},{"location":"Docker%20Apps/bookstack/#backup-and-restore","title":"Backup and Restore","text":"

Files Backup:

tar -czvf bookstack-files-backup.tar.gz public-uploads storage-uploads\n

Restore:

tar -xvzf bookstack-files-backup.tar.gz\n

Database backup:

sudo docker exec bookstack_mysql_1 /usr/bin/mysqldump -u root --password=secret bookstack > ./bookstack/bookstack_db.sql\n

Restore:

sudo docker exec -i bookstack_mysql_1 mysql -u root --password=secret bookstack < /$docker/bookstack/bookstack_db.sql\n
  • bookstack_mysql1 is the container name
  • password is secret or the database password
"},{"location":"Docker%20Apps/bookstack/#reverse-proxy","title":"Reverse Proxy","text":"

Use subdomain in proxy manager.

Backing Up and Restoring with LinuxServer.io container

Due to limits or Oracle Cloud free tier. The only arm image is from linuxserver io container, and it is different than solidnerd image.

Docker-Compose file

version: \"2\"\nservices:\n  bookstack:\n    image: lscr.io/linuxserver/bookstack\n    container_name: bookstack\n    environment:\n\n      - PUID=1001\n      - PGID=1001\n      - APP_URL=https://wiki.xxx.duckdns.org\n      - DB_HOST=bookstack_db\n      - DB_USER=bookstack\n      - DB_PASS=secret\n      - DB_DATABASE=bookstackapp\n    volumes:\n      - /home/ubuntu/bookstack:/config\n    ports:\n      - 6975:80\n    restart: unless-stopped\n    depends_on:\n      - bookstack_db\n\n  bookstack_db:\n    image: lscr.io/linuxserver/mariadb\n    container_name: bookstack_db\n    environment:\n\n      - PUID=1001\n      - PGID=1001\n      - MYSQL_ROOT_PASSWORD=secret\n      - TZ=Europe/London\n      - MYSQL_DATABASE=bookstackapp\n      - MYSQL_USER=bookstack\n      - MYSQL_PASSWORD=secret\n    volumes:\n      - /home/ubuntu/bookstack:/config\n    restart: unless-stopped\n

Notice: In Oracle cloud free tier, the default ubuntu user is 1001, not 1000. For database name, it it bookstackapp, keep in mind when executing restore command. The folder structure is also different. In the solidnerd container, the images are stored at /public-uploads while in LSIO container it is stored at /www/uploads

"},{"location":"Docker%20Apps/bookstack/#backing-up-from-home-pc","title":"Backing Up (from home PC)","text":"

Images

cd into /public-uploads and make a tar archive

tar -czvf images.tar.gz images\n

Backup the database

sudo docker exec bookstack_mysql_1 /usr/bin/mysqldump -u root --password=secret bookstack > ./bookstack_db.sql\n

Transfer to Oracle Cloud Server

scp -i oracle-arm-2.key images.tar.gz bookstack_db.sql ubuntu@$IPADDR:/home/ubuntu/bookstack/www/uploads\n

Take in consideration the location where LSIO image stores the images.

"},{"location":"Docker%20Apps/bookstack/#restore-into-oracle-cloud","title":"Restore (into Oracle Cloud)","text":"

Images (/home/ubuntu/bookstack/www/uploads)

tar -xvzf images.tar.gz\n

Database

The image url in the database still refers to old server url, it needs to be changed. The following command replace the subdomain in the sq1 dump.

sed -i 's/wiki.$home.duckdns.org/wiki.$oracle.duckdns.org/g' bookstack_db.sql\n

Restore the database.

sudo docker exec -i bookstack_db mysql -u root --password=secret bookstackapp < /home/ubuntu/bookstack/www/uploads/bookstack_db.sql\n
"},{"location":"Docker%20Apps/bookstack/#crontab","title":"Crontab","text":"

On Home PC

0 23 * * 2,5 /home/karis/bookstack.sh\n
#!/bin/bash\n\ncd ~/docker/bookstack/public-uploads #location of bookstack public uploads\ntar -czvf images.tar.gz images\nsudo docker exec bookstack_mysql_1 /usr/bin/mysqldump -u root --password=secret bookstack > ./bookstack_db.sql\nscp -i oracle-arm-2.key images.tar.gz bookstack_db.sql ubuntu@$ORACLEIP:/home/ubuntu/bookstack/www/uploads\n

Make sure to copy the oracle-arm-2.key to the appropriate location (~/docker/bookstack/public-uploads)

Also make sure the permission of oracle-arm-2.key is in correct permission (600). Especially changing the permission of public-uploads folder to allow write access.

Do a backup sequence in crontab at 11pm every Tuesday and Friday.

Oracle Cloud Server

0 8 * * 3,6 /home/ubuntu/bookstack.sh\n
#!/bin/bash\n\ncd ~/bookstack/www/uploads #directory where bookstack files scp from home are located\ntar -xvzf images.tar.gz\nsed -i 's/wiki.$homeip.duckdns.org/wiki.$oracle.duckdns.org/g' bookstack_db.sql\nsudo docker exec -i bookstack_db mysql -u root --password=secret bookstackapp < /home/ubuntu/bookstack/www/uploads/bookstack_db.sql\n

Restore the sequence after backup, every Wednesday and Saturday at 8am (need to consider the TZ between Vancouver, Edmonton and Toronto, or any the time zone of the remote server)

"},{"location":"Docker%20Apps/ddns-update/","title":"Dynamic DNS Updater Docker","text":"

Official Image: https://hub.docker.com/r/linuxserver/duckdns Custom Github Page: https://github.com/vttc08/docker-duckdns-dynu

This is a docker container that automatically updates the public IPv4 address of the server every 5 minutes to dynamic DNS services Dynu and DuckDNS. It is the fork of Linuxserver DuckDNS container.

"},{"location":"Docker%20Apps/ddns-update/#docker-compose","title":"Docker Compose","text":"
  services:\n      duckdns:\n        image: vttc08/docker-duckdns-dynu:latest\n        container_name: duckdns\n        env_file: ddns.env\n        environment:\n\n          - TZ=America/Vancouver\n          - PUID=1000\n          - PGID=1001\n        restart: unless-stopped\n

These need to be filled in the ddns.env

DYNU_HOST= # full name of dynu domains\nDYNU_PASS= # md5 hashed dynu login pass\nSUBDOMAINS= # DuckDNS domains without the duckdns.org part\nTOKEN= # DuckDNS token \n

  • token will be visible in DuckDNS dashboard
  • Dynu pass is the same as login; alternatively, it is possible to create a dedicated password just for IP update MD5 generator
    echo -n \"password\" | md5sum\n
  • when setting the IP to 10.0.0.0 in Dynu update API, dynu will automatically update the IP address to the IP address making that request
"},{"location":"Docker%20Apps/ddns-update/#other-usage","title":"Other Usage","text":"

docker restart duckdns will manually run IP update docker exec -it duckdns /app/debug.sh or other scripts, debug script will print out IP address of subdomains resolved by Cloudflare

"},{"location":"Docker%20Apps/epic-games-free-games/","title":"Epic Games Free Games","text":"

Buy Free Games from Epic Games

https://hub.docker.com/r/charlocharlie/epicgames-freegames

Config

NEED TO CHANGE

Email: email address

Password: password

Webhook URL: make a discord channel and click settings. Go to integrations, then webhook, copy webhook URL.

mentioned Users: right click your profile, and click Copy ID

TOTP

  1. Go here to login. https://www.epicgames.com/account/password Login with Epic Games account.
  2. Click \u201cenable authenticator app.\u201d
  3. In the section labeled \u201cmanual entry key,\u201d copy the key.
  4. Use your authenticator app to add scan the QR code.
  5. Activate 2FA by completing the form and clicking activate.
  6. Once 2FA is enabled, use the key you copied as the value for the TOTP parameter.

Docker

docker run -d -v /home/karis/docker/epicgames:/usr/app/config:rw -p 3000:3000 -m 2g --name epicgames --restart unless-stopped charlocharlie/epicgames-freegames:latest\n

Change the name of the container to a friendly name. Restart unless stopped so it restart automatically.

Copy and Paste

The default json configuration is located at /home/karis/docker/epicgames or $HOME/docker/epicgames.

Fix Login Issue Using Cookies

https://store.epicgames.com/en-US/

  1. Visit this site and make sure it\u2019s logged in.
  2. Install this extension EditThisCookie https://chrome.google.com/webstore/detail/editthiscookie/fngmhnnpilhplaeedifhccceomclgfbg/related
  3. Open the extension and change the url to epicgames.com/id as in screenshot below
  4. Export the cookie

  1. Go to $HOME/docker/epicgames and create a new file email@gmail.com-cookies.json
  2. If the json file is already there, truncate it with \u2013size 0
  3. Paste the cookie value to the json file
  4. Restart container.

Update

docker pull charlocharlie/epicgames-freegames:latest\ndocker rm -f epicgames\ndocker images | grep epicgames\n# use docker rmi to remote the corresponding image \n# re run the epicgames docker run command\n
"},{"location":"Docker%20Apps/filebrowser/","title":"Filebrowser","text":"

Filebrowser app on a webbrowser, port 4455. free-games-claimer

Docker-compose deployment

version: '3.9'\nservices:\n    filebrowser:\n        container_name: filebrowser\n        image: filebrowser/filebrowser\n        ports:\n\n            - '4455:80'\n        user: 1000:1000\n        restart: unless-stopped\n        volumes:\n            - '~/docker/filebrowser/.filebrowser.json:/.filebrowser.json'\n            - '~/docker/filebrowser/filebrowser.db:/database.db'\n            - '~/docker/filebrowser/branding:/branding'\n            - '~/docker:/srv/docker'\n            - '/mnt/data:/srv/data'\n            - '/mnt/nvme/share:/srv/nvme-share'\n

The first 3 bind mount are for configuration of filebrowser, eg. config, database and branding files. On first deployment, need to create an empty database.db file. The remaining bind mount are for the folders that need to be accessed, the folders should be bound under /srv.

This is the content of .filebrowser.json

{\n    \"port\": 80,\n    \"baseURL\": \"\",\n    \"address\": \"\",\n    \"log\": \"stdout\",\n    \"database\": \"/database.db\",\n    \"root\": \"/srv\"\n  }\n
"},{"location":"Docker%20Apps/filebrowser/#usershare","title":"User/Share","text":"

The user and share management in filebrowser is simple. The shares have a expiring time, and can optionally have a password. The recipient can view and download files in the share but cannot upload.

To create a new user, it\u2019s under settings -> User Management, and add a user and password accordingly, and give appropriate permission. The scope is where the root folder where the user have access to, since the docker data folder is bound at /srv/docker and /srv is defined as root folder in config, the folder name to put in scopes would be /docker. Only one scope is allowed.

It is also possible to add rules to prevent user access of files within a scope. Under rules, enter the path that is relative to the scope, for example /docker/minecraft/config would be /config

"},{"location":"Docker%20Apps/filebrowser/#personalization","title":"Personalization","text":"

Enable dark theme - Setting -> Global Settings -> Branding

  • also change the branding directory path to /branding which is bind mount in docker

Under the branding folder, create a file custom.csswhich is used for css customization. Then create a folder img and place logo.svg in it for custom icon. The icon is the same as egow entertainment and stored in OliveTin icon PSD file. Under the folder img, create a folder icons and use favicon generator site to create an icon archive and put all the content of that archive in the icons folder, the result should look like this.

Reverse Proxy/Homepage

Reverse proxy is normal procedure using NPM. To add bookmark to a file location, use browser/homepages bookmark function.

"},{"location":"Docker%20Apps/free-games-claimer/","title":"Free Games Claimer","text":"

https://github.com/vogler/free-games-claimer

This is the Github repo for the new and advanced free games claimer. This is implemented after Epicgames FreeGames keeps failing.

"},{"location":"Docker%20Apps/free-games-claimer/#configuration","title":"Configuration","text":"

Using Docker-Compose

In the folder structure

server: ~/docker/fgc$\ndocker-compose.yml\nfgc.env\n

fgc.env is the environment file for all the password/keys to login to different game services, fill it in manually or use a backup.

EG_OTPKEY=\nEG_EMAIL=\nEG_PASSWORD=\nNOTIFY=discord://123456/ABCD\nPG_EMAIL=\nPG_PASSWORD=\nGOG_EMAIL=\nGOG_PASSWORD=\nTIMEOUT=300\n

NOTIFY=discord://123456/ABCD if the webhook looks like this https://discord.com/api/webhooks/123456/ABCD

TIMEOUT=300 sets the timeout to 300s before the container skip and error out due to EpicGames captcha problems. However, the impact on prime gaming and GOG are not tested.

docker-compose.yml

services:\n  free-games-claimer:\n    container_name: FGC # is printed in front of every output line\n    image: ghcr.io/vogler/free-games-claimer # otherwise image name will be free-games-claimer-free-games-claimer\n    build: .\n    ports:\n\n      - \"5990:5900\" # VNC server\n      - \"5890:6080\" # noVNC (browser-based VNC client)\n    volumes:\n      - ~/docker/fgc:/fgc/data\n      - ~/docker/fgc/epic-games.js:/fgc/epic-games.js\n      - ~/docker/fgc/prime-gaming.js:/fgc/prime-gaming.js\n      - ~/docker/fgc/gog.js:/fgc/gog.js\n    command: bash -c \"node epic-games; node prime-gaming; node gog; echo sleeping; sleep 1d\"\n    env_file:\n      - fgc.env\n    restart: unless-stopped\n

This docker-compose file use the environment file fgc.env as indicated above and runs once every day. It also contains VNC server/web based client.

"},{"location":"Docker%20Apps/free-games-claimer/#missing-captcha-session","title":"Missing Captcha Session","text":"

This should no longer be needed. Edit the line to epicgames.js code and replace with the following message. When the captcha is missed, it will send a notification for manual claiming.

wait notify(`epic-games: got captcha challenge right before claim. Use VNC to solve it manually. Game link: \\n ${url}`)\n

EpicGames require a captcha to claim free games. If the 5 minute timeout window for EpicGames is missed, it is no longer possible to claim the games unless waiting for the next day, which due to the nature of discord notifications, there is a slim to none chance of catching the captcha at next day. To continuing claiming after acknowledging the missed session, use portainer, ConnectBot Android to temporarily restart the container to restore VNC session.

In order to restore the default time of claiming the games. Eg. waking up on Thurs or Fri and a predictable time and claim games, use the linux at command. Need to install at using apt.

at 9:20\n> docker restart FGC\n> <EOT>\n

This will run the command at 9:20 AM the next day. Ctrl-D to exit at prompt and verify the time is correct.

"},{"location":"Docker%20Apps/jlesage-vnc-apps/","title":"jlesage VNC Apps","text":"

VNC apps consists of desktop applications that have the GUI in a web browser, mostly from the creator jlesage.

"},{"location":"Docker%20Apps/jlesage-vnc-apps/#environments","title":"Environments","text":"

At least for apps from jlesage, it supports an environment variable. Create an environment file called vnc.env

The environment file can be reference in many docker images from jlesage using docker-compose. The current environment variable specify U/GID, time zone and make every app dark mode. It is also possible to set VNC passwords. This is the full list of environment variables.

USER_ID=1000\nGROUP_ID=1000\nTZ=America/Vancouver\nDARK_MODE=1\n

The jlesage apps have 2 ports, port 5800 for viewing the VNC app on a web browser on desktop; port 5900 is for VNC protocol that can be used in dedicated VNC viewer or mobile viewing.

"},{"location":"Docker%20Apps/jlesage-vnc-apps/#general-bind-mounts","title":"General Bind Mounts","text":"

The appdata bind mount is located in the ~/docker/vnc, as seen from the yml example, the vnc environment file vnc.env is placed in the appdata folder. For application requiring access to movie storage, the bind mount is on the corresponding hard drive or pool. As for applications requiring access to storage but not large media, it\u2019s best to put the files on a SSD.

This is an example of VNC container of MKVToolNix. The vnc.yml file is backed up elsewhere.

    mkvtoolnix:\n        image: jlesage/mkvtoolnix\n        env_file:\n\n            - ./vnc/vnc.env\n        volumes:\n            - '/mnt/data/nzbget:/storage:rw'\n            - '~/docker/vnc/mkvtoolnix:/config:rw'\n        ports:\n            - '5820:5800'\n            - '5920:5900'\n        container_name: mkvtoolnix\n
"},{"location":"Docker%20Apps/jlesage-vnc-apps/#ports","title":"Ports","text":"

The application port start from 5800/5900 for its corresponding access and add 10 for each application.

JDownloader: 5800

Firefox: 5810

MKVToolNix: 5820

MKVCleaver: 5840

MegaBasterd: 5860 (no VNC viewer 59xx port)

There are also some application specific setup. For applications accessing hard drive or intensive apps, it is best to stop when not used. Lazytainer and ContainerNursery and possibly using DNS server can help automate this process.

JDownloader

JDownloader Setup

"},{"location":"Docker%20Apps/minecraft/","title":"Minecraft","text":"

This chapter is about Minecraft setup in Docker server.

"},{"location":"Docker%20Apps/minecraft/#minecraft-prep-and-install","title":"Minecraft Prep and Install","text":"

Client Setup (Java + Online)

  1. Download Java
  2. Download OptiFine the latest version.
  3. On the official Minecraft client, go add a new installation and match the version with OptiFine.
  4. Download and try the official version, then install OptiFine with Java.
  5. Under Settings -> Keep the Launcher open while games are running

Client Setup (Java + Offline)

  1. Use the client PolyMC to enable offline play.
  2. Go to the right corner, manage accounts and create an offline account.
  3. Click on add an instance and follow the guide.
  4. To install OptiFine, need the official launcher first, then download OptiFine
  5. Extract OptiFine, the extracted file should be ending in _MOD.jar
  6. Open the jar file in WinRAR, then move the files from notch folder into the base folder. Save the jar archive.
  7. Go to PolyMC, right click on the instance, click Edit -> Versions -> Add to minecraft.jar and select the modified OptiFine.

Docker Server Setup

Docker-compose for minecraft server

version: \"3.9\"\nservices:\n  minecraft:\n    image: marctv/minecraft-papermc-server:latest\n    restart: unless-stopped\n    container_name: mcserver\n    environment:\n\n      - MEMORYSIZE=4G\n      - PAPERMC_FLAGS=\"\"\n      - PUID=1000\n      - PGID=1000\n    volumes:\n      - ~/docker/minecraft:/data:rw\n    ports:\n      - 25565:25565\n      - 19132:19132\n    stdin_open: true\n    tty: true\n

This downloads the latest version of Minecraft, to use another PaperMC version, need to build the image from scratch.

Warning: PaperMC cannot be downgraded, only newerversion of PaperMC can be installed after first run.

git clone https://github.com/mtoensing/Docker-Minecraft-PaperMC-Server\n# go edit the \"ARG version=1.xx.x\" to the correct version\ndocker build -t marctv/mcserver:1.xx.x\n

Folders and Plugins

Plugins are located in folder ./plugins some plugins have .yml files. To update or download plugins, use scp, wget on the server or VSCode.

The world folder consists of the save data. It is separated into world, nether, the_end.

Before starting the server, the eula.txt must have eula=true.

bukkit and spigot.yml in the root folder are configuration files for PaperMC.

Rcon Commands

To access the rcon-cli, use docker attach mcserver, to exit, use Ctrl-P and Q, if using VSCode may need to edit keyboard shortcut.

Editing VSCode Shortcut

Press Ctrl-Shift-P and search for keyboard shortcut json.

[\n    {\n        \"key\": \"ctrl+p\",\n        \"command\": \"ctrl+p\",\n        \"when\": \"terminalFocus\"\n    },\n\n    {\n        \"key\": \"ctrl+q\",\n        \"command\": \"ctrl+q\",\n        \"when\": \"terminalFocus\"\n    },\n\n    {\n        \"key\": \"ctrl+e\",\n        \"command\": \"ctrl+e\",\n        \"when\": \"terminalFocus\"\n    }\n\n]\n
"},{"location":"Docker%20Apps/minecraft/#useful-plugins","title":"Useful Plugins","text":"

WorldEdit

EssentialX

CoreProtect

ViaVersions - allow other similar version to join the server without conflict

Offline Mode/Mobile Bedrock

To allow offline play for PC version. Change server.properties and edit these lines

enforce-whitelist=false\nonline-mode=false\n

Refer to Minecraft Prep and Install to install offline client.

For bedrock compatibility, need the geyser plugin.

Geyser

To allows offline play for bedrock mobile version. Go to ./plugins/Geyser-Spigot/config.yml and change these lines. Do not install the plugin floodgate, if it\u2019s installed, removed the plugin. ViaVersions is also needed for mobile play.

auth-type: offline\nenable-proxy-connections: true\n

Now client can play without login to Xbox or Java.

WorldGuard

"},{"location":"Docker%20Apps/tesla-homepage/","title":"Tesla Homepage","text":"

This is a homepage that allows Tesla browser to enter full screen mode.

Docker-compose

services:\n  homepage-for-tesla:\n    image: jessewebdotcom/homepage-for-tesla:latest\n    container_name: homepage-for-tesla\n    environment:\n\n      - DEFAULT_THEME=13\n    volumes:\n      - ~/docker/tesla/public/bookmarks.json:/app/public/bookmarks.json\n      - ~/docker/tesla/public/images:/app/public/images\n    ports:\n      - \"3000:3000\"\n
"},{"location":"Docker%20Apps/Media%20Apps/audiobookshelf/","title":"Audiobookshelf","text":"

Audiobooks and podcasts.

Docker-compose, place it in the media apps compose media.yml

version: \"3.7\"\nservices:\n  audiobookshelf:\n    image: ghcr.io/advplyr/audiobookshelf:latest\n    environment:\n\n      - AUDIOBOOKSHELF_UID=99\n      - AUDIOBOOKSHELF_GID=100\n    ports:\n      - 13378:80\n    volumes:\n      - /mnt/m/Audios/audiobooks:/audiobooks # hard drive mount\n      - /mnt/m/Audios/podcasts:/podcasts # hard drive mount\n      - $HOME/audiobookshelf/config:/config\n      - $HOME/audiobookshelf/metadata:/metadata\n    restart: unless-stopped\n
"},{"location":"Docker%20Apps/Media%20Apps/audiobookshelf/#usage","title":"Usage","text":"

To add a library, go to settings, libraries and add the path as mounted in docker.

Go to Users, change the root password and create a new user. Note, the user cannot scan library, only the root can do that.

"},{"location":"Docker%20Apps/Media%20Apps/audiobookshelf/#adding-media","title":"Adding Media","text":"

Make sure the contents are in a separate folder. Follow naming like this. A cover image can also be created. The best bitrate should be under 128 kbps for smooth playback.

/audiobooks\n--- ./Author - Book\n---  --- ./cover.jpg\n---  --- ./book - 001 or book - chapter 1\n---  --- ./book - 002\n---  --- ./book - 003\n

In the WebUI, make sure logged in as root. Go to settings, library and scan. It will scan the newly added media.

If the media does not match or not have an image, go click the edit icon, go to Match, the best result is usually Audible.ca.

If the chapter does not match, chapters can be edited manually. Go to Chapter and Lookup.

"},{"location":"Docker%20Apps/Media%20Apps/audiobookshelf/#mobile-app","title":"Mobile App","text":"

https://play.google.com/store/apps/details?id=com.audiobookshelf.app

Mobile app also has download functionality, however, the directory cannot be changed, the default for download is /Internal Storage/Download/{Podcast or Audiobook}

The statistic of minutes listened is the actual minutes listened, not the minutes of audiobook progress listened (eg. playing at faster speed).

"},{"location":"Docker%20Apps/Media%20Apps/audiobookshelf/#scripting-windows","title":"Scripting (Windows)","text":"

ffmpeg detect audio silence (for splitting a large audio file into multiple chapters)

ffmpeg -i input.mp3 -af silencedetect=n=-50dB:d=1.5 -f null -\n
ffmpeg -i input.mp3 -af silencedetect=n=-50dB:d=1.5 -f null -loglevel debug 2>&1 - | findstr \"silence_duration\" | find /c /v \"\"\n

This will find silence parts below -50dB and duration threshold of 1.5s.

The second code (windows cmd only) for linux use grep -c, finds how many silence parts can be detected, this should correlate to number of chapters.

Once the optimal duration is set, use split.py.

ffmpeg that remove silence from audio

ffmpeg -i input.mp4 -af silenceremove=stop_periods=-1:stop_duration=4:stop_threshold=-50dB -b:a 96k output.mp3\n
  • stop_duration (threshold duration for removing silence part)
  • stop_periods = -1 (search for the entire audio track)

Use edge_reader.py to utilize Edge AI reader to read the audiobook if only the pdf book is provided.

After reading, put all the recorded files and pdf in the project folder and run processing.py twice.

"},{"location":"Docker%20Apps/Media%20Apps/rich-media/","title":"Rich Media","text":"

Hello Everyone

This is a demo consisting of medias.

Some Code

docker-compose up -d\n
import os\nimport time\n\nprint(\"hello world\")\nif a=b:\n  print(a)\nelif b=c:\n  try:\n    print(c)\n  except:\n    print(c+a)\nelse:\n  print(\"what is the meaning of life\")\n

More sample media

Portainer is a software for managing docker containers.

"},{"location":"Docker%20Apps/Minecraft/minecraft-prep-and-install/","title":"Minecraft Prep and Install","text":"

Client Setup (Java + Online)

  1. Download Java
  2. Download OptiFine the latest version.
  3. On the official Minecraft client, go add a new installation and match the version with OptiFine.
  4. Download and try the official version, then install OptiFine with Java.
  5. Under Settings -> Keep the Launcher open while games are running

Client Setup (Java + Offline)

  1. Use the client PolyMC to enable offline play.
  2. Go to the right corner, manage accounts and create an offline account.
  3. Click on add an instance and follow the guide.
  4. To install OptiFine, need the official launcher first, then download OptiFine
  5. Extract OptiFine, the extracted file should be ending in _MOD.jar
  6. Open the jar file in WinRAR, then move the files from notch folder into the base folder. Save the jar archive.
  7. Go to PolyMC, right click on the instance, click Edit -> Versions -> Add to minecraft.jar and select the modified OptiFine.

Docker Server Setup

Docker-compose for minecraft server

version: \"3.9\"\nservices:\n  minecraft:\n    image: marctv/minecraft-papermc-server:latest\n    restart: unless-stopped\n    container_name: mcserver\n    environment:\n\n      - MEMORYSIZE=4G\n      - PAPERMC_FLAGS=\"\"\n      - PUID=1000\n      - PGID=1000\n    volumes:\n      - ~/docker/minecraft:/data:rw\n    ports:\n      - 25565:25565\n      - 19132:19132\n    stdin_open: true\n    tty: true\n

This downloads the latest version of Minecraft, to use another PaperMC version, need to build the image from scratch.

Warning: PaperMC cannot be downgraded, only newer version of PaperMC can be installed after first run.

git clone https://github.com/mtoensing/Docker-Minecraft-PaperMC-Server\n# go edit the \"ARG version=1.xx.x\" to the correct version\ndocker build -t marctv/mcserver:1.xx.x\n

Folders and Plugins

Plugins are located in folder ./plugins some plugins have .yml files. To update or download plugins, use scp, wget on the server or VSCode.

The world folder consists of the save data. It is separated into world, nether, the_end.

Before starting the server, the eula.txt must have eula=true.

bukkit and spigot.yml in the root folder are configuration files for PaperMC.

Rcon Commands

To access the rcon-cli, use docker attach mcserver, to exit, use Ctrl-P and Q, if using VSCode may need to edit keyboard shortcut.

Editing VSCode Shortcut

Press Ctrl-Shift-P and search for keyboard shortcut json.

[\n    {\n        \"key\": \"ctrl+p\",\n        \"command\": \"ctrl+p\",\n        \"when\": \"terminalFocus\"\n    },\n\n    {\n        \"key\": \"ctrl+q\",\n        \"command\": \"ctrl+q\",\n        \"when\": \"terminalFocus\"\n    },\n\n    {\n        \"key\": \"ctrl+e\",\n        \"command\": \"ctrl+e\",\n        \"when\": \"terminalFocus\"\n    }\n\n]\n
"},{"location":"Docker%20Apps/Minecraft/useful-plugins/","title":"Useful Plugins","text":"

WorldEdit

EssentialX

CoreProtect

ViaVersions - allow other similar version to join the server without conflict

Offline Mode/Mobile Bedrock

To allow offline play for PC version. Change server.properties and edit these lines

enforce-whitelist=false\nonline-mode=false\n

Refer to Minecraft Prep and Install to install offline client.

For bedrock compatibility, need the geyser plugin.

Geyser

To allows offline play for bedrock mobile version. Go to ./plugins/Geyser-Spigot/config.yml and change these lines. Do not install the plugin floodgate, if it\u2019s installed, removed the plugin. ViaVersions is also needed for mobile play.

auth-type: offline\nenable-proxy-connections: true\n

Now client can play without login to Xbox or Java.

WorldGuard

"},{"location":"Linux%20Server/debian-based-server-setup/","title":"Debian-Based Server Setup","text":"

Debian 12 with Desktop

Run update and upgrade distro first. Install NTP package is there are errors with that. Reboot

Setup powertop and powersaving features

sudo apt install powertop\npowertop --auto-tune\n

Powersave governor and at reboot. Remember to run the command again

@reboot echo \"powersave\" | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor >/dev/null 2>&1\n

Ensure these packages are installedi

powertop htop iotop fio curl gnupg wget ntfs-3g neofetch ca-certificates lsb-release hdparm hd-idle openssh-server at\n

HDD

lsblk and blkid to get the ntfs hard drive /dev name and the /dev/by-uuid/\u2026

Edit the fstab to mount the drive, same entry for nvme drive

UUID=CC34294F34293E38 /mnt/data ntfs-3g 0 0\n

If the mounted device is HDD array, need to spindown disk with hdparm

hdparm -B 120 /dev/sdb # set the APM level\nhdparm -S 241 /dev/sdb\n

For the -S spindown, 0-240 is multiple of 5s, 241-255 is multiple of 30 min. The above command set spindown every 30min.

If hdparm does not work, hd-idle can be used. Edit the file in /etc/defaults/hd-idle

-i 60 -a disk/by-uuid/xxx -l /var/log/hd-idle.log\n

Sudo without password, go to visudo and add the lines to the bottom, replace $USER with the actual username.

$USER ALL=(ALL) NOPASSWD: ALL\n

Edit shortcuts in bashrc

source .bashrc\n

OpenSSH with Keys

Generate the key using the terminal

ssh-keygen\n
  • give a location to put the key pair
  • this generate a public (.pub) and private key pair
ssh-copy-id -i key.pub username@server\n
  • key.pub is the public key that was generated

The key is ready to use for authorization.

Generate keys using PuTTY software

  1. Copy the red part and use nano to add it in the server ~/.ssh/authorized_keys
  2. Make sure permissions are correct
  3. bash mkdir -p ~/.ssh chmod 700 ~/.ssh chmod 600 ~/.ssh/authorized_keys nano ~/.ssh/authorized_keys
  4. Save private key as ppk file on the root ssh folder.
  5. If the client with private key is Linux machine, need to change the permission of the private key.

    chmod 600 private.key\n
  6. Convert the private key Conversion > Export OpenSSH Keys and save the file to a folder OpenSSH Keys

Setting Up SMB

Refer to Samba(SMB) Setup to setup SMB server.

Desktop Environment Setup

Firefox

The location of firefox profile is at /home/$USER/.mozilla/firefox/xxxxx.default

Make a tarball and copy it and extract it in destination.

In the profile folder, look for compatibility.ini, go to a random profile in the dest machine and copy the compatibility.ini settings to the one that is copied over. This ensure compatibility so that the new profile works without warning.

Check the profile.ini with the name and the location of the new profile folder, firefox should be the same as before.

[Profile0]\nName=karis\nIsRelative=1\nPath=ims58kbd.default-esr-1\n

Themes

To backup/restore settings of cinnamon

Icons

The icons are located at these locations.

/usr/share/icons\n~/.icons\n

Scripts

Copy the scripts and put it into ~/script for organization and copy the old crontab for executing these scripts.

"},{"location":"Linux%20Server/olivetin/","title":"OliveTin","text":"

OliveTin exposes a webpage with buttons that execute shell command (eg. docker, scripts) on the server and allow others for easy access. It should be used internally only.

Installation

Download the correct file from this site. https://github.com/OliveTin/OliveTin/releases OliveTin_linux_amd64.deb

Go to the directory and install the package.

sudo dpkg -i OliveTin\u2026\u200bdeb\nsudo systemctl enable --now OliveTin\n
"},{"location":"Linux%20Server/olivetin/#configuration","title":"Configuration","text":"

The configuration file is located at <em>/etc/OliveTin/config.yaml\u00a0</em>

Example Configuration

listenAddressSingleHTTPFrontend: 0.0.0.0:1378 # set the port of OliveTin to 1378\n\n# Choose from INFO (default), WARN and DEBUG\nlogLevel: \"INFO\"\n\n# Actions (buttons) to show up on the WebUI:\nactions:\n  # This will run a simple script that you create.\n\n- title: Update Music\n  shell: /home/karis/scripts/script\n  icon: '&#127925'\n

More possible configurations (many are not possible on Docker)

Execute a shell command with textbox input.

- title: Restart a Docker CT\n  icon: '<img src = \"icons/restart.png\" width=\"48px\" />'\n  shell: docker restart {{ container }}\n  arguments:\n    - name: container\n      type: ascii\n
  • use {{ }} and give a variable
  • under arguments type, assign a type for it, ascii only allows letters and numbers

Execute a shell command with choices

- title: Manage Docker Stack Services\n  icon: \"&#128736;\"\n  shell: docker-compose -f /home/karis/docker/bookstack/docker-compose.yml {{ action }}\n  arguments:\n    - name: action\n      choices:\n        - title: Start Stack\n          value: up -d\n\n\n        - title: Stop Stack\n          value: down\n

This example give choices to start or stop a docker stack of a docker-compose file. If a argument is given the parameter choices, it will be in dropdown mode.

"},{"location":"Linux%20Server/olivetin/#icons-customization","title":"Icons Customization","text":"

The icons need to be placed in a folder in /var/www/[icon-folder]/icon.png. To use the icons, offline image or web address, it should be in HTML format. The size of 48px is the default size of OliveTin icons. Other CSS options such as <em>style=\"background-color: white;\"</em> also works.

icon: '<img src = \"icons/minecraft.png\" size=\"48px\" />'\n

Icon with emoji, to use emoji, need to use the html code. https://symbl.cc/en/emoji/

For example, &#9786; \ud83d\ude0a.

icon: \"&#9786;\"\n
"},{"location":"Linux%20Server/olivetin/#icon-management","title":"Icon Management","text":"

The default icon folder is /var/www/olivetin/icons

The icon folder of all homelab icons is in ~/icons/homelab

"},{"location":"Linux%20Server/olivetin/#api","title":"API","text":"

Simple action button.

curl -X POST \"http://mediaserver:1378/api/StartAction\" -d '{\"actionName\": \"Update Music\"}'\n

Action with Arguments.

curl -X POST 'http://mediaserver:1378/api/StartAction' -d '{\"actionName\": \"Rename Movies\", \"arguments\": [{\"name\": \"path\", \"value\": \"value\"}]}'\n
"},{"location":"Linux%20Server/sambasmb-setup/","title":"Samba(SMB) Setup","text":""},{"location":"Linux%20Server/sambasmb-setup/#setting-up-smb-server-on-linux","title":"Setting up SMB Server on Linux","text":"

Install the samba tool on Linux.

sudo apt update\nsudo apt install samba -y\n

Edit the /etc/samba/smb.conf

[nvme_share]\n   comment = NVMe Share\n   path = /mnt/nvme/share\n   browseable = yes\n   read only = no\n

nvme_share is the name of the Samba path which will appear in SMB clients and its path is accessed by \\\\192.168.0.1\\nvme_share

path is the location where the files are stored

browseable and read only are flags that are needed to make sure read/write access on the SMB share

Lastly, add the user and password for the SMB share

sudo smbpasswd -a $USER # enter the password twice\n

In the case when Windows fail to write files in the samba share for odd reason. Go to Manage Credentials -> Windows Credentials -> Add a Windows Credential and fill the necessary address, username and password.

"}]} \ No newline at end of file