Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transmission container plus volume management #160

Open
wants to merge 40 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
48b67d9
Update directoryfix.sh
kauffman-repo Jan 9, 2020
09351d6
Update directoryfix.sh
kauffman-repo Jan 9, 2020
a39d01c
Adds a Gitea container to IOTstack
Paraphraser Jan 12, 2020
eb9b982
Added filter.acl file to mosquitto/config
Tomasmerca Jan 15, 2020
c4adb55
Created empty file whit example filter
Tomasmerca Jan 15, 2020
89f4647
Added options for password and filter config
Tomasmerca Jan 15, 2020
72e6345
Update menu.sh
RavHilton Feb 6, 2020
1ff9ff3
line 61 fixed
DIYtechie Mar 5, 2020
cb3e1b0
added transmission container
gpongelli Mar 22, 2020
7751ed0
add volumes management from template
gpongelli Mar 22, 2020
98101cd
menu.sh: Fix small typo in msg "Restart Required"
gmacario Mar 23, 2020
da5a549
update menu.sh
peyanski Apr 4, 2020
b068114
first import
Willem-Dekker Apr 19, 2020
f6571ae
added index.md
Willem-Dekker Apr 19, 2020
96454c9
Updated styling of existing pages
Willem-Dekker Apr 19, 2020
7bf68e7
Updated styling of existing pages
Willem-Dekker Apr 19, 2020
1646a7a
Merge pull request #20 from Willem-Dekker/mkdocks
SensorsIot Apr 19, 2020
a9afc96
Update README.md
SensorsIot Apr 19, 2020
50cbf52
Update README.md
SensorsIot Apr 19, 2020
0b2385b
Merge pull request #16 from SensorsIot/pr/161
Willem-Dekker Apr 19, 2020
5d0c85a
Changed each menu item to be on a separate line to make merges easier.
Apr 19, 2020
4f8a47d
Merge branch 'armhf_keys_menu_lines' into pr/126
Apr 20, 2020
52ddbcb
Changed docker-compose yml file version to 3.6.
Apr 20, 2020
d464d0f
Merge pull request #26 from SensorsIot/armhf_keys_menu_lines
Willem-Dekker Apr 20, 2020
65e48e0
Merge pull request #7 from SensorsIot/pr/139
Willem-Dekker Apr 20, 2020
67e0056
Merge pull request #3 from SensorsIot/pr/126
Willem-Dekker Apr 20, 2020
15d840b
Update README.md
Slyke Apr 20, 2020
a3078f2
Merge pull request #27 from SensorsIot/readme-update
Willem-Dekker Apr 20, 2020
d86a5cd
Merge pull request #4 from SensorsIot/pr/130
Willem-Dekker Apr 20, 2020
a7d37c0
Fixes some inconsistencies and makes other tweaks to index page
Paraphraser Apr 21, 2020
e82dac8
Fixed typo in readme (PRi --> RPi)
Apr 21, 2020
7889207
Added docker version check
Apr 21, 2020
93b10ad
Merge pull request #31 from Paraphraser/20200421-doco-index
Willem-Dekker Apr 21, 2020
3e3b8fe
Merge pull request #32 from SensorsIot/docker-version-check
Slyke Apr 21, 2020
fec3260
Merge pull request #8 from SensorsIot/pr/143
Slyke Apr 21, 2020
c133968
Merge pull request #10 from SensorsIot/pr/151
Slyke Apr 21, 2020
c23d787
Merge pull request #17 from SensorsIot/pr/169
Slyke Apr 21, 2020
dc1c366
Merge branch 'master' into PR_add_volumes_to_script
gpongelli Apr 21, 2020
e95298f
Removed bookmark from transmission volumes.yml
gpongelli Apr 22, 2020
1936989
Create default directories on build
Apr 25, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: Publish docs via GitHub Pages
on:
push:
branches:
- master

jobs:
build:
name: Deploy docs
runs-on: ubuntu-latest
steps:
- name: Checkout master
uses: actions/checkout@v1

- name: Deploy docs
uses: mhausenblas/mkdocs-deploy-gh-pages@master
env:
PERSONAL_TOKEN: ${{ secrets.PERSONAL_TOKEN }}
1 change: 1 addition & 0 deletions .templates/gitea/gitea.env
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# initially empty
14 changes: 14 additions & 0 deletions .templates/gitea/service.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
gitea:
container_name: gitea
image: kapdap/gitea-rpi
restart: unless-stopped
user: "0"
ports:
- "7920:3000/tcp"
- "2222:22/tcp"
env_file:
- ./services/gitea/gitea.env
volumes:
- ./volumes/gitea/data:/data
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
6 changes: 6 additions & 0 deletions .templates/mosquitto/filter.acl
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
user admin
topic read #
topic write #

pattern read #
pattern write #
6 changes: 6 additions & 0 deletions .templates/mosquitto/mosquitto.conf
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@
persistence true
persistence_location /mosquitto/data/
log_dest file /mosquitto/log/mosquitto.log

#Uncomment to enable passwords
#password_file /mosquitto/config/pwfile
#allow_anonymous false

#Uncomment to enable filters
#acl_file /mosquitto/config/filter.acl
1 change: 1 addition & 0 deletions .templates/mosquitto/service.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,5 @@
- ./volumes/mosquitto/data:/mosquitto/data
- ./volumes/mosquitto/log:/mosquitto/log
- ./services/mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf
- ./services/mosquitto/filter.acl:/mosquitto/config/filter.acl

5 changes: 4 additions & 1 deletion .templates/python/directoryfix.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,12 @@

# Directoryfix for python

#current user
u=$(whoami)

if [ ! -d ./volumes/python/app ]; then
sudo mkdir -p ./volumes/python/app
sudo chown -R pi:pi ./volumes/python
sudo chown -R $u:$u ./volumes/python
echo 'print("hello world")' >./volumes/python/app/app.py

fi
7 changes: 7 additions & 0 deletions .templates/transmission/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/bin/bash

# create directories for named volumes
TRANSMISSION_BASEDIR=.volumes/transmission
mkdir -p $TRANSMISSION_BASEDIR/downloads
mkdir -p $TRANSMISSION_BASEDIR/watch
mkdir -p $TRANSMISSION_BASEDIR/config
16 changes: 16 additions & 0 deletions .templates/transmission/service.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
transmission:
image: linuxserver/transmission
container_name: transmission
environment:
- PUID=1000
- PGID=1000
- TZ=Europe/Rome
volumes:
- transm_config_volume:/config
- transm_download_volume:/downloads
- transm_watch_volume:/watch
ports:
- 9091:9091
- 51413:51413
- 51413:51413/udp
restart: unless-stopped
21 changes: 21 additions & 0 deletions .templates/transmission/volumes.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@

transm_download_volume:
driver: local
driver_opts:
o: bind
type: none
device: .volumes/transmission/downloads

transm_watch_volume:
driver: local
driver_opts:
o: bind
type: none
device: .volumes/transmission/watch

transm_config_volume:
driver: local
driver_opts:
o: bind
type: none
device: .volumes/transmission/config
205 changes: 21 additions & 184 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,198 +1,35 @@
# IOTStack
# IOT Stack
IOTstack is a builder for docker-compose to easily make and maintain IoT stacks on the Raspberry Pi.

IOTstack is a builder for docker-compose to easily make and maintain IoT stacks on the Raspberry Pi
## Documentation for the project:

## Announcements
https://sensorsiot.github.io/IOTstack/

The bulk of the README has moved to the Wiki. Please check it out [here](https://github.com/gcgarner/IOTstack/wiki)
## Video
https://youtu.be/a6mjt8tWUws

* 2019-12-19 Added python container, tweaked update script
* 2019-12-12 modified zigbee2mqtt template file
* 2019-12-12 Added Function to add custom containers to the stack
* 2019-12-12 PR cmskedgell: Added Homebridge
* 2019-12-12 PR 877dev: Added trimming of online backups
* 2019-12-03 BUGFIX Mosquitto: Fixed issue where mosquitto failed to start as a result of 11-28 change
* 2019-12-03 Added terminal for postgres, temporarily removed setfacl from menu
* 2019-11-28 PR @stfnhmplr added diyHue
* 2019-11-28 Fixed update notification on menu
* 2019-11-28 Fixed mosquitto logs and database not mapping correctly to volumes. Pull new template to fix
* 2019-11-28 added the option to disable swapfile by setting swappiness to 0
* 2019-11-28 PR @stfnhmplr fixed incorrect shegang on MariaDB terminal.sh
* 2019-11-28 Added native install for RPIEasy
* 2019-11-27 Additions: NextCloud, MariaDB, MotionEye, Mozilla Webthings, blynk-server (fixed issue with selection.txt)
* 2019-11-22 BUGFIX selection.txt failed on fresh install, added pushd IOTstack to menu to ensure correct path
* 2019-11-22 Added notification into menu if project update is available
* 2019-11-20 BUGFIX influxdb backup: Placing docker_backup in crontab caused influxdb backup not to execute correctly
* 2019-11-20 BUGFIX disable swap: swapfile recreation on reboot fixed. Re-run from menu to fix.
* Node-RED: serial port. New template adds privileged which allows acces to serial devices
* EspurinoHub: is available for testing see wiki entry
## Installation
1. On the (RPi) lite image you will need to install git first

***

## Highlighted topics

* [Bluetooth and Node-RED](https://github.com/gcgarner/IOTstack/wiki/Node-RED#using-bluetooth)
* [Saving files to disk inside containers](https://github.com/gcgarner/IOTstack/wiki/Node-RED#sharing-files-between-node-red-and-the-host)
* [Updating the Project](https://github.com/gcgarner/IOTstack/wiki/Updating-the-Project)

***

## Coming soon

* reverse proxy is now next on the list, I cant keep up with the ports
* Detection of arhcitecture for seperate stack options for amd64, armhf, i386
* autocleanup of backups on cloud
* Gitea (in testing branch)
* OwnCloud

***

## About

Docker stack for getting started on IoT on the Raspberry Pi.

This Docker stack consists of:

* Node-RED
* Grafana
* InfluxDB
* Postgres
* Mosquitto mqtt
* Portainer
* Adminer
* openHAB
* Home Assistant (HASSIO)
* zigbee2mqtt
* Pi-Hole
* TasmoAdmin (parial wiki)
* Plex media server
* Telegraf (wiki coming soon)
* RTL_433
* EspruinoHub (testing)
* MotionEye
* MariaDB
* Plex
* Homebridge

In addition, there is a write-up and some scripts to get a dynamic DNS via duckdns and VPN up and running.

Firstly what is docker? The correct question is "what are containers?". Docker is just one of the utilities to run a container.

A Container can be thought of as ultra-minimal virtual machines, they are a collection of binaries that run in a sandbox environment. You download a preconfigured base image and create a new container. Only the differences between the base and your "VM" are stored.
Containers don't have [GUI](https://en.wikipedia.org/wiki/Graphical_user_interface)s so generally the way you interact with them is via web services or you can launch into a terminal.
One of the major advantages is that the image comes mostly preconfigured.

There are pro's and cons for using native installs vs containers. For me, one of the best parts of containers is that it doesn't "clutter" your device. If you don't need Postgres anymore then just stop and delete the container. It will be like the container was never there.

The container will fail if you try to run the docker and native vesions as the same time. It is best to install this on a fresh system.

For those looking for a script that installs native applications check out [Peter Scargill's script](https://tech.scargill.net/the-script/)

## Tested platform

Raspberry Pi 3B and 4B Raspbian (Buster)

### Older Pi's

Docker will not run on a PiZero or A model 1 because of the CPU. It has not been tested on a Model 2. You can still use Peter Scargill's [script](https://tech.scargill.net/the-script/)

## Running under a virtual machine

For those wanting to test out the script in a Virtual Machine before installing on their Pi there are some limitations. The script is designed to work with Debian based distributions. Not all the container have x86_64 images. For example Portainer does not and will give an error when you try and start the stack. Please see the pinned issue [#29](https://github.com/gcgarner/IOTstack/issues/29), there is more info there.

## Feature Requests

Please direct all feature requests to [Discord](https://discord.gg/W45tD83)

## Youtube reference

This repo was originally inspired by Andreas Spiess's video on using some of these tools. Some containers have been added to extend its functionality.

[YouTube video](https://www.youtube.com/watch?v=JdV4x925au0): This is an alternative approach to the setup. Be sure to watch the video for the instructions. Just note that the network addresses are different, see the wiki under Docker Networks.

### YouTube guide

@peyanski (Kiril) made a YouTube video on getting started using the project, check it out [here](https://youtu.be/5JMNHuHv134)

## Download the project

1.On the lite image you will need to install git first

```bash
sudo apt-get install git
```
sudo apt-get install git -y
```

2.Download the repository with:

```bash
git clone https://github.com/gcgarner/IOTstack.git ~/IOTstack
2. Download the repository with:
```
git clone https://github.com/SensorsIot/IOTstack.git ~/IOTstack
```

Due to some script restraints, this project needs to be stored in ~/IOTstack

3.To enter the directory run:

```bash
cd ~/IOTstack
3. To enter the directory and run menu for installation options:
```
cd ~/IOTstack && bash ./menu.sh
```

## The Menu

I've added a menu to make things easier. It is good to familiarise yourself with the installation process.
The menu can be used to install docker and build the docker-compose.yml file necessary for starting the stack. It also runs a few common commands. I do recommend you start to learn the docker and docker-compose commands if you plan on using docker in the long run. I've added several helper scripts, have a look inside.

Navigate to the project folder and run `./menu.sh`

### Installing from the menu

Select the first option and follow the prompts

### Build the docker-compose file

docker-compose uses the `docker-compose.yml` file to configure all the services. Run through the menu to select the options you want to install.

### Docker commands

This menu executes shell scripts in the root of the project. It is not necessary to run them from the menu. Open up the shell script files to see what is inside and what they do.

### Miscellaneous commands

Some helpful commands have been added like disabling swap.

## Running Docker commands

From this point on make sure you are executing the commands from inside the project folder. Docker-compose commands need to be run from the folder where the docker-compose.yml is located. If you want to move the folder make sure you move the whole project folder.

## Starting and Stopping containers

to start the stack navigate to the project folder containing the docker-compose.yml file

To start the stack run:
`docker-compose up -d` or `./scripts/start.sh`

To stop:
`docker-compose stop`

The first time you run 'start' the stack docker will download all the images for the web. Depending on how many containers you selected and your internet speed this can take a long while.

The `docker-compose down` command stops the containers then deletes them.

## Persistent data

Docker allows you to map folders inside your containers to folders on the disk. This is done with the "volume" key. There are two types of volumes. Modification to the container are reflected in the volume.

## See Wiki for further info

[Wiki](https://github.com/gcgarner/IOTstack/wiki)

## Add to the project

Feel free to add your comments on features or images that you think should be added.

## Contributions

If you use some of the tools in the project please consider donating or contributing on their projects. It doesn't have to be monetary, reporting bugs and PRs help improve the projects for everyone.

### Thanks
4. Install docker with the menu, restart your system.

@mrmx, @oscrx, @brianimmel, @Slyke, @AugustasV, @Paulf007, @affankingkhan, @877dev, @Paraphraser, @stfnhmplr, @peyanski, @cmskedgell
5. Run menu again to select your build options, then start docker-compose with
```
docker-compose up -d
```
44 changes: 44 additions & 0 deletions docs/Accessing-your-Device-from-the-internet.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Accessing your device from the internet
The challenge most of us face with remotely accessing your home network is that you don't have a static IP. From time to time the IP that your ISP assigns to you changes and it's difficult to keep up. Fortunately, there is a solution, a DynamicDNS. The section below shows you how to set up an easy to remember address that follows your public IP no matter when it changes.

Secondly, how do you get into your home network? Your router has a firewall that is designed to keep the rest of the internet out of your network to protect you. Here we install a VPN and configure the firewall to only allow very secure VPN traffic in.

## DuckDNS
If you want to have a dynamic DNS point to your Public IP I added a helper script.
Register with duckdns.org and create a subdomain name. Then edit the `nano ~/IOTstack/duck/duck.sh` file and add your

```bash
DOMAINS="YOUR_DOMAINS"
DUCKDNS_TOKEN="YOUR_DUCKDNS_TOKEN"
```

first test the script to make sure it works `sudo ~/IOTstack/duck/duck.sh` then `cat /var/log/duck.log`. If you get KO then something has gone wrong and you should check out your settings in the script. If you get an OK then you can do the next step.

Create a cron job by running the following command `crontab -e`

You will be asked to use an editor option 1 for nano should be fine
paste the following in the editor `*/5 * * * * sudo ~/IOTstack/duck/duck.sh >/dev/null 2>&1` then ctrl+s and ctrl+x to save

Your Public IP should be updated every five minutes

## PiVPN
pimylifeup.com has an excellent tutorial on how to install [PiVPN](https://pimylifeup.com/raspberry-pi-vpn-server/)

In point 17 and 18 they mention using noip for their dynamic DNS. Here you can use the DuckDNS address if you created one.

Don't forget you need to open the port 1194 on your firewall. Most people won't be able to VPN from inside their network so download OpenVPN client for your mobile phone and try to connect over mobile data. ([More info.](https://en.wikipedia.org/wiki/Hairpinning))

Once you activate your VPN (from your phone/laptop/work computer) you will effectively be on your home network and you can access your devices as if you were on the wifi at home.

I personally use the VPN any time I'm on public wifi, all your traffic is secure.

## Zerotier
https://www.zerotier.com/

Zerotier is an alternative to PiVPN that doesn't require port forwarding on your router. It does however require registering for their free tier service [here](https://my.zerotier.com/login).

Kevin Zhang has written a how to guide [here](https://iamkelv.in/blog/2017/06/zerotier.html). Just note that the install link is outdated and should be:
```
curl -s 'https://raw.githubusercontent.com/zerotier/ZeroTierOne/master/doc/contact%40zerotier.com.gpg' | gpg --import && \
if z=$(curl -s 'https://install.zerotier.com/' | gpg); then echo "$z" | sudo bash; fi
```
12 changes: 12 additions & 0 deletions docs/Adminer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Adminer
## References
- [Docker](https://hub.docker.com/_/adminer)
- [Website](https://www.adminer.org/)

## About

This is a nice tool for managing databases. Web interface has moved to port 9080. There was an issue where openHAB and Adminer were using the same ports. If you have an port conflict edit the docker-compose.yml and under the adminer service change the line to read:
```
ports:
- 9080:8080
```
Loading