---

DSMR-reader - Docker

DSMR-protocol reader, telegram data storage, and energy consumption visualizer. Can be used for reading the smart meter DSMR (Dutch Smart Meter Requirements) P1 port yourself at your home. You will need a cable and hardware that can run Docker. Free for non-commercial use.

---

Table of Contents

• General Info
• Contribution
• Screenshots
• Technologies
• Releases
• Setup / Parameters
  • Public Access Warning
  • Settings
  • Nginx Related
  • DSMR Related
  • DB Related
  • DSMR Datalogger Related
  • Run with Docker Compose
  • Run with Docker Run
  • Environment Variables from Files (Docker Secrets)
• DSMR-Reader - Docker and Homewizard P1 Meter Integration
  • Prerequisites
  • Plugin Preparation
  • Docker Setup
  • Original Post
• Features
• Issues
• Inspiration
• Contact

---

General Info

The purpose of this project is to provide a simplified installation of DSMR-reader using all the benefits of Docker.

Contribution

Special thanks to the following persons for their great contribution(s):

• @dennissiemensma
• @bjw-s
• @Caroga
• mchangsp

---

Screenshots

---

Technologies

* Docker >= 20.x
* Python 3.x
* Postgres >= 13.x, max 16.x
* MySQL ??.?? (untested by me)

---

Releases

This is a multi-arch image and will also run on a Raspberry Pi or other Docker-enabled ARMv6/7/8 devices.

Image Tag	Architectures	Image OS
latest	amd64, arm32v6, arm32v7, arm64v8	Alpine Linux 3.19
development	amd64, arm32v6, arm32v7, arm64v8	Alpine Linux 3.19

Docker tags/releases can be found here.

---

Setup / Parameters

Public Access Warning

Exposing your DSMR-reader installation to the Internet? Consider additionally using HTTP Auth (see below) or enabling "Force password login everywhere" in the Frontend settings in DSMR-reader, to prevent public access.

Settings

For DSMR Reader specific environment settings, please refer to: DSMR-reader env settings docs

It's possible to set the following settings as environment variables, for example:

# Required (defaults are shown as value):
- DJANGO_TIME_ZONE=Europe/Amsterdam
- VIRTUAL_HOST=localhost
# It's possible to map a UID/GID with a user/group from your local system.
# This will not change the username, only match IDs to prevent issues with access rights!
- DUID=803
- DGID=803

Nginx Related

# Default inside the container nginx is running on port 80.
# In some cases (host network f.e.), some people requested to be able to change the default listen port.
# However, in most cases this setting isn't being used.
NGINX_LISTEN_PORT=80

# Default nginx generated access logs.
# In some cases you want to disable this, because e.g., you use a reverse proxy which also generates access logs
DISABLE_NGINX_ACCESS_LOGS=true

# Enables port 443 for nginx
# /etc/ssl/private/fullchain.pem and /etc/ssl/private/privkey.pem are required to be mounted!
ENABLE_NGINX_SSL=false

# ENABLE HSTS on SSL: https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security
ENABLE_NGINX_ENABLE_HSTS=false

# Redirect HTTP traffic to HTTPS
ENABLE_NGINX_SSL_REDIRECT=false

Nginx HTTP Auth:

ENABLE_HTTP_AUTH=false
HTTP_AUTH_USERNAME=
HTTP_AUTH_PASSWORD=

It's not possible to combine the following settings!!!:

ENABLE_NGINX_SSL
NGINX_LISTEN_PORT

DSMR Related (defaults are shown as value):

# Webinterface user:
DSMRREADER_ADMIN_USER=admin
# Webinterface user password:
DSMRREADER_ADMIN_PASSWORD=admin
# Loglevel. Valid values are ERROR, WARNING, DEBUG:
DSMRREADER_LOGLEVEL=WARNING
# Secret key for encryption:
DJANGO_SECRET_KEY=dsmrreader
# Ignore database size notifications:
DSMRREADER_SUPPRESS_STORAGE_SIZE_WARNINGS=True
# Plugins (custom) setup:
DSMRREADER_PLUGINS=dsmr_plugins.modules.plugin_name1,dsmr_plugins.modules.plugin_name2
# Enable IFrame support (e.g., for use in HASS).
ENABLE_IFRAME=false

DB Related (defaults are shown as value):

# Optional. Vacuum clean Postgres on startup:
VACUUM_DB_ON_STARTUP=false
# Required. Defaults are set to:
DJANGO_DATABASE_ENGINE=django.db.backends.postgresql
DJANGO_DATABASE_NAME=dsmrreader
DJANGO_DATABASE_USER=dsmrreader
DJANGO_DATABASE_PASSWORD=dsmrreader
DJANGO_DATABASE_HOST=dsmrdb
DJANGO_DATABASE_PORT=5432
DJANGO_DATABASE_CONN_MAX_AGE=60

DSMR Datalogger Related

When you are connecting to P1 via a network socket, you need to run DSMR Reader in standalone mode. Ignore the errors about /dev/ttyUSB* and head over to the DSMR Reader datalogger configuration in the admin panel and configure the setting so it matches your environment. More info: Issue #303

• DSMRREADER_OPERATION_MODE - Run DSMR Reader in one of the following modes (default is standalone with the serial flavor):
  • standalone - Run all processes, including backend, GUI, and datalogger. There are two flavors:
    • standalone - serial - Use a serial connection for the datalogger.
    • standalone - ipv4 - Use a network socket for the datalogger.
  • api_server - Run all processes, except the datalogger process. A remote datalogger is required to collect DSMR Reader telegrams.
  • api_client - Only start the datalogger client, which sends the P1 telegrams to the api_server. It is required to set up DATALOGGER_API_* environment variables.

DSMR Reader Datalogger - Standalone - Serial (default)

More info: DSMR-reader remote datalogger installation docs

The default startup values for DSMR Reader standalone are:

DSMRREADER_REMOTE_DATALOGGER_INPUT_METHOD=serial
DSMRREADER_REMOTE_DATALOGGER_SERIAL_PORT=/dev/ttyUSB0

# DSMR meter version 4/5
DSMRREADER_REMOTE_DATALOGGER_SERIAL_BAUDRATE=115200
DSMRREADER_REMOTE_DATALOGGER_SERIAL_BYTESIZE=8
DSMRREADER_REMOTE_DATALOGGER_SERIAL_PARITY=N

Some meters running on an older version can be set with the following values:

# DSMR meter version 2/3 settings
DSMRREADER_REMOTE_DATALOGGER_SERIAL_BAUDRATE=9600
DSMRREADER_REMOTE_DATALOGGER_SERIAL_BYTESIZE=7
DSMRREADER_REMOTE_DATALOGGER_SERIAL_PARITY=E

DSMR Reader Datalogger - Standalone - IPv4

More info: DSMR-reader remote datalogger installation docs

Instead of a serial connection, it's also possible to use a network socket. Define the following variables:

DSMRREADER_REMOTE_DATALOGGER_INPUT_METHOD=ipv4
DSMRREADER_REMOTE_DATALOGGER_NETWORK_HOST=127.0.0.1 # default
DSMRREADER_REMOTE_DATALOGGER_NETWORK_PORT=2000 # default

Remote DSMR Datalogger - API Client

More info: DSMR-reader remote datalogger installation docs

# Required. Instructs dsmr reader to start in api_client mode
DSMRREADER_OPERATION_MODE=api_client
# Required. Destination(s) of the DSMR Reader (Docker) host(s)
DSMRREADER_REMOTE_DATALOGGER_API_HOSTS=x
# Required. Add the API keys of the DSMR Reader (Docker) destination host(s)
DSMRREADER_REMOTE_DATALOGGER_API_KEYS=x

Remote DSMR Datalogger - API Server

More info: DSMR-reader remote datalogger installation docs

The configured api_client will push data to the api_server. The only difference between standalone and api_server is that the datalogger process isn't running.

# Required. Instructs dsmr reader to start in api_server mode, which means no datalogger process.
# All telegrams are coming in through the API
DSMRREADER_OPERATION_MODE=api_server

Remote DSMR Datalogger - Optional Settings

More info: DSMR-reader remote datalogger installation docs

DSMRREADER_REMOTE_DATALOGGER_TIMEOUT=x
DSMRREADER_REMOTE_DATALOGGER_SLEEP=x
DSMRREADER_REMOTE_DATALOGGER_DEBUG_LOGGING=false

Run with Docker Compose

An example docker-compose.yaml file can be found here.

Modify the docker-compose file with parameters that suit your environment, then run:

docker-compose up -d

After starting the containers with Docker Compose, the dashboard is reachable at:

http://<hostname>:7777

Don't forget to modify the default DSMR version (default is DSMR v4) after starting the containers:

http://<hostname>:7777/admin/dsmr_datalogger/dataloggersettings/

Run with Docker Run

Note: The example below only runs DSMR. You need to run a Postgres Docker container or traditional Postgres environment as well, since a database is needed.

docker run -d \
  --name dsmr \
  --restart always \
  -p 7777:80 \
  -p 7779:443 \
  -e DJANGO_TIME_ZONE=Europe/Amsterdam \
  -e DJANGO_DATABASE_HOST=x.x.x.x \
  -e DJANGO_DATABASE_USER=dsmrreader \
  -e DJANGO_DATABASE_PASSWORD=dsmrreader \
  -e VIRTUAL_HOST=localhost \
  --no-healthcheck \
  --device /dev/ttyUSB0:/dev/ttyUSB0 \
  xirixiz/dsmr-reader-docker

The --no-healthcheck argument should only be used when the containers function without presenting the DSMR Reader web interface, for example in datalogger sender mode. By default, this argument should not be used!

Environment Variables from Files (Docker Secrets)

You can set any environment variable from a file by using a special prepend FILE__.

Example:

services:
  some_service:
    image: some_image
    environment:
      FILE__SECRET: /run/secrets/a_secret_file
    secrets:
      - a_secret_file

secrets:
  a_secret_file:
    file: somedir/my_secret.txt

This setup mounts my_secret.txt as /run/secrets/a_secret_file. The secrets section under the service authorizes the service to use the a_secret_file secret. The environment variable FILE__SECRET tells the service what file to read to set/get the value of the environment variable SECRET.

Internal secrets:

secrets:
  a_secret_file:
    file: ./secrets/a_secret_file.txt  # Define the path to your secret file

External secrets (for example Docker Swarm):

secrets:
  a_secret_file:
    external: true

---

DSMR-Reader - Docker and Homewizard P1 Meter Integration

This guide explains how to install and configure a DSMR Reader plugin to read Homewizard P1 telegrams and inject them into DSMR Reader.

Prerequisites

• A working instance of DSMR Reader in Docker
• A working Homewizard P1 meter with Local API enabled
• The Homewizard P1 meter IP address

Plugin Preparation

Assumption: Your docker-compose.yaml file is in the folder /home/pi/dsmr

On your Docker host:

1. Create a folder /home/pi/dsmr/plugins

2. Inside the folder /home/pi/dsmr/plugins, create a file homewizard_p1.py with the following contents (replace 1.2.3.4 with the Homewizard P1 meter IP address):

   import requests
   from django.dispatch import receiver
   from dsmr_backend.signals import backend_called
   import dsmr_datalogger.services.datalogger
   
   HOMEWIZARD_ENDPOINT = 'http://1.2.3.4:80/api/v1/telegram'
   HOMEWIZARD_TIMEOUT = 5
   
   @receiver(backend_called)
   def handle_backend_called(**kwargs):
       response = requests.get(HOMEWIZARD_ENDPOINT, timeout=HOMEWIZARD_TIMEOUT)
   
       if response.status_code != 200:
           print(' [!] HomeWizard plugin: v1 telegram endpoint failed (HTTP {}): {}'.format(response.status_code, response.text))
           return
   
       dsmr_datalogger.services.datalogger.telegram_to_reading(data=response.text)

Docker Setup

1. Navigate to folder /home/pi/dsmr

2. Edit your docker-compose.yaml file

3. Add the following definition to the volumes: section:

   - ./plugins/homewizard_p1.py:/app/dsmr_plugins/modules/homewizard_p1.py

4. Add the following definitions to the environment: section:

   - DSMRREADER_OPERATION_MODE=api_server
   - DSMRREADER_PLUGINS=dsmr_plugins.modules.homewizard_p1

5. Save the docker-compose.yaml file

6. To stop DSMR Reader, run:

   docker-compose down

7. To start DSMR Reader, run:

   docker-compose up -d

Original Post

The original instructions are (partly in Dutch) on GitHub. The Python source code in the original post was missing an import statement. The instructions have been grouped together and translated into English.

For an alternative solution using Home Assistant automations, see this guide.

---

Features

DSMR Reader - Database Cleanup/Vacuum

You might receive a notification that the database is growing, like in this issue.

Clean up the Docker database by running the following command from the application container:

docker exec -ti dsmr bash -c '/app/cleandb.sh'

Or, to run verbose:

docker exec -ti dsmr bash -c '/app/cleandb.sh -v'

DSMR Reader - Plugins

DSMR Reader plugins (documentation) can be added by adding the plugin with a volume mapping and using it in the environmental variable to load it.

volumes:
  - ./modules/forward_telegram_to_api.py:/app/dsmr_plugins/modules/forward_telegram_to_api.py
environment:
  - DSMRREADER_PLUGINS=dsmr_plugins.modules.forward_telegram_to_api

Backup and Restore Mechanism 1

dsmrdb in docker-compose is configured to use a Docker volume. So when the application and Docker container have been removed, the Postgres data still persists.

You can easily create a backup. Values depend on Docker/docker-compose user and database variables:

docker-compose stop dsmr
docker exec -t dsmrdb pg_dumpall -c -U dsmrreader > dsmrreader.sql
docker-compose start dsmr

It's also possible to gzip:

docker exec -t dsmrdb pg_dumpall -c -U dsmrreader | /bin/gzip > dsmrreader.sql.gz

Or drop the database and restore a backup. Values depend on Docker/docker-compose user and database variables:

docker-compose stop dsmr
docker exec -t dsmrdb dropdb dsmrreader -U dsmrreader
docker exec -t dsmrdb createdb -O dsmrreader dsmrreader -U dsmrreader
cat dsmrreader.sql | docker exec -i dsmrdb psql -U dsmrreader
docker-compose start dsmr

Backup and Restore Mechanism 2

You can also use Docker's own volume backup and restore mechanism.

Backup:

docker run -it --rm -v dsmrdb:/volume -v /tmp:/backup alpine \
  tar -cjf /backup/dsmrdb.tar.bz2 -C /volume ./

Restore:

docker run -it --rm -v dsmrdb:/volume -v /tmp:/backup alpine \
  sh -c "rm -rf /volume/* /volume/..?* /volume/.[!.]* ; tar -C /volume/ -xjf /backup/dsmrdb.tar.bz2"

Backup Mechanism 3

Note: This backup is done via the dsmr container, not via the dsmrdb container!

Backup:

docker exec -ti dsmr bash -c 'PGPASSWORD=${DJANGO_DATABASE_PASSWORD} /usr/bin/pg_dump \
  -h "${DJANGO_DATABASE_HOST}" \
  -p "${DJANGO_DATABASE_PORT}" \
  -d "${DJANGO_DATABASE_NAME}" \
  -U "${DJANGO_DATABASE_USER}"'

Postgres Upgrade (Docker)

- Stop the DSMR Reader container ONLY
- Backup the dsmrdb database (see "Backup and restore mechanism" in the README.md)
- Validate the dsmrdb backup!
- Consider "vacuuming" the database following "DSMR Reader - Database cleanup/vacuum" in the README.md.
- Stop and remove the dsmrdb container
- Rename the db folder that is mounted in the Docker container, containing the database data, to something else (.old, etc.)
- Create a new db folder with the original mount name
- Update `docker-compose` or your Docker run command with the new Postgres version
- Ensure the client package is compatible with the Postgres version you're using. Check the DSMR image for supported versions.
- Start `dsmrdb` (it's an empty but valid Postgres DB now).
- Restore the database backup created in step 2 (see "Backup and restore mechanism" in the README.md)
- Restart the `dsmrdb` container
- Start the `dsmr` container

UID/GID

When using volumes (-v or --volume flags), permissions issues can occur between the host OS and the container. To avoid this, specify a user ID DUID and group ID DGID from the local system in the Docker container. Ensure the owner of the directory has all permissions on the volume you'd like to mount into the Docker container.

Identify a UID or GID by executing:

id xirixiz
uid=1000(xirixiz) gid=1000(xirixiz) groups=1000(xirixiz) 1001(docker)

---

Issues

MySQL Backend - Issue with DSMR Running on Top of MySQL

Problem: Configured timezone info tables result in "Data processing is lagging behind" messages on the web interface and only leading information in the logs if set to DEBUG: "Missing consumption data for:"

Solution: Drill down to find the issue: dsmrreader/dsmr-reader#909

Resolve by installing timezone info tables in MySQL: MySQL Timezone Documentation

Raspberry Pi

Issue: Caused by the upgrade from Alpine 3.12 to 3.13. Alpine 3.13 requires:

- Docker 19.03.9 or newer
- libseccomp 2.4.2 or newer

The libseccomp package hasn't been updated for Debian stable yet. Therefore, this image won't run on any Debian (or Raspbian) stable host.

Fix:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 04EE7237B7D453EC 648ACFD622F3D138
echo 'deb http://httpredir.debian.org/debian buster-backports main contrib non-free' | sudo tee -a /etc/apt/sources.list.d/debian-backports.list
sudo apt update
sudo apt install libseccomp2 -t buster-backports

Incorrect Timestamps

Mounting /etc/localtime:/etc/localtime often results in incorrect timestamps in DSMR Reader (+/- 1 hour). Removing the mount usually solves the problem.

Synology

For Synology or other NAS appliances, an additional driver is required:

• synokernel-usbserial
• The docker-compose file must be set to version 3.2 or lower.

Additional Links:

• Reddit Discussion

---

Inspiration

Project inspired by the hard work and effort of @dennissiemensma.

---

Contact

Created by @xirixiz - feel free to contact me!

---