[Req/WIP] Support environment variables in odoo.conf

[codebykyle]

Overview

Thanks for maintaining this, this is fantastic, and a very helpful resource. I literally use this container every day. I do not mean to come in and try to change things, but I have some thoughts about some improvements.

I'd like to rewrite this such that odoo.conf is merged with available environment variables at startup. This is very easily achievable with envsubst. This lets users pick and choose to override odoo.conf settings with environment variables.

Once generated, the wait-for-psql.py should load the configuration file with configparser to wait for the databases specified in the Odoo configuration to be available.

I am okay if you do not want to merge these changes, but I will contribute my changes in this direction. It would be helpful if anyone has any criticisms of this or ideas.

Proposal

We should change the way this script works so that it merges the environment variables into the odoo.conf file. wait-for-psql.py should be changed to read the resulting file, and should load relevant data from the odoo.conf file. The entrypoint.sh file should be responsible for generating the final odoo.conf, should execute Odoo without hard-coding command-line arguments, and database passwords should not be loaded into logs or the terminal.

Consider the following default configuration file:

[options]
addons_path = $ADDONS_PATH
data_dir = $DATA_DIR
db_host = $DB_HOST
db_port = $DB_PORT
db_user = $DB_USER
db_password = $DB_PASSWORD
; db_name = $DB_NAME

; admin_passwd = $ADMIN_PASSWD
; csv_internal_sep = $CSV_INTERNAL_SEP
; db_maxconn = $DB_MAXCONN
; db_template = $DB_TEMPLATE
; dbfilter = $DBFILTER
; debug_mode = $DEBUG_MODE
; email_from = $EMAIL_FROM
; limit_memory_hard = $LIMIT_MEMORY_HARD
; limit_memory_soft = $LIMIT_MEMORY_SOFT
; limit_request = $LIMIT_REQUEST
; limit_time_cpu = $LIMIT_TIME_CPU
; limit_time_real = $LIMIT_TIME_REAL
; list_db = $LIST_DB
; log_db = $LOG_DB
; log_handler = $LOG_HANDLER
; log_level = $LOG_LEVEL
; logfile = $LOGFILE
; longpolling_port = $LONGPOLLING_PORT
; max_cron_threads = $MAX_CRON_THREADS
; osv_memory_age_limit = $OSV_MEMORY_AGE_LIMIT
; osv_memory_count_limit = $OSV_MEMORY_COUNT_LIMIT
; smtp_password = $SMTP_PASSWORD
; smtp_port = $SMTP_PORT
; smtp_server = $SMTP_SERVER
; smtp_ssl = $SMTP_SSL
; smtp_user = $SMTP_USER
; workers = $WORKERS
; xmlrpc = $XMLRPC
; xmlrpc_interface = $XMLRPC_INTERFACE
; xmlrpc_port = $XMLRPC_PORT
; xmlrpcs = $XMLRPCS
; xmlrpcs_interface = $XMLRPCS_INTERFACE
; xmlrpcs_port = $XMLRPCS_PORT

A user can hard-code a value, or pull the value from the environment.

For example:

[options]
...
workers = 5
...

will be a hard-coded value in the config and remain untouched by these changes, but:

[options]
...
workers = $WORKERS
...

will set the value from the environment variable.

This can be accomplished by adding the gettext library to Ubuntu, which contains envsubst. To do this, edit the Dockerfile:

RUN apt-get update && \
    DEBIAN_FRONTEND=noninteractive \
    apt-get install -y --no-install-recommends \
...   
        gettext \
...
        ca-certificates \
        curl \
        npm \

We can then update the entrypoint.sh file to read these variables. You do not need to read them in this file, but this is how we can specify defaults for the configuration items. I have specified the defaults from the current odoo.conf:

#!/bin/bash

set -e

# Setup default configuration values
export ADDONS_PATH=${ADDONS_PATH:-/mnt/extra-addons}
export DATA_DIR=${DATA_DIR:-/var/lib/odoo}
export DB_HOST=${HOST:=${DB_PORT_5432_TCP_ADDR:='db'}}
export DB_PORT=${PORT:=${DB_PORT_5432_TCP_PORT:=5432}}
export DB_USER=${USER:=${DB_ENV_POSTGRES_USER:=${POSTGRES_USER:='odoo'}}}
export DB_PASSWORD=${PASSWORD:=${DB_ENV_POSTGRES_PASSWORD:=${POSTGRES_PASSWORD:='odoo'}}}
export DB_NAME=${DB_NAME:-'postgres'}
export ADMIN_PASSWD=${ADMIN_PASSWD:-admin}
export CSV_INTERNAL_SEP=${CSV_INTERNAL_SEP:-,}
export DB_MAXCONN=${DB_MAXCONN:-64}
export DB_TEMPLATE=${DB_TEMPLATE:-template1}
export DBFILTER=${DBFILTER:-.*}
export DEBUG_MODE=${DEBUG_MODE:-False}
export EMAIL_FROM=${EMAIL_FROM:-False}
export LIMIT_MEMORY_HARD=${LIMIT_MEMORY_HARD:-2684354560}
export LIMIT_MEMORY_SOFT=${LIMIT_MEMORY_SOFT:-2147483648}
export LIMIT_REQUEST=${LIMIT_REQUEST:-8192}
export LIMIT_TIME_CPU=${LIMIT_TIME_CPU:-60}
export LIMIT_TIME_REAL=${LIMIT_TIME_REAL:-120}
export LIST_DB=${LIST_DB:-True}
export LOG_DB=${LOG_DB:-False}
export LOG_HANDLER=${LOG_HANDLER:-[:INFO]}
export LOG_LEVEL=${LOG_LEVEL:-info}
export LOGFILE=${LOGFILE:-None}
export LONGPOLLING_PORT=${LONGPOLLING_PORT:-8072}
export MAX_CRON_THREADS=${MAX_CRON_THREADS:-2}
export OSV_MEMORY_AGE_LIMIT=${OSV_MEMORY_AGE_LIMIT:-1.0}
export OSV_MEMORY_COUNT_LIMIT=${OSV_MEMORY_COUNT_LIMIT:-False}
export SMTP_PASSWORD=${SMTP_PASSWORD:-False}
export SMTP_PORT=${SMTP_PORT:-25}
export SMTP_SERVER=${SMTP_SERVER:-localhost}
export SMTP_SSL=${SMTP_SSL:-False}
export SMTP_USER=${SMTP_USER:-False}
export WORKERS=${WORKERS:-0}
export XMLRPC=${XMLRPC:-True}
export XMLRPC_INTERFACE=${XMLRPC_INTERFACE:-}
export XMLRPC_PORT=${XMLRPC_PORT:-8069}
export XMLRPCS=${XMLRPCS:-True}
export XMLRPCS_INTERFACE=${XMLRPCS_INTERFACE:-}
export XMLRPCS_PORT=${XMLRPCS_PORT:-8071}

# Set the password file environment variable
if [ -v PASSWORD_FILE ]; then
    DB_PASSWORD="$(< $PASSWORD_FILE)"
fi

# Substitute environment variables into the config file
# and write them back to the Odoo config
export CONFIG_CONTENT;
CONFIG_CONTENT=$(envsubst < /etc/odoo/odoo.conf)
echo "$CONFIG_CONTENT" > /etc/odoo/odoo.conf

DB_ARGS=()

case "$1" in
    -- | odoo)
        shift
        if [[ "$1" == "scaffold" ]] ; then
            exec odoo "$@"
        else
            wait-for-psql.py --timeout=30
            exec odoo "$@"
        fi
        ;;
    -*)
        wait-for-psql.py --timeout=30
        exec odoo "$@"
        ;;
    *)
        exec "$@"
esac

exit 1

This results in the following configuration file:

[options]
addons_path = /mnt/extra-addons
data_dir = /var/lib/odoo
db_host = db
db_port = 5432
db_user = odoo
db_password = odoo
; db_name = postgres

; admin_passwd = admin
; csv_internal_sep = ,
; db_maxconn = 64
; db_template = template1
; dbfilter = .*
; debug_mode = False
; email_from = False
; limit_memory_hard = 2684354560
; limit_memory_soft = 2147483648
; limit_request = 8192
; limit_time_cpu = 60
; limit_time_real = 120
; list_db = True
; log_db = False
; log_handler = [:INFO]
; log_level = info
; logfile = None
; longpolling_port = 8072
; max_cron_threads = 2
; osv_memory_age_limit = 1.0
; osv_memory_count_limit = False
; smtp_password = False
; smtp_port = 25
; smtp_server = localhost
; smtp_ssl = False
; smtp_user = False
; workers = 0
; xmlrpc = True
; xmlrpc_interface = 
; xmlrpc_port = 8069
; xmlrpcs = True
; xmlrpcs_interface = 
; xmlrpcs_port = 8071

Users can uncomment or hard-code the value in the .conf file. This lets them pick what they want to come from the environment. By default, its just the database details.

You then need to change the wait-for-psql.py file to read those values instead of the args as it was doing before. You can read the resulting config file with the configparser library and get the database details directly from the odoo config. This way, if the user uploads their own odoo.conf they do not need to re-specify the details:

#!/usr/bin/env python3
import argparse
import os
import psycopg2
import sys
import time
import configparser
import logging

logging.basicConfig(
    format="%(asctime)s - %(levelname)s - %(message)s",
    level=logging.DEBUG
)

class DatabaseConnectionError(Exception):
    pass

if __name__ == '__main__':
    default_config_path = os.getenv('ODOO_RC', '/etc/odoo/odoo.conf')

    arg_parser = argparse.ArgumentParser()
    arg_parser.add_argument('--config', type=str, default=default_config_path)
    arg_parser.add_argument('--timeout', type=int, default=5)

    args = arg_parser.parse_args()

    config_path = args.config or default_config_path

    config = configparser.ConfigParser()
    config.read(config_path)

    db_host = config.get('options', 'db_host', fallback='localhost')
    db_port = config.get('options', 'db_port', fallback=5432)
    db_user = config.get('options', 'db_user', fallback='odoo')
    db_password = config.get('options', 'db_password', fallback='odoo')
    db_name = config.get('options', 'db_name', fallback='postgres')

    start_time = time.time()

    database_list = db_name.split(',')
    logging.info("Waiting for database(s) to be ready ...")
    logging.info(f"Host: {db_user}@{db_host}:{db_port}")
    logging.info(f"Database(s): {database_list}")
    logging.info(f"Timeout: {args.timeout} seconds")

    for database in database_list:
        logging.info(f"Checking database {database} ...")
        error = None

        while (time.time() - start_time) < args.timeout:
            try:
                conn = psycopg2.connect(
                    user=db_user,
                    host=db_host,
                    port=db_port,
                    password=db_password,
                    dbname=database
                )

                error = None
                conn.close()

                logging.info(f"Database {database} is ready.")
                break
            except psycopg2.OperationalError as e:
                error = e
                time.sleep(1)

        if error:
            raise DatabaseConnectionError(f"Database {database} connection failure: {error}")

    logging.info("🚀 Database(s) are ready.")

This version supports loading the configuration from the ODOO_RC environment variable, or will default to the /etc/odoo/odoo.conf

Finally, you can now configure basically everything from docker compose or ECS's in-browser environment variables, or you can specify the details from something like a secret manager or even aws s3.

Un-comment the configuration you want to come from the .env in the odoo.conf file, and you're good to go.

version: '3.8'

services:
  odoo:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: odoo
    environment:
      ADDONS_PATH: ${ADDONS_PATH:-/mnt/extra-addons}
      DATA_DIR: ${DATA_DIR:-/var/lib/odoo}
      DB_HOST: ${HOST:-${DB_PORT_5432_TCP_ADDR:-postgres}}
      DB_PORT: ${PORT:-${DB_PORT_5432_TCP_PORT:-5432}}
      DB_USER: ${USER:-${DB_ENV_POSTGRES_USER:-${POSTGRES_USER:-odoo}}}
      DB_PASSWORD: ${PASSWORD:-${DB_ENV_POSTGRES_PASSWORD:-${POSTGRES_PASSWORD:-odoo}}}
      DB_NAME: ${DB_NAME:-${DATABASE:-${DB_ENV_POSTGRES_DATABASE:-postgres}}}
      ADMIN_PASSWD: ${ADMIN_PASSWD:-admin}
      CSV_INTERNAL_SEP: ${CSV_INTERNAL_SEP:-,}
      DB_MAXCONN: ${DB_MAXCONN:-64}
      DB_TEMPLATE: ${DB_TEMPLATE:-template1}
      DBFILTER: ${DBFILTER:-.*}
      DEBUG_MODE: ${DEBUG_MODE:-False}
      EMAIL_FROM: ${EMAIL_FROM:-False}
      LIMIT_MEMORY_HARD: ${LIMIT_MEMORY_HARD:-2684354560}
      LIMIT_MEMORY_SOFT: ${LIMIT_MEMORY_SOFT:-2147483648}
      LIMIT_REQUEST: ${LIMIT_REQUEST:-8192}
      LIMIT_TIME_CPU: ${LIMIT_TIME_CPU:-60}
      LIMIT_TIME_REAL: ${LIMIT_TIME_REAL:-120}
      LIST_DB: ${LIST_DB:-True}
      LOG_DB: ${LOG_DB:-False}
      LOG_HANDLER: ${LOG_HANDLER:-[:INFO]}
      LOG_LEVEL: ${LOG_LEVEL:-info}
      LOGFILE: ${LOGFILE:-None}
      LONGPOLLING_PORT: ${LONGPOLLING_PORT:-8072}
      MAX_CRON_THREADS: ${MAX_CRON_THREADS:-2}
      OSV_MEMORY_AGE_LIMIT: ${OSV_MEMORY_AGE_LIMIT:-1.0}
      OSV_MEMORY_COUNT_LIMIT: ${OSV_MEMORY_COUNT_LIMIT:-False}
      SMTP_PASSWORD: ${SMTP_PASSWORD:-False}
      SMTP_PORT: ${SMTP_PORT:-25}
      SMTP_SERVER: ${SMTP_SERVER:-localhost}
      SMTP_SSL: ${SMTP_SSL:-False}
      SMTP_USER: ${SMTP_USER:-False}
      WORKERS: ${WORKERS:-0}
      XMLRPC: ${XMLRPC:-True}
      XMLRPC_INTERFACE: ${XMLRPC_INTERFACE:-}
      XMLRPC_PORT: ${XMLRPC_PORT:-8069}
      XMLRPCS: ${XMLRPCS:-True}
      XMLRPCS_INTERFACE: ${XMLRPCS_INTERFACE:-}
      XMLRPCS_PORT: ${XMLRPCS_PORT:-8071}
    ports:
      - "8069:8069"
      - "8071:8071"
      - "8072:8072"
    volumes:
      - odoo_data:/var/lib/odoo
      - ./addons:/mnt/extra-addons
    depends_on:
      - db

  db:
    image: postgres:13
    container_name: odoo_db
    environment:
      POSTGRES_USER: ${POSTGRES_USER:-odoo}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-odoo}
      POSTGRES_DB: ${POSTGRES_DATABASE:-postgres}
    volumes:
      - pg_data:/var/lib/postgresql/data
    ports:
      - "5432:5432"

volumes:
  odoo_data:
  pg_data:

I am wrapping up these changes and can submit this as a PR, but it would be good to have some additional thoughts about this, and if you're open to the change.

As it stands these changes seem to be working fine:

Does anyone else have any thoughts about this or why it shouldn't be done?