Install Manager

Install system packages

We assume that your system is configured with a sudoable admin user named devops.

Ubuntu

Ensure your system is up-to-date:

$ sudo apt-get update
$ sudo apt-get upgrade

Install dependencies for Backend.AI daemonization:

$ sudo apt-get install -y \
  ca-certificates nginx \
  redis-server redis-tools \
  supervisor

Install dependencies for Python builds:

$ sudo apt-get install -y \
  build-essential git-core \
  libreadline-dev libsqlite3-dev libssl-dev libbz2-dev libzmq3-dev tk-dev

Here are some optional but useful packages:

$ sudo apt-get install vim tmux htop

CentOS / RHEL

(TODO)

Install latest Python

You may choose your favorite method of Python installation here, but please ensure that it installs a version that is compatible to CPython 3.6 or higher.

We highly recommend pyenv for installing multiple Python versions side-by-side, which does not interfere with system-default Pythons.

Install pyenv

NOTE: Change ~/.bash_profile accroding to your shell configuration.

$ git clone https://github.com/pyenv/pyenv.git ~/.pyenv
$ echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bash_profile
$ echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bash_profile
$ echo 'eval "$(pyenv init -)"' >> ~/.bash_profile
$ source ~/.bash_profile
$ exec $SHELL -l

Install pyenv's virtualenv Plugin

$ git clone https://github.com/pyenv/pyenv-virtualenv.git ~/.pyenv/plugins/pyenv-virtualenv
$ echo 'eval "$(pyenv virtualenv-init -)"' >> ~/.bash_profile
$ exec $SHELL -l

Install Python via pyenv

$ pyenv install 3.6.4

Install Backend.AI Manager from Source

$ pyenv virtualenv 3.6.4 v3.6.4
$ git clone --branch=master "https://github.com/lablup/backend.ai-manager" "backend.ai-manager"
$ cd backend.ai-manager
$ pyenv local v3.6.4
$ pip install -U pip setuptools
$ pip install -U -r requirements.txt

Configure NGINX

$ sudo vi /etc/nginx/sites-available/gateway

ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
ssl_ciphers EECDH+CHACHA20:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;

map http_connection connection_upgrade {
    default upgrade;
    ''      close;
}
server {
    listen 443 ssl;
    server_name ENDPOINT;
    charset utf-8;
    client_max_body_size 32M;
    ssl_certificate SSLCERT;
    ssl_certificate_key SSLPRIVKEY;
    add_header Strict-Transport-Security "max-age=31536000; includeSubdomains";
    location / {
        proxy_pass http://127.0.0.1:8081;
        proxy_pass_request_headers on;
        proxy_set_header Host "ENDPOINT";
        proxy_redirect off;
        proxy_buffering off;
        proxy_read_timeout 600s;
    }
    location ~ ^/v\d+/stream/ {
        proxy_pass http://127.0.0.1:8081;
        proxy_pass_request_headers on;
        proxy_set_header Host "ENDPOINT";
        proxy_redirect off;
        proxy_buffering off;
        proxy_read_timeout 60s;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
    }
}

$ sudo cp /etc/nginx/sites-enabled/default ../nginx.site-enabled.default
$ sudo rm /etc/nginx/sites-enabled/default
$ ln -s ../sites-available/gateway /etc/nginx/sites-enabled/gateway
$ ln -s ../sites-available/gateway /etc/nginx/sites-enabled/gateway
$ sudo systemctl restart nginx

Install Datadog agent (optional)

$ DD_API_KEY=DDAPIKEY bash -c "$(curl -L https://raw.githubusercontent.com/DataDog/dd-agent/master/packaging/datadog-agent/source/install_agent.sh)"

Load initial etcd data

Edit image-metadata.sample.yml and image-aliases.sample.yml according to your setup. By default you can pull the images listed in the sample via docker pull lablup/kernel-xxxx:tag as they are hosted on the public Docker registry.

Load image registry metadata

$ python -m ai.backend.manager.cli etcd update-kernels \
  --namespace=NS --etcd-addr=ETCDADDR \
  -f image-metadata.sample.yml

Load image aliases

$ python -m ai.backend.manager.cli etcd update-aliases \
  --namespace=NS --etcd-addr=ETCDADDR \
  -f image-aliases.sample.yml

Set the default storage mount for virtual folders

$ ETCDCTL_API=3 etcdctl put /sorna/NS/volumes/_vfroot mystorage

Configure supervisord

$ sudo vi /etc/supervisor/conf.d/apps.conf

[program:backendai-manager]
user = devops
stopsignal = TERM
stopasgroup = true
command = /home/devops/run-manager.sh

$ sudo vi /home/devops/run-manager.sh

#!/bin/bash
source /home/devops/init-venv.sh
umask 0002
export AWS_ACCESS_KEY_ID="S3AKEY"
export AWS_SECRET_ACCESS_KEY="S3SKEY"
export DATADOG_API_KEY=DDAPIKEY
export DATADOG_APP_KEY=DDAPPKEY
export RAVEN_URI="SENTRYURL"
python -m ai.backend.gateway.server \
       --db-addr=DBHOST:DBPORT \
       --db-user=DBUSER \
       --db-password='DBPASS' \
       --db-name=backend \
       --etcd-addr ETCDHOST:ETCDPORT\
       --namespace NS \
       --redis-addr REDISHOST:REDISPORT \
       --events-port 5002 \
       --service-ip 127.0.0.1 \
       --service-port 8081

Initialize database and load intial fixtures

Create a new database

$ psql -h DBHOST -U DBPASS

postgres=# CREATE DATABASE backend;
postgres=# \q

Set up database schema

Backend.AI uses alembic to manage database schema and its migration during version upgrades. First, localize the sample config:

$ cp alembic.ini.sample alembic.ini

Modify the line where sqlalchemy.url is set. You may use the following shell command: (ensure that special characters in your password are properly escaped)

$ sed -i'' -e 's!^sqlalchemy.url = .*$!sqlalchemy.url = postgresql://DBUSER:DBPASS@DBHOST/backend!' alembic.ini

$ python -m ai.backend.manager.cli schema oneshot head

NOTE: All sub-commands under "schema" uses alembic.ini to establish database connections.

Load initial fixtures

Edit ai/backend/manager/models/fixtures.py so that you have a randomized admin keypair.

**(TODO: automate here!)**

Then pour it to the database:

$ python -m ai.backend.manager.cli \
  --db-addr=DBHOST:DBPORT --db-user=DBUSER --db-password=DBPASS --db-name=backend \
  fixture populate example_keypair

Finally, Run!

$ sudo supervisorctl start backendai-manager
$ sudo service nginx restart