This is an installation guide to setup and configure the aroskanalen display system.
The guide assumes that you have an installed linux based server with the following packages installed and at least the versions given.
- nginx 1.4.x
- redis 2.8.x
- php 5.5.x
- node 0.10.x
- elastic search 1.5.x
- supervisor 3.x
The system consists of four parts:
Admin - administration interface (symfony)
Screen - that serves content on the screens
Middleware - that handles web-sockets and connection between the ones above.
Search node - that interfaces with elastic search and provide fast content search in the administration interface.
The middleware and search node is only installed once on each server and then all administration interfaces and screens communicates with the them using API keys to identify them self and ensure content seperation.
We have chosen to use the following naming convention and place the application in /home/www/.
We have used the following naming in DNS:
- search-[server name].aroskanalen.dk
- middleware-[server name].aroskanalen.dk
- screen-[client name].aroskanalen.dk
- admin-[client name].aroskanalen.dk
And this will give the folders:
- /home/www/[client name]/admin
- /home/www/[client name]/screen
- /home/www/[client name]/logs
The Middleware and search node is located in:
- /home/www/middleware
- /home/www/search_node
If you already have a running middleware and search node on the server you can skip the steps and just add the need api-keys in both and add mappings in the search node. You can use the UI or edit the JSON files directly.
Start by cloning the git repository for the middleware and checkout the latest release tag (which can be found using git tag).
~$ cd /home/www ~$ git clone [email protected]:Indholdskanalen/middleware.git . ~$ cd middleware ~$ git checkout [v3.x.x]
The middleware is build as a NodeJS application and the different plugins require libraries to be installed. The application comes with an installation script that do this, just run it.
~$ ./install.sh
First we need to configure nginx in front of the middleware so it's accessible through normal web-ports and also proxy web-socket connections.
~$ nano -w /etc/nginx/sites-available/nodejs
In this file add an upstream connection to the middleware application. Later a connection for the search node also needs to be added to this file as well.
upstream nodejs_middleware {
server 127.0.0.1:3020;
}
To access the middleware administration and allow the screen to communication with the middleware a virtual host configuration is needed. You need to change the [server name] with the actual name of the server. You may also need to change the paths to the ssl certificates.
~$ nano -w /etc/nginx/sites-available/middleware_[server name]_aroskanalen_dk
server {
listen 80;
server_name middleware-[server name].aroskanalen.dk;
rewrite ^ https://$server_name$request_uri? permanent;
access_log /var/log/nginx/middleware_access.log;
error_log /var/log/nginx/middleware_error.log;
}
# HTTPS server
#
server {
listen 443;
server_name middleware-[server name].aroskanalen.dk;
access_log /var/log/nginx/middleware_access.log;
error_log /var/log/nginx/middleware_error.log;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_buffering off;
proxy_pass http://nodejs_middleware/;
proxy_redirect off;
}
location /socket.io/ {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_pass http://nodejs_middleware;
}
ssl on;
ssl_certificate /etc/ssl/certs2014/aroskanalen_dk.crt;
ssl_certificate_key /etc/ssl/certs2014/aroskanalen_dk.key;
ssl_session_timeout 5m;
ssl_session_cache shared:SSL:10m;
# https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
ssl_prefer_server_ciphers On;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS;
}
Next step is to configure the application by setting the secret (used to encode tokens) and admin password. If you have other applications on the server that's using Redis, you can change the default database used (1-15 is available).
~$ cd /home/www/middleware ~$ nano -w config.json
{
"port": 3020,
"secret": "[CHANGE ME]",
"log": {
"file": "messages.log",
"debug": false
},
"admin": {
"username": "admin",
"password": "[CHANGE ME]"
},
"cache": {
"port": "6379",
"host": "localhost",
"auth": null,
"db": 1
},
"apikeys": "apikeys.json"
}
To get the application running you need to create the apikeys.json file with an empty JSON object or optionally added the demo key below.
~$ cd /home/www/middleware ~$ nano -w apikeys.json
Demo content for the apikeys.json file.
{
"03303f8bf868b517bb732326aca86714": {
"name": "Demo",
"expire": 300,
"backend": "https://admin-demo.aroskanalen.dk"
}
}
Enable nginx configuration by adding symbolic links into sites-enabled folder and restarting the service.
~$ cd etc/nginx/sites-enabled/ ~$ ln -s ../sites-available/middleware_[server name]_aroskanalen_dk ~$ ln -s ../sites-available/nodejs ~$ service nginx restart
The middleware needs to be started at boot time which requires a Supervisor run script. Supervisor will also ensure that the node application is started again, if an error happens and it stops running.
~$ nano -w /etc/supervisor/conf.d/middleware.conf
The run script to keep the middleware running.
[program:middleware] command=/usr/bin/node /home/www/middleware/app.js autostart=true autorestart=true environment=NODE_ENV=production stderr_logfile=/var/log/middleware.err.log stdout_logfile=/var/log/middleware.out.log user=deploy
~$ service supervisor restart
@TODO: How to use the UI to add more configuration ap-keys.
The search node application is a general purpose application to provide a fast search engine through web-socket connections (using elasticsearch) and is designed to be used by other projects as well as aroskanalen. As a result of this, it will not follow the same version number as the other parts of this installation. It's also why the administration UI is somewhat complex to use when setting up the right mappings to be used with aroskanalen.
Start by cloning the git repository for the search node and checkout the latest release tag (which can be found using git tag).
~$ cd /home/www ~$ git clone [email protected]:Indholdskanalen/search_node.git home/www/search_node ~$ cd search_node ~$ git checkout [v1.x.x]
Search node, as the middleware, uses a plugin architecture that requires installation of libraries from node package manager. The application comes with an installation script to handle this, simply go to the root of the application and execute the script.
~$ ./install.sh
We need to configure nginx in front of the search node so it is accessible through normal web-ports and also proxy web-socket connections. So we add the upstream connection configuration in the nodejs config file used by the middleware
~$ nano -w /etc/nginx/sites-available/nodejs
The upstream connection definition.
upstream nodejs_search {
server 127.0.0.1:3010;
}
To access the search node administration and allow the aroskanalen administration interface to communication with the search node a virtual host configuration is needed. You need to change the [server name] with the actual name of the server. You may also need to change the paths to the ssl certificates.
~$ nano -w /etc/nginx/sites-available/search_[server name]_aroskanalen_dk
server {
listen 80;
server_name search-[server name].aroskanalen.dk;
rewrite ^ https://$server_name$request_uri? permanent;
access_log /var/log/nginx/search_access.log;
error_log /var/log/nginx/search_error.log;
}
# HTTPS server
#
server {
listen 443;
server_name search-[server name].aroskanalen.dk;
access_log /var/log/nginx/search_access.log;
error_log /var/log/nginx/search_error.log;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_buffering off;
proxy_pass http://nodejs_search/;
proxy_redirect off;
}
location /socket.io/ {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_pass http://nodejs_search;
}
ssl on;
ssl_certificate /etc/ssl/certs2014/aroskanalen_dk.crt;
ssl_certificate_key /etc/ssl/certs2014/aroskanalen_dk.key;
ssl_session_timeout 5m;
ssl_session_cache shared:SSL:10m;
# https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
ssl_prefer_server_ciphers On;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS;
}
Enable the configuration by adding a symbolic link for the search node. The nodejs configuration file should be linked under configuration of the middleware. Restart nginx to enable the configuration.
~$ cd /etc/nginx/sites-enabled/ ~$ ln -s ../sites-available/search_[server name] ~$ service nginx restart
Next the application needs to be configured by adding the following content to config.json in the root folder of the application /home/www/search___node. Remember to update the administration password and secret.
{
"port": 3010,
"secret": "[CHANGE ME]",
"admin": {
"username": "admin",
"password": "[PASSWORD]"
},
"log": {
"file": "messages.log",
"debug": false
},
"search": {
"hosts": [ "localhost:9200" ],
"mappings": "mappings.json"
},
"apikeys": "apikeys.json"
}
Before the application can be started the apikeys.json and mappings.json needs to exist and at least contain an empty JSON object.
As the search node is not specially created for aroskanalen the mappings (configuration for elasticsearch) can be somewhat complex. To get you started the mapping below can be used as a template for the configuration. Normally, go into the administration interface and add a new api key. Then go to the mappings in the interface and add a new empty mapping. Edit the mappings file and add the fields, tag and dates section as in the template. This is the fast way to do it, if you edit the mappings after adding the config to the JSON file you can see how it should look in the administration interface.
{
"5d437a016271077510c640e450bde9c3": {
"name": "demo",
"tag": "private",
"fields": [
{
"field": "title",
"type": "string",
"language": "da",
"country": "DK",
"default_analyzer": "string_index",
"sort": true,
"indexable": true
},
{
"field": "name",
"type": "string",
"language": "da",
"country": "DK",
"default_analyzer": "string_index",
"sort": true,
"indexable": true
}
],
"dates": [
"created_at",
"updated_at"
]
}
}
Add demo API key to apikeys.json where the indexes match the ID in the mappings.json file.
{
"77f0dd4a5ff65891a6f8e5c853295c7e": {
"name": "Demo",
"expire": 300,
"indexes": [
"5d437a016271077510c640e450bde9c3"
]
}
}
The search node needs to be started at boot time which requires a Supervisor run script. Supervisor will also ensure that the node application is started again, if an error happens and it stops running.
~$ nano -w /etc/supervisor/conf.d/search_node.conf
Supervisor run script for the search node.
[program:search-node] command=node /home/www/search_node/app.js autostart=true autorestart=true environment=NODE_ENV=production stderr_logfile=/var/log/search-node.err.log stdout_logfile=/var/log/search-node.out.log user=deploy
~$ service supervisor restart
@TODO: Activate search indexes.
@TODO: How to use the UI to add more configuration.
The administration user interface is the backend of the display system and used to add content and configure screens, etc.
~$ git clone [email protected]:Indholdskanalen/admin.git /home/www/[client name]/admin
It needs to have a SQL database and in this guide we assumes that it's MySQL or MariaDB, so start by creating a database, user and give the user privileges to use the database.
~$ mysql -u root -p -e "CREATE DATABASE [site name];" ~$ mysql -u root -p -e "CREATE USER '[site name]'@'localhost' IDENTIFIED BY '[RANDOM PASSWORD]';" ~$ mysql -u root -p -e "GRANT ALL PRIVILEGES ON [DB NAME].* TO '[USER NAME]'@'localhost';" ~$ mysql -u root -p -e "FLUSH PRIVILEGES;"
Install the site using composer and fill out the questions that you can and use default values for the rest. See the parameters example below to help filling out the values.
~$ cd /home/www/[client name]/admin ~$ composer install ~$ php app/check.php
The final app/config/parameters.yml file.
parameters: database_driver: pdo_mysql database_host: 127.0.0.1 database_port: null database_name: [DB NAME] database_user: [DB USERNAME] database_password: '[DB PASSWORD]' mailer_transport: sendmail mailer_host: 127.0.0.1 mailer_user: null mailer_password: null mailer_from_email: [email protected] mailer_from_name: 'Webmaster Indholdskanalen' locale: en secret: [CHANGE ME] debug_toolbar: false debug_redirects: false use_assetic_controller: true absolute_path_to_server: 'https://admin-[CLIENT NAME].aroskanalen.dk' zencoder_api: [ZENCODER API KEY] sharing_host: 'https://search-[SERVER NAME].aroskanalen.dk' sharing_path: /api sharing_apikey: [API KEY] sharing_enabled: false search_host: 'https://search-[SERVER NAME].aroskanalen.dk' search_path: /api search_apikey: [API KEY] search_index: [INDEX KEY] middleware_host: 'https://middleware-[SERVER NAME].aroskanalen.dk' middleware_path: /api middleware_apikey: [API KEY] site_title: 'Demo Aroskanalen' koba_apikey: [KOBA API KEY] koba_path: 'http://koba.aarhus.dk'
Now install the database tables by running the update command.
~$ php app/console doctrine:schema:update --force
Add an administrator user that can be used to test the site and create other users.
~$ php app/console fos:user:create [admin_username] [[email protected]] [p@ssword] --super-admin
Templates reside in the /web/templates folder. They are separated into screen and slide templates. Each template has its own folder and .json file. To import the templates into the database run the following command from the admin folder:
~$ cd /home/www/[CLIENT NAME]_aroskanalen_dk/admin ~$ php app/console ik:templates:load
In the administration menu there is a menu option to enable/disable imported templates.
If you change a .json file, you need to run the import command again.
As with the node application the administration needs a nginx configuration to be accessible.
~$ nano -w /etc/nginx/sites-available/admin_[client name]_aroskanalen_dk
You need to update the name, paths and ssl certificates.
server {
listen 80;
server_name admin-[CLIENT NAME].aroskanalen.dk;
rewrite ^ https://$server_name$request_uri? permanent;
access_log /home/www/[CLIENT NAME]_aroskanalen_dk/logs/backend_access.log;
error_log /home/www/[CLIENT NAME]_aroskanalen_dk/logs/backend_error.log;
}
# HTTPS server
#
server {
listen 443;
server_name admin-[CLIENT NAME].aroskanalen.dk;
root /home/www/[CLIENT NAME]_aroskanalen_dk/admin/web;
client_max_body_size 300m;
access_log /home/www/[CLIENT NAME]_aroskanalen_dk/logs/backend_access.log;
error_log /home/www/[CLIENT NAME]_aroskanalen_dk/logs/backend_error.log;
location / {
# try to serve file directly, fallback to rewrite
try_files $uri @rewriteapp;
}
location /templates/ {
add_header 'Access-Control-Allow-Origin' "*";
}
location @rewriteapp {
# rewrite all to app.php
rewrite ^(.*)$ /app.php/$1 last;
}
location ~ ^/(app|app_dev|config)\.php(/|$) {
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_split_path_info ^(.+\.php)(/.*)$;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param HTTPS off;
}
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
location ~ /\.ht {
deny all;
}
location /proxy/ {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_buffering off;
proxy_pass http://nodejs_search/;
proxy_redirect off;
}
location /socket.io/ {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_pass http://nodejs_search;
}
ssl on;
ssl_certificate /etc/ssl/certs2014/aroskanalen_dk.crt;
ssl_certificate_key /etc/ssl/certs2014/aroskanalen_dk.key;
ssl_session_timeout 5m;
ssl_session_cache shared:SSL:10m;
# https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
ssl_prefer_server_ciphers On;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS;
}
Enable the configuration by adding a symbolic link for the installation and restart nginx to enable the configuration.
~$ cd /etc/nginx/sites-enabled/ ~$ ln -s ../sites-available/admin_[client name]_aroskanalen_dk ~$ service nginx restart
Login to the site and create a new screen to test that it works, you will get search errors until theres is something in the search indexes.
@TODO: WRITE INTO TEXT.
First clone the screen code.
~$ git clone [email protected]:Indholdskanalen/screen.git /home/www/[client name]/screen ~$ cd /home/www/[client name]/screen
Edit the configuration file in app/config.js and use the template below to fill out the configuration. Note that this is not a JSON file but a javascript (.js) file.
window.config = {
"resource": {
"server": "//screen-[CLIENT NAME].aroskanalen.dk/",
"uri": "proxy"
},
"ws": {
"server": "https://screen-[CLIENT NAME].aroskanalen.dk/"
},
"backend": {
"address": "https://admin-[CLIENT NAME].aroskanalen.dk/"
},
"apikey": "[API KEY]",
"cookie": {
"secure": false
},
"debug": true
};
The screen client also needs to be configured to run i nginx.
~$ nano -w /etc/nginx/sites-available/screen_[client name]_aroskanalen_dk
server {
listen 80;
server_name screen-[CLIENT NAME].aroskanalen.dk;
rewrite ^ https://$server_name$request_uri? permanent;
access_log /home/www/[CLIENT NAME]_aroskanalen_dk/logs/screen_access.log;
error_log /home/www/[CLIENT NAME]_aroskanalen_dk/logs/screen_error.log;
}
# HTTPS server
#
server {
listen 443;
server_name screen-[CLIENT NAME].aroskanalen.dk;
root /home/www/[CLIENT NAME]_aroskanalen_dk/screen;
access_log /home/www/dokk1_aroskanalen_dk/logs/screen_access.log;
error_log /home/www/dokk1_aroskanalen_dk/logs/screen_error.log;
location / {
try_files $uri $uri/ /index.html;
}
location /templates/ {
add_header 'Access-Control-Allow-Origin' "*";
}
location /proxy/ {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_buffering off;
proxy_pass http://nodejs_middleware/;
proxy_redirect off;
}
location /socket.io/ {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_pass http://nodejs_middleware;
}
ssl on;
ssl_certificate /etc/ssl/certs2014/aroskanalen_dk.crt;
ssl_certificate_key /etc/ssl/certs2014/aroskanalen_dk.key;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 5m;
# https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
ssl_prefer_server_ciphers On;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS;
}
Enable the configuration by adding a symbolic link for the installation and restart nginx to enable the configuration.
~$ cd /etc/nginx/sites-enabled/ ~$ ln -s ../sites-available/screen_[client name]_aroskanalen_dk ~$ service nginx restart
- nginx -t can be used to test configuration file.