Merge pull request #133 from dlumbrer/integration-6.8.6

Add docker image for secured version and README with the details

README.md

@@ -1,54 +1,96 @@
-# Kibana
+# Kibiter 6.8.6
 
-Kibana is your window into the [Elastic Stack](https://www.elastic.co/products). Specifically, it's a browser-based analytics and search dashboard for Elasticsearch.
+Kibiter is a custom soft fork of [Kibana](https://github.com/elastic/kibana) which empowers [GrimoireLab](https://chaoss.github.io/grimoirelab/) Panels with metrics & data visualizations.
 
-- [Getting Started](#getting-started)
-  - [Using a Kibana Release](#using-a-kibana-release)
-  - [Building and Running Kibana, and/or Contributing Code](#building-and-running-kibana-andor-contributing-code)
-- [Documentation](#documentation)
-- [Version Compatibility with Elasticsearch](#version-compatibility-with-elasticsearch)
-- [Questions? Problems? Suggestions?](#questions-problems-suggestions)
+- [Installation](#installation)
+- [Features](#features)
+- [Compatibility with Elasticsearch](#compatibility-with-elasticsearch)
+- [Contributing](#contributing)
+- [License](#license)
 
-## Getting Started
+## Installation
 
-If you just want to try Kibana out, check out the [Elastic Stack Getting Started Page](https://www.elastic.co/start) to give it a whirl.
+There are several ways for installing Kibiter on your system: from releases, Docker images or source code.
 
-If you're interested in diving a bit deeper and getting a taste of Kibana's capabilities, head over to the [Kibana Getting Started Page](https://www.elastic.co/guide/en/kibana/current/getting-started.html).
+### Releases
 
-### Using a Kibana Release
+- Go to [release tab](https://github.com/chaoss/grimoirelab-kibiter/releases) and download the version you want.
 
-If you want to use a Kibana release in production, give it a test run, or just play around:
+### Docker images
 
-- Download the latest version on the [Kibana Download Page](https://www.elastic.co/downloads/kibana).
-- Learn more about Kibana's features and capabilities on the
-[Kibana Product Page](https://www.elastic.co/products/kibana).
-- We also offer a hosted version of Kibana on our
-[Cloud Service](https://www.elastic.co/cloud/as-a-service).
+There are four Docker images of Kibiter, they have the following tags:
 
-### Building and Running Kibana, and/or Contributing Code
+- `bitergia/kibiter:community-v6.8.6-X` (being X the version of the release), the image that corresponds to the community version of Kibiter.
+- `bitergia/kibiter:optimized-v6.8.6-X` (being X the version of the release), the image that corresponds to the optimized version of Kibiter.
+- `bitergia/kibiter:secured-v6.8.6-X` (being X the version of the release), the image that corresponds to the secured (with Search Guard) version of Kibiter.
 
-You may want to build Kibana locally to contribute some code, test out the latest features, or try
-out an open PR:
+#### Docker env variables for the secured version
 
-- [CONTRIBUTING.md](CONTRIBUTING.md) will help you get Kibana up and running.
-- If you would like to contribute code, please follow our [STYLEGUIDE.md](STYLEGUIDE.md).
-- Learn more about our UI code with [UI_SYSTEMS.md](src/ui/public/UI_SYSTEMS.md).
-- For all other questions, check out the [FAQ.md](FAQ.md) and
-[wiki](https://github.com/elastic/kibana/wiki).
+There are docker env variables for the secured image (`bitergia/kibiter:secured-v6.8.6-X`) that should be defined:
 
-## Documentation
+- `ELASTICSEARCH_URL`: This env variable defines the URL of the ElasticSearch that Kibiter will connect.
+- `BASE_PATH`: Enables you to specify a path to mount Kibiter at if you are running behind a proxy.
+- `PROJECT_NAME`: The name of the project that will be in the menu top bar and the page title.
+- `ELASTICSEARCH_USER`: Username that will use Kibiter to connect to ElasticSearch.
+- `ELASTICSEARCH_PASSWORD`: Password of the username that will use Kibiter to connect to ElasticSearch.
+- `ANONYMOUS_USER`: If true, the anonymous user will be activated. If not defined or false, the anonymous user will be deactivated. ElasticSearch with SearchGuard plugin must have activated the anonymous authentication (`ANONYMOUS_USER` if using [Bitergia](https://github.com/Bitergia/elasticsearch) ElasticSearch image).
+- `LOGIN_BRANDIMAGE`: This env variable must be an URL to an image, this image will be the logo of the login form. If not defined, the Bitergia logo will appear.
+- `LOGIN_TITLE`: A string (or HTML) that will define the title of the login. If not defined, the title will be: `Please login to Bitergia Analytics Dashboard`.
+- `LOGIN_SUBTITLE`: A string (or HTML) that will define the subtitle of the login. If not defined, the subtitle will be: `If you have forgotten your username or password, please contact the <a href="mailto:[email protected]?Subject=Credentials" target="_top">Bitergia staff</a>`.
 
-Visit [Elastic.co](http://www.elastic.co/guide/en/kibana/current/index.html) for the full Kibana documentation.
+### Source code
 
-For information about building the documentation, see the README in [elastic/docs](https://github.com/elastic/docs).
+Clone the repository from the branch `integration-6.8.6-<version>`, where version can be empty or //community//.
+```
+git clone https://github.com/chaoss/grimoirelab-kibiter -b integration-6.8.6-<version>
+```
 
-## Version Compatibility with Elasticsearch
+Install the npm dependencies
 
-Ideally, you should be running Elasticsearch and Kibana with matching version numbers. If your Elasticsearch has an older version number or a newer _major_ number than Kibana, then Kibana will fail to run. If Elasticsearch has a newer minor or patch number than Kibana, then the Kibana Server will log a warning.
+```
+cd grimoirelab-kibiter
+yarn kbn bootstrap
+```
 
-_Note: The version numbers below are only examples, meant to illustrate the relationships between different types of version numbers._
+Launch Kibiter
+```
+./bin/kibana --oss
+```
 
-| Situation                 | Example Kibana version     | Example ES version | Outcome |
+## Features
+
+Kibiter provides several features, not present in Kibana, that have been developed for GrimoireLab. The most important ones are described below.
+
+### Panel menu
+
+If you are using Kibiter with the [GrimoireLab](https://chaoss.github.io/grimoirelab/) tools, you will see a quick menu at the top of the page, like the one below:
+
+<img alt="Panel menu" src="https://i.imgur.com/6hO4aEV.png">
+
+
+This menu allows you to navigate through the GrimoireLab panels, see its structure below:
+
+<img alt="Panel menu opened" src="https://i.imgur.com/9yimD9m.png">
+
+### New visualization plugins
+
+Kibiter has several plugins installed by default, they improve the user customization and add more information to the dashboards. Clearly, they are all open source. The plugins are listed below:
+
+- [Network plugin](https://github.com/dlumbrer/kbn_network) supports data visualization in a graph-style way.
+- [Searchtables plugin](https://github.com/dlumbrer/kbn_searchtables) improves Kibiter tables by adding a search box to perform searches without applying filters.
+- [Radar plugin](https://github.com/dlumbrer/kbn_radar) allows to explore the data using radar visualizations.
+- [Dot plot plugin](https://github.com/dlumbrer/kbn_dotplot) empowers Kibiter with dot-plot visualizations, granting  to add metrics in both X and Y axis.
+- [Polar plugin](https://github.com/dlumbrer/kbn_network) enhances Kibiter with polar visualizations for your data.
+- [Enhanced table](https://github.com/fbaligand/kibana-enhanced-table) This Kibtier visualization plugin uses the Data Table, but with enhanced features like computed columns, filter bar and pivot table.
+
+## Version compatibility with Elasticsearch
+
+Following the Kibana docs, you should be running Elasticsearch and Kibiter with matching version numbers. However, Kibiter will run (and log a warning) in case your Elasticsearch has a newer minor or patch number. 
+Note that Kibiter won't be able to run, if your Elasticsearch has an older version number or a newer _major_ number. 
+
+The table below shows some examples to illustrate the relationships between different types of version numbers.
+
+| Situation                 | Example Kibiter version     | Example ES version | Outcome |
 | ------------------------- | -------------------------- |------------------- | ------- |
 | Versions are the same.    | 5.1.2                      | 5.1.2              | 💚 OK      |
 | ES patch number is newer. | 5.1.__2__                  | 5.1.__5__          | ⚠️ Logged warning      |
@@ -58,9 +100,12 @@ _Note: The version numbers below are only examples, meant to illustrate the rela
 | ES minor number is older. | 5.__1__.2                  | 5.__0__.0          | 🚫 Fatal error      |
 | ES major number is older. | __5__.1.2                  | __4__.0.0          | 🚫 Fatal error      |
 
-## Questions? Problems? Suggestions?
+## Contributing
+
+We happily accept contributions, and we will help you in case you need. We follow the same contribution process that Kibana provides, thus have a look at:
+
+- [CONTRIBUTING.md](CONTRIBUTING.md) if you want to get Kibiter up and running.
+- [STYLEGUIDE.md](STYLEGUIDE.md) if you plan to submit a pull request.
+- [GitHub issue tracker](https://github.com/chaoss/grimoirelab-kibiter/issues) for all other questions, we will answer you as soon as possible.
 
-- If you've found a bug or want to request a feature, please create a [GitHub Issue](https://github.com/elastic/kibana/issues/new).
-Please check to make sure someone else hasn't already created an issue for the same topic.
-- Need help using Kibana? Ask away on our [Kibana Discuss Forum](https://discuss.elastic.co/c/kibana) and a fellow community member or
-Elastic engineer will be glad to help you out.
+If you find a bug or want to request a new feature, please open a issue on [GitHub](https://github.com/chaoss/grimoirelab-kibiter/issues). To avoid duplicated issues, check the existing issues to make sure someone else hasn't already created a similar one.

docker/Dockerfile

@@ -0,0 +1,120 @@
+FROM debian:stable
+
+MAINTAINER David Moreno <[email protected]>
+# Image for developing Kibiter/Kibana
+
+# nvm environment variables
+ENV NVM_DIR /usr/local/nvm
+ENV NODE_VERSION 10.15.2
+
+# add our user and group first to make sure their IDs get assigned consistently
+RUN groupadd --system kibana && \
+    useradd --system --create-home --gid kibana kibana
+
+# backports used to install java 8 needed by elasticsearch 5
+# ruby gems are used for building binary packages
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends \
+            ca-certificates dirmngr gnupg \
+            curl \
+            wget \
+            unzip \
+            git \
+            python \
+            zip \
+            make \
+            g++ \
+            ruby-dev rpm rubygems \
+            libffi-dev
+    # Install nvm and npm 
+RUN curl -o /tmp/nvm-install.sh -sSL https://raw.githubusercontent.com/creationix/nvm/v0.31.0/install.sh && \
+    chmod +x /tmp/nvm-install.sh && \
+    bash -c /tmp/nvm-install.sh && \
+    . /root/.bashrc && \
+    nvm install $NODE_VERSION
+    # add node and npm to path so the commands are available
+ENV NODE_PATH $NVM_DIR/v$NODE_VERSION/lib/node_modules
+ENV PATH $NVM_DIR/versions/node/v$NODE_VERSION/bin:$PATH
+    # Install kibiter
+    # gems needed in Kibana > 5.x to build binary packages
+RUN gem install rake && \
+    gem install ffi && \
+    gem install fpm -v 1.5.0 && \
+    gem install pleaserun -v 0.0.24
+    # Download Kibiter release
+RUN curl -o /tmp/kibiter-6.8.6-linux-x86_64-1.tar.gz -sSL https://github.com/chaoss/grimoirelab-kibiter/releases/download/secured-v6.8.6-1/kibiter-6.8.6-linux-x86_64-1.tar.gz && \
+    tar xfz /tmp/kibiter-6.8.6-linux-x86_64-1.tar.gz --directory /opt && \
+    mv /opt/kibiter-6.8.6-linux-x86_64 /opt/kibana && \
+    chown -R kibana:kibana /opt/kibana
+    # Install dotplot plugin: https://github.com/dlumbrer/kbn_dotplot
+RUN cd /opt/kibana/plugins && \
+    git clone https://github.com/dlumbrer/kbn_dotplot.git -b 6-dev && \
+    cd kbn_dotplot && \
+    npm install
+    # Install polar plugin: https://github.com/dlumbrer/kbn_polar
+RUN cd /opt/kibana/plugins && \
+    git clone https://github.com/dlumbrer/kbn_polar.git -b 6-dev && \
+    cd kbn_polar && \
+    npm install
+    # Install radar plugin: https://github.com/dlumbrer/kbn_radar
+RUN cd /opt/kibana/plugins && \
+    git clone https://github.com/dlumbrer/kbn_radar.git -b 6-dev && \
+    cd kbn_radar && \
+    npm install
+    # Install network plugin: https://dlumbrer.github.io/kbn_network/
+RUN cd /opt/kibana/plugins && \
+    git clone https://github.com/dlumbrer/kbn_network.git network_vis -b 6-dev && \
+    cd network_vis && \
+    npm install
+    # Install kibana-enchanced table plugin: https://github.com/fbaligand/kibana-enhanced-table
+RUN cd /opt/kibana && \
+    ./bin/kibana-plugin install https://github.com/fbaligand/kibana-enhanced-table/releases/download/v1.8.0/enhanced-table-1.8.0_6.8.6.zip
+    # Install Search-Guard Custom Bitergia plugin: https://github.com/Bitergia/search-guard-kibana-plugin
+RUN cd /opt/kibana && \
+    ./bin/kibana-plugin install https://releases.floragunn.com/search-guard-kibana-plugin-6/6.8.6-19.0/search-guard-kibana-plugin-6-6.8.6-19.0.zip
+    # Add Search-Guard configuration to kibana.yml
+RUN echo 'elasticsearch.ssl.verificationMode: none' >> /opt/kibana/config/kibana.yml && \
+    echo 'searchguard.auth.type: "basicauth"' >> /opt/kibana/config/kibana.yml && \
+    echo '#searchguard.auth.anonymous_auth_enabled:' >> /opt/kibana/config/kibana.yml && \
+    echo '#searchguard.basicauth.login.brandimage:' >> /opt/kibana/config/kibana.yml && \
+    echo '#searchguard.basicauth.login.title:' >> /opt/kibana/config/kibana.yml && \
+    echo '#searchguard.basicauth.login.subtitle:' >> /opt/kibana/config/kibana.yml && \
+    cd /root
+    # cleanup to thin the final image
+RUN rm -rf /kibiter && \
+    apt-get purge -y g++ git make python zip wget && \
+    apt-get autoremove --purge -y && \
+    apt-get clean && \
+    find /var/lib/apt/lists -type f -delete && \
+    rm -rf /root/.npm /root/.node-gyp /root/.nvm /tmp/* /var/tmp/* && \
+    bash -c 'find /usr/share/locale -maxdepth 1 -mindepth 1 -type d | grep -v -e "en_US" | xargs rm -rfv' && \
+    bash -c 'localedef --list-archive | grep -v -e "en_US" | xargs localedef --delete-from-archive' && \
+    rm -rf /usr/share/doc/*
+
+# grab gosu for easy step-down from root
+RUN ( gpg --batch --keyserver hkps://hkps.pool.sks-keyservers.net  --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 \
+     || gpg --batch --keyserver pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 \ 
+     || gpg --batch --keyserver hkp://keys.gnupg.net:80  --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 )
+RUN arch="$(dpkg --print-architecture)" \
+	&& set -x \
+	&& curl -o /usr/local/bin/gosu -sSL "https://github.com/tianon/gosu/releases/download/1.3/gosu-$arch" \
+	&& curl -o /usr/local/bin/gosu.asc -sSL "https://github.com/tianon/gosu/releases/download/1.3/gosu-$arch.asc" \
+	&& gpg --verify /usr/local/bin/gosu.asc \
+	&& rm /usr/local/bin/gosu.asc \
+	&& chmod +x /usr/local/bin/gosu
+
+ENV PATH /opt/kibana/bin:$PATH
+
+# Remove kibana optimize so it is regenerated when we modify src files in containers
+# Don't delete because we don't modify anymore src in docker-entrypoint.sh
+RUN rm -rf /opt/kibana/optimize/*
+RUN cat /opt/kibana/config/kibana.yml
+
+RUN ./opt/kibana/bin/kibana | sleep 200 > /tmp/run_kibana_log.txt
+
+COPY ./docker_entrypoint.sh /
+RUN ["chmod", "+x", "/docker_entrypoint.sh"]
+
+EXPOSE 5601
+ENTRYPOINT ["/docker_entrypoint.sh"]
+CMD ["kibana"]

docker/docker_entrypoint.sh

@@ -0,0 +1,66 @@
+#!/bin/bash
+
+set -e
+
+# Add kibana as command if needed
+if [[ "$1" == -* ]]; then
+	set -- kibana "$@"
+fi
+
+# Run as user "kibana" if the command is "kibana"
+if [ "$1" = 'kibana' ]; then
+	if [ "$ELASTICSEARCH_URL" -o "$ELASTICSEARCH_PORT_9200_TCP" ]; then
+		: ${ELASTICSEARCH_URL:='http://elasticsearch:9200'}
+		sed -ri "s!^(\#\s*)?(elasticsearch\.hosts:).*!\2 ['$ELASTICSEARCH_URL']!" /opt/kibana/config/kibana.yml
+	else
+		echo >&2 'warning: missing ELASTICSEARCH_PORT_9200_TCP or ELASTICSEARCH_URL'
+		echo >&2 '  Did you forget to --link some-elasticsearch:elasticsearch'
+		echo >&2 '  or -e ELASTICSEARCH_URL=http://some-elasticsearch:9200 ?'
+		echo >&2
+	fi
+    sed -e "s|^#server.host: .*$|server.host: 0.0.0.0|" -i /opt/kibana/config/kibana.yml
+
+    if [ "$BASE_PATH" != "" ]; then
+        sed -e "s|^#server.basePath: \"\".*$|server.basePath: \"$BASE_PATH\"|" -i /opt/kibana/config/kibana.yml
+    fi
+
+    if [ "$PROJECT_NAME" != "" ]; then
+        sed -e "s/title: 'Kibana',$/title: '$PROJECT_NAME',/" -i /opt/kibana/src/legacy/core_plugins/kibana/index.js
+        sed -e "s|__PROJECT__|$PROJECT_NAME|" -i /opt/kibana/src/ui/ui_render/views/chrome.pug
+    fi
+
+    if [ "$ELASTICSEARCH_USER" != "" -a "$ELASTICSEARCH_PASSWORD" != "" ]; then
+        sed -e "s|^#elasticsearch.username:.*$|elasticsearch.username: \"$ELASTICSEARCH_USER\"|" -i /opt/kibana/config/kibana.yml
+        sed -e "s|^#elasticsearch.password:.*$|elasticsearch.password: \"$ELASTICSEARCH_PASSWORD\"|" -i /opt/kibana/config/kibana.yml
+	else
+        echo >&2 'error: ELASTICSEARCH_USER or/and ELASTICSEARCH_PASSWORD environment variables were not configured'
+        echo >&2 '  these two docker environment variables must be configured before running the container'
+        exit 1
+    fi
+
+    if [ "$ANONYMOUS_USER" != "" ]; then
+        sed -e "s|^#searchguard.auth.anonymous_auth_enabled:.*$|searchguard.auth.anonymous_auth_enabled: \"$ANONYMOUS_USER\"|" -i /opt/kibana/config/kibana.yml
+    fi
+
+    if [ "$LOGIN_BRANDIMAGE" != "" ]; then
+        sed -e "s|^#searchguard.basicauth.login.brandimage:.*$|searchguard.basicauth.login.brandimage: \"$LOGIN_BRANDIMAGE\"|" -i /opt/kibana/config/kibana.yml
+    else
+        sed -e "s|^#searchguard.basicauth.login.brandimage:.*$|searchguard.basicauth.login.brandimage: 'https://bitergia.com/assets/img/bitergia_logo-907x227.png'|" -i /opt/kibana/config/kibana.yml
+    fi
+
+    if [ "$LOGIN_TITLE" != "" ]; then
+        sed -e "s|^#searchguard.basicauth.login.title:.*$|searchguard.basicauth.login.title: \"$LOGIN_TITLE\"|" -i /opt/kibana/config/kibana.yml
+    else
+        sed -e "s|^#searchguard.basicauth.login.title:.*$|searchguard.basicauth.login.title: Please login to Bitergia Analytics Dashboard|" -i /opt/kibana/config/kibana.yml
+    fi
+
+    if [ "$LOGIN_SUBTITLE" != "" ]; then
+        sed -e "s|^#searchguard.basicauth.login.subtitle:.*$|searchguard.basicauth.login.subtitle: \"$LOGIN_SUBTITLE\"|" -i /opt/kibana/config/kibana.yml
+    else
+        sed -e 's|^#searchguard.basicauth.login.subtitle:.*$|searchguard.basicauth.login.subtitle: If you have forgotten your username or password, please contact the <a href="mailto:[email protected]?Subject=Credentials" target="_top">Bitergia staff</a>|' -i /opt/kibana/config/kibana.yml
+    fi
+
+	set -- gosu kibana "$@"
+fi
+
+exec "$@"