Elasticsearch Azure Marketplace offering

This repository consists of:

• src/mainTemplate.json - The main Azure Resource Management (ARM) template. The template itself is composed of many nested linked templates with the main template acting as the entry point.
• src/createUiDefinition - UI definition file for our Azure Marketplace offering. This file produces an output JSON that the ARM template can accept as input parameters.

Building

After pulling call npm install once, this will pull in all devDependencies.

You may edit build/allowedValues.json, which the build will use these to patch the arm template and ui definition.

Run npm run build, this will validate EditorConfig settings, validate JSON files, patch the allowedValues and then create a zip in the dist folder.

Development

New features should be developed on separate branches and merged back into master once complete. To aid in the development process, a gulp task is configured to update all of the github template urls to point at a specific branch so that UI definition and web based deployments can be tested. To run the task

npm run links

will update the links to point to the name of the current branch. Once ready to merge back into master, a specific branch name can be passed with

npm run links -- --branch master

Azure Marketplace

The Azure Marketplace Elasticsearch offering offers a simplified UI over the full power of the ARM template. It will always install a cluster complete with the X-Pack plugins Shield, Watcher and Marvel, Graph (for Elasticsearch 2.3.0+) and Reporting (for Elasticsearch 2.4.0+).

Additionally, the Azure Cloud plugin can be optionally installed to support snapshot and restore.

---

VERY IMPORTANT

This template does not configure SSL/TLS for communication with Kibana, or Elasticsearch through an external load balancer. It is strongly recommended that you secure communication before using in production.

Additionally, for Elasticsearch 2.4.0+, when X-Pack plugins and Kibana are installed, the Shield plugin is also installed in Kibana to provide login/logout functionality as well as allow the Graph UI to function correctly. The Shield plugin is configured to skip checking that requests are made over HTTPS and allow session cookies to be sent over HTTP by setting the following options in kibana.yml

shield.useUnsafeSessions: true
shield.skipSslCheck: true

It is strongly recommended that you remove these settings and secure communication to Kibana by enabling SSL before using in production.

---

You can view the UI in developer mode by clicking here. If you feel something is cached improperly use this client unoptimized link instead

Reporting bugs

Have a look at this screenshot to see how you can navigate to the deployment error status message. Please create an issue with that message and in which resource it occured on our github issues

ARM template

The output from the Azure Marketplace UI is fed directly to the ARM deployment template. You can use the ARM template on its own without going through the MarketPlace.

Parameters

Parameter	Type	Description
artifactsBaseUrl	string	The base url of the Elastic ARM template. Defaults to the raw content of the current branch.
esVersion	string	A valid supported Elasticsearch version see this list for supported versions
esClusterName	string	The name of the Elasticsearch cluster
loadBalancerType	string	The load balancer to set up to access the cluster. Can be internal or external. The default is internal. By choosing external, both internal and external load balancers will be deployed. Kibana communicates with the cluster through the internal load balancer. If you are setting up Elasticsearch on an external endpoint, you will need to secure your nodes with a product like Elastic's Shield as well as configure transport level security.
azureCloudPlugin	string	Either Yes or No to install the Azure Cloud plugin for snapshot/restore. Defaults to No. when set to Yes, both azureCloudeStorageAccountName and azureCloudStorageAccountKey should be specified to configure the plugin correctly.
azureCloudStorageAccountName	string	The name of an existing storage account to use for snapshots with Azure Cloud plugin. Must be a valid Azure Storage Account name.
azureCloudStorageAccountKey	securestring	The access key of an existing storage account to use for snapshots with Azure Cloud plugin.
esPlugins	string	Either Yes or No to install a trial license of the commercial X-Pack plugins: Shield, Watcher, Marvel and Graph (Elasticsearch 2.3.0+).
kibana	string	Either Yes or No provision an extra machine with a public IP that has Kibana installed on it. If you have opted to also install the Elasticsearch plugins using esPlugins then a trial license of the commercial X-Pack Kibana plugins as well as Sense Editor are also installed.
vmSizeKibana	string	Azure VM size of the Kibana instance. See this list for supported sizes. Defaults to Standard_A1.
jumpbox	string	Either Yes or No Optionally add a virtual machine to the deployment which you can use to connect and manage virtual machines on the internal network.
vmHostNamePrefix	string	The prefix to use for hostnames when naming virtual machines in the cluster. Hostnames are used for resolution of master nodes so if you are deploying a cluster into an existing virtual network containing an existing Elasticsearch cluster, be sure to set this to a unique prefix, to differentiate the hostnames of this cluster from an existing cluster. Can be up to 5 characters in length, must begin with an alphanumeric character and can contain alphanumeric and hyphen characters.
vmSizeDataNodes	string	Azure VM size of the data nodes. See this list for supported sizes
vmDataNodeCount	int	The number of data nodes you wish to deploy. Should be greater than 0. Defaults to 3.
dataNodesAreMasterEligible	string	Either Yes or No to make all data nodes master eligible. This can be useful for small Elasticsearch clusters however, for larger clusters it is recommended to have dedicated master nodes. When Yes no dedicated master nodes will be provisioned.
vmSizeMasterNodes	string	Azure VM size of dedicated master nodes. See this list for supported sizes. By default the template deploys 3 dedicated master nodes, unless dataNodesAreMasterEligible is set to Yes
vmClientNodeCount	int	The number of client nodes to provision. Defaults 0 and can be any positive integer. By default the data nodes are directly exposed on the loadbalancer. If you provision client nodes, only these will be added to the loadbalancer.
vmSizeClientNodes	string	Azure VM size of the client nodes see this list for supported sizes.
adminUsername	string	Admin username used when provisioning virtual machines. Must be a valid Linux username i.e. avoid any of the following usernames for Ubuntu
authenticationType	object	Either password or sshPublicKey
adminPassword	object	When authenticationType is password this sets the OS level user's password
sshPublicKey	object	When authenticationType is sshPublicKey this sets the OS level sshKey that can be used to login.
shieldAdminPassword	securestring	Shield password for the es_admin user with admin role, must be > 6 characters
shieldReadPassword	securestring	Shield password for the es_read user with user (read-only) role, must be > 6 characters
shieldKibanaPassword	securestring	Shield password for the es_kibana user with kibana4 role, must be > 6 characters
location	string	The location where to provision all the items in this template. Defaults to the special ResourceGroup value which means it will inherit the location from the resource group see this list for supported locations.
vNetNewOrExisting	string	Whether the Virtual Network is new or existing. An existing Virtual Network in another Resource Group in the same Location can be used. Defaults to new
vNetName	string	The name of the Virtual Network. Defaults to es-net
vNetSubnetName	string	The name of the subnet to which Elasticsearch nodes will be attached. Defaults to es-subnet
vNetLoadBalancerIp	string	The internal static IP address to use when configuring the internal load balancer. Must be an available IP address on the provided subnet name. Defaults to 10.0.0.4.
vNetExistingResourceGroup	string	The name of the Resource Group in which the Virtual Network resides when using an existing Virtual Network. Required when using an existing Virtual Network
vNetNewAddressPrefix	string	The address prefix when creating a new Virtual Network. Defaults to 10.0.0.0/16. Required when creating a new Virtual Network
vNetNewSubnetAddressPrefix	string	The address space of the subnet. Defaults to 10.0.0.0/24. Required when creating a new Virtual Network
userCompany	string	The name of your company.
userEmail	string	Your email address
userFirstName	string	Your first name
userLastName	string	Your last name
userJobTitle	string	Your job title. Pick the nearest one that matches from the list of job titles
userCountry	string	The country in which you are based.

Command line deploy

first make sure you are logged into azure

azure login

Then make sure you are in arm mode

azure config mode arm

Then create a resource group <name> in a <location> (e.g westeurope) where we can deploy too

azure group create <name> <location>

Next we can either use our published template directly using --template-uri

azure group deployment create --template-uri https://raw.githubusercontent.com/elastic/azure-marketplace/master/src/mainTemplate.json --parameters-file parameters/password.parameters.json -g name

or if your are executing commands from a clone of this repo using --template-file

azure group deployment create --template-file src/mainTemplate.json --parameters-file parameters/password.parameters.json -g name

<name> in these last two examples refers to the resource group you just created.

NOTE

The --parameters-file can specify a different location for the items that get provisioned inside of the resource group. Make sure these are the same prior to deploying if you need them to be. Omitting location from the parameters file is another way to make sure the resources get deployed in the same location as the resource group.

Web based deploy

The above button will take you to the autogenerated web based UI based on the parameters from the ARM template.

It should be pretty self explanatory except for password which only accepts a json object. Luckily the web UI lets you paste json in the text box. Here's an example:

{"sshPublicKey":null,"authenticationType":"password", "password":"Elastic12"}

License

This project is MIT Licensed and is based heavily on the Elasticsearch azure quick start arm template