- Deployment
- Durable cluster topology (via ZooKeeper)
- Web UI on scheduler port 8080
- Support deploying multiple Elasticsearch clusters to single Mesos cluster
- Fault tolerance
- High availability (master, indexer, replica)
- Upgrading configuration
- Scale cluster horizontally
- Scale cluster vertically
- Upgrade
- Rollback
- Snapshot and restore
- Faster task recovery with Mesos dynamic reservations (https://issues.apache.org/jira/browse/MESOS-1554)
- Persistent storage
- Local environment (Docker-machine)
- Rapid code + test (Mini Mesos)
- Build automation (Gradle)
- One click DCOS install
- One JSON post to marathon install
- DCOS Certified
We recommend that users install via marathon or via the DCOS command line (coming soon!).
This framework requires:
- A running Mesos cluster
- The use of Marathon is strongly recommended to provide resiliency against scheduler failover.
Create a Marathon file like the one below and fill in the IP addresses and other configuration.
{
"id": "elasticsearch-mesos-scheduler",
"container": {
"docker": {
"image": "mesos/elasticsearch-scheduler",
"network": "HOST"
}
},
"args": ["-n", "3", "-zk", "zk://ZOOKEEPER_IP_ADDRESS:2181/mesos", "-ram", "2048"],
"cpus": 0.2,
"mem": 512.0,
"env": {
"JAVA_OPTS": "-Xms128m -Xmx256m"
},
"instances": 1
}
Then post to marathon to instantiate the scheduler:
curl -k -XPOST -d @marathon.json -H "Content-Type: application/json" http://MARATHON_IP_ADDRESS:8080/v2/apps
Note: the JAVA_OPTS line is required. If this is not set, then the Java heap space will be incorrectly set.
The web based user interface is available on port 8080 of the scheduler by default. It displays real time information about the tasks running in the cluster and a basic configuration overview of the cluster.
The user interface uses REST API of the Elasticsearch Mesos Framework. You can find the API documentation here: docs.elasticsearchmesosui.apiary.io.
For developers, we have provided a range of tools for testing and running the project. Check out the mini-mesos project for an in-memory Mesos cluster for integration testing.
You can run Mesos-Elasticsearch using Mini Mesos, a containerized Mesos cluster for testing frameworks.
- Docker
$ ./gradlew build buildDockerImage system-test:main
- Docker Machine
$ docker-machine create -d virtualbox --virtualbox-memory 4096 --virtualbox-cpu-count 2 mesos-es
$ eval $(docker-machine env mesos-es)
$ ./gradlew build docker system-test:main
The project contains a system-test module which tests if the framework interacts correctly with Mesos, using Mini Mesos. We currently test Zookeeper discovery and the Scheduler's API by calling endpoints and verifying the results. As the framework grows we will add more system tests.
- Docker
$ ./gradlew build buildDockerImage system-test:systemTest
- Docker Machine
$ docker-machine create -d virtualbox --virtualbox-memory 4096 --virtualbox-cpu-count 2 mesos-es
$ eval $(docker-machine env mesos-es)
$ ./gradlew build docker system-test:systemTest
This project is sponsored by Cisco Cloud Services
Apache License 2.0