various optimizations for REST and windows build.

README.md

@@ -10,13 +10,17 @@ Automatically convert your movies and TV shows from one file format to another u
 * Take advantage of having multiple computers: Each node encodes a video, enabling parallelization.
 * Works as a single node too
 * No designated master. Whoever is "first" in the network, becomes master
-* Supports arbiter nodes for providing a quorum. Quorums are needed to prevent a split-brain. Useful if you have a spare Raspberry Pi that is just poor at encoding.
+* Supports arbiter nodes for providing a quorum. Quorums are needed to prevent a split-brain. Useful if you
+have a spare Raspberry Pi or NAS that is just poor at encoding.
 * Several and different cleanup strategies.
 * Supports Handbrake and ffmpeg
+* Basic REST API (more to come)
 
 ## Installation
 
-Currently, only on Docker. You have to build it using (Maven) if you prefer it another way. Works with Windows too (its Java), but you need to figure out who to deploy it reasonably. Better support is planned, but Docker takes priority.
+* The recommended platform is Docker.
+* Windows (download zip from releases tab).
+* Build it using Gradle if you prefer it another way.
 
 I hate long `docker run` commands with tons of arguments, so here is a docker-compose template:
 
@@ -35,16 +39,22 @@ services:
     volumes:
       - "/path/to/input:/input"
       - "/path/to/output:/output"
-      - "/path/to/config:/usr/src/clustercode/config"
+      - "/path/to/profiles:/profiles"
+# If you need modifications to the xml files, persist them:
+#      - "/path/to/config:/usr/src/clustercode/config"
     environment:
-      - CC_CLUSTER_JGROUPS_TCP_INITAL_HOSTS=your.other.docker.node[7600],another.one[7600]
+    # overwrite any settings from the default using env vars!
+      - CC_CLUSTER_JGROUPS_TCP_INITIAL_HOSTS=your.other.docker.node[7600],another.one[7600]
       - CC_CLUSTER_JGROUPS_EXT_ADDR=192.168.1.100
 ```
-The external IP address is needed so that other nodes will be available to contact the local node. Use the physical address of the docker host.
+The external IP address is needed so that other nodes will be available to
+contact the local node. Use the physical address of the docker host.
 
 ### Docker Compose, Swarm mode
 
-**This is untested**, as I don't have a Swarm. Just make sure that limit the CPU resources somehow, so that other containers still work reliably. I figure that encoding is a low-priority service that takes forever anyway.
+**This is untested**, as I don't have a Swarm. Just make sure to limit the CPU
+resources somehow, so that other containers still work reliably. I figure that
+encoding is a low-priority service that takes forever anyway.
 ```
 version: "3.2"
 services:
@@ -56,7 +66,9 @@ services:
     volumes:
       - "/path/to/input:/input"
       - "/path/to/output:/output"
-      - "/path/to/config:/usr/src/clustercode/config"
+      - "/path/to/profiles:/profiles"
+# If you need modifications to the xml files, persist them:
+#      - "/path/to/config:/usr/src/clustercode/config"
     deploy:
       restart_policy:
         condition: any
@@ -68,29 +80,29 @@ services:
           cpus: "3"
 ```
 
-
-### Don't use 'Latest'
-
-The latest builds are to be considered experimental with the newest features and bugs. If you only want to run the latest stable, use the 'stable' tag.
-
 ## Configuration
 
-When you first start the container using docker compose, it will create a default configuration file in `/usr/src/clustercode/config` (in the container). Be sure that you have mounted this dir from outside for persistence and it's empty. You can change the settings in the `clustercode.properties` file to modify the behaviour of the software.
-
-Alternatively you can configure all settings via Environment variables (same key/values syntax). Environment variables **always take precendence** over the ones in `clustercode.properties`.
+When you first start the container using docker compose, it will create a default configuration
+file in `/usr/src/clustercode/config` (in the container). You can view the settings in the
+`clustercode.properties` file and deviate from the default behaviour of the software. However, you should
+modify the settings via Environment variables (same key/values syntax). Environment variables **always take precedence**
+over the ones in `clustercode.properties`. If you made changes to the XML files, you need to mount a path from outside
+in order to have them persistent.
 
 ## Project status
 
 Active Development as of August/September 2017.
 
 ## Future Plans
 
-* Monitoring with a REST API.
-* [netdata](https://my-netdata.io/) plugin for monitoring.
-* Smooth-ier Windows deployment.
+- [x] Monitoring with a REST API.
+- [ ] More control with REST
+- [ ] [netdata](https://my-netdata.io/) plugin for monitoring.
+- [x] Smooth-ier Windows deployment.
+- [ ] Web-Admin (if I have spare time...)
 
 ## Docker Tags
 
-* latest: latest automated build of the master branch
-* stable: latest automated build of a tagged commit from a release
+* experimental: latest automated build of the master branch
+* latest: stable build of a tagged commit from a release
 * tagged: tags following the 1.x.x pattern are specific releases

build.gradle

@@ -115,6 +115,7 @@ task windowsZip(type: Zip) {
     from "${project.buildDir}/libs/${fullBuild.archiveName}"
     from "src/assembly/resources"
     from "src/assembly/windows"
+    classifier "win"
 }
 
 fullBuild.dependsOn test

docker/default/config/clustercode.properties

@@ -148,7 +148,7 @@ CC_CLUSTER_JGROUPS_TCP_INITIAL_HOSTS = localhost[7600]
 CC_CLUSTER_JGROUPS_HOSTNAME =
 
 # Integer. 0 <= x. Unit: Hours
-CC_CLUSTER_TASK_TIMEOUT = 24
+CC_CLUSTER_TASK_TIMEOUT = 6
 
 # Bool. true | false.
 CC_ARBITER_NODE = false

src/assembly/windows/clustercode.cmd

@@ -5,7 +5,8 @@ set java=java
 %java% -version
 IF %ERRORLEVEL% EQU 0 (
 	%java% -jar clustercode.jar
+	pause
 ) else (
-	echo Java is not installed or in PATH. Modify your environment variables.
+	echo Java is not installed or in PATH. Modify your environment variables or specify the absolute path in this CMD file.
 	pause
 )

src/assembly/windows/config/clustercode.properties

@@ -23,7 +23,7 @@ CC_PROFILE_DIR = profiles
 CC_MEDIA_INPUT_DIR = C:\\Users\\Public\\Videos
 
 # Path. Relative | Absolute path. Windows: use "\\" as path separators.
-CC_MEDIA_OUTPUT_DIR = C:\\Users\\Public\\Videos
+CC_MEDIA_OUTPUT_DIR = \\\\server\\a UNC\\Path
 
 #-------------------------------------------------------------------------------------------------
 #  TRANSCODER SETTINGS
@@ -148,7 +148,7 @@ CC_CLUSTER_JGROUPS_TCP_INITIAL_HOSTS = localhost[7600]
 CC_CLUSTER_JGROUPS_HOSTNAME =
 
 # Integer. 0 <= x. Unit: Hours
-CC_CLUSTER_TASK_TIMEOUT = 24
+CC_CLUSTER_TASK_TIMEOUT = 6
 
 # Bool. true | false.
 CC_ARBITER_NODE = false

src/assembly/resources/config/tcp.xml → src/assembly/windows/config/tcp.xml

@@ -2,7 +2,7 @@
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups.xsd">
     <TCP_NIO2
-              external_addr="${ext-addr:SITE_LOCAL}"
+              <!--external_addr="${ext-addr:SITE_LOCAL}"-->
               bind_addr="${jgroups.bind_addr:SITE_LOCAL}"
               bind_port="${jgroups.bind_port:7600}"
               recv_buf_size="${tcp.recv_buf_size:130k}"

src/main/java/net/chrigel/clustercode/Startup.java

@@ -79,8 +79,8 @@ public static void main(String[] args) throws Exception {
         log.info("Booting clustercode {}...", getApplicationVersion());
         Injector injector = Guice.createInjector(modules);
 
-        injector.getInstance(StateMachineService.class).initialize();
         injector.getInstance(RestApiServices.class).start();
+        injector.getInstance(StateMachineService.class).initialize();
     }
 
     private static String getApplicationVersion() {

src/main/java/net/chrigel/clustercode/api/RestApiServices.java

@@ -2,6 +2,8 @@
 
 public interface RestApiServices {
 
+    String REST_API_CONTEXT_PATH = "/api/v1";
+
     void start();
 
 }

src/main/java/net/chrigel/clustercode/api/impl/ApiModule.java

@@ -53,7 +53,6 @@ private void installJersey(int port) {
         JerseyConfiguration configuration = JerseyConfiguration.builder()
                 .addPackage("net.chrigel.clustercode.api")
                 .addPort(port)
-                .withContextPath("/api/v1")
                 .registerClasses(GensonJsonConverter.class)
                 .build();
 

src/main/java/net/chrigel/clustercode/api/rest/DocumentationApi.java

@@ -16,7 +16,6 @@
 
 @Path("/docs")
 @Api(description = "the swagger documentation")
-@XSlf4j
 public class DocumentationApi {
 
     @Path("api.html")
@@ -25,7 +24,7 @@ public class DocumentationApi {
     @ApiOperation(value = "Swagger API definition", notes = "Retrieves a html file that describes this API.", response
             = File.class, tags = {"api"})
     @ApiResponses(value = {
-            @ApiResponse(code = 200, message = "The API document", response = File.class),
+            @ApiResponse(code = 200, message = "OK", response = File.class),
             @ApiResponse(code = 500, message = "Unexpected error", response = ApiError.class)})
     public InputStream getApiHtml() {
         return getClass().getResourceAsStream("/swagger.html");
@@ -37,7 +36,7 @@ public InputStream getApiHtml() {
     @ApiOperation(value = "Swagger API definition", notes = "Retrieves a json file that describes this API.", response
             = File.class, tags = {"api"})
     @ApiResponses(value = {
-            @ApiResponse(code = 200, message = "The API document", response = File.class),
+            @ApiResponse(code = 200, message = "OK", response = File.class),
             @ApiResponse(code = 500, message = "Unexpected error", response = ApiError.class)})
     public InputStream getApiJson() {
         return getClass().getResourceAsStream("/swagger.json");

src/main/java/net/chrigel/clustercode/api/rest/ProgressApi.java

@@ -1,11 +1,8 @@
 package net.chrigel.clustercode.api.rest;
 
-import io.swagger.annotations.Api;
-import io.swagger.annotations.ApiOperation;
-import io.swagger.annotations.ApiResponse;
-import io.swagger.annotations.ApiResponses;
-import lombok.extern.slf4j.XSlf4j;
+import io.swagger.annotations.*;
 import net.chrigel.clustercode.api.ProgressReportAdapter;
+import net.chrigel.clustercode.api.RestApiServices;
 import net.chrigel.clustercode.api.dto.ApiError;
 import net.chrigel.clustercode.api.dto.FfmpegProgressReport;
 import net.chrigel.clustercode.api.dto.HandbrakeProgressReport;
@@ -21,9 +18,8 @@
 import javax.ws.rs.core.MediaType;
 import javax.ws.rs.core.Response;
 
-@Path("/progress")
+@Path(RestApiServices.REST_API_CONTEXT_PATH + "/progress")
 @Api(description = "The progress service API")
-@XSlf4j
 public class ProgressApi extends AbstractRestApi {
 
     private final TranscodingService transcodingService;
@@ -44,13 +40,13 @@ public class ProgressApi extends AbstractRestApi {
     @JSONP(queryParam = "callback")
     @ApiOperation(
             value = "Conversion progress",
-            notes = "Gets the percentage of the current encoding process.",
+            notes = "Gets the percentage of the current encoding process. Returns -1 if no conversion active.",
             response = Double.class,
             tags = {"Progress"})
     @ApiResponses(value = {
             @ApiResponse(
                     code = 200,
-                    message = "A percentage of the currently active task. Returns -1 if no conversion active.",
+                    message = "OK",
                     response = Double.class),
             @ApiResponse(
                     code = 500,

src/main/java/net/chrigel/clustercode/api/rest/StatusApi.java

@@ -4,7 +4,7 @@
 import io.swagger.annotations.ApiOperation;
 import io.swagger.annotations.ApiResponse;
 import io.swagger.annotations.ApiResponses;
-import lombok.extern.slf4j.XSlf4j;
+import net.chrigel.clustercode.api.RestApiServices;
 import net.chrigel.clustercode.api.StateMachineMonitor;
 import net.chrigel.clustercode.api.dto.ApiError;
 import net.chrigel.clustercode.api.dto.StatusReport;
@@ -18,8 +18,7 @@
 import javax.ws.rs.core.MediaType;
 import javax.ws.rs.core.Response;
 
-@XSlf4j
-@Path("/status")
+@Path(RestApiServices.REST_API_CONTEXT_PATH + "/status")
 @Api(description = "the status API")
 public class StatusApi extends AbstractRestApi {
 

src/main/java/net/chrigel/clustercode/api/rest/TasksApi.java

@@ -4,6 +4,7 @@
 import io.swagger.annotations.ApiOperation;
 import io.swagger.annotations.ApiResponse;
 import io.swagger.annotations.ApiResponses;
+import net.chrigel.clustercode.api.RestApiServices;
 import net.chrigel.clustercode.api.dto.ApiError;
 import net.chrigel.clustercode.api.dto.Task;
 import net.chrigel.clustercode.cluster.ClusterService;
@@ -19,8 +20,7 @@
 import java.sql.Date;
 import java.util.stream.Collectors;
 
-@Path("/tasks")
-
+@Path(RestApiServices.REST_API_CONTEXT_PATH + "/tasks")
 @Api(description = "the tasks API")
 public class TasksApi extends AbstractRestApi {
 

src/main/java/net/chrigel/clustercode/constraint/impl/AbstractConstraint.java

@@ -14,7 +14,7 @@ public abstract class AbstractConstraint implements Constraint {
     protected final XLogger log = XLoggerFactory.getXLogger(getClass());
 
     protected AbstractConstraint() {
-        log.info("Enabled {} constraint.", getClass().getSimpleName());
+        log.info("Enabled {}.", getClass().getSimpleName());
     }
 
     /**

src/main/java/net/chrigel/clustercode/statemachine/actions/InitializeAction.java

@@ -1,5 +1,6 @@
 package net.chrigel.clustercode.statemachine.actions;
 
+import lombok.extern.slf4j.XSlf4j;
 import net.chrigel.clustercode.cluster.ClusterService;
 import net.chrigel.clustercode.statemachine.Action;
 import net.chrigel.clustercode.statemachine.StateContext;
@@ -8,6 +9,7 @@
 
 import javax.inject.Inject;
 
+@XSlf4j
 public class InitializeAction extends Action {
 
     private final ClusterService clusterService;
@@ -20,8 +22,20 @@ public class InitializeAction extends Action {
     @Override
     protected StateEvent doExecute(State from, State to, StateEvent event, StateContext context) {
         log.entry(from, to, event, context);
+        log.info("Initializing state machine...");
         clusterService.joinCluster();
 
+        if (clusterService.getSize() == 1) {
+            log.info("We are the only member in the cluster. Let's wait 15 seconds before we continue");
+            log.info("in order to make sure that there wasn't a connection problem and we can join");
+            log.info("an existing cluster.");
+            try {
+                Thread.sleep(15000);
+            } catch (InterruptedException e) {
+                log.warn(e.getMessage());
+            }
+        }
+
         return StateEvent.FINISHED;
     }
 }

src/test/resources/log4j2.xml

@@ -8,10 +8,13 @@
         </Console>
     </Appenders>
     <Loggers>
-        <Logger name="org.jgroups" level="debug" additivity="false">
+        <Logger name="org.jgroups" level="info" additivity="false">
             <AppenderRef ref="Console"/>
         </Logger>
-        <Logger name="org.jgroups.protocols" level="debug" additivity="false">
+        <Logger name="org.jgroups.protocols.pbcast.GMS" level="debug" additivity="false">
+            <AppenderRef ref="Console"/>
+        </Logger>
+        <Logger name="org.jgroups.protocols" level="info" additivity="false">
             <AppenderRef ref="Console"/>
         </Logger>
         <Logger name="org.squirrelframework.foundation.util" level="debug" additivity="false">
@@ -23,7 +26,7 @@
         <Logger name="org.eclipse.jetty" level="info" additivity="false">
             <AppenderRef ref="Console"/>
         </Logger>
-        <Root level="debug" additivity="false">
+        <Root level="info" additivity="false">
             <AppenderRef ref="Console"/>
         </Root>
     </Loggers>