Snapshots and Resource Compatibility Checking

Introduction

Due to the fact that resources and clients can be upgraded separately, it is very important to developers that they be notified of any changes they make that could break backwards or forwards compatibility issues. To that end, Rest.li uses a form of expanded IDLs, called Snapshots, to keep track of the state of resources and check compatibility between resource iterations.

Running the Compatibility Checker

The Snapshot Compatibility checker will be automatically run during a basic gradle build. However, if you wish to run the compatibility checker as a stand-alone on a particular target, you can do so by running gradle :[target]:checkRestModel. Additionally, there are four compatibility levels that the checker can be run on, by adding the argument -Prest.model.compatibility=[compatibility-level] to the gradle command.

The compatibility checker generates .snapshot.json files.

Compatibility Levels

There are four levels of compatibility. From least permissive to most, they are:

equivalent If the check is run in equivalent mode, no changes to Resources or PDSCs will pass.

backwards Changes that are considered backwards compatible will pass, otherwise, changes will fail.

ignore The compatibility checker is run, but all changes will pass. All changes, backwards compatible and backwards incompatible, will be printed.

off The compatibility checker will not be run at all.

By default, the compatibility checker will be run on backwards compatibility mode. There are two ways to change the mode the compatibility checker is run on. For one, you can add a flag to the gradle build itself: --Prest.model.compatibility=<compatLevel>

Alternately, if you want to change the default compatibility level for all builds on a machine, you can create or edit the file ~/.gradle/gradle.properties to contain the line rest.model.compatibility=<compatLevel>

Why is adding to an enum considered backwards incompatible?

Many developers are surprised that adding to an enum is considered a backwards incompatible change. However, if some service A using a newer version of your code and sends this enum to some service B using an older version of your code, service B may not handle the unknown case gracefully or correctly.

It is safe to ignore the incompatibility message if you are sure this will not happen, or you can work with your clients to make sure that it doesn't. In general, we only provide notifications about backwards incompatible changes as a tool for the developer. You are always free to ignore backwards-incompatible change messages if you know that the change will not cause problems, or are willing to take steps to ensure that it will not.

Error Messages

There are a number of error messages that can appear during compatibility checking. They will look something like this:

idl compatibility report between published "/publishedlocation/com.namespace.resource.snapshot.json" and current "/currentlocation/com.namespace.resource.snapshot.json":
  Incompatible changes:
    1) /location : detailed error message

This will tell you what resource is causing the problem, whether the change is backwards incompatible or not, and where exactly in the resource the problem is. You can then either use this information to fix the problem, or ignore it and re-run the build with a compatibility level that will cause it to pass.

Location of Snapshots/IDLs

The canonical snapshots for a set of resources will be located in the api project associated with those resources. If this project is located in some directory project-api, then the snapshots will be located in project-api/src/main/snapshot. Similarly, canonical idls will be located in project-api/src/main/idl. Though the files in these directories are generated, we strongly suggest that they be checked in. The IDLs must be checked in to correctly generate builders, and checking in the snapshots ensures that developers will receive accurate compatibility messages when making changes.

The temporary snapshots for a set of resources will be located within the same project as the java Resource files themselves. If the project for the Resource file is in a directory project-impl, then the snapshots will be located in project-impl/src/mainGeneratedRest/snapshot. Again, similarly, the temporary idls will be located in project-impl/src/mainGeneratedRest/idl. Unlike the canonical files, these generated files should NOT be checked in.

The Compatibility Checker and IDLs

The compatibility checker will also do limited IDL checks. By default, IDLs will only be checked to make sure there are no orphan IDLs from newly removed Resources and no missing IDLs from newly added Resources. A compatibility message may be printed saying that a resource has been added or removed. If a resource has been removed, you will need to remove the canonical IDL yourself. (This is also true for Snapshots).

Continuous Integration Environments

If you are running a continuous integration environment on a rest.li project, you will want to run your compatibility checker on equivalent. This will prevent your canonical IDLs and Snapshots from getting out of sync with the Resources they represent. You can do this either by running each build with the flag -Prest.model.compatibility=equivalent, or by creating or editing the ~/.gradle/gradle.properties file to contain the line rest.model.compatibility=equivalent