Packaging Rock Core Updates

This is intended for internal use. This page explains how official Rock-Update packages should be made and it also includes design notes and instructions regarding exporting pages, blocks, etc. from Rock. This is for Trello card #34 and Trello card #133.

Forward: When Creating a New 1.x Release

we'll branch develop to release-X.Y
we'll create a hotfix-X.Y.Z branch
any bugs we find MUST be fixed in the hotfix-X.Y.Z branch and should be merged back to the develop branch

Packaging Design Overview

An official Rock update (v1.0.3 for example) will consist of two NuGet packages. One will have "Rock" as the Id, and the other will have a unique Id (such as RockUpdate-1-0-3). The Rock package will simply have a dependency on the RockUpdate-* package, and the RockUpdate-* package contents will include any changed file items for that update/release/patch.

Why, you ask? Why not just put all those items into one single Rock package? Due to the way NuGet works, each Rock package would have to include every Rock file. When NuGet installs an updated package, it removes all the previous package file items and adds all the new package file items. We want small delta patches of only the files that changed since the last release.

With our approach, each release of the Rock package will have a dependency on the single, latest RockUpdate-* package and then that package will include a dependency on the immediate/prior RockUpdate-* package (recursively, all the way back to the first update. You can see these relationships depicted in the following diagram:

As time goes on, a Rock installation will be comprised of a single Rock package and all the delta/patch packages (RockUpdate-1-0-0, RockUpdate-1-0-1, RockUpdate-1-0-2, etc.) that lead up to that release. Once installed the packages will be found inside the App_Data\packages folder (as defined in the WebProjectManager classes GetWebRepositoryDirectory() method.

So, the NuGet package metadata for a couple of updates might look something like this:

[ Caption ][img1] [img1]: Attachments/Exporting-and-Packaging/rock-package-metadata.png

The RockUpdate block handles the update process.

Regarding Uninstall

Because Rock packages may occasionally contain embedded migrations inside the Rock.dll, it would be quite difficult to support a back-out or restore-to-previous-version type of system.

Major, Minor, Patch Release

As mentioned in the semantic versioning specification, "patches" and "minor" releases should always be backward compatible. A "major" release should only occur when Rock runs into the situation where backwards compatibility cannot be maintained. Because of this, we need to put safe guards into the RockUpdate block to prevent a user simply from attempting to do a 'simple update' to a greater major version number (for example, from version 1.x.x to version 2.x.x or greater).

There should be no problems using this RockUpdate block approach to upgrade any minor or patch release of Rock.

The official "product version fullname" of Rock is noted inside the AssemblyInfo.cs. This is the friendly name of the release/patch that the user sees on certain pages and can be obtained by calling the Rock.Version's GetRockProductVersionFullName() method.

    [assembly: AssemblyInformationalVersion( "Rock Humphreys 0.1.0 (alpha)" )]

To get only the official version number call the Rock.Version's GetRockProductVersionNumber() method.

Rock Package MetaData

Id: - This must always be "Rock"
Version: - This is the semantic version of the release.
Title: - This should be a friendly name for the release (if any; otherwise Rock Update will suffice).
Summary/Description: - This will be displayed to the user so they can understand more about the release.
Tags: - Use the format "requires-A.B.C" to denote that the A.B.C Rock package MUST be installed prior to this package installation.
Release notes: - These will be displayed immediately following the install (if successful).
Dependencies: - This should refer to the corresponding RockUpdate* package.

RockUpdate Package MetaData & Package Contents

Id: - This should be something that starts with "RockUpdate". The Id for each update should be unique unless you intend on it completely replacing the previous update.
Version: - This is largely unused at this point, but it may make sense to simply use the same versioning as the corresponding Rock package.
Title: - unused.
Package contents: -
- the Rock.Version.DLL should always be included in the lib folder along with any other assemblies required with the release/patch.
- All other content should be included under the NuGet content folder.
- Any files that are to be deleted must be listed in a App_Data\deletefile.lst file. After installing, the RockUpdate block will remove each file listed in that file and then remove the deletefile.lst.
- Any XML files (such as foo\example.config) that are to be transformed must have a corresponding *.rock.xdt file (as in foo\example.config.rock.xdt). Details regarding the Transformation Syntax can be found on the MSDN site. (There is also a testing tool over at appHarbor) After installing, the system will perform the transform and then remove the *.rock.xdt files.

Steps for Creating a Rock Update (release)

The RockPackageBuilder will do most of the work when it comes to packaging. It will get list of all files added/changed/deleted since last tag, build a new nuget package (Id: RockUpdate-X-Y-Z) with a dependency to the last RockUpdate, and build a Rock package (Id: Rock) with a dependency to the current RockUpdate-X-Y-Z package. However, there are still some steps that humans must perform.

These are the steps we'll follow when creating an official update:

Merge-Master Role (full release)
1. Verify that the AssemblyVersion and AssemblyFileVersion version numbers in the AssemblyInfo.cs for each project have the correct version (the version you are packaging):
  1. Rock
  2. Rock.Version (and the AssemblyInformationalVersion version)
  3. Rock.Migrations
  4. Rock.Rest
  5. CheckScannerUtility
  6. Rock.Wpf
  7. StatementGenerator
  8. RockJobSchedulerService
  9. CheckinClient
2. Verify all migrations rollups have been added
3. Add a final migration that nulls out the [dbo].[__MigrationHistory].[Model] column for the previous update's migrations.
4. Verify all migrations include [dbo] owner when creating stored procedures
5. Verify that installers for the following have been built (if needed), added to azure storage, and a migration includes the new download links
  1. Check Scanner
  2. Check-in Client
  3. Job Scheduler Service
  4. Statement Generator
6. Verify that there is a [migration that includes the latest version of the installers](Creating a migration for updated Installers).
7. Create a version specific migration folder and move migrations
8. Fix any compiler warnings
9. Perform Code-Gen operation (see David and then document below).
10. Commit these changes.
11. Create a new release-x.y.z branch from the develop branch and push
12. Tag the release branch as 'X.Y.Z'.
13. Switch back to develop branch and increment the AssemblyVersion and AssemblyFileVersion version numbers in the AssemblyInfo.cs for each project to the next future release (so that develop always has the next version):
14. Rock
15. Rock.Version (and the AssemblyInformationalVersion version)
16. Rock.Migrations
17. Rock.Rest
18. CheckScannerUtility
19. Rock.Wpf
20. StatementGenerator
21. RockJobSchedulerService
22. CheckinClient
Merge-Master Role (hotfix)
1. Increment the AssemblyVersion and AssemblyFileVersion version numbers in the AssemblyInfo.cs for each project that changed since the previous release:
  1. Open SmartGit Log.
  2. Check only the hotfix branch and the release branch.
  3. Control-click the hotfix commit and the earlier release- commit.
  4. In the Details window, sort by Relative Directory to determine which projects/DLLs have been updated since the release.
  5. For each changed projects, Increment the AssemblyVersion and AssemblyFileVersion version numbers in the AssemblyInfo.cs.
  6. Update the same and the AssemblyInformationalVersion in the Rock.Version project.
2. Verify all migrations rollups have been added to the Rock/Plugin/HotFixes AND THAT THEY SUCCESSFULLY RUN (and remember to comment out and create a regular migration in the develop branch -- see Merging Hotfix to Develop below).
3. Rebuild solution and fix any compiler warnings.
4. Perform Code-Gen operation (see David and then document below).
5. Commit these changes.
6. Create a new hotfix-x.y.z branch from the release branch and push
7. If a 'X.Y.Z' tag already exists, delete it and then Add Tag this hotfix branch as 'X.Y.Z'.
8. While on the hotfix-x.y.z branch, create a new hotfix-x.y.z+1 branch and push (for any possible fixes people are going to commit for the X.Y.Z+1 release.)
Package-Master Role
1. Git pull and switch to Release-x.y.z (or hotfix) branch (may have to select option to configure tracking for local branch).
2. Verify that the version numbers in the Rock, Rock.Version, Rock.Migrations, and Rock.Rest projects are correct.
3. Build the Rock solution with RELEASE (not Debug) configuration.
4. Pull the latest from the Rock-UpdatePackageBuilder
  1. Open the Rock-UpdatePackageBuilder solution and re-build
  2. Change to the Rock-UpdatePackageBuilder\RockPackageBuilder\bin\Debug folder
  3. Run the RockPackageBuilder from the command line (example: RockPackageBuilder.exe -r C:\Misc\Rock -l 1.6.1 -c 1.6.2 -n 1.6.1
    1. Set the -r arg to be the location of your local repository
    2. Set the -l arg to be the last version (ie. the tag) that was built
    3. Set the -c arg to be the current version that is being built
    4. Set the -n arg to be the last tag to compare with the current tag to build the release notes.
  4. Open the Rock.#.#.#.nupkg created during the previous step.
    1. Edit the release notes.
    2. Edit the description and summary.
    3. Verify it has a dependency to the current RockUpdate-X-Y-Z package.
  5. Open the RockUpdate-X-Y-Z.x.y.z.nupkg created during the earlier step.
    1. Verify that the App_Data\Run.Migration file exists.
    2. Verify the files listed in the App_Data\deletefile.lst (if it exists).
    3. Verify it has a dependency to the previous RockUpdate package.
    4. If there are any web.config (or other .xml) transformations needed, add any necessary *.rock.xdt files in the appropriate folder under content as described in the Package MetaData section above. Also keep a copy of them in the Rock-UpdatePackageBuilder\NuGetLocal\artifacts\<x.y.z> folder
  6. Publish the two packages to alpha (https://www.myget.org/F/rockalpha/) and then to beta (https://www.myget.org/F/rockbeta/) in the following order:
    1. RockUpdate-X-Y-Z.x.y.z.nupkg
    2. Rock.x.y.z.nupkg
    3. Test the update on http://RockBeta.azurewebsites.net
  7. If previous test was success, publish the two packages to https://www.myget.org/F/rock/ in the following order:
    1. RockUpdate-X-Y-Z.x.y.z.nupkg
    2. Rock.x.y.z.nupkg
  8. Notify all Rock instances via new Spark broadcast.
Package-Master Role
1. Merge the develop (or other appropriate branch) into the master branch (causes issues to auto-close and email the issue reporters)
  1. Switch to the master branch
  2. Select Merge (verify that the latest release is selected) and press Create Merge-Commit
  3. You should be prompted to just "Fast-Forward" (if not somethings not right), press Fast Forward
  4. The changes have been committed so now just Push.

RockUpdater Design Overview

Using the Rock.Services.NuGet.WebProjectManager class, the RockUpdate block simply lists any pending updates and displays the results of an update. All the real work to perform an update is done by the Rock.Services.NuGet.WebProjectManager (and related WebProjectSystem and RockProjectManager classes). The RockUpdate block uses the UpdateServerUrl Global Attribute when constructing a WebProjectManager to determine the location of the NuGet package repository.

RockUpdate

From a high level, when an update is performed by the RockUpdate block, it will:

Write an app offline page.
Use the WebProjectManager to perform an update.
If successful:
1. Move any new files found in the App_Data\<version> folder to their respective folders.
2. Record the new version to the "RockInstanceId" System Setting.
3. Register any new REST controllers.
4. Send statistics to the www.rockrms.com server (if successful).
if unsuccessful:
1. Display the errors.
Remove the app offline page.

The RockUpdate block also removes any old *.rdelete files found under app root every time it loads.

WebProjectManager

From a high level, when the WebProjectManager UpdatePackageAndBackup method is called, it will:

Backup the old package file (Rock.x.y.z.nupkg) to App_Data\PackageRestore.
Use the RockProjectManager to invoke the NuGet UpdatePackageReference method. NuGet will then invokes the RockProjectManager's ExtractPackageFilesToProject method which:
1. Extracts any *.rock.xdt files into a respective subfolder in the App_Data\PackageRestore\xdt folder
2. Runs the transform against the intended target and copies the output to the respective App_Data\PackageRestore\xdtfolder.
3. Calls the NuGet base ExtractPackageFilesToProject method to continue the update which invokes the WebProjectSystem's AddFile method for each new file that is being added. In AddFile:
  1. If file is a DLL that is not in the bin folder, it moves the existing one to a *.rdelete file.
  2. Calls the NuGet base AddFile method if the file is not a *.rock.xdt file.
  3. If file is the App_Data\deletefile.lst, performs the delete of all files referenced.
4. Moves the transform files to the respective folder under app root.
5. Deletes the App_Data\PackageRestore folder.
If exceptions:
1. Restore the old package file back to the App_Data\Packages folder.

Notes

The RockInstanceId key (public static readonly string ROCK_INSTANCE_ID = "RockInstanceId";) is used by the SystemSettings.GetRockInstanceId() method to retrieve a globally unique identifier for each installation of Rock. The value comes from the Guid of the RockInstanceId Attribute:

   SELECT [Guid]
   FROM [Attribute]
   WHERE [Key] = 'RockInstanceId'

Merging Hotfix to Develop

Once you're finished with a hotfix branch, finish it by doing the following:

Switch to develop branch.
Click Merge.
Select the hotfix commit and select Create Merge-Commit.
Create a single new regular migration for each of the plugin hotfix migrations:
1. Add-Migration MIGRATIONNAME
2. Add the SQL to a file called timestamp_MIGRATIONNAME.sql in the Migrations folder
  1. Open the RockMigrationSQL.resx file and drag-drop the timestamp_MIGRATIONNAME.sql, then save it.
3. Add a comment before each section referencing the plugin hotfix migration number.
4. Comment out the hotfix migration and include a reference to the regular migration.
5. TEST

Code-Gen Operation

TBD

Provide feedback

Saved searches

Use saved searches to filter your results more quickly