ci: Cache nuget packages (Github actions)

[nikcio]

closes #857

This adds a cache step to the GitHub actions using packages.lock.json files. These files are generated on build and keep track of changes in the dependencies of projects. Thereby acting as cache keys. I've also added the dotnet restore step separately from the build and pack steps so that it's easier to see how much time is spent on restoring vs building in an action.

[NightOwl888]

Thanks for the PR.

One issue we had when we tried this on Azure DevOps was it wasn't set up right to correctly detect an upgrade or change of a dependency and when we did a "normal" upgrade of a dependency it failed. The only way I found out of that scenario was to remove the caching and add an explicit download location in NuGet.config.

I have read somewhere that packages.lock.json isn't consistently generated across machines, which makes them change too often to keep in source control because they show up as changed files even when no dependencies have changes. Also, having these extra files seems a bit odd when we have a dependencies.props file that can be used as part of the cache key (we would also need to include all *.csproj and all *.targets files to get all of the references).

Whatever the case, we need a solid procedure for upgrading or changing dependencies so we don't shoot ourselves in the foot (again) when upgrading. We are likely going to be juggling dependencies around more often from now until the release than we have done so in the past. If we stick with the packages.lock.json files, it would be nice to know how to update them and when to commit them.

[nikcio]

Hey @NightOwl888

I've used packages.lock.json for checking the cache of NuGet packages on a few projects over the past 2 years and havn't had any problems with it. But these projects were nowhere near the scale of this project.

The packages.lock.json are automatically updated on dotnet build. So you will need to build the project locally before pushing for it to work.

Does this answer you questions?

[NightOwl888]

Actually, what I am looking for is a step-by-step procedure for updating a dependency and making sure the cache gets invalidated so the new dependency version is used. The blog article about it makes it clear what the packages.lock.json file is for and how to configure it, but not how to perform a dependency update (which is the important bit). Of course, without 3rd party docs this might boil down to doing experimentation.

Do note that we have a condition on the ContinuousIntegrationBuild property so it only is set to true in Azure DevOps and GitHub Actions. This property is also used by the Source Link functionality and that is the way they recommended to use it.

My guess is the procedure is something along the lines of:

1. Update the dependencies
2. Run the build (in VS? on the command line?)
3. Check the diff of the packages.lock.json files for any issues
4. Commit the changes to Git
5. Validate the changes worked in GitHub Actions
6. And after the functionality has been added...validate the changes worked in Azure DevOps

Putting the procedure here is okay, but ideally we would add it to a new page in the contrbuting section. And when Azure DevOps has caching we can add those instructions there as well (if we use packages.lock.json chances are there will be some overlap in the procedures).

The packages.lock.json being flakey may have been due to early buggy build tools that have now been fixed, not sure. I think there were differences in the way the lock file was generated on the command line vs the IDE at some point. At least this article recommends using a lock file but also goes into some of the issues with doing so.

My hope is to have this whole procedure tested out prior to running it, since there is no way (at least in Azure DevOps) to invalidate the cache. The only option is to give the key a different name (which I know now, but didn't know then). It was a major showstopper having the cache put in place on Azure DevOps without it detecting when there was a change to the dependencies and then having to try to figure out how to work around the problem. We ended up having to remove the cache and then add a NuGet.config because for some reason it was no longer looking in NuGet.org for the dependencies.