dotnet --version fails if global.json specifies a newer SDK version

[martincostello]

I've been using dotnet --version for some time in build environments to determine whether or not the version of the .NET Core SDK I need for my build is already installed. If not, I download it and install it locally.

With .NET Core SDK 2.1.200 and later, the behaviour has changed so that if global.json in the current directory specifies a newer version that the version of dotnet that is invoked, rather than just returning its own version, it now errors with the following message:

A compatible SDK version for global.json version: [{version in global.json}] from [{working directory}\global.json] was not found
Did you mean to run dotnet SDK commands? Please install dotnet SDK from:
  http://go.microsoft.com/fwlink/?LinkID=798306&clcid=0x409

Given that dotnet --version is such a simple command, it should not matter what is in global.json, particularly for scenarios such as this where it is likely that a newer version is wanted and the command is just being used for version detection.

[livarcocc]

I do not believe we have changed the behavior of the host in 2.1.200. Did you install 2.1.300 by any chance and are targeting 2.1.200 using a global.json?

Also, a bit of implementation detail here, dotnet --version is implemented by the SDK itself and not by the host, as such, the code that validates the global.json option runs way sooner than the one that handles --version.

[martincostello]

So the CI agent had 2.1.104 on it originally and the project targets 2.1.300. It would detect the mismatch and install 2.1.300 using the PowerShell obtain script. Here's an example of a line output by a build from Tuesday:

[09:05:53]	[Step 7/14] PowerShell arguments: -NonInteractive, -ExecutionPolicy, ByPass, -File, C:\BuildAgent\work\7e9caab035072cf9\Build.ps1
[09:05:54]	[Step 7/14] The required version of the .NET Core SDK is not installed. Expected 2.1.300 but 2.1.104 was found.
[09:05:54]	[Step 7/14] dotnet-install: Downloading link: https://dotnetcli.azureedge.net/dotnet/Sdk/2.1.300/dotnet-sdk-2.1.300-win-x86.zip
[09:05:59]	[Step 7/14] dotnet-install: Extracting zip from https://dotnetcli.azureedge.net/dotnet/Sdk/2.1.300/dotnet-sdk-2.1.300-win-x86.zip
[09:06:04]	[Step 7/14] dotnet-install: Binaries of dotnet can be found in C:\BuildAgent\work\7e9caab035072cf9\.dotnetcli\
[09:06:04]	[Step 7/14] dotnet-install: Installation finished

Since this morning the agent appears to have acquired 2.1.200, and now the build does this:

[08:50:27]	[Step 7/14] PowerShell arguments: -NonInteractive, -ExecutionPolicy, ByPass, -File, C:\BuildAgent\work\2bda9dae9152226c\Build.ps1
[08:50:29]	[Step 7/14] A compatible SDK version for global.json version: [2.1.300] from [C:\BuildAgent\work\2bda9dae9152226c\global.json] was not found
[08:50:30]	[Step 7/14] Error message is logged
[08:50:29]	[Step 7/14] Did you mean to run dotnet SDK commands? Please install dotnet SDK from:
[08:50:29]	[Step 7/14]   http://go.microsoft.com/fwlink/?LinkID=798306&clcid=0x409
[08:50:29]	[Step 7/14] The required version of the .NET Core SDK is not installed. Expected 2.1.300 but  was found.
[08:50:31]	[Step 7/14] dotnet-install: Downloading link: https://dotnetcli.azureedge.net/dotnet/Sdk/2.1.300/dotnet-sdk-2.1.300-win-x86.zip
[08:50:34]	[Step 7/14] dotnet-install: Extracting zip from https://dotnetcli.azureedge.net/dotnet/Sdk/2.1.300/dotnet-sdk-2.1.300-win-x86.zip
[08:50:38]	[Step 7/14] dotnet-install: Binaries of dotnet can be found in C:\BuildAgent\work\2bda9dae9152226c\.dotnetcli\
[08:50:38]	[Step 7/14] dotnet-install: Installation finished

Here's a snippet of the part of the script that does the version check pre-download:

$installedDotNetVersion = (dotnet --version | Out-String).Trim()

if ($installedDotNetVersion -ne $dotnetVersion) {
    Write-Host "The required version of the .NET Core SDK is not installed. Expected $dotnetVersion but $installedDotNetVersion was found."
    $installDotNetSdk = $true
}
if ($installDotNetSdk -eq $true) {

}

I can also replicate this with 2.1.300 by changing the version in global.json to something like 7.1.300.

[livarcocc]

We never did that. The behavior if you have a global.json and specify a version and we can't find that version (or a patch version for that major.minor) was always to fail. Also, dotnet --version has always been implemented by the CLI and as such, if the version selection fails, the version command will also fail.

There must be something else going on in your agent that allowed this to work. In any case, can't you consider the exit code for dotnet --version as a way to determine if the version you want is installed or not?

[livarcocc]

cc @nguerrera @johnbeisner in case one of them has any ideas how this was working before. They may remember some change we made that I am not remembering now.

[yaakovp]

I can see the problem on my machine.

[davidje13]

From what I can see, this change has also broken the behaviour of dotnet install-sdk.

Everything I see online says that running dotnet install-sdk in a folder with a global.json will cause it to install the correct version (after running dotnet tool install -g installsdkglobaltool, of course). But when I use it, it just errors with "A compatible SDK version for global.json version: [2.2.105] from [(path)\global.json] was not found". This is clearly not the intended behaviour, so something must have broken in the meantime.

In my case, I want dotnet tool install -g installsdkglobaltool and dotnet install-sdk to ignore global.json, so this seems bigger than just dotnet --version. If the regression can't be identified / fixed, can a flag be added to explicitly ignore global.json to allow these use cases? The whole point here (in both my case and the OP's) is to ensure the correct version is installed before continuing.

[nguerrera]

There were versions of the host that:

1. Didn't know about 2.1.Nxx convention and thought that it was plain semver with the third part being a patch.

2. If exact match was not found, used latest patch, but allowed that to be lower than what was specified in global.json, causing a roll backwards (!) in some cases.

This would have allowed a specific example like 2.1.104 being used when you specified 2.1.300. This was a bug. If you didn't have this logic to download the sdk when this condition occurred and tried to build, you would have had to debug subtle issues with the build just trying to use 2.1.104.

Relying on that bug was also a dead end. It still would have failed if 2.2.100 was specified and only 2.1.104 was installed. The best thing to do is to check if dotnet --version fails and also consider the SDK you need not installed in that case. It appears the OP's scripts do just that because they do proceed to download the right version after the message that the version in global.json was not matched.

[nguerrera]

Everything I see online says that running dotnet install-sdk in a folder with a global.json

We don't maintain that tool, but except for a few old bug cases, I don't see how this would work if you invoke it as dotnet install-sdk. I suggest opening an issue on https://github.com/JosephWoodward/DotNetInstallSdkGlobalTool/

It would likely work if you invoked it as dotnet-install-sdk, which would just launch the tool directly without going through the sdk. I think if this tool wants to work irrespective of global.json, it should not be trying to be a dotnet sdk command. Instead, it could install a tool that is invoked as install-dotnet-sdk or something.

[davidje13]

@nguerrera yes I already raised josephwoodward/DotNetInstallSdkGlobalTool#14

The explanation makes sense, but it feels like the missing feature is the ability to explicitly ignore global.json. It's good that it's strict by default, but clearly there are cases where non-versioned commands are useful.

Of course, the real missing feature is the ability to auto-install the correct version if it isn't already present, a-la mavenw / gradlew / rbenv / etc.

The suggested workaround causes another issue. Maybe it's unrelated, but I suspect dotnet does more than just delegate to the other executable (maybe it sets some environment variables, for example?)

[nguerrera]

but except for a few old bug cases.

Sorry, and the more obvious case of installing an older version than you have.

[matkoch]

Adding my experience:

1. The behavior of calling dotnet --version in presence of a non-matching global.json has definitely changed at some point. I can't tell an exact version, unfortunately, but at some point it was just outputting the available version, which is why I've ended up with this comparison. I was pretty happy with that, as it was an exact match (compared to what @nguerrera mentioned).

2. Then I switched from 2.2.101 to 3.0.100. Calling dotnet --version now outputted the error message, as well as a list of available SDKs. So I had to select the first line for comparison. This worked on the TeamCity CI build.

3. However, it doesn't work on AppVeyor with the Visual Studio 2017 image (it does work with Visual Studio 2019, comparison is here).

For the love of god, I can't figure out what's the difference. Why on one system the message halts PowerShell, and not on the other. The $ErrorActionPreference is the same on both.

Build log AppVeyor
Build log TeamCity

Solving this is crucial for me, as I'm maintaing a build system. So I can't cheap out by just changing the image :/ I hope someone can help.

[matkoch]

@nguerrera can you please give advice on this?