Unable to get diff for corelib

[alexcovington]

I have two builds of dotnet/runtime that I would like to compare using jit-diff. One is just a build off of the master branch, runtime-master, and the other is my own local build with some changes I've made to the JIT, runtime.

Whenever I try to do a diff of the two builds on System.Private.Corelib.dll, I get the following error:

PS C:\Users\acovingt\source\repos\runtime> jit-diff diff --output C:\Users\acovingt\Documents\diffs --base C:\Users\acovingt\source\repos\runtime-master\artifacts\bin\coreclr\windows.x64.Release\ --diff C:\Users\acovingt\source\repos\runtime\artifacts\bin\coreclr\windows.x64.Release\ --core_root C:\Users\acovingt\source\repos\runtime-master\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root\ --corelib                                                                                                                                                   
Beginning Crossgen CodeSize Diffs for System.Private.CoreLib.dll                                                                                                                                                                                                                        
Unhandled exception. System.ArgumentOutOfRangeException: startIndex cannot be larger than length of string. (Parameter 'startIndex')                                                                                                                                                     
  at System.String.Substring(Int32 startIndex, Int32 length)                                                                                                                                                                                                                              
  at ManagedCodeGen.jitdiff.DiffTool.IdentifyAssemblies(String basePath, String rootPath, Config config, Boolean recursive, String searchPattern) in C:\Users\acovingt\source\repos\jitutils\src\jit-diff\diff.cs:line 532                                                              
  at ManagedCodeGen.jitdiff.DiffTool.GenerateAssemblyWorklist(Config config) in C:\Users\acovingt\source\repos\jitutils\src\jit-diff\diff.cs:line 466                                                                                                                                   
  at ManagedCodeGen.jitdiff.DiffTool.DiffCommand(Config config) in C:\Users\acovingt\source\repos\jitutils\src\jit-diff\diff.cs:line 332                                                                                                                                                
  at ManagedCodeGen.jitdiff.Main(String[] args) in C:\Users\acovingt\source\repos\jitutils\src\jit-diff\jit-diff.cs:line 1335

I've tried re-cloning and rebuilding both dotnet/jitutils and dotnet/runtime, but this error persists.

Here is dotnet --info output to help with troubleshooting:

PS C:\Users\acovingt\source\repos\runtime> dotnet --info
.NET SDK (reflecting any global.json):
 Version:   5.0.100-rc.2.20479.15
 Commit:    da7dfa8840

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.18363
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.100-rc.2.20479.15\

Host (useful for support):
  Version: 6.0.0-alpha.1.20454.24
  Commit:  4fd87bc4ce

.NET SDKs installed:
  3.1.403 [C:\Program Files\dotnet\sdk]
  5.0.100-rc.2.20479.15 [C:\Program Files\dotnet\sdk]
  6.0.100-alpha.1.20459.9 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.21 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.21 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.0-rc.2.20458.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.0-rc.2.20475.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.21 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.0-preview.8.20407.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.0-rc.2.20475.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.0-alpha.1.20454.24 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.7 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.0-preview.8.20411.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.0-rc.1.20417.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.0-rc.2.20475.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

To install additional .NET runtimes or SDKs:
  https://aka.ms/dotnet-download

Please let me know if I can provide any other information or try any other steps. Thanks!

[AndyAyersMS]

You are perhaps running into #197 / #217 -- if you specify both base and diff in one go then only the base SPC and runtime are used.

In order to make diffing SPC work you need to run base and diff separately, each in its own respective core_root, and then invoke jit-analyze afterwards.

Something like this should work

jit-diff diff --output C:\Users\acovingt\Documents\diffs --base C:\Users\acovingt\source\repos\runtime-master\artifacts\bin\coreclr\windows.x64.Release\  --core_root C:\Users\acovingt\source\repos\runtime-master\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root\ --corelib          

jit-diff diff --output C:\Users\acovingt\Documents\diffs --diff C:\Users\acovingt\source\repos\runtime\artifacts\bin\coreclr\windows.x64.Release\   --core_root C:\Users\acovingt\source\repos\runtime\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root\ --corelib 

jit-analyze --base  C:\Users\acovingt\Documents\diffs\base --diff C:\Users\acovingt\Documents\diffs\diff                                                                                                                                  

[alexcovington]

Hmm, I'm still getting the same error even when running the base and diff separately.

PS C:\Users\acovingt> jit-diff diff --output C:\Users\acovingt\Documents\diffs --base C:\Users\acovingt\source\repos\runtime-master\artifacts\bin\coreclr\windows.x64.Release\  --core_root C:\Users\acovingt\source\repos\runtime-master\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root\ --corelib
Beginning Crossgen CodeSize Diffs for System.Private.CoreLib.dll
Unhandled exception. System.ArgumentOutOfRangeException: startIndex cannot be larger than length of string. (Parameter 'startIndex')
   at System.String.Substring(Int32 startIndex, Int32 length)
   at ManagedCodeGen.jitdiff.DiffTool.IdentifyAssemblies(String basePath, String rootPath, Config config, Boolean recursive, String searchPattern) in C:\Users\acovingt\source\repos\jitutils\src\jit-diff\diff.cs:line 532
   at ManagedCodeGen.jitdiff.DiffTool.GenerateAssemblyWorklist(Config config) in C:\Users\acovingt\source\repos\jitutils\src\jit-diff\diff.cs:line 466
   at ManagedCodeGen.jitdiff.DiffTool.DiffCommand(Config config) in C:\Users\acovingt\source\repos\jitutils\src\jit-diff\diff.cs:line 332
   at ManagedCodeGen.jitdiff.Main(String[] args) in C:\Users\acovingt\source\repos\jitutils\src\jit-diff\jit-diff.cs:line 1335

[AndyAyersMS]

Hmm. Can you try adding --verbose ?

[alexcovington]

Sure, here's the output:

PS C:\Users\acovingt> jit-diff diff --output C:\Users\acovingt\Documents\diffs --diff C:\Users\acovingt\source\repos\runtime\artifacts\bin\coreclr\windows.x64.Release\   --core_root C:\Users\acovingt\source\repos\runtime\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root\ --corelib --verbose
After setting defaults:
--core_root C:\Users\acovingt\source\repos\runtime\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root\
--output C:\Users\acovingt\Documents\diffs
--diff C:\Users\acovingt\source\repos\runtime\artifacts\bin\coreclr\windows.x64.Release\
Beginning Crossgen CodeSize Diffs for System.Private.CoreLib.dll
Scanning: C:\Users\acovingt\source\repos\runtime\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root\System.Private.CoreLib.dll
Unhandled exception. System.ArgumentOutOfRangeException: startIndex cannot be larger than length of string. (Parameter 'startIndex')
   at System.String.Substring(Int32 startIndex, Int32 length)
   at ManagedCodeGen.jitdiff.DiffTool.IdentifyAssemblies(String basePath, String rootPath, Config config, Boolean recursive, String searchPattern) in C:\Users\acovingt\source\repos\jitutils\src\jit-diff\diff.cs:line 532
   at ManagedCodeGen.jitdiff.DiffTool.GenerateAssemblyWorklist(Config config) in C:\Users\acovingt\source\repos\jitutils\src\jit-diff\diff.cs:line 466
   at ManagedCodeGen.jitdiff.DiffTool.DiffCommand(Config config) in C:\Users\acovingt\source\repos\jitutils\src\jit-diff\diff.cs:line 332
   at ManagedCodeGen.jitdiff.Main(String[] args) in C:\Users\acovingt\source\repos\jitutils\src\jit-diff\jit-diff.cs:line 1335

I did confirm that C:\Users\acovingt\source\repos\runtime\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root\System.Private.CoreLib.dll exists.

[alexcovington]

Oh, well this is interesting. I slightly modified my command to drop the trailing backslash from the Core_Root path, I get a different error message:

PS C:\Users\acovingt> jit-diff diff --output C:\Users\acovingt\Documents\diffs --base C:\Users\acovingt\source\repos\runtime-master\artifacts\bin\coreclr\windows.x64.Release\  --core_root C:\Users\acovingt\source\repos\runtime-master\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root --corelib --verbose
After setting defaults:
--core_root C:\Users\acovingt\source\repos\runtime-master\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root
--output C:\Users\acovingt\Documents\diffs
--base C:\Users\acovingt\source\repos\runtime-master\artifacts\bin\coreclr\windows.x64.Release\
Beginning Crossgen CodeSize Diffs for System.Private.CoreLib.dll
Scanning: C:\Users\acovingt\source\repos\runtime-master\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root\System.Private.CoreLib.dll
Started dasm command "--platform C:\Users\acovingt\source\repos\runtime-master\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root --verbose --crossgen C:\Users\acovingt\source\repos\runtime-master\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root\crossgen.exe --jit C:\Users\acovingt\source\repos\runtime-master\artifacts\bin\coreclr\windows.x64.Release\clrjit.dll --output C:\Users\acovingt\Documents\diffs\dasmset_2\base C:\Users\acovingt\source\repos\runtime-master\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root\System.Private.CoreLib.dll"
Dasm command: jit-dasm --platform C:\Users\acovingt\source\repos\runtime-master\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root --verbose --crossgen C:\Users\acovingt\source\repos\runtime-master\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root\crossgen.exe --jit C:\Users\acovingt\source\repos\runtime-master\artifacts\bin\coreclr\windows.x64.Release\clrjit.dll --output C:\Users\acovingt\Documents\diffs\dasmset_2\base C:\Users\acovingt\source\repos\runtime-master\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root\System.Private.CoreLib.dll
- Finished 0/1 Base 0/0 Diff [0.1 sec]
assembly name: System.Private.CoreLib.dll
set COMPlus_NgenDisasm=*
set COMPlus_NgenUnwindDump=*
set COMPlus_NgenEHDump=*
set COMPlus_JitDiffableDasm=1
set COMPlus_JitEnableNoWayAssert=1
set COMPlus_JitNoForceFallback=1
set COMPlus_JitRequired=1
set COMPlus_JitStdOutFile=C:\Users\acovingt\Documents\diffs\dasmset_2\base\System.Private.CoreLib.dasm
Running: C:\Users\acovingt\source\repos\runtime-master\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root\crossgen.exe /silent /nologo /JitPath C:\Users\acovingt\source\repos\runtime-master\artifacts\bin\coreclr\windows.x64.Release\clrjit.dll /Platform_Assemblies_Paths C:\Users\acovingt\source\repos\runtime-master\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root /out C:\Users\acovingt\Documents\diffs\dasmset_2\base\System.Private.CoreLib.ni.dll C:\Users\acovingt\source\repos\runtime-master\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root\System.Private.CoreLib.dll
Unhandled exception. System.IO.FileNotFoundException: Could not find file 'C:\Users\acovingt\Documents\diffs\dasmset_2\base\System.Private.CoreLib.log'.
File name: 'C:\Users\acovingt\Documents\diffs\dasmset_2\base\System.Private.CoreLib.log'
   at System.IO.File.Move(String sourceFileName, String destFileName, Boolean overwrite)
   at ManagedCodeGen.jitdasm.DisasmEngine.GenerateAsm() in C:\Users\acovingt\source\repos\jitutils\src\jit-dasm\jit-dasm.cs:line 573
   at ManagedCodeGen.jitdasm.Main(String[] args) in C:\Users\acovingt\source\repos\jitutils\src\jit-dasm\jit-dasm.cs:line 228
Dasm command "jit-dasm --platform C:\Users\acovingt\source\repos\runtime-master\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root --verbose --crossgen C:\Users\acovingt\source\repos\runtime-master\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root\crossgen.exe --jit C:\Users\acovingt\source\repos\runtime-master\artifacts\bin\coreclr\windows.x64.Release\clrjit.dll --output C:\Users\acovingt\Documents\diffs\dasmset_2\base C:\Users\acovingt\source\repos\runtime-master\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root\System.Private.CoreLib.dll" returned with -532462766 failures
/ Finished 1/1 Base 0/0 Diff [5.5 sec]

Dasm commands returned -532462766 base failures, 0 diff failures.
Completed Crossgen CodeSize Diffs for System.Private.CoreLib.dll in 5.47s
Diffs (if any) can be viewed by comparing: C:\Users\acovingt\Documents\diffs\dasmset_2\base C:\Users\acovingt\Documents\diffs\dasmset_2\diff

git diff --no-index --diff-filter=M --exit-code --numstat C:\Users\acovingt\Documents\diffs\dasmset_2\diff C:\Users\acovingt\Documents\diffs\dasmset_2\base


Warning: Failures detected generating asm: -532462766 base, 0 diff

Edit: Updated to verbose output

[AndyAyersMS]

Thanks... still no idea what's going wrong. @dotnet/jit-contrib anyone have any ideas?

[BruceForstall]

I tried the same command line (with paths changed to my local paths) but didn't get any failure.

This wouldn't cause the failure, but you're not going to get any diffs by specifying a Release JIT unless you have replaced the JIT in the Release directory with a Checked or Debug build of the JIT.

Do you have write permission to the "C:\Users\acovingt\Documents\diffs" directory? Did jit-diff create any directories there?

You might have to run the tool under the debugger.

[BruceForstall]

Oh, looks like I was out of date. Hopefully #302 fixes it.

[alexcovington]

Yes, looks like #302 fixed it. Thanks so much everyone!