[Prometheus]: Build Tools

[RobBryce]

Contact Details

[email protected]

What happened?

Prometheus uses Visual Studio 2019 as the host/base compilation tools. Prometheus also uses the Intel OneAPI compiler to build portions of the binary where performance is absolutely needed.

We have encountered migration issues as the Prometheus project has evolved with various generations of Microsoft compiler tools. We have also encountered compatibility issues between the Intel suite of compilers and the Microsoft compilers.

Visual Studio 2019 is now marked for strictly bug and security issue fixes, and has recommended migrating all projects to Visual Studio 2022, which was formally release (no longer beta) months ago.

There are always benefits in migration to new versions of these tools. However, there are also pain points along the way, with these migrations. We've encountered these in the past, with UI issues, some issues with generation of statistics, and compilation and linking issues that need to be resolved along the way to simply build a program that will run.

There remains work to be done in Prometheus, so I'm asking for direction on whether we should stay with VS2019, or follow recommendations (which also impact prerequisites, etc.) and migrate the projects to VS2022.

Version

(Dev) 2021.12.03

What version of Windows are you seeing the problem on?

Windows 11

Relevant log output

Note that the list of versions of Windows includes 32-bit targets.  These are impossible since we haven't build a 32-bit version of Prometheus for a very long time.

Code of Conduct

• I agree to follow this project's Code of Conduct

[spydmobile]

from Exec: @RobBryce We will be building Both Windows and Linux versions of W.I.S.E.

[spydmobile]

This must be GitHub compliant too!!

[RobBryce]

Prometheus won't be built in Github, and this ticket is for Prometheus, not PSaaS.

Github does offer Visual Studio compilers for Windows targets, and we are using Linux compilers for the Linux target. Github does not yet offer the Intel OneAPI compiler suite, that we use for all builds on Windows.

[spydmobile]

I may not understand the repercussions but I think we should stay current and go VS2022

[spydmobile]

also @RobBryce I created #141 to address the 32-bit windows in the Issues templates.

[BadgerOnABike]

I would imagine moving to VS2022 would be a good move for the final release of Prometheus. I really don't feel good about having to provide an out-of-date intel compiler that is not using their OneAPI system. My thoughts would be that the extensibility of OneAPI would provide longer serviceability of the tool before it versions out than sitting on an out-of-date compiler.

[spydmobile]

I believe Exec decided that we do not want to continue hard coding to specific versions unless we have no other choice, Didnt we?. So i think we are steering away from it, Meanwhile, lets just remember this is only for EOL Promethues.

[RobBryce]

Intel is proving to be a moving target, with compiler options and warning messages coming and going with different releases. But they are investing heavily into it and I believe it's a good/great product.

Microsoft has traditionally been very stable, and there's been a migratory path and tools to update projects during that migration. However, each version of the compiler is also tied to a major OS release, so VS2022 is for Win11 (more/less). With that, will come minor updates to MFC (which will need testing, since we've encountered changes in UI behaviour before). And they will have updated their implementation for newer versions of C++, and in some instances we have discovered changes in behaviour where we got erroneous outputs. So migrating to a new version of the compiler does require validation.

There's also compatibility concerns between Microsoft and Intel, and Microsoft and Java, and Microsoft and Microsoft.

[spydmobile]

Do we migrate from VS2019 to VS2022 or not?
@RobBryce thinks - "I prefer to use the latest but if we do, Prometheus needs rigorous testing to ensure bugs in VS2022 do not cause regressions in Prometheus."
Moving to 2022
Pros

• Run Prometheus EOL on Windows 11
• Best version IDE and Compiler with latest OS Support
• Older bugs in IDE and Compiler are fixed.
  Con
• Level of regression testing that SOMEONE must do - this is testing the effect of the compiler on the code, NOT Robs coding which does not change by his hand, but by interpretation of the compiler. This has happened before.

[spydmobile]

@nealmcloughlin and @BadgerOnABike can you please share your thoughts so we can either action this or not.

[spydmobile]

This should be considered for both Prometheus and WISE but also for what is in GitHub

[RobBryce]

I did some exploratory work and I can get debug versions (v6 and v7) to run. However, I'm having linking issues with the release targets. I'm using Visual Studio 2022 17.4.2 (current), but Intel reports it should be compatible with 17.3. https://www.intel.com/content/www/us/en/developer/articles/reference-implementation/intel-compilers-compatibility-with-microsoft-visual-studio-and-xcode.html

If linking issues can be resolved, then things look promising. However, linking issues are some of the most difficult to resolve and can be a time sink to find the correct combination of compiler and linker options.

[BadgerOnABike]

I still think we go ahead.

Does this require heavy regression testing? Yes

Can we generate these regression tests as fgmjs within WISE? Hopefully also yes?

We have a unique capacity now that we've never had before in that we can use our job specification tool to generate a constant set of tests for testing against rather than trusting that humans are doing a good job of performing every action the same every time.

UI components can be tested relatively easily, but the computational effort should be pseudo-automated.

[RobBryce]

To be clear, I believe that most of the issues with migration to C++17 have been identified and we've isolated them appropriately. We've also tightened up the code a lot along the way, using the new language keywords to help. There are still a few files that we need to watch (e.g. functionality for John Braun's work for percentiles) that need tested carefully.

[RobBryce]

Updating to VS2022 and Intel 2023.

[RobBryce]

https://devblogs.microsoft.com/dotnet/net-framework-4-5-2-4-6-4-6-1-will-reach-end-of-support-on-april-26-2022/ This is important. We are now one year past reviewing versions of .Net. This has been rolled ahead to .Net 4.8 for the current Prometheus build, to match what's available (by default) with the Visual Studio 2022 tool.

It looks like v4.8 may reach end of life with Windows 10. It's unclear and I'm not finding definite dates here. If that's the case, than that will occur in 2025.

This will require some additional testing, as we use .Net for the stat's view and some controls in some dialogs.