Error while setting up environment for the application

[peter8777555]

OS : Windows 11 (10.0.22631.2715 23H2)

OK:
C:\Windows\system32\cmd.exe and run any DOS programs

NG:
C:\Windows\SysWOW64\cmd.exe and run any DOS programs

---

16 bit MS-DOS Subsystem

Administrator: C:\Windows\SysWOW64\cmd.exe - pc
Error while setting up environment for the application. Choose 'Close' to terminate the application.

Close(C) Ignore(I)

LOG.zip

[peter8777555]

OS : Windows 11 (10.0.22631.2715 23H2)

OK:
C:\Windows\system32\cmd.exe and run any DOS programs

NG:
Explorer.exe double click run any DOS programs.

16 bit MS-DOS Subsystem

G:\PC.EXE
NTVDM has encountered a System Error
Access denied。
Choose 'Close' to terminate the application.

Close(C) Ingore(I)

I use Administrator account.

By the way,
Windows 7 X64 has no any problems.

Log.zip

[pcmicro]

Could this be caused because Microsoft removed the "Legacy console" feature in December 2023?

[leecher1337]

So RegisterConsoleVDM call failed. Could be due to Legacy Console removal, as pcmicro said.
You can try to reset symbols first with Release\reg\reloadsym.reg just to be sure that it is not related to some bad sybols which often causes weird problems.
Otherwise a debugger would be needed to step into mentioned function and also on attached conhost process to check where it fails in particular.

I cannot reproduce the problem, tried on Windows 11 23H2 Build 22631 without any issues (but conhost V1 also still works for me on this release).

[XxVanHelsingxX]

Hello I have setup ntvdmx64 without issues on windows 11 but I also get Error while setting up environment for the application. Choose 'Close' to terminate the application.
On to other machines I got my dos program running without problems - also windows 11
I don't know how to find the issues - all files are there and the registry keys seem to be OK too

[leecher1337]

Did you install latest NTVDMx64 release which installs conhostv1 properly?
581da17

You need at least this patch level so that it forces installation of ConhostV1 under Windows 11, as it became an optional component in latest Win11 release.

[XxVanHelsingxX]

I compiled the newest version today but I always get the same error
ConhostV1 is present in System32

Also tested on another machine 24H2 today and got it working
I have no clue why I get the error on this machine - only difference would be secure boot is enabled at point of install on my problem machine

I see - on 16Bit support install
systemprofile\AppData\Local\Microsoft\Windows\INetCache\Content.IE5*: Access is being processed. A processing error has occurred for 1 file.

Seems like an access denied for all files that get edited in "C:\Windows\SysWOW64**" error when placing files for 16 bit support
I run as Administrator and tried as NT\System
I am able to copy the files over to SysWow64 but the error for systemprofile\AppData\Local\Microsoft\Windows\INetCache\Content.IE5* remains

Thanks for your help!

[leecher1337]

The error usually indicates some problem with the Process Environment or with the call to GetNextVDMCommand().
Usually, the loader puts the application to start into a list of applications to execute which is then fetched via GetNextVDMCommand Windows-API.
To determine the exact error, a Stacktrace from a debug build may be needed. So you need the checked build (ccpu-chk, which is default anyway). After installation on NTVDMx64, you need to install the debug symbols with inst-sym.cmdfrom the installer directory.
Then you can use a debugger like x64dbg.
To let ntvdmx64 break on startup, run reg\yoda.reg from the installer directory.
Then run symbols\vdmdebug.exe from the installer directory.
Then try to run your DOS application. You will break into yoda in vdmdbg.exe window.
Open x32dbg.exe and attach to ntvdm.exe process
Go to symbols / ntvdm.exe and place a breakpoint on cmdGetNextCmd with F2 on the right window with the symbols, then continue with F9

Also place a breakopint on RcErrorDialogBox:

Go to yoda window and enter c to continue execution. If you are receiving a break on exception, continue with SHIFT+F9 until you end up in cmdGetNextCmd:

Now you can stop through with F8 and check where it goes to the RcErrorDialogBox and send a screenshot from it.
Next time you know, you can break a Breakpoint with F2 before the function that returns the failure and step into it with f7 and then step through it with F8 to find out where it breaks.
There are a few possible locations where it can exit with an environment error and we need to isolate the section with the debugger in order to find out more.

When finished, you can remove the break into YODA again by deleting
the Key named YODA under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\WOW\CpuEnv in the registry (you can reenable it with the .reg file whenever you want).
If you don't remove the YODA key and you don't have vdmdebug.exe running when starting NTVDM, then NTVDM.exe will lock up in an endless loop while trying to display YODA-> Prompt (then has to be kille via taskmanager).
Also YODA key doesn't like multiple concurrent NTVDMs running, so it's just for debugging and not for production use.