strange output to console on Windows about vk_raytrace

[mklefrancois]

This is strange indeed and I don't see this locally.
Is this under Linux? What is the OS and compiler used?

[tigrazone]

Windows 10.
I try vs 2019 and mingw.
Output is same

[tigrazone]

seems console properties determines wrong about support ANSI escape codes for colored output

[NBickford-NV]

Hi @tigrazone ! Yes, these are ANSI escape codes; it turns out cmd.exe doesn't format these by default! (Windows Terminal and Visual Studio's Debugger Command Prompt do.) It sounds like you may have figured this out: if you create a registry key in Computer\HKEY_CURRENT_USER\Console with name VirtualTerminalLevel, type DWORD, and value 1, it should enable ANSI color codes in cmd.exe.

Here's the content of a .reg file for it:

[HKEY_CURRENT_USER\Console]
"VirtualTerminalLevel"=dword:00000001

It looks like we can also improve this on the nvpro_core side by enabling ENABLE_VIRTUAL_TERMINAL_PROCESSING using SetConsoleMode(); I'm going to look more into this.

Thanks!

[tigrazone]

why If on users computer there is no described record in registry is showed ←[0m like in my issue record?
Is it real to determine if ANSI escape codes is off and dont show text with escapes?
Looks like issue in determine routine

[NBickford-NV]

That would relate to the ENABLE_VIRTUAL_TERMINAL_PROCESSING approach; I'm currently looking into that.

[tigrazone]

I tested fix:
add after https://github.com/nvpro-samples/nvpro_core/blob/500dea27f7043bfca3e82b5e246fe5f8c946cc54/nvh/nvprint.cpp#L242

#ifdef _WIN32
    HANDLE handleOut = GetStdHandle(((1 << level) & LOGBITS_ERRORS) ? STD_ERROR_HANDLE : STD_OUTPUT_HANDLE);
    DWORD consoleMode;
    GetConsoleMode( handleOut , &consoleMode);
    consoleMode |= ENABLE_VIRTUAL_TERMINAL_PROCESSING;
    consoleMode |= DISABLE_NEWLINE_AUTO_RETURN;            
    SetConsoleMode( handleOut , consoleMode );
#endif