MathLoadTest_autosimd crash Illegal instruction vmState=0x00000000 Compiled_method= about openj9

[pshipton]

@hzongaro fyi

[pshipton]

This is a 0.45 release build.
https://openj9-jenkins.osuosl.org/job/Test_openjdk22_j9_special.system_x86-64_windows_Release_testList_3/7/ - win2012x64-openj9-1a
MathLoadTest_autosimd_special_5m_2
-Xgcpolicy:optthruput -Xjit:count=0,optlevel=hot,gcOnResolve,rtResolve -Xnocompressedrefs

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk22_j9_special.system_x86-64_windows_Release_testList_3/7/system_test_output.tar.gz

13:44:42  MLT stderr Type=Illegal instruction vmState=0x00000000
13:44:42  MLT stderr Windows_ExceptionCode=c000001d J9Generic_Signal=00000010 ExceptionAddress=00007FFCFDA0D1C3 ContextFlags=0010005f
13:44:42  MLT stderr Handler1=00007FFD12C715B0 Handler2=00007FFD15B6ABA0
13:44:42  MLT stderr RDI=0000000000000000 RSI=00007FF6A8F03EF8 RAX=00007FF6A8F03EA8 RBX=00007FF6A8F03F58
13:44:42  MLT stderr RCX=0000000000000008 RDX=00007FF6A8F03F48 R8=0000000000000001 R9=00007FF6A8F03F08
13:44:42  MLT stderr R10=00007FF6A8F03EB8 R11=0000000000000000 R12=0000000000000007 R13=0000000000000000
13:44:42  MLT stderr R14=0000000000000007 R15=00007FF6A8F03EA8
13:44:42  MLT stderr RIP=00007FFCFDA0D1C3 RSP=000000C115107910 RBP=000000C113E11500 EFLAGS=0000000000010202
13:44:42  MLT stderr FS=0053 ES=002B DS=002B
13:44:42  MLT stderr XMM0 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:44:42  MLT stderr XMM1 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:44:42  MLT stderr XMM2 bf9e6ef750d2d67c (f: 1355994752.000000, d: -2.972018e-02)
13:44:42  MLT stderr XMM3 3f37a76fde89167d (f: 3733526016.000000, d: 3.609322e-04)
13:44:42  MLT stderr XMM4 3fd38a432efa283e (f: 788146240.000000, d: 3.053139e-01)
13:44:42  MLT stderr XMM5 3fa7dd0a579694c2 (f: 1469486336.000000, d: 4.660828e-02)
13:44:42  MLT stderr XMM6 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:44:42  MLT stderr XMM7 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:44:42  MLT stderr XMM8 3fee8524488267d7 (f: 1216505856.000000, d: 9.537527e-01)
13:44:42  MLT stderr XMM9 3f6fb81eaa4a9143 (f: 2857013504.000000, d: 3.871975e-03)
13:44:42  MLT stderr XMM10 3fb7dd0a579694c2 (f: 1469486336.000000, d: 9.321656e-02)
13:44:42  MLT stderr XMM11 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:44:42  MLT stderr XMM12 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:44:42  MLT stderr XMM13 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:44:42  MLT stderr XMM14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:44:42  MLT stderr XMM15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:44:42  MLT stderr Module=
13:44:42  MLT stderr Module_base_address=00007FFCFD000000 Offset_in_DLL=0000000000a0d1c3
13:44:42  MLT stderr 
13:44:42  MLT stderr Compiled_method=net/adoptopenjdk/test/autosimd/AutoSIMDTestDouble.simdSub([D[D[DI)V
13:44:42  MLT stderr Target=2_90_20240428_11 (Windows Server 2012 R2 6.3 build 9600)
13:44:42  MLT stderr CPU=amd64 (4 logical CPUs) (0x3fff77000 RAM)
13:44:42  MLT stderr ----------- Stack Backtrace -----------
13:44:42  MLT stderr (0x00007FFCFDA0D1C3)
13:44:42  MLT stderr (0x000000C115117B00)
13:44:42  MLT stderr (0x00007FF6A8F03E98)
13:44:42  MLT stderr (0x00007FF6A8F03F48)
13:44:42  MLT stderr (0x00007FF6A8F03EF8)
13:44:42  MLT stderr (0x00007FF6A8F03EA8)
13:44:42  MLT stderr (0x00007FFCFDA0C815)
13:44:42  MLT stderr (0x00007FF6A945E280)
13:44:42  MLT stderr (0x00007FF6A92A0260)
13:44:42  MLT stderr ---------------------------------------

[pshipton]

Dup of #19377?

These are all 64-bit JVMs.

[hzongaro]

@BradleyWood, may I ask you to look at this one as well?

[pshipton]

https://openj9-jenkins.osuosl.org/job/Test_openjdk17_j9_sanity.functional_x86-64_windows_Nightly_testList_0/709 - win2012x64-openj9-1a
SIMDCommonedAddressTest_0

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk17_j9_sanity.functional_x86-64_windows_Nightly_testList_0/709/functional_test_output.tar.gz

23:46:14  Unhandled exception
23:46:14  Type=Illegal instruction vmState=0x00040000
23:46:14  Windows_ExceptionCode=c000001d J9Generic_Signal=00000010 ExceptionAddress=00007FFC82C00102 ContextFlags=0010005f
23:46:14  Handler1=00007FFC9B4DC5D0 Handler2=00007FFC9A41ABA0
23:46:14  RDI=0000000000000532 RSI=00000007FFE70110 RAX=0000000000000000 RBX=0000000000000000
23:46:14  RCX=0000000000000542 RDX=00000007FFE71620 R8=00000007FFE70110 R9=00000007FFE71628
23:46:14  R10=0000000000000541 R11=00007FFC9A645397 R12=0000000000000000 R13=0000003242AD9EA8
23:46:14  R14=0000000000000000 R15=000000324234CC40
23:46:14  RIP=00007FFC82C00102 RSP=0000000000105D20 RBP=0000000000017000 EFLAGS=0000000000010293
23:46:14  FS=0053 ES=002B DS=002B
23:46:14  XMM0 000000324cc6f009 (f: 1288105984.000000, d: 1.067362e-312)
23:46:14  XMM1 0000000000000000 (f: 0.000000, d: 0.000000e+00)
23:46:14  XMM2 0000000000000000 (f: 0.000000, d: 0.000000e+00)
23:46:14  XMM3 0000000000000000 (f: 0.000000, d: 0.000000e+00)
23:46:14  XMM4 0000000000000000 (f: 0.000000, d: 0.000000e+00)
23:46:14  XMM5 0000000000000000 (f: 0.000000, d: 0.000000e+00)
23:46:14  XMM6 0000000000105de8 (f: 1072616.000000, d: 5.299427e-318)
23:46:14  XMM7 0000000000105de8 (f: 1072616.000000, d: 5.299427e-318)
23:46:14  XMM8 0000000000000000 (f: 0.000000, d: 0.000000e+00)
23:46:14  XMM9 0000000000000000 (f: 0.000000, d: 0.000000e+00)
23:46:14  XMM10 0000000000000000 (f: 0.000000, d: 0.000000e+00)
23:46:14  XMM11 0000000000000000 (f: 0.000000, d: 0.000000e+00)
23:46:14  XMM12 0000000000000000 (f: 0.000000, d: 0.000000e+00)
23:46:14  XMM13 0000000000000000 (f: 0.000000, d: 0.000000e+00)
23:46:14  XMM14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
23:46:14  XMM15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
23:46:14  Module=
23:46:14  Module_base_address=00007FFC82C00000 Offset_in_DLL=0000000000000102
23:46:14  
23:46:14  Compiled_method=jit/test/tr/SIMDOpts/SIMDCommonedAddressTest.testSIMDCommonedAddress([I[II)V
23:46:14  Target=2_90_20240430_754 (Windows Server 2012 R2 6.3 build 9600)
23:46:14  CPU=amd64 (4 logical CPUs) (0x3fff77000 RAM)
23:46:14  ----------- Stack Backtrace -----------
23:46:14  (0x00007FFC82C00102)
23:46:14  J9_GetInterface+0x19217 (0x00007FFC9B502D67 [j9vm29+0xf2d67])
23:46:14  (0x00000007FFEECBD0)
23:46:14  ---------------------------------------

[pshipton]

See also #19424 (cmdLineTester_loopReduction_0)
and #19377 (MathLoadTest_autosimd_5m_2 on win32)

All the failures occur on win2012x64-openj9-1a

[BradleyWood]

We definitely have a problem with generating AVX-512 on 32-bit JVMs in 64-bit machine. I think we have two issues here since there is a failure on a 64-bit jvm. @pshipton Could you get me the cpuid info of the machine that failed this test?

[BradleyWood]

The javacore states PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 85 which is one of cooperlake, skylake, or cascadelate (server). Each of these support AVX-512 so we must be looking at a problem in addition to #19377.

The javacore also states this, which makes no sense to me. Don't know how avx512dq can exist without avx512f.
JITFEATURE CPU features (JIT): fpu cx8 cmov mmx sse sse2 sse3 ssse3 fma sse4_1 sse4_2 popcnt aesni osxsave avx fdp_excptn_only avx512dq rdseed sha avx512vl null

[pshipton]

I think @AdamBrousseau will have to obtain the cpuid info of win2012x64-openj9-1a

[pshipton]

Is is a virtual machine, so perhaps it's messed up somehow.

[pshipton]

I've disabled https://openj9-jenkins.osuosl.org/computer/win2012x64%2Dopenj9%2D1a/ in jenkins since we don't need tests running on it and crashing.

Also opened infrastructure/issues/9283

[BradleyWood]

@pshipton Has anything similar happened on any other machine?

This is the instruction causing problems. It is valid on AVX-512 supported hardware.

62d17e086f441300     vmovdqu32 xmm0, xmmword ptr [r11 + rdx]

@AdamBrousseau Could you get me the cpuid info for win2012x64-openj9-1a

[pshipton]

Has anything similar happened on any other machine?

No, I checked all the failures and they were on win2012x64-openj9-1a

[AdamBrousseau]

[BradleyWood]

@AdamBrousseau I need the list of instruction set extensions supported by that CPU. Whatever command would be equivalent to lscpu on linux.

[AdamBrousseau]

Hopefully this helps

$ wmic cpu list /format:list

AddressWidth=64
Architecture=9
Availability=3
Caption=Intel64 Family 6 Model 85 Stepping 4
ConfigManagerErrorCode=
ConfigManagerUserConfig=
CpuStatus=1
CreationClassName=Win32_Processor
CurrentClockSpeed=2300
CurrentVoltage=
DataWidth=64
Description=Intel64 Family 6 Model 85 Stepping 4
DeviceID=CPU0
ErrorCleared=
ErrorDescription=
ExtClock=
Family=1
InstallDate=
L2CacheSize=
L2CacheSpeed=
LastErrorCode=
Level=6
LoadPercentage=1
Manufacturer=GenuineIntel
MaxClockSpeed=2300
Name=Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz
OtherFamilyDescription=
PNPDeviceID=
PowerManagementCapabilities=
PowerManagementSupported=FALSE
ProcessorId=1FCBFBFF00050654
ProcessorType=3
Revision=21764
Role=CPU
SocketDesignation=CPU 1
Status=OK
StatusInfo=3
Stepping=
SystemCreationClassName=Win32_ComputerSystem
SystemName=WIN2012R2-X86-1
UniqueId=
UpgradeMethod=1
Version=
VoltageCaps=0

https://www.intel.com/content/www/us/en/products/sku/120485/intel-xeon-gold-6140-processor-24-75m-cache-2-30-ghz/specifications.html
https://en.wikichip.org/wiki/intel/xeon_gold/6140#google_vignette

[BradleyWood]

@AdamBrousseau So the cpu in question does support AVX-512, and therefore the instruction in this issue. But that output does not tell me if it is enabled or not.