Support Windows x64 about data-race-test

[GoogleCodeExporter]

Windows x64 is not supported by TSan(v1) at this moment.
Technically, there are no known roadblocks; we've just never aimed at Win x64 
support and don't plan to as of today.

One day I got some Win x64-related patches, and I already committed the most 
obvious bits into TSan sources.
Unfortunately, I've never heard back from the author.
That said, your patches are welcome!

I'd optimistically estimate the effort as "a few days to read the existing and 
understand the Windows bits" and "a week to fix all problems in the 
x64-specific parts".
One of the reasons we don't work on this is that we don't have a decent 
multi-threaded Win x64 app to test TSan on.
It should be easier for you guys as you do have such an app.
If you can provide new minimized test cases for racecheck_unittest.cc that show 
some problems on Win x64 I might help you with some advice on how to fix them.

---
Also, there's TSan v2 in active development now:
http://code.google.com/p/thread-sanitizer/
It should be doable to add support for Win_x64 as soon as Clang works there 
(hm, that's not close now) and with some effort to port the runtime.

Original comment by [email protected] on 15 Nov 2012 at 9:42

• Changed title: Support Windows x64
• Added labels: OpSys-Windows, Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

That sounds great - I'd like to pursue this. We definitely have a good test app 
- about 500K lines of heavily multi-threaded C++ (closed source). I downloaded 
the source, but the build process appears to be all Linux-oriented. I am 
familiar with makefiles etc. on Linux though. Are there instructions somewhere 
for the 32-bit Windows build?

Original comment by [email protected] on 15 Nov 2012 at 1:53

[GoogleCodeExporter]

Hm, I'm pretty sure we had a separate wiki page for that, not sure why it was 
deleted [was out of date?]

See how it's built on our buildbot:
http://build.chromium.org/p/client.tsan/builders/buildbot-win7
-> pick the latest build
-> hit on the "stdio" link just below "build tsan with pin" or "build debug 
tsan with pin"
-> look for the "argv" line at the top.

Please note that we're currently using an slightly out-of-date version of PIN 
which has bugs on Win x64, you should probably rebase the TSan sources to work 
with a newer PIN.

Original comment by [email protected] on 15 Nov 2012 at 2:00

[GoogleCodeExporter]

See
http://code.google.com/p/data-race-test/source/diff?spec=svn3607&old=3606&r=3607
&format=side&path=%2Fwiki%2FThreadSanitizerPin.wiki
for the old version of the instruction.

Original comment by [email protected] on 15 Nov 2012 at 2:02

[GoogleCodeExporter]

Ah, sorry - it was a Linux PIN page.

If you need help besides c#3, please ping me.
Otherwise, it'd be good if you could send me some text to add to a Windows 
subsection of 
http://code.google.com/p/data-race-test/wiki/BuildingThreadSanitizer

Original comment by [email protected] on 15 Nov 2012 at 2:04

[GoogleCodeExporter]

Okay - thanks Timur. I should have time to get started on that today. I'll let 
you know if I run into any issues.

Jan

Original comment by [email protected] on 15 Nov 2012 at 2:10

[GoogleCodeExporter]

I installed Cygwin and I'm trying the 32-bit make, but it isn't finding pin.H. 
I wonder if it's because the cl command is using /I instead of -I. Here's the 
make command and output, with an "ls" at the bottom to show that pin.H is 
really there.

$ make -C tsan -j4 VALGRIND_ROOT= PIN_ROOT=/cygdrive/e/pin w32d
make: Entering directory `/cygdrive/e/tsanv1/tsan'
make all OS=windows ARCH=x86 DEBUG=1
make[1]: Entering directory `/cygdrive/e/tsanv1/tsan'
VALGRIND_ROOT is not set. Not building the Valgrind-based variant.
cl -nologo -c -Zi -MT -EHs- -EHa- -wd4530 -D_CRT_SECURE_NO_DEPRECATE 
-D_SECURE_SCL=0 -D_HAS_ITERATOR_DEBUGGING=0 -Gy -Ox -GL 
/I/cygdrive/e/pin/source/include /I/cygdrive/e/pin/source/include/gen 
/I/cygdrive/e/pin/extras/xed2-ia32/include /DTARGET_WINDOWS 
/DBIGARRAY_MULTIPLIER=1 /DUSING_XED /DTARGET_IA32 /DHOST_IA32a /D_SECURE_SCL=0 
-DTS_PIN=1 -Fobin/x86-windows-debug-pinmp-ts_pin.obj -c ts_pin.cc 
-DTS_VERSION=\"0\" -DDEBUG=1 -I../dynamic_annotations -DTS_SERIALIZED=0
ts_pin.cc
ts_pin.cc(32) : fatal error C1083: Cannot open include file: 'pin.H': No such 
file or directory
Makefile:253: recipe for target `bin/x86-windows-debug-pinmp-ts_pin.obj' failed
make[1]: *** [bin/x86-windows-debug-pinmp-ts_pin.obj] Error 2
make[1]: Leaving directory `/cygdrive/e/tsanv1/tsan'
Makefile:145: recipe for target `w32d' failed
make: *** [w32d] Error 2
make: Leaving directory `/cygdrive/e/tsanv1/tsan'

$ ls /cygdrive/e/pin/source/include
compiler_version_check2.H  level_core.PLH        pin.H        pin_profile.H   
pin-errtype.h
foundation.PLH             level_foundation.PLH  pin_basic.h  pin_ucontext.h  
pintool.exp
gen                        level_pinclient.PLH   pin_cache.H  pin_util.H      
pintool.ver
level_base.PLH             level_vmapi.PLH       pin_isa.H    pinapp.h        
portability.H

Original comment by [email protected] on 15 Nov 2012 at 10:16

[GoogleCodeExporter]

I tried entering the cl command by hand with -I but that didn't work either:

$ cl -nologo -c -Zi -MT -EHs- -EHa- -wd4530 -D_CRT_SECURE_NO_DEPRECATE 
-D_SECURE_SCL=0 -D_HAS_ITERATOR_DEBUGGING=0 -Gy -Ox -GL 
-I/cygdrive/e/pin/source/include -I/cygdrive/e/pin/source/include/gen 
-I/cygdrive/e/pin/extras/xed2-ia32/include -DTARGET_WINDOWS 
-DBIGARRAY_MULTIPLIER=1 -DUSING_XED -DTARGET_IA32 -DHOST_IA32a -D_SECURE_SCL=0 
-DTS_PIN=1 -Fobin/x86-windows-debug-pinmp-ts_pin.obj -c ts_pin.cc 
-DTS_VERSION=\"0\" -DDEBUG=1 -I../dynamic_annotations -DTS_SERIALIZED=0
ts_pin.cc
ts_pin.cc(32) : fatal error C1083: Cannot open include file: 'pin.H': No such 
file or directory

Original comment by [email protected] on 15 Nov 2012 at 10:21

[GoogleCodeExporter]

[FTR the problem has been solved over e-mail by replacing "-I/cygdrive/e/" with 
"-IE:/" ]

Original comment by [email protected] on 20 Nov 2012 at 9:25

[GoogleCodeExporter]

Is Win64 support in TSAN v1 still being actively worked on?

Original comment by [email protected] on 28 Feb 2013 at 1:53

[GoogleCodeExporter]

Not that I know of. We also mostly abandoned tsan1 on other platforms and 
switched to tsan2 (code.google.com/p/thread-sanitizer/)

Original comment by [email protected] on 28 Feb 2013 at 4:47

[GoogleCodeExporter]

I was able to get tsan1 building on Win64 and I got the unit tests to pass, but 
there are a lot of problems running it (false positives in the Windows runtime 
mainly), and I don't have the time or expertise to fix that. When will tsan2 be 
ready to try? 

Original comment by [email protected] on 28 Feb 2013 at 2:35

[GoogleCodeExporter]

The main question is -- do you able to build your codebase with clang/gcc?
Porting of tsan2 runtime to windows should not be a rocket science. But we do 
not invest in it ATM because we are unable to build our codebase with clang/gcc 
on windows. 

Original comment by [email protected] on 28 Feb 2013 at 2:55

[GoogleCodeExporter]

Right - so tsan2 doesn't apply to most Windows apps, such as mine. Hopefully 
somebody will maintain tsan1 then, and help get the Win64 support working! 

Original comment by [email protected] on 28 Feb 2013 at 3:13

[GoogleCodeExporter]

RE: codegen on Windows
I'm currently working on this for ASan for Win x86 (see issue 
address-sanitizer:56) and once we're there it shouldn't be too hard to add x64 
support and TSan v2 runtime.

However, I wouldn't expect quick results (the task is huge), so Windows is 
tsanv1-only for now.

Original comment by [email protected] on 1 Mar 2013 at 8:36