kejiewei / thread-sanitizer Goto Github PK

Implement builtin deadlock detector in tsan.
This will give us feature parity with Helgrind.

Most modern concurrent software is built around the concept of "tasks" rather 
than raw threads. Tsan could understand the concept of tasks natively. This 
would allow to (1) print better reports (task creation stacks instead of thread 
creation stacks) and (2) eliminate false negatives due to shared worker threads 
(a task is synchronized with something just because a previous task on the same 
worked thread was synchronized with that).

The proposed interface:
https://codereview.appspot.com/6904068/diff/6001/rtl/tsan_interface_task.h

---------- Forwarded message ----------
From: Christian Holler
Date: Tue, Nov 19, 2013 at 3:35 AM
Subject: Problem with cmov/wrong instrumented code?
To: [email protected]

Hi,

it would be really nice if some TSan developers could take a look at
https://bugzilla.mozilla.org/show_bug.cgi?id=939788 . In particular
comment 14 suggests that there could be a bug in the way TSan generates
it's instrumented code. Basically we're checking if we are the main
thread with a function called NS_IsMainThread() and then read a value if
that is true. However, with TSan, we always seem to read the value
(unconditionally), leading to a race report.

It would be nice if someone could figure out what exactly is going wrong
there and how we (or you) can fix it.


Thanks,

Chris

What steps will reproduce the problem?

following instructions from:
http://www.chromium.org/developers/testing/threadsanitizer-tsan-v2

GYP_GENERATORS=ninja GYP_DEFINES='tsan=1 linux_use_tcmalloc=0 
release_extra_cflags="-gline-tables-only" disable_nacl=1' gclient runhooks
ninja -C out/Release base_unittests
export G_SLICE=always-malloc
export NSS_DISABLE_ARENA_FREE_LIST=1
export NSS_DISABLE_UNLOAD=1
TSAN_OPTIONS="suppressions=tools/valgrind/tsan_v2/suppressions.txt 
report_signal_unsafe=0 report_thread_leaks=0" out/Release/base_unittests 
--gtest_filter=-ProcessUtilTest.GetAppOutputRestrictedNoZombies 2>&1 | tee log


What is the expected output? What do you see instead?

actual:

FATAL: ThreadSanitizer can not mmap the shadow memory (something is mapped at 
0x555555554000 < 0x7cf000000000)
FATAL: Make sure to compile with -fPIE and to link with -pie.

expected:
tests run

What version of the product are you using? On what operating system?

trunk (r173381 of chromium)

ubuntu 12.04
Linux 3.2.0-35-generic #55-Ubuntu SMP Wed Dec 5 17:42:16 UTC 2012 x86_64 x86_64 
x86_64 GNU/Linux

Repro by Julien Tinnes, 
https://code.google.com/p/chromium/issues/detail?id=258655

$ cat sig.cc 

#include <assert.h>
#include <stdlib.h>
#include <signal.h>
#include <unistd.h>

static bool signal_handler_ran = false;

void do_nothing_signal_handler(int signum) {
  write(2, "BBBBBBB\n", 8);
  signal_handler_ran = true;
}

int main() {
  int kSignalToTest = SIGSYS;
  assert(SIG_ERR != signal(kSignalToTest, do_nothing_signal_handler));
  sigset_t empty_set;
  assert(0 == sigemptyset(&empty_set));
  sigset_t one_signal = empty_set;
  assert(0 == sigaddset(&one_signal, kSignalToTest));
  sigset_t old_set;
  assert(0 == sigprocmask(SIG_BLOCK, &one_signal, &old_set));
  raise(kSignalToTest);
  assert(!signal_handler_ran);
  sigset_t all_but_one;
  assert(0 == sigfillset(&all_but_one));
  assert(0 == sigdelset(&all_but_one, kSignalToTest));
  sigsuspend(&all_but_one);
  assert(signal_handler_ran);

  // Restore the original set.
  assert (0 == sigprocmask(SIG_SETMASK, &old_set, NULL));
}
$ bin/clang++ sig.cc   -o sig
$ ./sig 
BBBBBBB
$ bin/clang++ sig.cc -fsanitize=thread  -o sig -g
$ ./sig 
==================
WARNING: ThreadSanitizer: signal-unsafe call inside of a signal (pid=11791)
    #0 write /usr/local/google/asan/llvm/projects/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:232 (exe+0x000000018344)
    #1 do_nothing_signal_handler(int) /usr/local/google/asan/llvm/llvm_cmake_build/sig.cc:9 (exe+0x000000063d43)
    #2 __tsan::ProcessPendingSignals(__tsan::ThreadState*) /usr/local/google/asan/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:1877 (exe+0x00000000d2d4)
    #3 __libc_start_main /build/buildd/eglibc-2.15/csu/libc-start.c:226 (libc.so.6+0x00000002176c)

SUMMARY: ThreadSanitizer: signal-unsafe call inside of a signal 
/usr/local/google/asan/llvm/llvm_cmake_build/sig.cc:9 
do_nothing_signal_handler(int)
==================
BBBBBBB
sig: sig.cc:28: int main(): Assertion `signal_handler_ran' failed.
Aborted (core dumped)

On Linux, it is possible for a user to define a name for a thread via 
non-standard pthread_setname_np function. It would be nice if ThreadSanitizer 
would show these names in the information about data races (in addition to 
"thread T%d" text). GDB, for example, provides these names in the output of 
'info threads' command.

Test added to llvm repo: tsan/lit_tests/aligned_vs_unaligned_race.cc

uint64_t Global[2];

void *Thread1(void *x) {
  Global[1]++;
  return NULL;
}

void *Thread2(void *x) {
  char *p1 = reinterpret_cast<char *>(&Global[0]);
  uint64_t *p4 = reinterpret_cast<uint64_t *>(p1 + 1);
  (*p4)++;
  return NULL;
}

tsan currently does not find this race. 
I don't see an easy way to do it w/o sacrificing performance for the
common case.
This bug is just for the record, I don't think we should do anything 
about it, at least in the short term. 

both helgrind and tsan v1 find this race, but the code that allows to do it
slows down the common path. 

Similar issue in asan: 
https://code.google.com/p/address-sanitizer/issues/detail?id=100

At Chrome we're building Clang using `make` (see 
http://src.chromium.org/viewvc/chrome/trunk/src/tools/clang/scripts/update.sh?re
vision=188142&view=markup), and this results in the runtime having no debug 
info (e.g. the report summaries contain '??:0' if the top frame contains a 
function interceptor).
This needs to be fixed, at least by adding -g to the appropriate makefiles (the 
best fix will involve dropping Makefile.old in favor of the Clang buildsystem 
configs)

This is very helpful for projects that are not error-clean yet (e.g. Chromium)

What steps will reproduce the problem?
1.
2.
3.

What is the expected output? What do you see instead?


What version of the product are you using? On what operating system?


Please provide any additional information below.

As of compiler-rt r182843, when building ASan RTL with MSVC -W2:
$ cl -nologo -W2 -I.. -I../../include -c asan_allocator2.cc
asan_allocator2.cc
..\sanitizer_common/sanitizer_quarantine.h(150) : warning C4146: unary minus 
operator applied to unsigned type, result still unsigned
        ..\sanitizer_common/sanitizer_quarantine.h(145) : while compiling class template member function '__sanitizer::QuarantineBatch *__sanitizer::QuarantineCache<Callback>::DequeueBatch(void)'
        with
        [
            Callback=__asan::QuarantineCallback
        ]
        ..\sanitizer_common/sanitizer_quarantine.h(79) : see reference to class template instantiation '__sanitizer::QuarantineCache<Callback>' being compiled
        with
        [
            Callback=__asan::QuarantineCallback
        ]
        asan_allocator2.cc(253) : see reference to class template instantiation '__sanitizer::Quarantine<Callback,Node>' being compiled
        with
        [
            Callback=__asan::QuarantineCallback,
            Node=__asan::AsanChunk
        ]

TSan fails to detect the following UAF:

struct MyClass {
 virtual ~MyClass() {
   LOG(INFO) << "~MyClass()";
 }
 virtual void Doit() {
   LOG(INFO) << "Doit()";
 }
};

int main() {
 char data[sizeof(MyClass)];
 MyClass *c = new(data) MyClass();
 c->~MyClass();
 c->Doit();
}

To fix this we can emit an additional vptr write at the end of every dtor.

$ TSAN_OPTIONS="history_size=7 
external_symbolizer_path=third_party/llvm-build/Release+Asserts/bin/llvm-symboli
zer" out/Release/base_unittests 
--gtest_filter=ThreadCollisionTest.MTB*:ThreadLocalStorageTest.TLS*
Note: Google Test filter = ThreadCollisionTest.MTB*:ThreadLocalStorageTest.TLS*
[==========] Running 2 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 1 test from ThreadLocalStorageTest
[ RUN      ] ThreadLocalStorageTest.TLSDestructors
[       OK ] ThreadLocalStorageTest.TLSDestructors (98 ms)
[----------] 1 test from ThreadLocalStorageTest (99 ms total)

[----------] 1 test from ThreadCollisionTest
[ RUN      ] ThreadCollisionTest.MTBookCriticalSectionTest
==================
WARNING: ThreadSanitizer: data race (pid=6979)
  Write of size 8 at 0x7fff1da08148 by main thread:
    #0 ThreadCollisionTest_MTBookCriticalSectionTest_Test::TestBody() /usr/local/google/chrome-asan/src/out/Release/../../base/threading/thread_collision_warner_unittest.cc:163 (base_unittests+0x0000004fb113)
    #1 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /usr/local/google/chrome-asan/src/out/Release/../../testing/gtest/src/gtest.cc:2051 (base_unittests+0x0000005cf26e)
    #2 testing::Test::Run() /usr/local/google/chrome-asan/src/out/Release/../../testing/gtest/src/gtest.cc:2067 (base_unittests+0x0000005cf0d0)
    #3 testing::TestInfo::Run() /usr/local/google/chrome-asan/src/out/Release/../../testing/gtest/src/gtest.cc:2244 (base_unittests+0x0000005cfe06)
    #4 testing::TestCase::Run() /usr/local/google/chrome-asan/src/out/Release/../../testing/gtest/src/gtest.cc:2351 (base_unittests+0x0000005d07e8)
    #5 testing::internal::UnitTestImpl::RunAllTests() /usr/local/google/chrome-asan/src/out/Release/../../testing/gtest/src/gtest.cc:4177 (base_unittests+0x0000005d8734)
    #6 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /usr/local/google/chrome-asan/src/out/Release/../../testing/gtest/src/gtest.cc:2051 (base_unittests+0x0000005d7ffe)
    #7 testing::UnitTest::Run() /usr/local/google/chrome-asan/src/out/Release/../../testing/gtest/src/gtest.cc:3810 (base_unittests+0x0000005d7ecd)
    #8 base::TestSuite::Run() /usr/local/google/chrome-asan/src/out/Release/../../base/test/test_suite.cc:195 (base_unittests+0x000000818117)
    #9 main /usr/local/google/chrome-asan/src/out/Release/../../base/test/run_all_unittests.cc:10 (base_unittests+0x0000006a08b8)

  Previous read of size 4 at 0x7fff1da0814c by thread T4:
    #0 base::(anonymous namespace)::ThreadLocalStorageCleanup(void*) /usr/local/google/chrome-asan/src/out/Release/../../base/threading/thread_local_storage_unittest.cc:64 (base_unittests+0x0000004fd7c4)
    #1 start_thread <null>:0 (libpthread.so.0+0x000000006a60)

==================
==================
WARNING: ThreadSanitizer: data race (pid=6979)
  Write of size 8 at 0x7fff1da08140 by main thread:
    #0 scoped_ptr /usr/local/google/chrome-asan/src/out/Release/../../base/memory/scoped_ptr.h:147 (base_unittests+0x0000004fb123)
    #1 scoped_ptr /usr/local/google/chrome-asan/src/out/Release/../../base/memory/scoped_ptr.h:147 (base_unittests+0x0000004fb123)
    #2 NonThreadSafeQueue /usr/local/google/chrome-asan/src/out/Release/../../base/threading/thread_collision_warner_unittest.cc:131 (base_unittests+0x0000004fb123)
    #3 ThreadCollisionTest_MTBookCriticalSectionTest_Test::TestBody() /usr/local/google/chrome-asan/src/out/Release/../../base/threading/thread_collision_warner_unittest.cc:165 (base_unittests+0x0000004fb123)
    #4 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /usr/local/google/chrome-asan/src/out/Release/../../testing/gtest/src/gtest.cc:2051 (base_unittests+0x0000005cf26e)
    #5 testing::Test::Run() /usr/local/google/chrome-asan/src/out/Release/../../testing/gtest/src/gtest.cc:2067 (base_unittests+0x0000005cf0d0)
    #6 testing::TestInfo::Run() /usr/local/google/chrome-asan/src/out/Release/../../testing/gtest/src/gtest.cc:2244 (base_unittests+0x0000005cfe06)
    #7 testing::TestCase::Run() /usr/local/google/chrome-asan/src/out/Release/../../testing/gtest/src/gtest.cc:2351 (base_unittests+0x0000005d07e8)
    #8 testing::internal::UnitTestImpl::RunAllTests() /usr/local/google/chrome-asan/src/out/Release/../../testing/gtest/src/gtest.cc:4177 (base_unittests+0x0000005d8734)
    #9 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /usr/local/google/chrome-asan/src/out/Release/../../testing/gtest/src/gtest.cc:2051 (base_unittests+0x0000005d7ffe)
    #10 testing::UnitTest::Run() /usr/local/google/chrome-asan/src/out/Release/../../testing/gtest/src/gtest.cc:3810 (base_unittests+0x0000005d7ecd)
    #11 base::TestSuite::Run() /usr/local/google/chrome-asan/src/out/Release/../../base/test/test_suite.cc:195 (base_unittests+0x000000818117)
    #12 main /usr/local/google/chrome-asan/src/out/Release/../../base/test/run_all_unittests.cc:10 (base_unittests+0x0000006a08b8)

  Previous read of size 4 at 0x7fff1da08140 by thread T1:
    #0 base::(anonymous namespace)::ThreadLocalStorageCleanup(void*) /usr/local/google/chrome-asan/src/out/Release/../../base/threading/thread_local_storage_unittest.cc:64 (base_unittests+0x0000004fd7c4)
    #1 start_thread <null>:0 (libpthread.so.0+0x000000006a60)

==================
[       OK ] ThreadCollisionTest.MTBookCriticalSectionTest (136 ms)
[----------] 1 test from ThreadCollisionTest (136 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 2 test cases ran. (235 ms total)
[  PASSED  ] 2 tests.

Because the TLSDestructors test does the maximum possible number of its 
destructor's iterations, TSan is unable to jump in after the last one.
This leads to false race reports between the TSD destructor and the next user 
of the same TSD slot.

Running a test under ThreadSanitizer I see the following log:

$ /usr/local/google/chrome-asan/src$ out/Release/net_unittests 
--gtest_filter=DiskCacheEntryTest.SimpleCacheSizeAtCreate 2>&1 | tee log

==================
WARNING: ThreadSanitizer: data race (pid=8375)
  Read of size 4 at 0x7d1800004ff0 by thread T2:
    #0 StripTrailingSeparatorsInternal /usr/local/google/chrome-asan/src/out/Release/../../base/files/file_path.cc:1246 (exe+0x0000020ac959)
    #1 DirName base/files/file_path.cc:307 (exe+0x0000020abe72)
...
  Previous atomic write of size 4 at 0x7d1800004ff0 by thread T3:
    #0 __tsan_atomic32_fetch_add /work/chromium/src/third_party/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interface_atomic.cc:465 (exe+0x00000024f8b7)
    #1 __exchange_and_add_dispatch /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/ext/atomicity.h:80 (exe+0x0000020ab10d)
    #2 ~BindState /usr/local/google/chrome-asan/src/out/Release/../../base/bind_internal.h:2620 (exe+0x000001dc7f57)
...
==================

The address 0x0000020ac959 most certainly corresponds to _M_is_leaked(), which 
touches the string refcounter in a racy way.
The debug info for the __tsan_read4 call looks fine, but for some reason the 
debug info for the actual memory access looks different:

$ objdump --start-address=0x20ac90a --stop-address=0x20acf00 -dS 
out/Release/net_unittests | grep -C 5 "20ac959" 
    _M_is_leaked() const
        { return this->_M_refcount < 0; }
 20ac950:   49 8d 7d f8             lea    -0x8(%r13),%rdi
 20ac954:   e8 17 2c 1b fe          callq  25f570 <__tsan_read4>
       pos > start && IsSeparator(path_[pos - 1]);
 20ac959:   4d 8d 77 ff             lea    -0x1(%r15),%r14
 20ac95d:   41 83 7d f8 00          cmpl   $0x0,-0x8(%r13)
 20ac962:   78 17                   js     20ac97b <_ZN4base8FilePath31StripTrailingSeparatorsInternalEv+0xbb>
 20ac964:   48 8b 5d d0             mov    -0x30(%rbp),%rbx


Here also are the line numbers printed by llvm-symbolizer:

$ third_party/llvm-build/Release+Asserts/bin/llvm-symbolizer 
out/Release/net_unittests 0x20ac958
_M_rep
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/bits/basic_string.
h:187:0
_M_leak
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/bits/basic_string.
h:311:0
IsSeparator
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/bits/basic_string.
h:841:0
base::FilePath::StripTrailingSeparatorsInternal()
/usr/local/google/chrome-asan/src/out/Release/../../base/files/file_path.cc:1246
:0

out/Release/net_unittests 0x20ac959
base::FilePath::StripTrailingSeparatorsInternal()
/usr/local/google/chrome-asan/src/out/Release/../../base/files/file_path.cc:1246
:0

out/Release/net_unittests 0x20ac95d
_M_rep
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/bits/basic_string.
h:187:0
_M_leak
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/bits/basic_string.
h:311:0
IsSeparator
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/bits/basic_string.
h:841:0
base::FilePath::StripTrailingSeparatorsInternal()
/usr/local/google/chrome-asan/src/out/Release/../../base/files/file_path.cc:1246
:0

What steps will reproduce the problem?
$ cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON \
 -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \
 -DCMAKE_EXPORT_COMPILE_COMMANDS=ON /path/to/llvm -G Ninja
$ ninja check-tsan

What is the expected output? What do you see instead?
Failing Tests (7):
    ThreadSanitizer :: fd_close_norace.cc
    ThreadSanitizer :: fd_dup_norace.cc
    ThreadSanitizer :: fd_pipe_norace.cc
    ThreadSanitizer :: fd_socket_connect_norace.cc
    ThreadSanitizer :: fd_socket_norace.cc
    ThreadSanitizer :: fd_socketpair_norace.cc
    ThreadSanitizer :: ignore_race.cc

  Expected Passes    : 130
  Unexpected Failures: 7

All tests fail with:
--
FileCheck error: '-' is empty.
--

Compiler annotations such as acquired_before and acquired_after under 
-Wthread-safety in clang give information on expected lock order, but they 
aren't enforced in runtime.

We could enforce this in runtime by instrumenting code to add edges to the 
deadlock detector (a class constructor for instance would be instrumented to 
add edges to annotated member locks when created).

Add system test for suppressions

#include <pthread.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

void *threadfn(void*) {
  return NULL;
}
void *allocate_stack(pthread_attr_t &a, size_t n = 655360) {
  void *stack = malloc(n);
  pthread_attr_init(&a);
  pthread_attr_setstack(&a, stack, n);
  return stack;
}

int main(void) {
  pthread_attr_t a;
  allocate_stack(a);
  pthread_t t;
  pthread_create(&t, &a, threadfn, NULL);
}

# clang++ 1.cc -o 1 -lpthread -fPIE -pie -fsanitize=thread -O0 -g
# ./1
Segmentation fault (core dumped)

Debugging is diffucult:
# gdb ./1
(gdb) set disable-randomization off
(gdb) run

[...]

Program received signal SIGSEGV, Segmentation fault.
memset () at ../sysdeps/x86_64/multiarch/../memset.S:1285
1285    ../sysdeps/x86_64/multiarch/../memset.S: No such file or directory.
(gdb) bt
#0  memset () at ../sysdeps/x86_64/multiarch/../memset.S:1285
#1  0x00007f4e2f1e5ef7 in __GI__dl_allocate_tls_init (result=0x7f4e2e1e1700) at 
dl-tls.c:437
#2  0x00007f4e2efbf6f3 in allocate_stack (
../../gdb-7.5.x/gdb/dwarf2read.c:10202: internal-error: 
dwarf2_record_block_ranges: Assertion `dwarf2_per_objfile->ranges.readin' 
failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Quit this debugging session? (y or n) 


Note that allocate_stack in the stack trace above refers to a function in glibc 
(called from pthread_create).

Increasing allocated stack size even further "fixes" the problem.

Even after the "too many RAM" fix in
http://code.google.com/p/data-race-test/source/detail?r=4338 ,
the test added
http://code.google.com/p/data-race-test/source/detail?r=4334

takes too much time to run:
26 seconds for N=500 under tsanv2,
0.015 seconds for N=500 natively.

This can be estimated as 50ms for a thread to shut down, which is not very 
little.

Currently the matched suppressions printed aren't attributed to their bug IDs, 
so it's hard to check whether a particular bug is fixed or not.
I suggest to allow free-form text annotations in the one-line suppressions that 
can be printed together with the matched suppressions.
We can either extend the semantics of the # symbol to allow inline comments in 
the suppressions:
  race:foobar  # bug 30

or can add some other form of annotations, e.g.:
  {bug 30}race:foobar

What steps will reproduce the problem?
1. Compile the testcase below one with ASan and one with TSan
2. Observe that with TSan, two tests fail.

What is the expected output? What do you see instead?

With ASan, all tests pass (this test is part of the ASan test suite). With 
TSan, the tests fail because the function always returns 0. Note that with 
neither ASan nor TSan, the tests also fail because malloc_usable_size returns 
sizes larger than the expected ones (which is allowed).


What version of the product are you using? On what operating system?

llvm/clang/compiler-rt r175901 on Ubuntu Linux 12.04 LTS


Testcase to reproduce the problem:

#include <cstdio>
#include <cstdlib>
#include <malloc.h>

int main() {
  const size_t kArraySize = 100;
  char *array = (char*)malloc(kArraySize);
  int *int_ptr = new int;

  size_t mus_null = malloc_usable_size(NULL);
  if (mus_null != 0) {
        fprintf(stderr, "Test failed: malloc_usable_size(NULL) returned %lu, expected 0\n", mus_null);
  }

  size_t mus_arr = malloc_usable_size(array);
  if (mus_arr != kArraySize) {
        fprintf(stderr, "Test failed: malloc_usable_size(array) returned %lu, expected %lu\n", mus_arr, kArraySize);
  }

  size_t mus_ptr = malloc_usable_size(int_ptr);
  if (mus_ptr != sizeof(int)) {
        fprintf(stderr, "Test failed: malloc_usable_size(int_ptr) returned %lu, expected %lu\n", mus_ptr, sizeof(int));
  }

  return 0;
}

This file contains duplicate strings and manually aligned spaces. Can it be 
auto-generated?

% cat atomic_stress.cc 
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char **argv) {
  int n = argc == 1 ? 100 : atoi(argv[1]);
  int n_iter = 10000;
  int *a = new int [n];
  printf("n: %5d:  ", n);
  for (int i = 0; i < n_iter; i++) {
    for (int j = 0; j < n; j++) {
      __sync_lock_test_and_set(a + j, 0);
    }
  }
  return 0;
}
% clang -pie -fPIE -fsanitize=thread -O atomic_stress.cc
% for((i=128; i<=1024*1024; i*=2)); do (TIMEFORMAT=%R;  time ./a.out $i) ; done
n:   128:  0.328
n:   256:  0.690
n:   512:  4.380
n:  1024:  19.285
n:  2048:  87.791


Profile: 
    87.93%    a.out  a.out              [.] __tsan::SyncTab::GetAndLock(__tsan::ThreadState*, unsigned long, unsigned long, bool, bool)
     2.59%    a.out  a.out              [.] __tsan_atomic32_exchange
     1.31%    a.out  a.out              [.] __tsan::Mutex::Unlock()

The code: 
  if (PrimaryAllocator::PointerIsMine((void*)addr)) {
    MBlock *b = user_mblock(thr, (void*)addr);
    CHECK_NE(b, 0);
    MBlock::ScopedLock l(b);
    SyncVar *res = 0;
    int iter = 0;
    for (res = b->ListHead(); res; res = res->next) {
      iter++;
      if (res->addr == addr)
        break;
    }


Obviously, if a given heap-allocated block has N objects accessed atomically, 
we will spend O(N) time handling each of them. 
This was previously known, but one of our users has hit it in a real test.

We may need a more efficient data structure here :( 

$ cat t.c
#include <pthread.h>
#include <stdio.h>

int main() {
  pthread_cond_t cond;
  pthread_condattr_t condAttr;
  int result = pthread_condattr_init(&condAttr);
  if (result != 0) {
        fprintf(stderr, "HERE: %s:%d\n", __FILE__, __LINE__);
        return -1;
  }
  result = pthread_condattr_setclock(&condAttr, CLOCK_MONOTONIC);
  if (result != 0) {
    fprintf(stderr, "HERE: %s:%d\n", __FILE__, __LINE__);
    return -1;
  }
  fprintf(stderr, "pthread_cond_init(%p, %p)\n", &cond, &condAttr);
  result = pthread_cond_init(&cond, &condAttr);
  if (result != 0) {
    fprintf(stderr, "HERE: %s:%d\n", __FILE__, __LINE__);
    fprintf(stderr, "result: %d\n", result);
    return -1;
  }
  return 0;
}

$ $CC t.c -o t -fsanitize=thread -pie -fPIC && ./t
pthread_cond_init(0x7fffbb707118, 0x7fffbb707110)
HERE: t.c:20
result: 22
$ $CC t.c -o t -lpthread && ./t
pthread_cond_init(0x7fff6915cf78, 0x7fff6915cf70)

It turns out that an incorrect version of pthread_cond_init (there are two of 
them) is picked from libpthread.
This particular problem can be solved by removing the pthread_cond_init wrapper 
(thus issue type set to Enhancement)
However chances are that other pthread_cond_* functions will also behave 
incorrectly.

... and should call ParseCommonFlagsFromString in __tsan_init

Currently it's impossible to suppress some mutex reports and maybe others.
Originally reported here:
http://llvm.org/bugs/show_bug.cgi?id=18378

It took me a while to debug 
https://code.google.com/p/chromium/issues/detail?id=324489, where TSan 
deadlocked in a child process that was calling an async-signal-unsafe routine 
soon after calling fork() in a multi-threaded program.

It would be useful if TSan notified me about the child process taking locks 
after fork() if there've been threads prior to fork() (TSan's own background 
thread should be ignored).

% cat memcpy16.cc 
#include <string.h>
void foo(int *x, int *y) {
  memcpy(x, y, 16);
}

% clang -fsanitize=thread -O -S -o -  memcpy16.cc
        callq   __tsan_func_entry
        movups  (%r14), %xmm0
        movups  %xmm0, (%rbx)
        callq   __tsan_func_exit

The simplest way is to replace mem intrins with __tsan_memset/etc, 
like we do in msan.

Ideally, the instrumentation module should know when the intrinsic 
will be inlined. But that's not trivial. 

It's currently only possible to set the default TSan options in the client 
program by declaring the __tsan::OverrideFlags() function that modifies the 
flags.
This approach requires the client to be aware of the layout of the 
__tsan::Flags struct, which changes over time. For some clients it may be 
inconvenient, e.g. it's hard to support two different compiler versions with 
different flags.
We need to introduce the 'char *__tsan_default_options()' function similar to 
that used by ASan and MSan instead.

Full story described here https://github.com/OpenImageIO/oiio/issues/720.

Short story : 
https://github.com/OpenImageIO/oiio/blob/master/src/include/thread.h l.542 
implements a spin lock.
TSan reports data race with one write l.568
> // Fastest way to do it is with a store with "release" semantics
> __asm__ __volatile__("": : :"memory");
> m_locked = 0;  <<<< this line

and a compare and exchange l.597
> return atomic_compare_and_exchange (&m_locked, 0, 1);

m_locked is of type `volatile int`.

There is already a debate in the above mentioned thread about the usefulness of 
volatile. I'm not a low level concurrency expert so I actually don't know if 
the asm code has the correct semantic or not.

Environment:
- clang version 3.3 (tags/RELEASE_33/final)
- Arch Linux x86_64, kernel 3.11.6-1-ARCH

[ RUN      ] PositiveTests.LibcStringFunctions
==================
WARNING: ThreadSanitizer: missed expected data race
  Write0, CheckStrlenResult addr=7d040000b372 racecheck_unittest.cc:8430
==================
==================
WARNING: ThreadSanitizer: missed expected data race
  Write0, CheckMemcmpResult addr=7d040000b312 racecheck_unittest.cc:8430
==================
==================
WARNING: ThreadSanitizer: missed expected data race
  Write0, CheckStrcmpResult addr=7d040000b2f2 racecheck_unittest.cc:8430
==================
==================
WARNING: ThreadSanitizer: missed expected data race
  Write0, CheckStrncmpResult addr=7d040000b2d2 racecheck_unittest.cc:8430
==================
[       OK ] PositiveTests.LibcStringFunctions (5 ms)

Disabled in the bot config until fixed.
https://code.google.com/p/address-sanitizer/source/browse/trunk/build/scripts/sl
ave/test_tsan.sh

E.g. it should be clear from a race report that a write has occasionally 
happened under a reader lock.

See https://code.google.com/p/chromium/issues/detail?id=270675

[ RUN      ] HostResolverImplTest.MultipleAttempts
==================
WARNING: ThreadSanitizer: data race (pid=17661)
  Write of size 8 at 0x7d2400004770 by thread T4: 
    #0 free /usr/local/google/asan/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:475 (net_unittests+0x0000002082dc)
    #1 <null> <null>:0 (libc.so.6+0x000000119a77)
    #2 net::RuleBasedHostResolverProc::Resolve(std::string const&, net::AddressFamily, int, net::AddressList*, int*) net/dns/mock_host_resolver.cc:372 (net_unittests+0x0000017ddaef)
    #3 net::HostResolverProc::ResolveUsingPrevious(std::string const&, net::AddressFamily, int, net::AddressList*, int*) net/dns/host_resolver_proc.cc:79 (net_unittests+0x000001e8e120)
    #4 net::(anonymous namespace)::LookupAttemptHostResolverProc::Resolve(std::string const&, net::AddressFamily, int, net::AddressList*, int*) net/dns/host_resolver_impl_unittest.cc:373 (net_unittests+0x0000007c3769)
    #5 net::HostResolverImpl::ProcTask::DoLookup(base::TimeTicks const&, unsigned int) net/dns/host_resolver_impl.cc:671 (net_unittests+0x000001e7bb00)
    #6 Run base/bind_internal.h:248 (net_unittests+0x000001e817dd)
...
  Previous write of size 8 at 0x7d2400004770 by thread T2: 
    #0 malloc /usr/local/google/asan/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:429 (net_unittests+0x000000207e03)
    #1 <null> <null>:0 (libc.so.6+0x0000001195c8)
    #2 net::RuleBasedHostResolverProc::Resolve(std::string const&, net::AddressFamily, int, net::AddressList*, int*) net/dns/mock_host_resolver.cc:372 (net_unittests+0x0000017ddaef)
    #3 net::HostResolverProc::ResolveUsingPrevious(std::string const&, net::AddressFamily, int, net::AddressList*, int*) net/dns/host_resolver_proc.cc:79 (net_unittests+0x000001e8e120)
    #4 net::(anonymous namespace)::LookupAttemptHostResolverProc::Resolve(std::string const&, net::AddressFamily, int, net::AddressList*, int*) net/dns/host_resolver_impl_unittest.cc:373 (net_unittests+0x0000007c3769)
    #5 net::HostResolverImpl::ProcTask::DoLookup(base::TimeTicks const&, unsigned int) net/dns/host_resolver_impl.cc:671 (net_unittests+0x000001e7bb00)
    #6 Run base/bind_internal.h:248 (net_unittests+0x000001e817dd)
...

The relevant piece of Chromium source is here:

https://code.google.com/p/chromium/codesearch#chromium/src/net/dns/host_resolver
_proc.cc&q=SystemHostResolverCall&sq=package:chromium&type=cs&l=122

Hi I've managed to build and run everything now with tsan, but I'm blocked by a 
problem that I've tracked down to how tsan intercepts mmap. It seems it doesn't 
support MAP_32BIT. Usually that's not a big deal because it's not an often used 
flag. However, luajit uses MAP_32BIT on x64 for all allocations. Here's a 
sample program that demonstrates the problem:

#include <sys/mman.h>
#include <iostream>

int main()
{
    void *ptr = mmap(NULL, 131072, PROT_READ|PROT_WRITE, MAP_32BIT|MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
    std::cout << ptr << std::endl;    
    return 0;
}

Is this a show stopper, or could support be added for this? I'd be happy to do 
the work if someone pointed me in the right direction.

Cheers,
Dan

For some race reports the best suppression may contain a very short string, 
e.g. the name of the global ("mem0") or the top frame in the stack trace. It's 
quite likely that these strings will accidentally match other globals or other 
stack frames, which may lead to new races being unnoticed.

I see two possible ways to fix this:
 - introduce the "^" and "$" to match the beginning and the end of an entity (path, function name, global name), so that the users will have to write "race:^mem0$" if they want to restrict their suppressions. Obviously, nobody will
 - make "race:mem0" not imply "race:*mem0*", so that the users will have to explicitly denote that they want their pattern to match a substring.

See the demo test
http://code.google.com/p/data-race-test/source/detail?r=4334
with the comments

Hi,
I am obviously misunderstanbding the pthread_barrier idiom as I have an issue 
where the main thread + n other threasds wait on a barrier and then main 
continues. The thread receiving the PTHREAD_BARRIER_SERIAL_THREAD calls detroy. 
However, it seem that this causes threadsanitizer to report an error. I cannot 
see how this is a race given my understanding of the barrier call. A simple 
program to demonstrate is below.

What steps will reproduce the problem?
1. See program below

#include <pthread.h>
#include <stdio.h>
#include <stddef.h>
#include <unistd.h>

pthread_barrier_t B;


void *Thread(void *x) {
    if (pthread_barrier_wait(&B) == PTHREAD_BARRIER_SERIAL_THREAD){
          pthread_barrier_destroy(&B);
      }
  return NULL;
}



int main() {
  pthread_t t,t1;
  pthread_barrier_init(&B, 0, 3);
  pthread_create(&t, NULL, Thread, NULL);
  pthread_create(&t1, NULL, Thread, NULL);

  if (pthread_barrier_wait(&B) == PTHREAD_BARRIER_SERIAL_THREAD){
      pthread_barrier_destroy(&B);
  }

  pthread_join(t,NULL);
  pthread_join(t1,NULL);



  return 0;
}

What is the expected output? What do you see instead?
I expect this not to be an issue with threadsanitizer

What version of the product are you using? On what operating system?
gcc (GCC) 4.8.0 20130502 (prerelease)
on linux 3.9.3.1

If I wait until the join completes then call destroy in main there does not 
seem to be an issue (in the real app this will not really be possible unless I 
defer to overall shutdown - which seems a bit wrong.)

regards
Steve

According to http://crbug.com/258451, the contents of /proc/self/maps cannot be 
read atomically, and reading the file line by line may produce a garbled memory 
map.
We need to improve our /p/s/m reader by doing something similar to 
https://chromiumcodereview.appspot.com/18661009

Using Dmitry's atomics patch for tsan, I eventually managed to build my project 
with clang and libc++, it was not easy on linux! 

I have some linker errors remaining that look like this in calls to 
compare_exchange_strong:

error: undefined reference to `__tsan_atomic64_compare_exchange_val'

I see in the tsan binary, the symbols:

__tsan_atomic64_compare_exchange_strong
__tsan_atomic64_compare_exchange_weak

Are defined, but not _val, not sure if the problem there is in libc++ or in 
tsan, but no doubt you guys would know best. Let me know if you need any more 
information.

I'm eager to continue trying tsan against this codebase, which utilizes a lot 
of tricky synchronization and lock-free algorithms.

Cheers,
Dan

LLVM r159521

% cat tsan-bug.cc 
struct AAA              {  virtual long aaa (); };
struct BBB: virtual AAA { unsigned long bbb; };
struct CCC: virtual AAA { };
struct DDD: CCC, BBB { DDD (); };
DDD::DDD()  { }


% clang -O2 -c -fthread-sanitizer tsan-bug.cc 
clang: /home/kcc/llvm/lib/VMCore/Constants.cpp:1366: static llvm::Constant 
*llvm::ConstantExpr::getCast(unsigned int, llvm::Constant *, llvm::Type *): 
Assertion `CastInst::castIsValid(opc, C, Ty) && "Invalid constantexpr cast!"' 
failed.


The assertion happens here: 

bool ThreadSanitizer::instrumentLoadOrStore(Instruction *I) {                   


...
  if (IsWrite && isVtableAccess(I)) {                                                                                                                                                               
    Value *StoredValue = cast<StoreInst>(I)->getValueOperand();                                                                                                                                     
    IRB.CreateCall2(TsanVptrUpdate,                                                                                                                                                                 
                    IRB.CreatePointerCast(Addr, IRB.getInt8PtrTy()),                                                                                                                                
                    IRB.CreatePointerCast(StoredValue, IRB.getInt8PtrTy())); <<<<<<<<<<<<<<<<

The IR looks like this: 
  store i64 ptrtoint (i8** getelementptr inbounds ([9 x i8*]* @_ZTC3DDD8_3BBB, i64 0, i64 8) to i64), i64* %add.ptr.i, align 8, !tbaa !0

!0 = metadata !{metadata !"vtable pointer", metadata !1}


So, we have a store which is marked as "vtable pointer" but which is actually 
not a pointer store.

Investigating. 

Ran TSAN_OPTIONS=detect_deadlocks=1 gdb -ex 'set disable-randomization off' -ex 
'set print thread-events off' out/Debug/video_engine_tests

where video_engine_tests is built from a code.webrtc.org checkout (in trunk):

$ ./webrtc/build/gyp_webrtc -D tsan=1 -D clang_use_chrome_plugins=0
$ ninja -C out/Debug video_engine_tests

What is the expected output? What do you see instead?

A lot of mutex inversion error reports, but no internal tsan crashes.

After a lot of error reports I instead get a CHECK failure:

FATAL: ThreadSanitizer CHECK failed: 
../projects/compiler-rt/lib/sanitizer_common/sanitizer_deadlock_detector1.cc:102
 "((!dd.isHeld(&lt->dd, m->id))) != (0)" (0x0, 0x0)
    #0 __tsan::PrintCurrentStackSlow() /usr/local/google/home/pbos/llvm/build/../projects/compiler-rt/lib/tsan/rtl/tsan_rtl_report.cc:724 (video_engine_tests+0x0000000bc30c)
    #1 __tsan::TsanCheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) /usr/local/google/home/pbos/llvm/build/../projects/compiler-rt/lib/tsan/rtl/tsan_rtl_report.cc:44 (video_engine_tests+0x0000000bc3e3)
    #2 __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) /usr/local/google/home/pbos/llvm/build/../projects/compiler-rt/lib/sanitizer_common/sanitizer_common.cc:76 (video_engine_tests+0x0000000c1de3)
    #3 __sanitizer::DDetectorImpl::MutexLock(__sanitizer::DDPhysicalThread*, __sanitizer::DDLogicalThread*, __sanitizer::DDMutex*, bool, bool) /usr/local/google/home/pbos/llvm/build/../projects/compiler-rt/lib/sanitizer_common/sanitizer_deadlock_detector1.cc:102 (video_engine_tests+0x0000000c3f30)
    #4 __tsan::MutexReadLock(__tsan::ThreadState*, unsigned long, unsigned long, bool) /usr/local/google/home/pbos/llvm/build/../projects/compiler-rt/lib/tsan/rtl/tsan_rtl_mutex.cc:184 (video_engine_tests+0x0000000b91b8)
    #5 pthread_rwlock_rdlock /usr/local/google/home/pbos/llvm/build/../projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:1041 (video_engine_tests+0x00000006a112)
    #6 webrtc::RWLockPosix::AcquireLockShared() /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/system_wrappers/source/rw_lock_posix.cc:44 (video_engine_tests+0x0000002bbc09)
    #7 webrtc::ViEManagerBase::ReadLockManager() const /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/video_engine/vie_manager_base.cc:27 (video_engine_tests+0x0000004d865a)
    #8 webrtc::ViEManagerScopedBase::ViEManagerScopedBase(webrtc::ViEManagerBase const&) /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/video_engine/vie_manager_base.cc:45 (video_engine_tests+0x0000004d889f)
    #9 webrtc::ViEChannelManagerScoped::ViEChannelManagerScoped(webrtc::ViEChannelManager const&) /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/video_engine/vie_channel_manager.cc:532 (video_engine_tests+0x000000492503)
    #10 webrtc::ViECodecImpl::GetSendSideDelay(int, int*, int*) const /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/video_engine/vie_codec_impl.cc:743 (video_engine_tests+0x00000044692f)
    #11 webrtc::internal::VideoSendStream::GetSendSideDelay(webrtc::VideoSendStream::Stats*) /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/video/video_send_stream.cc:270 (video_engine_tests+0x0000002b5304)
    #12 non-virtual thunk to webrtc::internal::VideoSendStream::GetSendSideDelay(webrtc::VideoSendStream::Stats*) /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/video/video_send_stream.cc:272 (video_engine_tests+0x0000002b5397)
    #13 webrtc::SendStatisticsProxy::GetStats() const /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/video/send_statistics_proxy.cc:47 (video_engine_tests+0x0000002a4344)
    #14 webrtc::internal::VideoSendStream::GetStats() const /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/video/video_send_stream.cc:266 (video_engine_tests+0x0000002b5200)
    #15 webrtc::VideoSendStreamTest_ProducesStats_Test::TestBody()::StatsObserver::OnSendRtcp(unsigned char const*, unsigned long) /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/video/video_send_stream_tests.cc:1092 (video_engine_tests+0x000000152175)
    #16 webrtc::test::RtpRtcpObserver::PacketTransport::SendRtcp(unsigned char const*, unsigned long) /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/test/rtp_rtcp_observer.h:149 (video_engine_tests+0x00000012e34e)
    #17 webrtc::internal::TransportAdapter::SendRTCPPacket(int, void const*, int) /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/video/transport_adapter.cc:36 (video_engine_tests+0x0000002a8aae)
    #18 webrtc::ViESender::SendRTCPPacket(int, void const*, int) /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/video_engine/vie_sender.cc:129 (video_engine_tests+0x0000004f0bf4)
    #19 webrtc::RTCPSender::SendToNetwork(unsigned char const*, unsigned short) /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/modules/rtp_rtcp/source/rtcp_sender.cc:2108 (video_engine_tests+0x00000022e154)
    #20 webrtc::RTCPSender::SendRTCP(webrtc::RTCPSender::FeedbackState const&, unsigned int, int, unsigned short const*, bool, unsigned long) /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/modules/rtp_rtcp/source/rtcp_sender.cc:1736 (video_engine_tests+0x00000021c6d4)
    #21 webrtc::RTCPSender::SetSendingStatus(webrtc::RTCPSender::FeedbackState const&, bool) /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/modules/rtp_rtcp/source/rtcp_sender.cc:310 (video_engine_tests+0x00000021c225)
    #22 webrtc::ModuleRtpRtcpImpl::SetSendingStatus(bool) /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/modules/rtp_rtcp/source/rtp_rtcp_impl.cc:501 (video_engine_tests+0x00000076630f)
    #23 webrtc::ViEChannel::StopSend() /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/video_engine/vie_channel.cc:1577 (video_engine_tests+0x00000047d1ac)
    #24 webrtc::ViEBaseImpl::StopSend(int) /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/video_engine/vie_base_impl.cc:323 (video_engine_tests+0x000000436b01)
    #25 webrtc::internal::VideoSendStream::StopSending() /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/video/video_send_stream.cc:215 (video_engine_tests+0x0000002b482b)
    #26 webrtc::VideoSendStreamTest_ProducesStats_Test::TestBody() /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/video/video_send_stream_tests.cc:1141 (video_engine_tests+0x000000151888)
    #27 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../testing/gtest/src/gtest.cc:1989 (video_engine_tests+0x0000001d85f0)
    #28 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../testing/gtest/src/gtest.cc:2042 (video_engine_tests+0x0000001a1c19)
    #29 testing::Test::Run() /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../testing/gtest/src/gtest.cc:2061 (video_engine_tests+0x00000018d836)
    #30 testing::TestInfo::Run() /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../testing/gtest/src/gtest.cc:2237 (video_engine_tests+0x00000018e737)
    #31 testing::TestCase::Run() /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../testing/gtest/src/gtest.cc:2344 (video_engine_tests+0x00000018f350)
    #32 testing::internal::UnitTestImpl::RunAllTests() /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../testing/gtest/src/gtest.cc:4065 (video_engine_tests+0x000000197447)
    #33 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../testing/gtest/src/gtest.cc:1989 (video_engine_tests+0x0000001c1f40)
    #34 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../testing/gtest/src/gtest.cc:2042 (video_engine_tests+0x0000001a6b5f)
    #35 testing::UnitTest::Run() /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../testing/gtest/src/gtest.cc:3697 (video_engine_tests+0x000000196d1c)
    #36 RUN_ALL_TESTS() /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../testing/gtest/include/gtest/gtest.h:2231 (video_engine_tests+0x00000028787d)
    #37 webrtc::test::RunAllTests() /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/test/run_tests.cc:16:10 (video_engine_tests+0x0000002876b5)
    #38 main /usr/local/google/home/pbos/webrtc/trunk/out/Debug/../../webrtc/test/test_main.cc:22 (video_engine_tests+0x00000017f42d)
    #39 __libc_start_main /build/buildd/eglibc-2.15/csu/libc-start.c:226 (libc.so.6+0x00000002176c)
    #40 _start <null>:0 (video_engine_tests+0x0000000d0260)                      

What version of the product are you using? On what operating system?

Tip of tree built yesterday (2014-03-03), Ubuntu 12.04.

BackgroundThread is responsible for various repeated jobs like symbolizer cache 
cleanup. If the user program calls fork() when the thread holds a lock, the 
child process may deadlock trying to acquire that lock.

A program under tsan does much more than a normal one, even memory accesses are 
not fully async-signal-safe. So a correct program that uses fork can hang under 
tsan.
This needs to be fixed one way or another.
There are 2 competing proposals.
One is to lock all internal mutexes before fork:
http://llvm-reviews.chandlerc.com/D2470
Another is to disable memory access and sync action processing after fork in 
multithreaded program (expecting it to exec soon). For single-threaded programs 
we can continue to work as usual.

ThreadSanitizer allows to specify that data races inside a particular
memory region are benign. However, it is impossible to remove a region
from the list of those with benign data races. Once the region is
reallocated and used for a different purpose, real data races on it may
be missed.

I propose adding such a "deannotation" functionality to ThreadSanitizer.
A possible patch:

https://github.com/yegord/compiler-rt/commit/DeannotateBenignRace

When compile mozilla-central with Clang tip and thread-sanitizer enabled, Clang 
crashes during compilation of xpcom/io/nsStreamUtils.cpp. I verified that this 
only happens with thread-sanitizer. If I just build without sanitizers, this 
builds fine.

Attached is the log of the crash, as well as the cpp/sh files written by Clang.

I used LLVM r195890 on Ubuntu LTS 64 bit.

$ cat t.c
#include <pthread.h>
#include <stdio.h>

void *thread(void *unused) {
  char *line = NULL;
  size_t size;
  getline(&line, &size, stdin);
  return NULL;
}

int main() {
  pthread_t t;
  pthread_create(&t, NULL, thread, NULL);
  return 0;
}
$ clang t.c -o t -lpthread
$ ./t
(everything ok)
$ clang t.c -o t -lpthread -fsanitize=thread
% ./t
(hangs)

ThreadSanitizer has more runtime flags than documented on
http://code.google.com/p/thread-sanitizer/wiki/Flags

Missing flags include suppress_equal_stacks, suppress_equal_addresses,
report_atomic_races, which are useful not only for those, who read
TSan's sources.

Could you, please, add documentation on missing flags to the wiki?

tsan may miss a racy use-after-return: 

  1. Thread1 passes a pointer to a local var to Thread2
  2. Thread1 exits the current function
  3a. Thread2 touches the now-stale pointer
  3b. Thread1 enters or exits a function thus writing/reading stack 

3a and 3b may race.

A possible solution is to treat __tsan_func_entry as write and __tsan_func_exit 
as read.
The tricky parts are what memory range to access on entry.exit,
how that will affect performance,
and how to test the feature reliably. 
We'll need to make sure that even an empty function gets instrumented. 

This feature should be guarder by a run-time flag. 


% cat ~/tmp/racy_uar.cc
#include <pthread.h>
#include <assert.h>
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>

int *g, *g1;
pthread_t t;

void *Thread(void*) {
  assert(g);
  sleep(1);
  *g = 0xabcd1234;
  return 0;
}

void foo() {
  int x, y, z;
  g = &x;
  printf("& x,y,z: %p %p %p; Thread pc: %p\n", &x, &y, &z, Thread);
  sleep(1);
  pthread_create(&t, 0, Thread, 0);
//  printf("x: %x\n", x);
}

void bar() {
  int x, y, z = 0;
  g1 = &x;
  printf("& x,y,z: %p %p %p\n", &x, &y, &z);
  sleep(2);
  pthread_join(t, 0);
  printf("x: %x\n", x);
}

int main() {
  foo();
  bar();
}
% clang++ -g  -fPIE -pie -O1 -fsanitize=thread  ~/tmp/racy_uar.cc -lpthread  && 
./a.out 
& x,y,z: 0x7fffd7c7114c 0x7fffd7c71148 0x7fffd7c71144; Thread pc: 0x7fbcbc83f390
& x,y,z: 0x7fffd7c7113c 0x7fffd7c71138 0x7fffd7c71134
x: 7fbc
% 

Note: x should be 0, instead it contains part of the pc (7fbc)

Note: AddressSanitizer finds this bug reliably in use-after-return mode 
and less reliably in the default mode.

Build with tsan and run this: https://pastebin.mozilla.org/3271918

The simplest possible fix is this:
--- lib/Transforms/Instrumentation/ThreadSanitizer.cpp  (revision 192795)
+++ lib/Transforms/Instrumentation/ThreadSanitizer.cpp  (working copy)
@@ -358,7 +358,7 @@
   // (e.g. variables that do not escape, etc).

   // Instrument memory accesses.
-  if (ClInstrumentMemoryAccesses)
+  if (ClInstrumentMemoryAccesses && 
F.hasFnAttribute(Attribute::SanitizeThread))
     for (size_t i = 0, n = AllLoadsAndStores.size(); i < n; ++i) {
       Res |= instrumentLoadOrStore(AllLoadsAndStores[i]);
     }


Or maybe we should check the attribute near 
  if (BL->isIn(F)) return false;

Anyway, Dmitry, please fix (with lit tests)

tsan currently produces false positives in c++11 mode if the c++ library is not 
instrumented. One such case is discussed here: 
http://lists.cs.uiuc.edu/pipermail/cfe-dev/2014-February/035408.html

We need: 

1. Start building instrumented libcxx as part of the tsan build
2. Add lit tests that use c++11 threading (like the one above)