emeryberger / hoard

The Hoard Memory Allocator: A Fast, Scalable, and Memory-efficient Malloc for Linux, Windows, and Mac.

The Hoard Memory Allocator

by Emery Berger

The Hoard memory allocator is a fast, scalable, and memory-efficient memory allocator that works on a range of platforms, including Linux, Mac OS X, and Windows.

Hoard is a drop-in replacement for malloc that can dramatically improve application performance, especially for multithreaded programs running on multiprocessors and multicore CPUs. No source code changes necessary: just link it in or set one environment variable (see Building Hoard, below).

Press

• "If you'll be running on multiprocessor machines, ... use Emery Berger's excellent Hoard multiprocessor memory management code. It's a drop-in replacement for the C and C++ memory routines and is very fast on multiprocessor machines."

  • Debugging Applications for Microsoft .NET and Microsoft Windows, Microsoft Press

• "(To improve scalability), consider an open source alternative such as the Hoard Memory Manager..."

  • Windows System Programming, Addison-Wesley

• "Hoard dramatically improves program performance through its more efficient use of memory. Moreover, Hoard has provably bounded memory blowup and low synchronization costs."

  • Principles of Parallel Programming, Addison-Wesley

Users

Companies using Hoard in their products and servers include AOL, British Telecom, Blue Vector, Business Objects (formerly Crystal Decisions), Cisco, Credit Suisse, Entrust, InfoVista, Kamakura, Novell, Oktal SE, OpenText, OpenWave Systems (for their Typhoon and Twister servers), Pervasive Software, Plath GmbH, Quest Software, Reuters, Royal Bank of Canada, SAP, Sonus Networks, Tata Communications, and Verite Group.

Open source projects using Hoard include the Asterisk Open Source Telephony Project, Bayonne GNU telephony server, the Cilk parallel programming language, the GNU Common C++ system, the OpenFOAM computational fluid dynamics toolkit, and the SafeSquid web proxy.

Hoard is now a standard compiler option for the Standard Performance Evaluation Corporation's CPU2006 benchmark suite for the Intel and Open64 compilers.

Licensing

Hoard has now been released under the widely-used and permissive Apache license, version 2.0.

Why Hoard?

There are a number of problems with existing memory allocators that make Hoard a better choice.

Contention

Multithreaded programs often do not scale because the heap is a bottleneck. When multiple threads simultaneously allocate or deallocate memory from the allocator, the allocator will serialize them. Programs making intensive use of the allocator actually slow down as the number of processors increases. Your program may be allocation-intensive without you realizing it, for instance, if your program makes many calls to the C++ Standard Template Library (STL). Hoard eliminates this bottleneck.

False Sharing

System-provided memory allocators can cause insidious problems for multithreaded code. They can lead to a phenomenon known as "false sharing": threads on different CPUs can end up with memory in the same cache line, or chunk of memory. Accessing these falsely-shared cache lines is hundreds of times slower than accessing unshared cache lines. Hoard is designed to prevent false sharing.

Blowup

Multithreaded programs can also lead the allocator to blowup memory consumption. This effect can multiply the amount of memory needed to run your application by the number of CPUs on your machine: four CPUs could mean that you need four times as much memory. Hoard is guaranteed (provably!) to bound memory consumption.

Installation

Homebrew (Mac OS X)

You can use Homebrew to install the current version of Hoard as follows:

brew tap emeryberger/hoard
brew install --HEAD emeryberger/hoard/libhoard

This not only installs the Hoard library, but also creates a hoard command you can use to run Hoard with anything at the command-line.

hoard myprogram-goes-here

---

Building Hoard from source (Mac OS X, Linux, and Windows WSL2)

On Linux, you may need to first install the appropriate version of libstdc++-dev (e.g., libstdc++-12-dev):

   sudo apt install libstdc++-dev

Now, to build Hoard from source, do the following:

    git clone https://github.com/emeryberger/Hoard
    cd src
    make

You can then use Hoard by linking it with your executable, or by setting the LD_PRELOAD environment variable, as in

    export LD_PRELOAD=/path/to/libhoard.so

or, in Mac OS X:

    export DYLD_INSERT_LIBRARIES=/path/to/libhoard.dylib

---

Building Hoard (Windows)

Change into the src directory and build the Windows version:

C:\hoard\src> nmake

To use Hoard, link your executable with source\uselibhoard.cpp and libhoard.lib. You must use the /MD flag.

Example:

C:\hoard\src> cl /Ox /MD yourapp.cpp source\uselibhoard.cpp libhoard.lib

To run yourapp.exe, you will need to have libhoard.dll in your path.

Benchmarks

The directory benchmarks/ contains a number of benchmarks used to evaluate and tune Hoard.

Technical Information

Hoard has changed quite a bit over the years, but for technical details of the first version of Hoard, read Hoard: A Scalable Memory Allocator for Multithreaded Applications, by Emery D. Berger, Kathryn S. McKinley, Robert D. Blumofe, and Paul R. Wilson. The Ninth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS-IX). Cambridge, MA, November 2000.

hoard's Issues

Use a specific commit of Heap-Layers

Looks like Hoard's Makefile and GNUmakefile clone master of Heap-Layers. Unless you never change Heap-Layers, it seems likely that at some point you will introduce a change to Heap-Layers that will require a corresponding change to Hoard. When that happens, old versions of Hoard (which might be present in package management systems) will no longer be able to build. This problem might not always be noticed right away, leaving your software in broken state in that package management system for an unbounded period of time.

For this reason, you should download a specific version of Heap-Layers that is known to work with this version of Hoard.

I think git already behaved that way before when you were using submodules, so you could get this behavior back by reverting 5275686 and also making the same change to GNUmakefile.

Add caching for large objects to reduce mmap/munmap pressure.

Sample code for timing from Phong Vo:

#include    <errno.h>
#include    <stdlib.h>
#include    <string.h>
#include    <pthread.h>

#define N_THREAD    256
#define N_ALLOC     1000000
static int      Nthread = N_THREAD;
static int      Nalloc = N_ALLOC;
static size_t       Minsize = 10;
static size_t       Maxsize = 1024;

int error(char* mesg)
{
    write(2, mesg, strlen(mesg));
    exit(1);
}

void* allocate(void* arg)
{
    int     k, p, q, c;
    size_t      sz, nalloc, len;
    char        **list;
    size_t      *size;
    int     thread = (int)((long)arg);

    unsigned int    rand = 0; /* use a local RNG so that threads work uniformly */
#define FNV_PRIME   ((1<<24) + (1<<8) + 0x93)
#define FNV_OFFSET  2166136261
#define RANDOM()    (rand = rand*FNV_PRIME + FNV_OFFSET)

    nalloc = Nalloc/Nthread; /* do the same amount of work regardless of #threads */

    if(!(list = (char**)malloc(nalloc*sizeof(char*))) )
        error("failed to allocate space for list of objects\n");
    if(!(size = (size_t*)malloc(nalloc*sizeof(size_t))) )
        error("failed to allocate space for list of sizes\n");
    memset(list, 0, nalloc*sizeof(char*));
    memset(size, 0, nalloc*sizeof(size_t));

    for(k = 0; k < nalloc; ++k)
    {   
        /* get a random size favoring smaller over larger */
        len = Maxsize-Minsize+1;
        for(;;)
        {   sz = RANDOM() % len; /* pick a random size in [0,len-1] */
            if((RANDOM()%100) >= (100*sz)/len) /* this favors a smaller size */
                break;
            len = sz; /* the gods want a smaller length, try again */
        }
        sz += Minsize;

        if(!(list[k] = malloc(sz)) )
            error("malloc failed\n");
        else
        {   size[k] = sz;
            for(c = 0; c < 10; ++c)
                list[k][c*sz/10] = 'm';
        }

        if(k < 1000)
            continue;

        /* get an interval to check for free and realloc */
        if((p = RANDOM()%k) > (q = RANDOM()%k) )
            { c = p; p = q; q = c; }

        for(; p <= q; ++p)
        {   if(list[p])
            {   if(RANDOM()%2 == 0 ) /* 50% chance of being freed */
                {   free(list[p]);
                    list[p] = 0;
                    size[p] = 0;
                }
                else if(RANDOM()%4 == 0 ) /* survived free, check realloc */
                {   sz = size[p] > Maxsize ? size[p]/4 : 2*size[p];
                    if(!(list[p] = realloc(list[p], sz)) )
                        error("realloc failed\n");
                    else
                    {   size[p] = sz;
                        for(c = 0; c < 10; ++c)
                            list[p][c*sz/10] = 'r';
                    }
                }
            }
        }
    }

    free(list);
    free(size);

    return (void*)0;
}

int main(int argc, char* argv[])
{
    int     i, rv;
    void        *status;
    pthread_t   th[N_THREAD];

    for(; argc > 1; --argc, ++argv)
    {   if(argv[1][0] != '-')
            continue;
        else if(argv[1][1] == 'a') /* # malloc calls */
            Nalloc = atoi(argv[1]+2);
        else if(argv[1][1] == 't') /* # threads */
            Nthread = atoi(argv[1]+2);
        else if(argv[1][1] == 'z') /* min block size */
            Minsize = atoi(argv[1]+2);
        else if(argv[1][1] == 'Z') /* max block size */
            Maxsize = atoi(argv[1]+2);
    }

    if(Nalloc <= 0 || Nalloc > N_ALLOC)
        Nalloc = N_ALLOC;
    if(Nthread <= 0 || Nthread > N_THREAD)
        Nthread = N_THREAD;
    if(Minsize <= 0)
        Minsize = 1;
    if(Maxsize < Minsize)
        Maxsize = Minsize;

    for(i = 0; i < Nthread; ++i)
    {   if((rv = pthread_create(&th[i], NULL, allocate, (void*)((long)i))) != 0 )
            error("Failed to create thread\n");
    }

    for(i = 0; i < Nthread; ++i)
    {   if((rv = pthread_join(th[i], &status)) != 0 )
            error("Failed waiting for thread\n");
    }

    return 0;
}

hoard revers result

I could not compile hoard from source in fedora 17 x64 ( #5 I compiled it with compat gcc 34 and produced libhoard.so. After that tried to compile and test cache-scratch with and without hoard.so the result was weird as below.could you tell me where is the problem?.I also download libhoard.so for 64bit llinux from <a href=http://plasma.cs.umass.edu/emery/index.php?page=download-hoard>Emery but the result was the same.

g++ /home/src/Hoard/benchmarks/cache-scratch/cache-scratch.cpp -I../common/ /home/javad/src/Hoard/libhoard.so -o cache-scratch-hoard -lpthread

./cache-scratch 1 1000 1 1000000
Time elapsed = 2.383032 seconds.
./cache-scratch-hoard 1 1000 1 1000000
Time elapsed = 10.085448 seconds.

FreeBSD 10.1 loaded issue

Hello,

I just install hoard on FreeBSD 10.1 when I set libhoard. Any idea what I'm doing wrong?

root@localhost:~ # ldd /bin/ls
/bin/ls:
libutil.so.9 => /lib/libutil.so.9 (0x800824000)
libncurses.so.8 => /lib/libncurses.so.8 (0x800a36000)
libc.so.7 => /lib/libc.so.7 (0x800c83000)
root@localhost:~ # setenv LD_PRELOAD /usr/local/lib/libhoard.so.1
root@localhost:~ # ldd /bin/ls
/bin/ls:
libutil.so.9 => /lib/libutil.so.9 (0x800a83000)
libncurses.so.8 => /lib/libncurses.so.8 (0x800c95000)
libc.so.7 => /lib/libc.so.7 (0x800ee2000)
libc++.so.1 => /usr/lib/libc++.so.1 (0x80128b000)
libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x80154b000)
libm.so.5 => /lib/libm.so.5 (0x801767000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x80198f000)
libthr.so.3 => /lib/libthr.so.3 (0x801b9d000)
root@localhost:~ #

Turn off banner in STDERR

I don't really find it any useful or even common for allocators to report they're being put in use, what's the logical reasoning behind it?

difficulties building from src, linux x86_64 gcc 4.6.3

hitting an error and some warnings attempting to build

• libhoard-3.9.tar.gz
• Ubuntu Linux 12.04 (x86_64)
• g++ (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3
• make linux-gcc-x86-64
  Relevant tail of build output below. Suggestions?

...
heaplayers/wrappers/gnuwrapper.cpp:68:42: error: conflicting declaration 'void (* __malloc_initialize_hook)()'
/usr/include/malloc.h:170:38: error: '__malloc_initialize_hook' has a previous declaration as 'void (* volatile __malloc_initialize_hook)()'

heaplayers/wrappers/gnuwrapper.cpp: In function 'void my_init_hook()':
heaplayers/wrappers/gnuwrapper.cpp:72:23: warning: '__malloc_hook' is deprecated (declared at /usr/include/malloc.h:176) [-Wdeprecated-declarations]
heaplayers/wrappers/gnuwrapper.cpp:73:21: warning: '__free_hook' is deprecated (declared at /usr/include/malloc.h:173) [-Wdeprecated-declarations]
heaplayers/wrappers/gnuwrapper.cpp:74:24: warning: '__realloc_hook' is deprecated (declared at /usr/include/malloc.h:179) [-Wdeprecated-declarations]
heaplayers/wrappers/gnuwrapper.cpp:75:25: warning: '__memalign_hook' is deprecated (declared at /usr/include/malloc.h:183) [-Wdeprecated-declarations]
heaplayers/wrappers/gnuwrapper.cpp:78:5: warning: '__malloc_hook' is deprecated (declared at /usr/include/malloc.h:176) [-Wdeprecated-declarations]
heaplayers/wrappers/gnuwrapper.cpp:79:5: warning: '__free_hook' is deprecated (declared at /usr/include/malloc.h:173) [-Wdeprecated-declarations]
heaplayers/wrappers/gnuwrapper.cpp:80:5: warning: '__realloc_hook' is deprecated (declared at /usr/include/malloc.h:179) [-Wdeprecated-declarations]
heaplayers/wrappers/gnuwrapper.cpp:81:5: warning: '__memalign_hook' is deprecated (declared at /usr/include/malloc.h:183) [-Wdeprecated-declarations]

Build issue on Visual Studio 2017

I downloaded the latest code from github and ran nmake from a VS 2017 command line. I got the following error:

    cl /I. /Iinclude /Iinclude/util /Iinclude/hoard /Iinclude/superblocks /IHeap-Layers /D "NDEBUG" /D "_WINDOWS" /D "_WINDLL" /D "_WINRT_DLL" /D "_UNICODE" /D "UNICODE" /Zi /Ox /MD /nologo /W1 /WX- /Ox /Oi /Oy- /Gm- /EHsc /MD /GS /Gy /Zc:wchar_t /Zc:forScope /Gd /errorReport:queue "source\libhoard.cpp" "Heap-Layers\wrappers\winwrapper.cpp" "source\wintls.cpp" /GL /link /DLL /subsystem:console /OUT:libhoard.dll

libhoard.cpp
c:\users\jeff\desktop\hoard-master\src\heap-layers\utility\bins.h(35): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory
winwrapper.cpp
wintls.cpp
c:\users\jeff\desktop\hoard-master\src\heap-layers\utility\bins.h(35): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.15.26726\bin\HostX64\x64\cl.EXE"' : return code '0x2'
Stop.

Compilation fails on Mac OS X 10.8.4

At commit 7e8647e, "make macos" fails with numerous errors. Some are in Heap-Layers, some not. Transcript below. You may well be aware of all this :)

I downloaded the 3.9 tarball and it built (with many warnings). It would be nice if there were Git tags for the releases, assuming you have the history in Git.

Thanks!

$ uname -a
Darwin silverbird.local 12.4.0 Darwin Kernel Version 12.4.0: Wed May  1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64
$ clang --version
Apple clang version 4.0 (tags/Apple/clang-421.0.60) (based on LLVM 3.1svn)
Target: x86_64-apple-darwin12.4.0
Thread model: posix
$ git clone --recursive git://github.com/emeryberger/Hoard.git
$ cd Hoard/src
$ make macos
clang++ -arch i386 -arch x86_64 -pipe -g -O3 -Wall -DNDEBUG -I. -Iinclude -Iinclude/util -Iinclude/hoard -Iinclude/superblocks -IHeap-Layers -D_REENTRANT=1 -compatibility_version 1 -current_version 1 -dynamiclib -D'CUSTOM_PREFIX(x)=xx##x' source/libhoard.cpp Heap-Layers/wrappers/macwrapper.cpp source/mactls.cpp -o libhoard.dylib -ldl -lpthread 
In file included from source/libhoard.cpp:33:
In file included from Heap-Layers/heaplayers.h:103:
In file included from Heap-Layers/heaps/all.h:1:
In file included from Heap-Layers/heaps/buildingblock/all.h:1:
Heap-Layers/heaps/buildingblock/adaptheap.h:51:9: error: use of undeclared
      identifier 'assert'
        assert (SuperHeap::getSize(ptr) >= sizeof(dict));
        ^
Heap-Layers/heaps/combining/strictsegheap.h:102:50: note: in instantiation of
      member function 'HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::malloc' requested here
        ptr = SuperHeap::myLittleHeap[sizeClass].malloc (realSize);
                                                 ^
include/hoard/thresholdsegheap.h:42:31: note: in instantiation of member
      function 'HL::StrictSegHeap<80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::malloc' requested here
      void * ptr = SuperHeap::malloc (sz);
                              ^
Heap-Layers/heaps/threads/lockedheap.h:43:21: note: in instantiation of member
      function 'Hoard::ThresholdSegHeap<20, 65536, 80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::malloc' requested here
      return Super::malloc (sz);
                    ^
Heap-Layers/heaps/threads/threadheap.h:66:28: note: in instantiation of member
      function 'HL::LockedHeap<HL::MacLockType, Hoard::ThresholdSegHeap<20,
      65536, 80, size2class, class2size, HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>> >::malloc' requested here
      return getHeap(tid)->malloc (sz);
                           ^
Heap-Layers/heaps/combining/hybridheap.h:82:17: note: in instantiation of member
      function 'HL::ThreadHeap<64, HL::LockedHeap<HL::MacLockType,
      Hoard::ThresholdSegHeap<20, 65536, 80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>> >::malloc' requested here
      return bm.malloc (sz);
                ^
Heap-Layers/heaps/combining/hybridheap.h:57:15: note: in instantiation of member
      function 'HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128,
      Hoard::PerThreadHoardHeap>, Hoard::BigHeap>::slowPath' requested here
        ptr = slowPath (sz);
              ^
Heap-Layers/wrappers/ansiwrapper.h:52:31: note: in instantiation of member
      function 'HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128,
      Hoard::PerThreadHoardHeap>, Hoard::BigHeap>::malloc' requested here
      void * ptr = SuperHeap::malloc (sz);
                              ^
include/superblocks/tlab.h:102:33: note: in instantiation of member function
      'HL::ANSIWrapper<Hoard::IgnoreInvalidFree<HL::HybridHeap<8192,
      Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>,
      Hoard::BigHeap>> >::malloc' requested here
      void * ptr = _parentHeap->malloc (sz);
                                ^
source/libhoard.cpp:116:21: note: in instantiation of member function
      'Hoard::ThreadLocalAllocationBuffer<11, getSizeClass, getClassSize, 256,
      262144, Hoard::HoardSuperblock<HL::MacLockType, 65536, Hoard::SmallHeap>,
      65536, Hoard::HoardHeapType>::malloc' requested here
    void * ptr = h->malloc (sz);
                    ^
In file included from source/libhoard.cpp:33:
In file included from Heap-Layers/heaplayers.h:103:
In file included from Heap-Layers/heaps/all.h:1:
In file included from Heap-Layers/heaps/buildingblock/all.h:1:
Heap-Layers/heaps/buildingblock/adaptheap.h:59:9: error: use of undeclared
      identifier 'assert'
        assert (SuperHeap::getSize(ptr) >= sizeof(dict));
        ^
Heap-Layers/heaps/combining/strictsegheap.h:127:50: note: in instantiation of
      member function 'HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::free' requested here
        SuperHeap::myLittleHeap[objectSizeClass].free (ptr);
                                                 ^
include/hoard/thresholdsegheap.h:55:18: note: in instantiation of member
      function 'HL::StrictSegHeap<80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::free' requested here
      SuperHeap::free (ptr);
                 ^
Heap-Layers/heaps/threads/lockedheap.h:48:14: note: in instantiation of member
      function 'Hoard::ThresholdSegHeap<20, 65536, 80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::free' requested here
      Super::free (ptr);
             ^
Heap-Layers/heaps/threads/threadheap.h:73:21: note: in instantiation of member
      function 'HL::LockedHeap<HL::MacLockType, Hoard::ThresholdSegHeap<20,
      65536, 80, size2class, class2size, HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>> >::free' requested here
      getHeap(tid)->free (ptr);
                    ^
Heap-Layers/heaps/combining/hybridheap.h:68:12: note: in instantiation of member
      function 'HL::ThreadHeap<64, HL::LockedHeap<HL::MacLockType,
      Hoard::ThresholdSegHeap<20, 65536, 80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>> >::free' requested here
        bm.free (ptr);
           ^
include/superblocks/ignoreinvalidfree.h:49:13: note: in instantiation of member
      function 'HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128,
      Hoard::PerThreadHoardHeap>, Hoard::BigHeap>::free' requested here
        SuperHeap::free (ptr);
                   ^
Heap-Layers/wrappers/ansiwrapper.h:59:19: note: in instantiation of member
      function 'Hoard::IgnoreInvalidFree<HL::HybridHeap<8192,
      Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>,
      Hoard::BigHeap> >::free' requested here
        SuperHeap::free (ptr);
                   ^
include/superblocks/tlab.h:132:23: note: in instantiation of member function
      'HL::ANSIWrapper<Hoard::IgnoreInvalidFree<HL::HybridHeap<8192,
      Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>,
      Hoard::BigHeap>> >::free' requested here
          _parentHeap->free (ptr);
                       ^
source/libhoard.cpp:121:22: note: in instantiation of member function
      'Hoard::ThreadLocalAllocationBuffer<11, getSizeClass, getClassSize, 256,
      262144, Hoard::HoardSuperblock<HL::MacLockType, 65536, Hoard::SmallHeap>,
      65536, Hoard::HoardHeapType>::free' requested here
    getCustomHeap()->free (ptr);
                     ^
In file included from source/libhoard.cpp:33:
In file included from Heap-Layers/heaplayers.h:103:
In file included from Heap-Layers/heaps/all.h:6:
In file included from Heap-Layers/heaps/combining/all.h:2:
Heap-Layers/heaps/combining/segheap.h:189:15: error: no member named 'clear' in
      'Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>'
      bigheap.clear();
      ~~~~~~~ ^
include/hoard/thresholdsegheap.h:63:15: note: in instantiation of member
      function 'HL::SegHeap<80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::clear' requested here
          SuperHeap::clear();
                     ^
Heap-Layers/heaps/threads/lockedheap.h:48:14: note: in instantiation of member
      function 'Hoard::ThresholdSegHeap<20, 65536, 80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::free' requested here
      Super::free (ptr);
             ^
Heap-Layers/heaps/threads/threadheap.h:73:21: note: in instantiation of member
      function 'HL::LockedHeap<HL::MacLockType, Hoard::ThresholdSegHeap<20,
      65536, 80, size2class, class2size, HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>> >::free' requested here
      getHeap(tid)->free (ptr);
                    ^
Heap-Layers/heaps/combining/hybridheap.h:68:12: note: in instantiation of member
      function 'HL::ThreadHeap<64, HL::LockedHeap<HL::MacLockType,
      Hoard::ThresholdSegHeap<20, 65536, 80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>> >::free' requested here
        bm.free (ptr);
           ^
include/superblocks/ignoreinvalidfree.h:49:13: note: in instantiation of member
      function 'HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128,
      Hoard::PerThreadHoardHeap>, Hoard::BigHeap>::free' requested here
        SuperHeap::free (ptr);
                   ^
Heap-Layers/wrappers/ansiwrapper.h:59:19: note: in instantiation of member
      function 'Hoard::IgnoreInvalidFree<HL::HybridHeap<8192,
      Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>,
      Hoard::BigHeap> >::free' requested here
        SuperHeap::free (ptr);
                   ^
include/superblocks/tlab.h:132:23: note: in instantiation of member function
      'HL::ANSIWrapper<Hoard::IgnoreInvalidFree<HL::HybridHeap<8192,
      Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>,
      Hoard::BigHeap>> >::free' requested here
          _parentHeap->free (ptr);
                       ^
source/libhoard.cpp:121:22: note: in instantiation of member function
      'Hoard::ThreadLocalAllocationBuffer<11, getSizeClass, getClassSize, 256,
      262144, Hoard::HoardSuperblock<HL::MacLockType, 65536, Hoard::SmallHeap>,
      65536, Hoard::HoardHeapType>::free' requested here
    getCustomHeap()->free (ptr);
                     ^
In file included from source/libhoard.cpp:33:
In file included from Heap-Layers/heaplayers.h:103:
In file included from Heap-Layers/heaps/all.h:1:
In file included from Heap-Layers/heaps/buildingblock/all.h:1:
Heap-Layers/heaps/buildingblock/adaptheap.h:81:18: error: no member named
      'clear' in 'Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType,
      65536, Hoard::BigHeap>, 65536, Hoard::MmapSource>'
      SuperHeap::clear();
                 ^
Heap-Layers/heaps/combining/segheap.h:184:25: note: in instantiation of member
      function 'HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::clear' requested here
        myLittleHeap[i].clear();
                        ^
include/hoard/thresholdsegheap.h:63:15: note: in instantiation of member
      function 'HL::SegHeap<80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::clear' requested here
          SuperHeap::clear();
                     ^
Heap-Layers/heaps/threads/lockedheap.h:48:14: note: in instantiation of member
      function 'Hoard::ThresholdSegHeap<20, 65536, 80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::free' requested here
      Super::free (ptr);
             ^
Heap-Layers/heaps/threads/threadheap.h:73:21: note: in instantiation of member
      function 'HL::LockedHeap<HL::MacLockType, Hoard::ThresholdSegHeap<20,
      65536, 80, size2class, class2size, HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>> >::free' requested here
      getHeap(tid)->free (ptr);
                    ^
Heap-Layers/heaps/combining/hybridheap.h:68:12: note: in instantiation of member
      function 'HL::ThreadHeap<64, HL::LockedHeap<HL::MacLockType,
      Hoard::ThresholdSegHeap<20, 65536, 80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>> >::free' requested here
        bm.free (ptr);
           ^
include/superblocks/ignoreinvalidfree.h:49:13: note: in instantiation of member
      function 'HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128,
      Hoard::PerThreadHoardHeap>, Hoard::BigHeap>::free' requested here
        SuperHeap::free (ptr);
                   ^
Heap-Layers/wrappers/ansiwrapper.h:59:19: note: in instantiation of member
      function 'Hoard::IgnoreInvalidFree<HL::HybridHeap<8192,
      Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>,
      Hoard::BigHeap> >::free' requested here
        SuperHeap::free (ptr);
                   ^
include/superblocks/tlab.h:132:23: note: in instantiation of member function
      'HL::ANSIWrapper<Hoard::IgnoreInvalidFree<HL::HybridHeap<8192,
      Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>,
      Hoard::BigHeap>> >::free' requested here
          _parentHeap->free (ptr);
                       ^
source/libhoard.cpp:121:22: note: in instantiation of member function
      'Hoard::ThreadLocalAllocationBuffer<11, getSizeClass, getClassSize, 256,
      262144, Hoard::HoardSuperblock<HL::MacLockType, 65536, Hoard::SmallHeap>,
      65536, Hoard::HoardHeapType>::free' requested here
    getCustomHeap()->free (ptr);
                     ^
4 errors generated.
make: *** [macos] Error 1

Windows 10 version works for 64-bit but not 32-bit builds

It remains to be seen whether this is a problem in practice but it would be good to continue to maintain 32-bit support if possible.

"Assertion `c2s(left) >= sz' failed." on large array allocation.

Hi!

I tried the git master version of Hoard and I am pretty amazed by the performance. My first test where all right. But my tests with larger problem sizes failed. Attached I send you the output of gdb after a assertion raised.

include/hoard/geometricsizeclass.h:56: static int Hoard::GeometricSizeClass<MaxOverhead, Alignment>::size2class(size_t) [with int MaxOverhead = 20; int Alignment = 16; size_t = long unsigned int]: Assertion `c2s(left) >= sz' failed.

Program received signal SIGABRT, Aborted.
0x00007ffff6eae1c9 in raise () from /usr/lib/libc.so.6
(gdb) where
#0  0x00007ffff6eae1c9 in raise () from /usr/lib/libc.so.6
#1  0x00007ffff6eaf5c8 in abort () from /usr/lib/libc.so.6
#2  0x00007ffff6ea7356 in __assert_fail_base () from /usr/lib/libc.so.6
#3  0x00007ffff6ea7402 in __assert_fail () from /usr/lib/libc.so.6
#4  0x00007ffff7b6f1d9 in Hoard::GeometricSizeClass<20, 16>::size2class (sz=314956304)
    at include/hoard/geometricsizeclass.h:56
#5  0x00007ffff7b719e6 in Hoard::ThresholdSegHeap<25, 1048576, 80, &Hoard::GeometricSizeClass<20, 16>::size2class, &Hoard::GeometricSizeClass<20, 16>::class2size, HL::AdaptHeap<HL::DLList, Hoard::objectSource>, Hoard::objectSource>::malloc (this=0x7ffff7dc57b0 <getMainHoardHeap()::thBuf+244688>, sz=314956304) at include/hoard/thresholdsegheap.h:37
#6  0x00007ffff7b71232 in HL::LockedHeap<HL::SpinLockType, Hoard::ThresholdSegHeap<25, 1048576, 80, &Hoard::GeometricSizeClass<20, 16>::size2class, &Hoard::GeometricSizeClass<20, 16>::class2size, HL::AdaptHeap<HL::DLList, Hoard::objectSource>, Hoard::objectSource> >::malloc (this=0x7ffff7dc57b0 <getMainHoardHeap()::thBuf+244688>, sz=314956304)
    at Heap-Layers/heaps/threads/lockedheap.h:43
#7  0x00007ffff7b70cc4 in HL::ThreadHeap<64, HL::LockedHeap<HL::SpinLockType, Hoard::ThresholdSegHeap<25, 1048576, 80, &Hoard::GeometricSizeClass<20, 16>::size2class, &Hoard::GeometricSizeClass<20, 16>::class2size, HL::AdaptHeap<HL::DLList, Hoard::objectSource>, Hoard::objectSource> > >::malloc (this=0x7ffff7db0678 <getMainHoardHeap()::thBuf+158360>, 
    sz=314956304) at Heap-Layers/heaps/./threads/threadheap.h:66
#8  0x00007ffff7b7063e in HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>, Hoard::BigHeap>::slowPath (this=0x7ffff7d89be0 <getMainHoardHeap()::thBuf>, sz=314956304)
    at Heap-Layers/heaps/./combining/hybridheap.h:82
#9  0x00007ffff7b700a2 in HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>, Hoard::BigHeap>::malloc (this=0x7ffff7d89be0 <getMainHoardHeap()::thBuf>, sz=314956304)
    at Heap-Layers/heaps/./combining/hybridheap.h:57
#10 0x00007ffff7b6f9f9 in HL::ANSIWrapper<Hoard::IgnoreInvalidFree<HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>, Hoard::BigHeap> > >::malloc (this=0x7ffff7d89be0 <getMainHoardHeap()::thBuf>, 
    sz=314956304) at Heap-Layers/wrappers/ansiwrapper.h:52
#11 0x00007ffff7b6f4fb in Hoard::ThreadLocalAllocationBuffer<11, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536>::getSizeClass, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536>::getClassSize, 256u, 262144u, Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536u, Hoard::HoardHeapType>::malloc (this=0x7ffff7fd0770, sz=314956304) at include/superblocks/tlab.h:102
#12 0x00007ffff7b6e7ec in xxmalloc (sz=314956304) at source/libhoard.cpp:116
#13 0x00007ffff7b77909 in malloc (sz=314956304) at Heap-Layers/wrappers/gnuwrapper.cpp:265
#14 0x000000000040680a in utils_read_metis_edges (mg=0x7ffff7f90070, 
    file_name=0x7fffffffe9c9 "../../Graph-Daten/pokec/nodes-0.7-1/pokec-clean-0.7-1.metis") at utils.c:47
#15 0x000000000040c072 in main (argc=2, argv=0x7fffffffe698) at community-find.c:26

The problematic malloc at my side leads to the allocation of 314956304 bytes. Malloc of smaler arrays went fine.

How may I use Hoard for large memory allocs?

Verify Linux build works fine out of the box (with optimizations).

Updated the Makefile to use CPPFLAGS; verify that this does the right thing on Linux builds. If not, modify to include appropriate optimizations.

Completion of error handling

Would you like to add more error handling for return values from functions like the following?

• dlsym ⇒ thr_create
• LoadLibrary ⇒ ReferenceHoard

Please remove the -g flag in the GNUMakefile for non debug builds.

Please remove the -g flag in the GNUMakefile for non debug builds. Could see a performance improvement without that. (also looks like a typo in section LINUX_GCC_x86_64_COMPILE ).

Header only version of Hoard?

Hi @bertwesarg , thanks for the great work. Is it possible to provide a single-header version of Hoard?

shbench.cpp license apparently forbids cloning

The leader comment of benchmarks/shbench/shbench.cpp says:

 * This source code, or source code derived from it, may not be redistributed
 * without express written permission of the copyright owner.

Could you clarify whether we (interested devs on GitHub) have such permission? I don't mean here in this issue discussion so much as in the file itself.

This license declaration seems at odds with "Hoard is distributed under the GPL".

Thanks!

Interposition not working on FreeBSD

Try to get working for FreeBSD for version 4 release.

errors on ArchLinux and python3.9

Hello

I am trying to use scalene on some code.

I cannot post the whole code, but here is what it does, at least:

• creating threads, processes, using the 'forkserver' start method, on linux, and multiprocessing.get_context()
• overriding threading.Thread.run
• creating multiprocessing,queues

I tried to create a simple reproducer code, but I couldn't find what causes the error.

When my code creates new processes, I get these errors:

Exception ignored in: <function _after_fork at 0x559984ec9880>
Traceback (most recent call last):
  File "/usr/lib/python3.9/threading.py", line 1496, in _after_fork
    thread._reset_internal_locks(False)
  File "/usr/lib/python3.9/threading.py", line 827, in _reset_internal_locks
    self._started._at_fork_reinit()
  File "/usr/lib/python3.9/threading.py", line 527, in _at_fork_reinit
    self._cond._at_fork_reinit()
  File "/usr/lib/python3.9/threading.py", line 253, in _at_fork_reinit
    self._lock._at_fork_reinit()
AttributeError: 'ReplacementLock' object has no attribute '_at_fork_reinit'

Is there a way to get more details about this exception?

Thank you

Compilation issues on Solaris SunOS 5.10

Hoard 3.12 is not compiling on SunOS 5.10 with compiler version CC: Sun C++ 5.8

Errors:

CC -dalign -xbuiltin=%all -fast -xO5 -DNDEBUG -mt -g -xildoff -xthreadvar=dynamic -L/usr/lib/lwp -R/usr/lib/lwp -I. -Iinclude -Iinclude/util -Iinclude/hoard -Iinclude/superblocks -IHeap-Layers -D_REENTRANT=1 -G -PIC source/libhoard.cpp source/unixtls.cpp Heap-Layers/wrappers/wrapper.cpp Heap-Layers/wrappers/arch-specific/sparc-interchange.il -o libhoard_32.so -lthread -ldl -lCrun
source/libhoard.cpp:
"include/superblocks/tlab.h", line 90: Error: Cannot use HL::SLList::Entry* to initialize int_.
"source/libhoard.cpp", line 131: Where: While instantiating "Hoard::ThreadLocalAllocationBuffer<11, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536>::getSizeClass, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536>::getClassSize, 256, 2097152, Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536, Hoard::HoardHeapType>::malloc(unsigned)".
"source/libhoard.cpp", line 131: Where: Instantiated from non-template code.
"include/superblocks/tlab.h", line 102: Error: Cannot use void_ to initialize int_.
"source/libhoard.cpp", line 131: Where: While instantiating "Hoard::ThreadLocalAllocationBuffer<11, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536>::getSizeClass, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536>::getClassSize, 256, 2097152, Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536, Hoard::HoardHeapType>::malloc(unsigned)".
"source/libhoard.cpp", line 131: Where: Instantiated from non-template code.
2 Error(s) detected.
source/unixtls.cpp:
"source/unixtls.cpp", line 82: Error: "," expected instead of "attribute".
"source/unixtls.cpp", line 83: Error: "," expected instead of "attribute".
"source/unixtls.cpp", line 87: Error: "{" expected instead of "attribute".
"source/unixtls.cpp", line 87: Error: constructor is not defined.
"source/unixtls.cpp", line 89: Error: Use ";" to terminate declarations.
"source/unixtls.cpp", line 99: Error: Use ";" to terminate declarations.
"source/unixtls.cpp", line 100: Error: Cannot return bool from a function that should return Hoard::ThreadLocalAllocationBuffer<11, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536>::getSizeClass, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536>::getClassSize, 256, 2097152, Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536, Hoard::HoardHeapType>.
"source/unixtls.cpp", line 103: Error: Use ";" to terminate declarations.
"source/unixtls.cpp", line 185: Error: Linkage specifications are allowed only at file level.
"source/unixtls.cpp", line 185: Error: Identifier expected instead of "{".
"source/unixtls.cpp", line 185: Error: Use ";" to terminate declarations.
"source/unixtls.cpp", line 200: Error: Use ";" to terminate declarations.
"source/unixtls.cpp", line 215: Error: Linkage specifications are allowed only at file level.
"source/unixtls.cpp", line 215: Error: Identifier expected instead of "{".
"source/unixtls.cpp", line 215: Error: Use ";" to terminate declarations.
"source/unixtls.cpp", line 216: Error: "inline" is not allowed here.
"source/unixtls.cpp", line 216: Error: Use ";" to terminate declarations.
"source/unixtls.cpp", line 219: Error: threadFunctionType is not defined.
"source/unixtls.cpp", line 220: Error: threadFunctionType is not defined.
"source/unixtls.cpp", line 220: Error: a is not defined.
"source/unixtls.cpp", line 220: Error: Cannot use int to initialize int.
"source/unixtls.cpp", line 222: Error: threadFunctionType is not defined.
"source/unixtls.cpp", line 222: Error: int is not a structure type.
"source/unixtls.cpp", line 223: Error: int is not a structure type.
"source/unixtls.cpp", line 226: Error: The operation "_ int" is illegal.
Compilation aborted, too many Error messages.
Heap-Layers/wrappers/wrapper.cpp:
make: *** [SunOS-sunw-sparc] Error 3

Allow macOS architectures to be configured

GNUmakefile hardcodes the architectures for which to compile on macOS in the MACOS_COMPILE and MACOS_COMPILE_DEBUG variables. You should make this configurable via another makefile variable. Currently you're hardcoding the architectures i386 and x86_64, but Apple is phasing out 32-bit support and you can't compile for i386 anymore as of the macOS 10.14 SDK.

Crashes everything on ubuntu 18.04

I did the following, not sure why but this seems to break everything on Ubuntu 18.04

git clone https://github.com/emeryberger/Hoard
make
echo "export LD_PRELOAD=$PWD/libhoard.so" >> ~/.bashrc

Opening up a new terminal and trying to do anything like ls, vim, chmod, etc. causes a seg fault

Sys Info Below

uname -a
Linux 4.15.0-39-generic #42-Ubuntu SMP Tue Oct 23 15:48:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

g++ -v
Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/7/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 7.3.0-27ubuntu1~18.04' --with-bugurl=file:///usr/share/doc/gcc-7/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr --with-gcc-major-version-only --program-suffix=-7 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 7.3.0 (Ubuntu 7.3.0-27ubuntu1~18.04) 

Last Commit on master
de06b8b

This is the same as #49 but I could not re-open that issue

Extending the HoardSuperblock to support different headers

Hi,

I'd like to be able to add additional fields to the HoardSuperblock header. If the Header were passed as a template parameter, that would be easy.

Thanks,
Irene

Crashes with Large Superblock Sizes

Hoard crashes when SUPERBLOCK_SIZE is #define'd with values larger than 256M.

We think it's during program initialization. Here's the backtrace:

Hoard::ThreadLocalAllocationBuffer<15, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 524288, Hoard::SmallHeap>, 524288ul>::getSizeClass, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 524288, Hoard::SmallHeap>, 524288ul>::getClassSize, 256ul, 2097152ul, Hoard::HoardSuperblock<HL::SpinLockType, 524288, Hoard::SmallHeap, Hoard::HoardSuperblockHeader>, 524288u, Hoard::HoardHeapType>::free(Hoard::ThreadLocalAllocationBuffer<15, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 524288, Hoard::SmallHeap>, 524288>::getSizeClass, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 524288, Hoard::SmallHeap>, 524288>::getClassSize, 256, 2097152, Hoard::HoardSuperblock<HL::SpinLockType, 524288, Hoard::SmallHeap, HoardSuperblockHeader>, 524288, Hoard::HoardHeapType> * this, void * ptr) (/home/.../include/superblocks/tlab.h:114)
__GI__IO_setb(_IO_FILE * f, char * b, char * eb, int a) (/build/glibc-yWQXbR/glibc-2.24/libio/genops.c:382)
_IO_new_file_close_it(_IO_FILE * fp) (/build/glibc-yWQXbR/glibc-2.24/libio/fileops.c:193)
_IO_new_fclose(_IO_FILE * fp) (/build/glibc-yWQXbR/glibc-2.24/libio/iofclose.c:58)
[Unknown/Just-In-Time compiled code] (Unknown Source:0)
ld-linux-x86-64.so.2!call_init(struct link_map * l, int argc, char ** argv, char ** env) (/build/glibc-yWQXbR/glibc-2.24/elf/dl-init.c:72)
ld-linux-x86-64.so.2!call_init(char ** env, char ** argv, int argc, struct link_map * l) (/build/glibc-yWQXbR/glibc-2.24/elf/dl-init.c:30)
ld-linux-x86-64.so.2!_dl_init(struct link_map * main_map, int argc, char ** argv, char ** env) (/build/glibc-yWQXbR/glibc-2.24/elf/dl-init.c:120)
ld-linux-x86-64.so.2!_dl_start_user (Unknown Source:0)
[Unknown/Just-In-Time compiled code] (Unknown Source:0)

Thanks in advance!

Compilation fails for win32

Tried compiling commit 5996bcf (current master) for Windows (VC2008) and it failed on hoardsuperblockheader.h line 287 with an error about it not being legal to use "this" there.

Fix is to assign sizeof(*this) to a local const size_t variable; this can then be used in the sassert template without errors.

Support DESTDIR

You currently install your library with e.g.

cp libhoard.dylib $(PREFIX)

This assumes $(PREFIX) is writable. It might not be in the context of a package management system that wants to stage the files in a temporary location first.

You should support the standard $(DESTDIR) variable for this, by changing these copy commands to e.g.

cp libhoard.dylib $(DESTDIR)$(PREFIX)

Invariant in your paper

Hi Emery,

In your Hoard paper, should the condition of line 9 in free pseudocode be or instead of and?

cannot allocate memory for thread-local data: ABORT

it works in CentOS with some error,how to solve this problem...thanks

Can`t build Hoard library on Windows

I clone Horad by usage the link:

https://github.com/emeryberger/Hoard.git

Then as descibed in documentation I change directory to

C:\hoard\src

Then I try to build it:

C:\hoard\src> nmake

And receive error:

git clone https://github.com/emeryberger/Heap-Layers
fatal: destination path 'Heap-Layers' already exists and is not an empty directory.

Its bug - the Heap-Layers already exist when I clone Hoard repo. Ok, I delete Heap-Layers folder and try to build again:

C:\hoard\src> nmake

And receive followed error:

source\libhoard.cpp(34): fatal error C1083: stdalign.h: No such file or directory

So at now the sources can not be compiled on the Windows platform. Could you fix it?

New release?

Hi, I'm trying to update the out of date hoard port in MacPorts to the latest version 3.12 but I'm getting this build failure, on macOS 10.13.6 with Xcode 9.4.1:

In file included from source/libhoard.cpp:33:
In file included from Heap-Layers/heaplayers.h:106:
In file included from Heap-Layers/heaps/all.h:2:
In file included from Heap-Layers/heaps/./debug/all.h:4:
In file included from Heap-Layers/heaps/./debug/sanitycheckheap.h:13:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/map:818:5: error: static_assert failed "Allocator::value_type must be same type as value_type"
    static_assert((is_same<typename allocator_type::value_type, value_type>::value),
    ^             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Heap-Layers/heaps/./debug/sanitycheckheap.h:110:13: note: in instantiation of template class 'std::__1::map<void *, unsigned long, std::__1::less<void *>, HL::STLAllocator<std::__1::pair<const void *, unsigned long>, HL::FreelistHeap<HL::ZoneHeap<HL::MmapHeap, 16384> > > >' requested here
    mapType allocatedObjects;
            ^
Heap-Layers/heaps/./debug/sanitycheckheap.h:64:16: error: no type named 'iterator' in 'std::__1::map<void *, unsigned long, std::__1::less<void *>, HL::STLAllocator<std::__1::pair<const void *, unsigned long>, HL::FreelistHeap<HL::ZoneHeap<HL::MmapHeap, 16384> > > >'
      mapType::iterator i;
      ~~~~~~~~~^
Heap-Layers/heaps/./debug/sanitycheckheap.h:88:16: error: no type named 'iterator' in 'std::__1::map<void *, unsigned long, std::__1::less<void *>, HL::STLAllocator<std::__1::pair<const void *, unsigned long>, HL::FreelistHeap<HL::ZoneHeap<HL::MmapHeap, 16384> > > >'
      mapType::iterator i;
      ~~~~~~~~~^

I tried master (Hoard May 24 2018, Heap-Layers Oct 17 2018) and it builds, but I would rather not update the MacPorts port to some random point in time of your repositories. Is a new stable release planned at some point to which I could then update the port?

Would be great if, when you make a new release, you could provide a tarball containing the complete source code of Hoard and Heap-Layers and attach it to the GitHub release, like you did before for version 3.10, so we don't have to clone from git.

Investigate use of new hooking libraries

http://code.google.com/p/easyhook-continuing-detours/

Failed to allocate more than 2147483647 bytes of memory

I wanted to evaluate the Hoard for one of the project but i observed that Hoard fails to allocate more than 2GB of memory. I built the Hoard 3.12 with g++ version 4.9.0 on Linux. Is there any limitation with the Hoard for the memory allocation. If not what could be wrong?
The allocation for 2147483647 bytes succeeds and allocation for 2147483648 onward fails.

Below are the g++ and Linux details:

	g++ --version
	g++ (GCC) 4.9.0 20130520 (experimental)
	Copyright (C) 2013 Free Software Foundation, Inc.
	This is free software; see the source for copying conditions.  There is NO
	warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

	uname -a
	Linux xxx.xxx.com 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

Attached here AllocMem.zip the sample program. Below the sample program output for allocations:

With Default gcc allocator:

	./AllocMem 2147483648
	Trying to allocate 2147483648 bytes of memory
	Successfully allocated 2147483648 bytes of memory

With Hoard Allocator:

	export LD_PRELOAD=/data/local/MemoryAllocators/Hoard/libhoard.so
	./AllocMem 2147483648
	Trying to allocate 2147483648 bytes of memory
	Failed to allocate 2147483648 bytes of memory

uselibhoard.cpp cannot be compiled on Windows x64 platform.

When you add a uselibhoard.cpp file at your project on windows x64 platform, it occurs link error.
because of _WIN32 define.
_WIN32 define in uselibhoard.cpp has to be removed.

Build failed: error: there are no arguments to ‘getpid’ ...

I tried to build hoard from source on linux Fedora core 17 (64bit) gcc version 4.7.2 20120921 (Red Hat 4.7.2-2) (GCC) I did git clone --recursive but it give out below error. here is some part of compilation output which contained errors.

Thanks

g++ -g -W -Wconversion -Wall -m64 -I/usr/include/nptl -pipe -fPIC -O3 -finline-limit=20000 -finline-functions -DNDEBUG -I. -Iinclude -Iinclude/util -Iinclude/hoard -Iinclude/superblocks -IHeap-Layers -D_REENTRANT=1 -shared source/libhoard.cpp source/unixtls.cpp Heap-Layers/wrappers/gnuwrapper.cpp -Bsymbolic -o libhoard.so -ldl -lpthread
In file included from Heap-Layers/heaps/debug/all.h:3:0,
from Heap-Layers/heaps/all.h:2,
from Heap-Layers/heaplayers.h:105,
from source/libhoard.cpp:33:
Heap-Layers/heaps/debug/logheap.h: In constructor ‘HL::Log<Obj, MAX_ENTRIES>::Log()’:
Heap-Layers/heaps/debug/logheap.h:35:46: error: there are no arguments to ‘getpid’ that depend on a template parameter, so a declaration of ‘getpid’ must be available [-fpermissive]
Heap-Layers/heaps/debug/logheap.h:35:46: note: (if you use ‘-fpermissive’, G++ will accept your code, but allowing the use of an undeclared name is deprecated)
.
.
.
source/unixtls.cpp:47:6: warning: multi-line comment [-Wcomment]
In file included from Heap-Layers/heaps/debug/all.h:3:0,
from Heap-Layers/heaps/all.h:2,
from Heap-Layers/heaplayers.h:105,
from include/hoard/hoardheap.h:31,
from include/hoard/hoardtlab.h:39,
from source/unixtls.cpp:56:
Heap-Layers/heaps/debug/logheap.h: In constructor ‘HL::Log<Obj, MAX_ENTRIES>::Log()’:
Heap-Layers/heaps/debug/logheap.h:35:46: error: there are no arguments to ‘getpid’ that depend on a template parameter, so a declaration of ‘getpid’ must be available [-fpermissive]
.
.
.
Heap-Layers/heaps/debug/logheap.h: In constructor ‘HL::Log<Obj, MAX_ENTRIES>::Log()’:
Heap-Layers/heaps/debug/logheap.h:35:46: error: there are no arguments to ‘getpid’ that depend on a template parameter, so a declaration of ‘getpid’ must be available [-fpermissive]
.
.
.Heap-Layers/heaps/general/kingsleyheap.h: In function ‘int Kingsley::ceilLog2(size_t)’:
Heap-Layers/heaps/general/kingsleyheap.h:89:63: warning: conversion to ‘int’ from ‘long unsigned int’ may alter its value [-Wconversion]
Heap-Layers/wrappers/gnuwrapper.cpp: In function ‘void my_init_hook()’:
Heap-Layers/wrappers/gnuwrapper.cpp:85:25: warning: ‘__malloc_hook’ is deprecated (declared at /usr/include/malloc.h:176) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:86:23: warning: ‘__free_hook’ is deprecated (declared at /usr/include/malloc.h:173) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:87:26: warning: ‘__realloc_hook’ is deprecated (declared at /usr/include/malloc.h:179) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:88:27: warning: ‘__memalign_hook’ is deprecated (declared at /usr/include/malloc.h:183) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:91:7: warning: ‘__malloc_hook’ is deprecated (declared at /usr/include/malloc.h:176) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:92:7: warning: ‘__free_hook’ is deprecated (declared at /usr/include/malloc.h:173) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:93:7: warning: ‘realloc_hook’ is deprecated (declared at /usr/include/malloc.h:179) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:94:7: warning: ‘memalign_hook’ is deprecated (declared at /usr/include/malloc.h:183) [-Wdeprecated-declarations]
In file included from Heap-Layers/heaps/top/mmapheap.h:50:0,
from Heap-Layers/heaps/debug/sanitycheckheap.h:18,
from Heap-Layers/heaps/debug/all.h:4,
from Heap-Layers/heaps/all.h:2,
from Heap-Layers/heaplayers.h:105,
from Heap-Layers/wrappers/gnuwrapper.cpp:23:
Heap-Layers/utility/myhashmap.h: In instantiation of ‘Value HL::MyHashMap<Key, Value, Allocator>::get(Key) [with Key = void; Value = long unsigned int; Allocator = HL::MmapHeap::MyHeap]’:
Heap-Layers/heaps/top/mmapheap.h:178:33: required from here
Heap-Layers/utility/myhashmap.h:69:67: warning: conversion to ‘unsigned int’ from ‘long unsigned int’ may alter its value [-Wconversion]
Heap-Layers/utility/myhashmap.h: In instantiation of ‘void HL::MyHashMap<Key, Value, Allocator>::erase(Key) [with Key = void; Value = long unsigned int; Allocator = HL::MmapHeap::MyHeap]’:
Heap-Layers/heaps/top/mmapheap.h:195:23: required from here
Heap-Layers/utility/myhashmap.h:82:67: warning: conversion to ‘unsigned int’ from ‘long unsigned int’ may alter its value [-Wconversion]
Heap-Layers/utility/myhashmap.h: In instantiation of ‘void HL::MyHashMap<Key, Value, Allocator>::insert(Key, Value) [with Key = void; Value = long unsigned int; Allocator = HL::MmapHeap::MyHeap]’:
Heap-Layers/utility/myhashmap.h:65:7: required from ‘void HL::MyHashMap<Key, Value, Allocator>::set(Key, Value) [with Key = void; Value = long unsigned int; Allocator = HL::MmapHeap::MyHeap]’
Heap-Layers/heaps/top/mmapheap.h:170:25: required from here
Heap-Layers/utility/myhashmap.h:108:67: warning: conversion to ‘unsigned int’ from ‘long unsigned int’ may alter its value [-Wconversion]
make: *** [linux-gcc-x86-64] Error 1

Heap-Layers not included in release archives

Is this intentional?

Simple hello world program crashes when linked to hoard

Hi,

I have compiled Hoard according to the instructions under Windows 10 with Compiler "Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24218.1 for x64".

I have then created a new dir containing

• hello.cpp
• make.bat
• libhoard.lib
• libhoard.dll
• uselibhoard.cpp

I am attaching the text files as "hg_test.zip" to this post:
hg_test.zip

I have then called "make" which creates "hello.exe", with this output on the console:
``
Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24218.1 for x64
Copyright (C) Microsoft Corporation. All rights reserved.

hello.cpp
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\xlocale(341): warning C4530: C++ exception handler used, but unwind semantics are not enabled. Specify /EHsc
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\exception(359): warning C4577: 'noexcept' used with no exception handling mode specified; termination on exception is not guaranteed. Specify /EHsc
uselibhoard.cpp
Generating Code...
Microsoft (R) Incremental Linker Version 14.00.24218.1
Copyright (C) Microsoft Corporation. All rights reserved.

/out:hello.exe
hello.obj
uselibhoard.obj
libhoard.lib
``

(OK - I have specified /EHsc in addition to get rid of those warnings, but this is irrelevant here as the crash still happens then.)

When I now execute "hello.exe", a popup window appears: "Appication Error - The application was unable to start correctly (0xc0000142). Click OK to close the application."

What am I doing wrong?

Best regards,
Hagen

GPL license text still in source files.

The project has been re-licensed as Apache 2, but all the source files still have the GPL license text.

Crashing on Windows while doing make_shared for a shared_ptr

Am using Hoard in my application to run on both Linux and Windows. While it runs perfectly fine on Linux on Windows after some allocations it crashes at a std::make_shared for an object. Its gets a "Exception thrown at 0x00007FF6E568FCDE in : Access violation writing location 0x00007FF6E5778078". Does it require any specific configuration for Windows? I have followed the exact instructions to link to libhoard.dll and also my application loads it too but after some random number of allocations it crashes the application. The application works fine without Hoard. Am using Visual Studio 2015 community version on Windows 10 host.

Does not compile on VS-2010

I am in the process of reviewing this library for use in a multi-threaded utility that makes extensive use of the STL. When I attempt to compile the source using Visual Studio 2010 I get the following error from the command line:

cl /I. /Iinclude /Iinclude/util /Iinclude/hoard /Iinclude/superblocks /IHeap-Layers /Zi /nologo /W1 /WX- /Ox /Oi /Oy- /GL /D "_WINRT_DLL" /D "NDEBUG" /D "_WINDOWS" /D "_WINDLL" /D "_UNICODE" /D "UNICODE" /D "WINAPI_FAMILY=2" /Gm- /EHsc /MD /GS /Gy /fp:precise /Zc:wchar_t /Zc:forScope /Gd /errorReport:queue "source\libhoard.cpp" "Heap-Layers\wrappers\winwrapper.cpp" "source\wintls.cpp" /link /DLL /subsystem:console /OUT:libhoard.dll

libhoard.cpp
c:\work\hoard\src\heap-layers\heaps/special/zoneheap.h(53) : error C2247: 'Alignment' not accessible because 'HL::MmapHeap' uses 'private' to inherit from 'HL::PrivateMmapHeap'
c:\work\hoard\src\heap-layers\heaps/top/mmapheap.h(79) : see declaration of 'Alignment'
c:\work\hoard\src\heap-layers\heaps/top/mmapheap.h(140) : see declaration of 'HL::MmapHeap'
c:\work\hoard\src\heap-layers\heaps/top/mmapheap.h(73) : see declaration of 'HL::PrivateMmapHeap'
c:\work\hoard\src\heap-layers\heaps/buildingblock/freelistheap.h(49) : see reference to class template instantiation 'HL::ZoneHeap<SuperHeap,ChunkSize>' being compiled
with
[
SuperHeap=HL::MmapHeap,
ChunkSize=16384
]
c:\work\hoard\src\heap-layers\wrappers/stlallocator.h(63) : see reference to class template instantiation 'HL::FreelistHeap' being compiled
with
[
SuperHeap=HL::ZoneHeapHL::MmapHeap,16384
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\map(26) : see reference to class template instantiation 'HL::STLAllocator<T,Super>' being compiled
with
[
T=HL::SanityCheckHeap::objType,
Super=HL::SanityCheckHeap::heapType
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\xtree(451) : see reference to class template instantiation 'std::_Tmap_traits<_Kty,_Ty,_Pr,_Alloc,_Mfl>' being compiled
with
[
_Kty=void *,
_Ty=size_t,
_Pr=HL::SanityCheckHeap::localComparator,
_Alloc=HL::SanityCheckHeap::localAllocator,
_Mfl=false
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\xtree(520) : see reference to class template instantiation 'std::_Tree_nod<_Traits>' being compiled
with
[
_Traits=std::_Tmap_traits<void *,size_t,HL::SanityCheckHeap::localComparator,HL::SanityCheckHeap::localAllocator,false>
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\xtree(659) : see reference to class template instantiation 'std::_Tree_val<_Traits>' being compiled
with
[
_Traits=std::_Tmap_traits<void *,size_t,HL::SanityCheckHeap::localComparator,HL::SanityCheckHeap::localAllocator,false>
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\map(81) : see reference to class template instantiation 'std::_Tree<_Traits>' being compiled
with
[
_Traits=std::_Tmap_traits<void *,size_t,HL::SanityCheckHeap::localComparator,HL::SanityCheckHeap::localAllocator,false>
]
c:\work\hoard\src\heap-layers\heaps\debug\sanitycheckheap.h(110) : see reference to class template instantiation 'std::map<_Kty,_Ty,_Pr,_Alloc>' being compiled
with
[
_Kty=void *,
_Ty=size_t,
_Pr=HL::SanityCheckHeap::localComparator,
_Alloc=HL::SanityCheckHeap::localAllocator
]
c:\work\hoard\src\heap-layers\heaps\debug\sanitycheckheap.h(111) : see reference to class template instantiation 'HL::SanityCheckHeap' being compiled
c:\work\hoard\src\heap-layers\heaps\threads\threadspecificheap.h(34) : fatal error C1189: #error : "This functionality currently is not implemented for Windows."

winwrapper.cpp
wintls.cpp
c:\work\hoard\src\heap-layers\heaps/special/zoneheap.h(53) : error C2247: 'Alignment' not accessible because 'HL::MmapHeap' uses 'private' to inherit from 'HL::PrivateMmapHeap'
c:\work\hoard\src\heap-layers\heaps/top/mmapheap.h(79) : see declaration of 'Alignment'
c:\work\hoard\src\heap-layers\heaps/top/mmapheap.h(140) : see declaration of 'HL::MmapHeap'
c:\work\hoard\src\heap-layers\heaps/top/mmapheap.h(73) : see declaration of 'HL::PrivateMmapHeap'
c:\work\hoard\src\heap-layers\heaps/buildingblock/freelistheap.h(49) : see reference to class template instantiation 'HL::ZoneHeap<SuperHeap,ChunkSize>' being compiled
with
[
SuperHeap=HL::MmapHeap,
ChunkSize=16384
]
c:\work\hoard\src\heap-layers\wrappers/stlallocator.h(63) : see reference to class template instantiation 'HL::FreelistHeap' being compiled
with
[
SuperHeap=HL::ZoneHeapHL::MmapHeap,16384
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\map(26) : see reference to class template instantiation 'HL::STLAllocator<T,Super>' being compiled
with
[
T=HL::SanityCheckHeap::objType,
Super=HL::SanityCheckHeap::heapType
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\xtree(451) : see reference to class template instantiation 'std::_Tmap_traits<_Kty,_Ty,_Pr,_Alloc,_Mfl>' being compiled
with
[
_Kty=void *,
_Ty=size_t,
_Pr=HL::SanityCheckHeap::localComparator,
_Alloc=HL::SanityCheckHeap::localAllocator,
_Mfl=false
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\xtree(520) : see reference to class template instantiation 'std::_Tree_nod<_Traits>' being compiled
with
[
_Traits=std::_Tmap_traits<void *,size_t,HL::SanityCheckHeap::localComparator,HL::SanityCheckHeap::localAllocator,false>
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\xtree(659) : see reference to class template instantiation 'std::_Tree_val<_Traits>' being compiled
with
[
_Traits=std::_Tmap_traits<void *,size_t,HL::SanityCheckHeap::localComparator,HL::SanityCheckHeap::localAllocator,false>
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\map(81) : see reference to class template instantiation 'std::_Tree<_Traits>' being compiled
with
[
_Traits=std::_Tmap_traits<void *,size_t,HL::SanityCheckHeap::localComparator,HL::SanityCheckHeap::localAllocator,false>
]
c:\work\hoard\src\heap-layers\heaps\debug\sanitycheckheap.h(110) : see reference to class template instantiation 'std::map<_Kty,_Ty,_Pr,_Alloc>' being compiled
with
[
_Kty=void *,
_Ty=size_t,
_Pr=HL::SanityCheckHeap::localComparator,
_Alloc=HL::SanityCheckHeap::localAllocator
]
c:\work\hoard\src\heap-layers\heaps\debug\sanitycheckheap.h(111) : see reference to class template instantiation 'HL::SanityCheckHeap' being compiled
c:\work\hoard\src\heap-layers\heaps\threads\threadspecificheap.h(34) : fatal error C1189: #error : "This functionality currently is not implemented for Windows."

I would like to directly compile the library into the program to eliminate the need to distribute another file if necessary, but I cannot even built the library as retrieved from the source. What am I missing? Thanks.

why performance decline？

when i compile this program, type "make" instruction, so i change "-std=c++14" to "-std=c++11".
then i export envorment instr,
OK. when i run the program, find the performance decline ? So How to solve this problem?

Linking in windows does not seem to work

Program: Executable / with /MD flag / x64 / Release / VC10
Code: Compiled from source / x64 / Release. / VC10

In the binary project I've updated the linker options to link to the appropriate usewinhoard.obj and winhoard.lib. I've placed the winhoard.dll next to the executable. I've validated that it is required during runtime.
The runtime on my program is not improving and I know that there is alot of room for improvement as I have verified this using other allocators. The dll does get hooked, I have checked this by std::cout-ing some indicators.
Do I need to explicitly call hoard_malloc? As I read it Hoard should hijack any new, malloc calls. If that is the case how does it work in the presence of another pair of allocation hooks (this is what my code has at the moment). Do I need to take those out for hoard's hooks to kick in?

Thanks.

Issues replacing malloc() on Windows

I'm evaluating the possibility to replace malloc() for LLVM.
I hit some problems I'd like to point out here:
The README is wrong:

c:\Users\fluttershy\work\Hoard\src>nmake windows

Microsoft (R) Program Maintenance Utility Version 14.00.23918.0
Copyright (C) Microsoft Corporation.  All rights reserved.

NMAKE : fatal error U1073: don't know how to make 'windows'
Stop.

The correct command seems to be just nmake:

c:\Users\fluttershy\work\Hoard\src>nmake

Microsoft (R) Program Maintenance Utility Version 14.00.23918.0
Copyright (C) Microsoft Corporation.  All rights reserved.

*****
This Makefile is for Windows only. For other systems, use gmake.
*****
        cl /I. /Iinclude /Iinclude/util /Iinclude/hoard /Iinclude/superblocks /I
Heap-Layers /D "NDEBUG" /D "_WINDOWS" /D "_WINDLL" /D "_WINRT_DLL" /D "_UNICODE"
 /D "UNICODE" /Zi /Ox /MD /nologo /W1 /WX- /Ox /Oi /Oy- /Gm- /EHsc /MD /GS /Gy /
Zc:wchar_t /Zc:forScope /Gd /errorReport:queue "source\libhoard.cpp" "Heap-Layer
s\wrappers\winwrapper.cpp" "source\wintls.cpp" /GL /link /DLL /subsystem:console
 /OUT:libhoard.dll
libhoard.cpp
winwrapper.cpp
wintls.cpp
   Creating library libhoard.lib and object libhoard.exp
Generating code
Finished generating code
        cl /I. /Iinclude /Iinclude/util /Iinclude/hoard /Iinclude/superblocks /I
Heap-Layers /D "NDEBUG" /D "_WINDOWS" /D "_WINDLL" /D "_WINRT_DLL" /D "_UNICODE"
 /D "UNICODE" /Zi /Ox /MD /nologo /W1 /WX- /Ox /Oi /Oy- /Gm- /EHsc /MD /GS /Gy /
Zc:wchar_t /Zc:forScope /Gd /errorReport:queue /c "source\uselibhoard.cpp"
uselibhoard.cpp

The malloc() replacement example doesn't quite work, in basic cases:

$ cat patatino.cpp
#include <stdlib.h>

int main (void) {
        malloc(16);
        return (0);
}

If I try to run the way it's described on the website:

c:\Users\fluttershy\work\Hoard\src>cl /Ox /MD patatino.cpp source\libhoard.cpp libhoard.lib
Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23918 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

patatino.cpp
libhoard.cpp
source\libhoard.cpp(33): fatal error C1083: Cannot open include file: 'heaplayer
s.h': No such file or directory
Generating Code...

It seems the issue is that it's not able to locate the headers, something like this might fix:

diff --git a/src/source/libhoard.cpp b/src/source/libhoard.cpp
index 54d3713..cb39efc 100644
--- a/src/source/libhoard.cpp
+++ b/src/source/libhoard.cpp
@@ -30,7 +30,7 @@
  * @author Emery Berger <http://www.cs.umass.edu/~emery>
  */

-#include "heaplayers.h"
+#include "../Heap-Layers/heaplayers.h"
 using namespace HL;

 #include <new>

But immediately after I hit another problem:

c:\users\fluttershy\work\hoard\src\heap-layers\heaps\combining\hybridheap.h(32):
 fatal error C1083: Cannot open include file: 'heaplayers.h': No such file or di
rectory

Any ideas on how to fix? I think the instructions on the website need some love.

Documentation says use "linux-gcc-x86-64" which is incorrect build target

Please update the documentation with the correct build target "linux-gcc-x86_64"

Poor performance with large objects in benchmark

It's possible that this is a quirk or unrealism of the particular benchmark design rather than a failing in hoard as such, but I tried running it through the t-test1.c test case from the ptmalloc family and the results were not good. (Tests run on Windows 7 high-end quad-core machine; each size change was run as a separate process.)

It performed reasonably well (better than the OS allocator and a ptmalloc-derivative allocator) with allocation sizes up to 8192, but when using size parameters of 16384 and above it performed abysmally. (eg. 0.3s at 8192 size, 22s at 16384 size; note that the system allocator scored 1.1s for hoard's 22s, and this difference only got worse as the size increased.)

There was also significantly higher CPU usage (to the point of pegging all processors to 100%, making the system unusable) when running those particular test values.

compilation fails for linux-gcc-x86 or -64

Heap-Layers/utility/align.h:17:49: warning: unused variable ‘isPowerOfTwo’ [-Wunused-variable]
Heap-Layers/utility/align.h: In instantiation of ‘size_t HL::align(size_t) [with long unsigned int Alignment = 65536ul; size_t = long unsigned int]’:
include/util/alignedmmap.h:83:7: required from ‘void* Hoard::AlignedMmapInstance<Alignment_>::malloc(size_t) [with long unsigned int Alignment_ = 65536ul; size_t = long unsigned int]’
Heap-Layers/heaps/threads/lockedheap.h:42:31: required from ‘void* HL::LockedHeap<LockType, Super>::malloc(size_t) [with LockType = HL::SpinLockType; Super = Hoard::AlignedMmapInstance<65536ul>; size_t = long unsigned int]’
include/util/exactlyoneheap.h:42:34: required from ‘void* Hoard::ExactlyOneHeap::malloc(size_t) [with Heap = HL::LockedHeap<HL::SpinLockType, Hoard::AlignedMmapInstance<65536ul> >; size_t = long unsigned int]’
include/superblocks/alignedsuperblockheap.h:54:70: required from ‘void* Hoard::SuperblockStore<SuperblockSize, TheLockType, MmapSource>::malloc(size_t) [with long unsigned int SuperblockSize = 65536ul; TheLockType = HL::SpinLockType; MmapSource = Hoard::MmapSource; size_t = long unsigned int]’
include/util/fixedrequestheap.h:43:44: required from ‘void* Hoard::FixedRequestHeap<RequestSize, SuperHeap>::malloc(size_t) [with long unsigned int RequestSize = 65536ul; SuperHeap = Hoard::SuperblockStore<65536ul, HL::SpinLockType, Hoard::MmapSource>; size_t = long unsigned int]’
Heap-Layers/heaps/threads/lockedheap.h:42:31: required from ‘void* HL::LockedHeap<LockType, Super>::malloc(size_t) [with LockType = HL::SpinLockType; Super = Hoard::FixedRequestHeap<65536ul, Hoard::SuperblockStore<65536ul, HL::SpinLockType, Hoard::MmapSource> >; size_t = long unsigned int]’
include/hoard/hoardmanager.h:354:49: required from ‘void* Hoard::HoardManager<SourceHeap, ParentHeap, SuperblockType_, EmptinessClasses, LockType, thresholdFunctionClass, HeapType>::getAnotherSuperblock(size_t) [with SourceHeap = Hoard::AlignedSuperblockHeap<HL::SpinLockType, 65536ul, Hoard::MmapSource>; ParentHeap = Hoard::GlobalHeap<65536ul, 8, Hoard::MmapSource, HL::SpinLockType>; SuperblockType_ = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>; int EmptinessClasses = 8; LockType = HL::SpinLockType; thresholdFunctionClass = Hoard::hoardThresholdFunctionClass; HeapType = Hoard::SmallHeap; size_t = long unsigned int]’
include/hoard/hoardmanager.h:298:4: required from ‘void* Hoard::HoardManager<SourceHeap, ParentHeap, SuperblockType_, EmptinessClasses, LockType, thresholdFunctionClass, HeapType>::slowPathMalloc(size_t) [with SourceHeap = Hoard::AlignedSuperblockHeap<HL::SpinLockType, 65536ul, Hoard::MmapSource>; ParentHeap = Hoard::GlobalHeap<65536ul, 8, Hoard::MmapSource, HL::SpinLockType>; SuperblockType_ = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>; int EmptinessClasses = 8; LockType = HL::SpinLockType; thresholdFunctionClass = Hoard::hoardThresholdFunctionClass; HeapType = Hoard::SmallHeap; size_t = long unsigned int]’
include/hoard/hoardmanager.h:93:2: required from ‘void* Hoard::HoardManager<SourceHeap, ParentHeap, SuperblockType_, EmptinessClasses, LockType, thresholdFunctionClass, HeapType>::malloc(size_t) [with SourceHeap = Hoard::AlignedSuperblockHeap<HL::SpinLockType, 65536ul, Hoard::MmapSource>; ParentHeap = Hoard::GlobalHeap<65536ul, 8, Hoard::MmapSource, HL::SpinLockType>; SuperblockType_ = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>; int EmptinessClasses = 8; LockType = HL::SpinLockType; thresholdFunctionClass = Hoard::hoardThresholdFunctionClass; HeapType = Hoard::SmallHeap; size_t = long unsigned int]’
include/hoard/hoardheap.h:171:31: required from here
Heap-Layers/utility/align.h:17:49: warning: unused variable ‘isPowerOfTwo’ [-Wunused-variable]
In file included from Heap-Layers/heaps/top/mmapheap.h:49:0,
from Heap-Layers/heaps/debug/sanitycheckheap.h:18,
from Heap-Layers/heaps/debug/all.h:4,
from Heap-Layers/heaps/all.h:2,
from Heap-Layers/heaplayers.h:105,
from include/hoard/hoardheap.h:31,
from include/hoard/hoardtlab.h:39,
from source/unixtls.cpp:56:
Heap-Layers/utility/myhashmap.h: In instantiation of ‘Value HL::MyHashMap<Key, Value, Allocator>::get(Key) [with Key = void_; Value = long unsigned int; Allocator = Hoard::AlignedMmapInstance<65536ul>::SourceHeap]’:
include/util/alignedmmap.h:127:44: required from ‘void Hoard::AlignedMmapInstance<Alignment_>::free(void_) [with long unsigned int Alignment_ = 65536ul]’
Heap-Layers/heaps/threads/lockedheap.h:47:7: required from ‘void HL::LockedHeap<LockType, Super>::free(void_) [with LockType = HL::SpinLockType; Super = Hoard::AlignedMmapInstance<65536ul>]’
include/util/exactlyoneheap.h:45:7: required from ‘void Hoard::ExactlyOneHeap::free(void_) [with Heap = HL::LockedHeap<HL::SpinLockType, Hoard::AlignedMmapInstance<65536ul> >]’
include/superblocks/addheaderheap.h:85:7: required from ‘void Hoard::AddHeaderHeap<SuperblockType, SuperblockSize, SuperHeap>::free(void_) [with SuperblockType = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::BigHeap>; long unsigned int SuperblockSize = 65536ul; SuperHeap = Hoard::MmapSource]’
Heap-Layers/heaps/threads/lockedheap.h:47:7: required from ‘void HL::LockedHeap<LockType, Super>::free(void_) [with LockType = HL::SpinLockType; Super = Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::BigHeap>, 65536ul, Hoard::MmapSource>]’
Heap-Layers/heaps/combining/hybridheap.h:69:2: required from ‘void HL::HybridHeap<BigSize, SmallHeap, BigHeap>::free(void_) [with int BigSize = 8192; SmallHeap = Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>; BigHeap = Hoard::BigHeap]’
include/superblocks/ignoreinvalidfree.h:49:2: required from ‘void Hoard::IgnoreInvalidFree::free(void_) [with SuperHeap = HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>, Hoard::BigHeap>]’
Heap-Layers/wrappers/ansiwrapper.h:54:8: required from ‘void HL::ANSIWrapper::free(void_) [with SuperHeap = Hoard::IgnoreInvalidFree<HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>, Hoard::BigHeap> >]’
include/superblocks/tlab.h:145:4: required from ‘void Hoard::ThreadLocalAllocationBuffer<NumBins, getSizeClass, getClassSize, LargestObject, LocalHeapThreshold, SuperblockType, SuperblockSize, ParentHeap>::clear() [with int NumBins = 11; unsigned int (_ getSizeClass)(size_t) = HL::bins<Header, 65536>::getSizeClass<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap> >; size_t (* getClassSize)(unsigned int) = HL::bins<Header, 65536>::getClassSize<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap> >; unsigned int LargestObject = 256u; unsigned int LocalHeapThreshold = 262144u; SuperblockType = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>; unsigned int SuperblockSize = 65536u; ParentHeap = Hoard::HoardHeapType]’
source/unixtls.cpp:172:15: required from here
Heap-Layers/utility/myhashmap.h:68:67: warning: conversion to ‘unsigned int’ from ‘long unsigned int’ may alter its value [-Wconversion]
Heap-Layers/utility/myhashmap.h: In instantiation of ‘void HL::MyHashMap<Key, Value, Allocator>::erase(Key) [with Key = void_; Value = long unsigned int; Allocator = Hoard::AlignedMmapInstance<65536ul>::SourceHeap]’:
include/util/alignedmmap.h:136:7: required from ‘void Hoard::AlignedMmapInstance<Alignment_>::free(void_) [with long unsigned int Alignment_ = 65536ul]’
Heap-Layers/heaps/threads/lockedheap.h:47:7: required from ‘void HL::LockedHeap<LockType, Super>::free(void_) [with LockType = HL::SpinLockType; Super = Hoard::AlignedMmapInstance<65536ul>]’
include/util/exactlyoneheap.h:45:7: required from ‘void Hoard::ExactlyOneHeap::free(void_) [with Heap = HL::LockedHeap<HL::SpinLockType, Hoard::AlignedMmapInstance<65536ul> >]’
include/superblocks/addheaderheap.h:85:7: required from ‘void Hoard::AddHeaderHeap<SuperblockType, SuperblockSize, SuperHeap>::free(void_) [with SuperblockType = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::BigHeap>; long unsigned int SuperblockSize = 65536ul; SuperHeap = Hoard::MmapSource]’
Heap-Layers/heaps/threads/lockedheap.h:47:7: required from ‘void HL::LockedHeap<LockType, Super>::free(void_) [with LockType = HL::SpinLockType; Super = Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::BigHeap>, 65536ul, Hoard::MmapSource>]’
Heap-Layers/heaps/combining/hybridheap.h:69:2: required from ‘void HL::HybridHeap<BigSize, SmallHeap, BigHeap>::free(void_) [with int BigSize = 8192; SmallHeap = Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>; BigHeap = Hoard::BigHeap]’
include/superblocks/ignoreinvalidfree.h:49:2: required from ‘void Hoard::IgnoreInvalidFree::free(void_) [with SuperHeap = HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>, Hoard::BigHeap>]’
Heap-Layers/wrappers/ansiwrapper.h:54:8: required from ‘void HL::ANSIWrapper::free(void_) [with SuperHeap = Hoard::IgnoreInvalidFree<HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>, Hoard::BigHeap> >]’
include/superblocks/tlab.h:145:4: required from ‘void Hoard::ThreadLocalAllocationBuffer<NumBins, getSizeClass, getClassSize, LargestObject, LocalHeapThreshold, SuperblockType, SuperblockSize, ParentHeap>::clear() [with int NumBins = 11; unsigned int (_ getSizeClass)(size_t) = HL::bins<Header, 65536>::getSizeClass<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap> >; size_t (* getClassSize)(unsigned int) = HL::bins<Header, 65536>::getClassSize<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap> >; unsigned int LargestObject = 256u; unsigned int LocalHeapThreshold = 262144u; SuperblockType = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>; unsigned int SuperblockSize = 65536u; ParentHeap = Hoard::HoardHeapType]’
source/unixtls.cpp:172:15: required from here
Heap-Layers/utility/myhashmap.h:81:67: warning: conversion to ‘unsigned int’ from ‘long unsigned int’ may alter its value [-Wconversion]
Heap-Layers/utility/myhashmap.h: In instantiation of ‘void HL::MyHashMap<Key, Value, Allocator>::insert(Key, Value) [with Key = void_; Value = long unsigned int; Allocator = Hoard::AlignedMmapInstance<65536ul>::SourceHeap]’:
Heap-Layers/utility/myhashmap.h:64:7: required from ‘void HL::MyHashMap<Key, Value, Allocator>::set(Key, Value) [with Key = void_; Value = long unsigned int; Allocator = Hoard::AlignedMmapInstance<65536ul>::SourceHeap]’
include/util/alignedmmap.h:73:2: required from ‘void* Hoard::AlignedMmapInstance<Alignment_>::malloc(size_t) [with long unsigned int Alignment_ = 65536ul; size_t = long unsigned int]’
Heap-Layers/heaps/threads/lockedheap.h:42:31: required from ‘void* HL::LockedHeap<LockType, Super>::malloc(size_t) [with LockType = HL::SpinLockType; Super = Hoard::AlignedMmapInstance<65536ul>; size_t = long unsigned int]’
include/util/exactlyoneheap.h:42:34: required from ‘void* Hoard::ExactlyOneHeap::malloc(size_t) [with Heap = HL::LockedHeap<HL::SpinLockType, Hoard::AlignedMmapInstance<65536ul> >; size_t = long unsigned int]’
include/superblocks/alignedsuperblockheap.h:54:70: required from ‘void* Hoard::SuperblockStore<SuperblockSize, TheLockType, MmapSource>::malloc(size_t) [with long unsigned int SuperblockSize = 65536ul; TheLockType = HL::SpinLockType; MmapSource = Hoard::MmapSource; size_t = long unsigned int]’
include/util/fixedrequestheap.h:43:44: required from ‘void* Hoard::FixedRequestHeap<RequestSize, SuperHeap>::malloc(size_t) [with long unsigned int RequestSize = 65536ul; SuperHeap = Hoard::SuperblockStore<65536ul, HL::SpinLockType, Hoard::MmapSource>; size_t = long unsigned int]’
Heap-Layers/heaps/threads/lockedheap.h:42:31: required from ‘void* HL::LockedHeap<LockType, Super>::malloc(size_t) [with LockType = HL::SpinLockType; Super = Hoard::FixedRequestHeap<65536ul, Hoard::SuperblockStore<65536ul, HL::SpinLockType, Hoard::MmapSource> >; size_t = long unsigned int]’
include/hoard/hoardmanager.h:354:49: required from ‘void* Hoard::HoardManager<SourceHeap, ParentHeap, SuperblockType_, EmptinessClasses, LockType, thresholdFunctionClass, HeapType>::getAnotherSuperblock(size_t) [with SourceHeap = Hoard::AlignedSuperblockHeap<HL::SpinLockType, 65536ul, Hoard::MmapSource>; ParentHeap = Hoard::GlobalHeap<65536ul, 8, Hoard::MmapSource, HL::SpinLockType>; SuperblockType_ = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>; int EmptinessClasses = 8; LockType = HL::SpinLockType; thresholdFunctionClass = Hoard::hoardThresholdFunctionClass; HeapType = Hoard::SmallHeap; size_t = long unsigned int]’
include/hoard/hoardmanager.h:298:4: required from ‘void* Hoard::HoardManager<SourceHeap, ParentHeap, SuperblockType_, EmptinessClasses, LockType, thresholdFunctionClass, HeapType>::slowPathMalloc(size_t) [with SourceHeap = Hoard::AlignedSuperblockHeap<HL::SpinLockType, 65536ul, Hoard::MmapSource>; ParentHeap = Hoard::GlobalHeap<65536ul, 8, Hoard::MmapSource, HL::SpinLockType>; SuperblockType_ = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>; int EmptinessClasses = 8; LockType = HL::SpinLockType; thresholdFunctionClass = Hoard::hoardThresholdFunctionClass; HeapType = Hoard::SmallHeap; size_t = long unsigned int]’
include/hoard/hoardmanager.h:93:2: required from ‘void* Hoard::HoardManager<SourceHeap, ParentHeap, SuperblockType_, EmptinessClasses, LockType, thresholdFunctionClass, HeapType>::malloc(size_t) [with SourceHeap = Hoard::AlignedSuperblockHeap<HL::SpinLockType, 65536ul, Hoard::MmapSource>; ParentHeap = Hoard::GlobalHeap<65536ul, 8, Hoard::MmapSource, HL::SpinLockType>; SuperblockType_ = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>; int EmptinessClasses = 8; LockType = HL::SpinLockType; thresholdFunctionClass = Hoard::hoardThresholdFunctionClass; HeapType = Hoard::SmallHeap; size_t = long unsigned int]’
include/hoard/hoardheap.h:171:31: required from here
Heap-Layers/utility/myhashmap.h:107:67: warning: conversion to ‘unsigned int’ from ‘long unsigned int’ may alter its value [-Wconversion]
Heap-Layers/wrappers/gnuwrapper.cpp: In function ‘void my_init_hook()’:
Heap-Layers/wrappers/gnuwrapper.cpp:79:23: warning: ‘__malloc_hook’ is deprecated (declared at /usr/include/malloc.h:176) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:80:21: warning: ‘__free_hook’ is deprecated (declared at /usr/include/malloc.h:173) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:81:24: warning: ‘__realloc_hook’ is deprecated (declared at /usr/include/malloc.h:179) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:82:25: warning: ‘__memalign_hook’ is deprecated (declared at /usr/include/malloc.h:183) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:85:5: warning: ‘__malloc_hook’ is deprecated (declared at /usr/include/malloc.h:176) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:86:5: warning: ‘__free_hook’ is deprecated (declared at /usr/include/malloc.h:173) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:87:5: warning: ‘__realloc_hook’ is deprecated (declared at /usr/include/malloc.h:179) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:88:5: warning: ‘__memalign_hook’ is deprecated (declared at /usr/include/malloc.h:183) [-Wdeprecated-declarations]
make: *** [linux-gcc-x86-64] Error 1

Fix SIGSEGV

3.12 is just failing when LD_PRELOADed on Ubuntu Linux 16.04 LTS — on literally any executables' run.

Is Hoard alive at all? I understand that tcmalloc outperforms it a bit (and is under active development for sure) but it'd be great to know the status for sure as well.

Nmake error

\Hoard\src>nmake

Microsoft (R) Program Maintenance Utility Version 14.00.24210.0
Copyright (C) Microsoft Corporation.  All rights reserved.

GNUmakefile Heap-Layers include Makefile README.md source test test.sh
This Makefile is for Windows only. For other systems, use gmake.
GNUmakefile Heap-Layers include Makefile README.md source test test.sh
        git submodule update --init --checkout --recursive
        cl /I. /Iinclude /Iinclude/util /Iinclude/hoard /Iinclude/superblocks /IHeap-Layers /D "NDEBUG" /D "_WINDOWS" /D "_WINDLL" /D "_WINRT_DLL" /D "_UNICODE" /D "UNICODE" /Zi /Ox /MD /nologo /W1 /WX- /Ox /Oi /Oy- /Gm- /EHsc /MD /GS /Gy /Zc:wchar_t /Zc:forScope /Gd /errorReport:queue "source\libhoard.cpp" "Heap-Layers\wrappers\winwrapper.cpp" "source\wintls.cpp" /GL /link /DLL /subsystem:console /OUT:libhoard.dll
libhoard.cpp
\Hoard\src\Heap-Layers\heaplayers.h(41): fatal error C1083: Cannot open include file: 'assert.h': No such file or directory
winwrapper.cpp
Heap-Layers\wrappers\winwrapper.cpp(38): fatal error C1083: Cannot open include file: 'windows.h': No such file or directory
wintls.cpp
source\wintls.cpp(37): fatal error C1083: Cannot open include file: 'new': No such file or directory
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\cl.EXE"' : return code '0x2'
Stop.

Not very sure where to go with this. Any help would be appreciated.
Another question while I'm at it how effective is hoard with games?

Unused pages are not released

Hello,

I am using your allocator in a memory intensive application. If the application allocates and deallocates many small objects, then not enough free space for a big memory block remains. It seems that the allocator doesn't free any unused page, but this should be part of the task of your blowup prevention.

Test system:
Windows 7 x64
Visual Studio 2005
32bit application
Hoard 3.8

test application:

#include "hoard-38/src/libhoard.cpp"

class CNode
{
public:
    double dummy;
    CNode *next;

    CNode(double v)
    {
        this->dummy = v;
        this->next = NULL;
    }
};

int main()
{
    // Init TLS, ...
    DllMain(NULL, DLL_PROCESS_ATTACH, NULL);

    int bigSize = 600000000;
    char *big;

    printf("allocate big memory block, use it, free it ... should be okay\n");
    big = new char[bigSize];
    memset(big, 0, bigSize);
    delete[] big;
    big = NULL;

    printf("allocate many small objects ... should be okay\n");
    // memory usage of the process increases
    int smallElementCount = 100000000;
    CNode head(0), *current;
    current = &head;

    for (int i = 1; i < smallElementCount; i++)
    {
        current->next = new CNode(i);
        current = current->next;
    }

    printf("free all small objects ... should be okay\n");
    // memory usage of the process does NOT decrease
    current = head.next;
    while(current)
    {
        CNode* t = current->next;
        delete current;
        current = t;
    }

    printf("allocate another big memory block, use it, free it ... should be CRASH (bad_alloc)!!!\n"); 
    big = new char[bigSize];
    memset(big, 0, bigSize);
    delete[] big;
    big = NULL;

    return 0;
}

Can you check this issue, please. This would be helpful.

Thanks
Bastian

Use -install_name on macOS

Your library is being built without the install_name being set, which means that it cannot be used unless the user manually fixes the install_name later using install_name_tool.

You have instructions about setting DYLD_INSERT_LIBRARIES to the path where the library is, which would be unnecessary if you set the install_name to the place where the library will be installed.

You could do this by adding -install_name $(PREFIX)/libhoard.dylib to MACOS_COMPILE and MACOS_COMPILE_DEBUG in GNUmakefile.

can not cross compile in arm

when i cross compilt it, it is error:
{standard input}: Assembler messages:
{standard input}:1118: Error: misaligned branch destination
{standard input}:1128: Error: misaligned branch destination
{standard input}:1132: Error: misaligned branch destination
{standard input}: Assembler messages:
{standard input}:356: Error: misaligned branch destination
{standard input}:366: Error: misaligned branch destination
{standard input}:370: Error: misaligned branch destination

any idea?

Runtime issues with Visual Studio 2015

I tried to build libhoard on windows with VS2015 using nmake. It built successfully. But when I try to run a sample program using hoard, It crashes in the following call stack :

ntdll.dll!RtlReportCriticalFailure()    Unknown
ntdll.dll!RtlpReportHeapFailure()   Unknown
ntdll.dll!RtlpHeapHandleError() Unknown
ntdll.dll!RtlpLogHeapFailure()  Unknown
ntdll.dll!NtdllpFreeStringRoutine() Unknown
ntdll.dll!RtlFreeUnicodeString()    Unknown
ntdll.dll!LdrpSnapThunk()   Unknown
ntdll.dll!LdrGetProcedureAddressEx()    Unknown
ntdll.dll!LdrGetProcedureAddress()  Unknown
KernelBase.dll!GetProcAddress() Unknown
libhoard.dll!PatchMe() Line 594 C++
libhoard.dll!DllMain(void * hinstDLL, unsigned long fdwReason, void * lpreserved) Line 122

It's a simple program to allocate memory :

int main()
{
    char *str = NULL;
    int rc = ERROR;

    printf("Allocating a buffer...\n");
    str = (char*)malloc(MAX_BUF_SIZE*sizeof(char));

    if (str == NULL)
    {
        printf("Allocation of buffer failed !!! Exiting... \n");
        return ERROR;
    }

    strncpy(str, "Hello", 6);
    printf("Entered string is : %s\n", str);
    return OK;
}

It's a visual studio project where I added source/uselibhoard.cpp as additional source file and libhoard.lib in the linker options. Please check the attached screenshots.

always_inline breaks on Windows

#54 seems to be breaking the Windows build as MSVC won't understand always_inline or gnuwrapper code.