peter-lawrey / java-thread-affinity Goto Github PK

View Code? Open in Web Editor NEW

381.0 53.0 81.0 402 KB

Control thread affinity for Java

C 8.53% Java 88.94% Makefile 2.53%

java-thread-affinity's Introduction

This is an old version of this library kept here for historical purposes. The latest version is available here

https://github.com/OpenHFT/Java-Thread-Affinity

----

[ Project Development Summary ]

https://www.ohloh.net/p/Java-Thread-Affinity

[ Version History ]
Verions 3.x - moved to https://github.com/OpenHFT/Java-Thread-Affinity

Version 1.6.3 - Added AffinityLock.isAllocated() and isBound() so you can check whether a CPU could be allocated.

Version 1.6.2 - Initial implementation for Mac OS for its thread policies. This is not the same as affinity but may be equivalent.

Version 1.6.1 - Improved behaviour on machines without thread affinity support esp. Macs.

Version 1.6 - First formal release available under https://github.com/peter-lawrey/Java-Thread-Affinity/tree/master/repository

Version 1.5.5 - Added native support for Linux/amd64, thank you akerbos.

Version 1.5.4 - Added native support extracted from JAR and native build for Linux/i386, thank you isopov.

Version 1.5.3 - Build on Windows cleanly. Added a Unix profile for Unix enhanced build.

Version 1.5.2 - Added Windows support and Javadoc for all public methods.

Version 1.5.1 - Add changes to support i386 and Intel i3
#9
#10

Version 1.5 - Add support for efficient pause()ing, whileEqual and whileLessThan with limited busy waiting.

Version 1.4.1 - Add an AffinityThreadFactory to support ExecutorService

Version 1.4 - Support binding of a whole core for hyper-threaded systems. AffinityLock.acquireCore()

Version 1.3 - Support thread layout strategies for using the same/different socket or cores.

Version 1.2.2 - Improve logging and handling of edge cases

Version 1.2.1 - Fix a critical bug and added a unit test.

Version 1.2 - Restructure to separate Thread Affinity and nanoTime() functionality.

Verison 1.1 - Add JNA implementation

Verison 1.0 - JNI implementation

[ Contributers ]

peter.lawrey - Lead Developer.

java-thread-affinity's People

Contributors

Stargazers

Watchers

java-thread-affinity's Issues

Please push to maven central

I can help with this if you like.

rdtsc versus System.currentTimeMillis() versus System.nanoTime()

Hi Peter,

I have a call to System.currentTimeMillis() on my high-performance selector. I haven't measured how much latency it introduces but I was thinking about substituting it for your rdtsc implementation. Have you measured how much faster is rdtsc when compared to System.currentTimeMillis() and System.nanoTime() ?

Thanks!

-Sergio

Mispelled error message for missing affinity

Cannot run affinity test as no threads gave been reserved.
Use isocpus= in grub.conf or use -Daffinity.reserved={hex mask}

Note the missing 'l' from isolcpus.

Are strategies not effective?

I do not quite see through the code in this regard, so I might be wrong: are the strategies obeyed with respect to all already reserved cpus or only to the one last reserved? The latter is definitely not sufficient, therefor I ask.

acquireLock not going through all cores

It seems that the following code in AffinityLock:

private static AffinityLock acquireLock(boolean bind, int cpuId, AffinityStrategy... strategies) {
    synchronized (AffinityLock.class) {
        for (AffinityStrategy strategy : strategies) {
            for (int i = PROCESSORS - 1; i < 0; i--) {
                AffinityLock al = LOCKS[i];
                if (al.canReserve() && strategy.matches(cpuId, i)) {
                    al.assignCurrentThread(bind, false);
                    return al;
                }
            }
        }
    }

should be changed to

private static AffinityLock acquireLock(boolean bind, int cpuId, AffinityStrategy... strategies) {
    synchronized (AffinityLock.class) {
        for (AffinityStrategy strategy : strategies) {
            for (int i = PROCESSORS - 1; i >= 0; i--) {
                AffinityLock al = LOCKS[i];
                if (al.canReserve() && strategy.matches(cpuId, i)) {
                    al.assignCurrentThread(bind, false);
                    return al;
                }
            }
        }
    }

Am I right?

CPU 0 should be used

Currently, the cpu with id 0 is not used with the rationale being that this one is often reserved for OS processes.

However, there is a basic scenario for which this assumption is void, namely if the whole Java process is restricted to some otherweise unused cores. Java will only see those it can use, so the library prevents me from using a perfectly usable CPU.

The easiest solution is, of course, to just use CPU 0. It is the last one considered, anyway, so I think this should not cause problems in most situations.

Affinity.getCpu() doesn't work

Code part

package net.openhft.affinity;
import java.util.BitSet;

import static net.openhft.affinity.AffinityStrategies.*;
//import static net.openhft.affinity.Affinity.*;
/**
 * @author peter.lawrey
 */
public final class AffinityLockBindMainTest {
    private AffinityLockBindMainTest() {
        throw new InstantiationError( "Must not instantiate this class" );
    }

    public static void main(String... args) throws InterruptedException {
        AffinityLock al = AffinityLock.acquireLock();

        int threadId = AffinitySupport.getThreadId();
        System.out.println("threadId of main "+ threadId);

        int cpuId = Affinity.getCpu();
        System.out.println("cpuId of main: " + cpuId);

        // current cpu bitmap
        BitSet currentAffinity = Affinity.getAffinity();
        StringBuilder s = new StringBuilder();
        for( int i = 0; i < currentAffinity.length();  i++ )
        {
            s.append( currentAffinity.get( i ) == true ? 1: 0 );
        }
        System.out.println( s );

        //Affinity.setAffinity( 1L << 5 ); // lock to CPU 5.

        try {
            // find a cpu on a different socket, otherwise a different core.
            AffinityLock readerLock = al.acquireLock(DIFFERENT_SOCKET, DIFFERENT_CORE);
            new Thread(new SleepRunnable(readerLock, false), "reader").start();

            // find a cpu on the same core, or the same socket, or any free cpu.
            //AffinityLock writerLock = readerLock.acquireLock(SAME_CORE, SAME_SOCKET, ANY);
            //new Thread(new SleepRunnable(writerLock, false), "writer").start();
            int cpuId_ = Affinity.getCpu();
            System.out.println("cpuId of thread: " + cpuId_);

            Thread.sleep(200);
        } finally {
            al.release();
        }

        // allocate a whole core to the engine so it doesn't have to compete for resources.
        //al = AffinityLock.acquireCore(false);
        //new Thread(new SleepRunnable(al, true), "engine").start();

        Thread.sleep(200);
        System.out.println("\nThe assignment of CPUs is\n" + AffinityLock.dumpLocks());
    }

    static class SleepRunnable implements Runnable {
        private final AffinityLock affinityLock;
        private final boolean wholeCore;

        SleepRunnable(AffinityLock affinityLock, boolean wholeCore) {
            this.affinityLock = affinityLock;
            this.wholeCore = wholeCore;
        }

        public void run() {
            affinityLock.bind(wholeCore);

            int threadId = AffinitySupport.getThreadId();
            System.out.println("threadId of one SleepRunnable "+ threadId);

            int cpuId = Affinity.getCpu();
            System.out.println("cpuId of thread: " + cpuId);

            // current cpu bitmap
            BitSet currentAffinity = Affinity.getAffinity();
            StringBuilder s = new StringBuilder();
            for( int i = 0; i < currentAffinity.length();  i++ )
            {
                s.append( currentAffinity.get( i ) == true ? 1: 0 );
            }
            System.out.println( s );

            try {
                Thread.sleep(1000);
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
            } finally {
                affinityLock.release();
            }
        }
    }
}

Output

[main] INFO net.openhft.affinity.AffinityLock - Assigning cpu 7 to Thread[main,5,main]
[reader] INFO net.openhft.affinity.AffinityLock - Assigning cpu 6 to Thread[reader,5,main]
threadId of main 9973760
cpuId of main: -1

cpuId of thread: -1
threadId of one SleepRunnable 14413824
cpuId of thread: -1

[main] INFO net.openhft.affinity.LockInventory - Releasing cpu 7 from Thread[main,5,main]

The assignment of CPUs is
0: CPU not available
1: Reserved for this application
2: Reserved for this application
3: Reserved for this application
4: Reserved for this application
5: Reserved for this application
6: Thread[reader,5,main] alive=true
7: Reserved for this application

[reader] INFO net.openhft.affinity.LockInventory - Releasing cpu 6 from Thread[reader,5,main]

Process finished with exit code 0

Thread Migration Behaviour

Hi!

i wanted to use thread affinity in order to force Java threads migrate actually migrate from one core to another.
Does this code capable of such thing? I mean when I acquire a lock for a thread, does it really make the tread to move (migrate) from one core to another? or when a thread have a lock, the migration will happen at the discretion of VM (migration will happen when VM decides to)?

I'm looking forward to your answer :)

Test and document how build this library on Solaris

I don't have a Solaris development environment.

Native library for 64bit Gnu/Linux

Please include libaffinity into the library for newer architectures.

[removed invalid link]

This one was compiled under:

> uname -a
Linux name 2.6.35-32-generic #66-Ubuntu SMP Mon Feb 13 21:04:32 UTC 2012 x86_64 GNU/Linux

Fix JNA errno support

The modern way to handle errno from JNA is to declare your native methods as "throws LastErrorException." Then you can access getErrorCode on a caught LEE. For example, https://github.com/ryanking/apache-cassandra/blob/trunk/src/java/org/apache/cassandra/utils/CLibrary.java (which has some instanceof dancing that you may not care about if you only support versions of JNA new enough to define LEE).

Calling errno directly is problematic (it's often a macro), and even Native.getLastError (JNA's old way to expose it semi-portably) doesn't always work (apparently it can itself trigger syscalls that reset errno...). LastErrorException does work reliably and is the recommended way to handle errno now.

Maven install -DskipTests failure on OSX

[INFO] --- maven-gpg-plugin:1.4:sign (sign-artifacts) @ affinity ---
GPG Passphrase: *
gpg: no default secret key: secret key not available
gpg: signing failed: secret key not available
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 42.395s
[INFO] Finished at: Tue Jul 02 23:50:40 SGT 2013
[INFO] Final Memory: 9M/81M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-gpg-plugin:1.4:sign (sign-artifacts) on project affinity: Exit code: 2 -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Core ids from cpuinfo are not in strict succession

I have two cores, In cpuinfo first have core id 0, and second have core id 2. Thats why the following assertion fails:

assert cpuDetails.size() == sockets * cores * threads; //cores here has value 3 instead of 2

How about using string "cpu cores : 2" to know number of cores?

Complete /proc/cpuinfo:

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 37
model name : Intel(R) Core(TM) i3 CPU 530 @ 2.93GHz
stepping : 2
cpu MHz : 1200.000
cache size : 4096 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 2
apicid : 0
initial apicid : 0
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx rdtscp lm constant_tsc arch_perfmon pebs bts xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm arat tpr_shadow vnmi flexpriority ept vpid
bogomips : 5866.15
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 37
model name : Intel(R) Core(TM) i3 CPU 530 @ 2.93GHz
stepping : 2
cpu MHz : 1200.000
cache size : 4096 KB
physical id : 0
siblings : 4
core id : 2
cpu cores : 2
apicid : 4
initial apicid : 4
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx rdtscp lm constant_tsc arch_perfmon pebs bts xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm arat tpr_shadow vnmi flexpriority ept vpid
bogomips : 5866.64
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

processor : 2
vendor_id : GenuineIntel
cpu family : 6
model : 37
model name : Intel(R) Core(TM) i3 CPU 530 @ 2.93GHz
stepping : 2
cpu MHz : 1200.000
cache size : 4096 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 2
apicid : 1
initial apicid : 1
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx rdtscp lm constant_tsc arch_perfmon pebs bts xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm arat tpr_shadow vnmi flexpriority ept vpid
bogomips : 5866.62
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

processor : 3
vendor_id : GenuineIntel
cpu family : 6
model : 37
model name : Intel(R) Core(TM) i3 CPU 530 @ 2.93GHz
stepping : 2
cpu MHz : 1200.000
cache size : 4096 KB
physical id : 0
siblings : 4
core id : 2
cpu cores : 2
apicid : 5
initial apicid : 5
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx rdtscp lm constant_tsc arch_perfmon pebs bts xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm arat tpr_shadow vnmi flexpriority ept vpid
bogomips : 5866.65
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

Linux Mint build question

I am trying to build the project in Linux Mint, but there are many errors. I'm new to JNA and maven, but so I'm hoping this is something that I am doing wrong. This is what I get from maven:

Results :

Tests in error: 
  dumpLocksI3(com.higherfrequencytrading.affinity.AffinityLockTest): 
  dumpLocksCoreDuo(com.higherfrequencytrading.affinity.AffinityLockTest): Could not initialize class com.higherfrequencytrading.affinity.AffinityLock
  assignReleaseThread(com.higherfrequencytrading.affinity.AffinityLockTest): Could not initialize class com.higherfrequencytrading.affinity.AffinityLock
  testIssue21(com.higherfrequencytrading.affinity.AffinityLockTest): Could not initialize class com.higherfrequencytrading.affinity.AffinityLock
  testIssue19(com.higherfrequencytrading.affinity.AffinityLockTest): Could not initialize class com.higherfrequencytrading.affinity.AffinityLock
  dumpLocksI7(com.higherfrequencytrading.affinity.AffinityLockTest): Could not initialize class com.higherfrequencytrading.affinity.AffinityLock
  com.higherfrequencytrading.affinity.impl.PosixJNAAffinityTest: Could not initialize class com.higherfrequencytrading.affinity.impl.PosixJNAAffinity

Tests run: 21, Failures: 0, Errors: 7, Skipped: 1

In the surefire reports for individual tests I get the following errors. JNA seems to be up to date on my machine, so is it possible I should be using an older version?

Thank you for your time.

-------------------------------------------------------------------------------
Test set: com.higherfrequencytrading.affinity.impl.PosixJNAAffinityTest
-------------------------------------------------------------------------------
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0 sec <<< FAILURE!
com.higherfrequencytrading.affinity.impl.PosixJNAAffinityTest  Time elapsed: 0.014 sec  <<< ERROR!
java.lang.NoClassDefFoundError: Could not initialize class com.higherfrequencytrading.affinity.impl.PosixJNAAffinity
    at com.higherfrequencytrading.affinity.impl.PosixJNAAffinityTest.checkJniLibraryPresent(PosixJNAAffinityTest.java:29)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
    at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
    at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:172)
    at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:104)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:70)

-------------------------------------------------------------------------------
Test set: com.higherfrequencytrading.affinity.AffinityLockTest
-------------------------------------------------------------------------------
Tests run: 6, Failures: 0, Errors: 6, Skipped: 0, Time elapsed: 0.053 sec <<< FAILURE!
dumpLocksI3(com.higherfrequencytrading.affinity.AffinityLockTest)  Time elapsed: 0.035 sec  <<< ERROR!
java.lang.Error: 

There is an incompatible JNA native library installed on this system.
To resolve this issue you may do one of the following:
 - remove or uninstall the offending library
 - set the system property jna.nosys=true
 - set jna.boot.library.path to include the path to the version of the 
   jnidispatch library included with the JNA jar file you are using

    at com.sun.jna.Native.<clinit>(Native.java:142)
    at com.higherfrequencytrading.affinity.impl.PosixJNAAffinity$CLibrary.<clinit>(PosixJNAAffinity.java:49)
    at com.higherfrequencytrading.affinity.impl.PosixJNAAffinity.getAffinity(PosixJNAAffinity.java:74)
    at com.higherfrequencytrading.affinity.impl.PosixJNAAffinity.<clinit>(PosixJNAAffinity.java:64)
    at com.higherfrequencytrading.affinity.AffinitySupport.<clinit>(AffinitySupport.java:45)
    at com.higherfrequencytrading.affinity.AffinityLock.<clinit>(AffinityLock.java:44)
    at com.higherfrequencytrading.affinity.AffinityLockTest.dumpLocksI3(AffinityLockTest.java:73)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
    at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
    at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:172)
    at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:104)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:70)
assignReleaseThread(com.higherfrequencytrading.affinity.AffinityLockTest)  Time elapsed: 0 sec  <<< ERROR!
java.lang.NoClassDefFoundError: Could not initialize class com.higherfrequencytrading.affinity.AffinityLock
    at com.higherfrequencytrading.affinity.AffinityLockTest.assignReleaseThread(AffinityLockTest.java:115)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
    at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
    at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:172)
    at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:104)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:70)
testIssue21(com.higherfrequencytrading.affinity.AffinityLockTest)  Time elapsed: 0.002 sec  <<< ERROR!
java.lang.NoClassDefFoundError: Could not initialize class com.higherfrequencytrading.affinity.AffinityLock
    at com.higherfrequencytrading.affinity.AffinityLockTest.testIssue21(AffinityLockTest.java:140)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
    at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
    at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:172)
    at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:104)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:70)
testIssue19(com.higherfrequencytrading.affinity.AffinityLockTest)  Time elapsed: 0 sec  <<< ERROR!
java.lang.NoClassDefFoundError: Could not initialize class com.higherfrequencytrading.affinity.AffinityLock
    at com.higherfrequencytrading.affinity.AffinityLockTest.testIssue19(AffinityLockTest.java:159)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
    at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
    at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:172)
    at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:104)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:70)
dumpLocksCoreDuo(com.higherfrequencytrading.affinity.AffinityLockTest)  Time elapsed: 0.003 sec  <<< ERROR!
java.lang.NoClassDefFoundError: Could not initialize class com.higherfrequencytrading.affinity.AffinityLock
    at com.higherfrequencytrading.affinity.AffinityLockTest.dumpLocksCoreDuo(AffinityLockTest.java:97)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
    at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
    at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:172)
    at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:104)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:70)
dumpLocksI7(com.higherfrequencytrading.affinity.AffinityLockTest)  Time elapsed: 0.01 sec  <<< ERROR!
java.lang.NoClassDefFoundError: Could not initialize class com.higherfrequencytrading.affinity.AffinityLock
    at com.higherfrequencytrading.affinity.AffinityLockTest.dumpLocksI7(AffinityLockTest.java:36)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
    at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
    at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:172)
    at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:104)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:70)

ArrayIndexOutOfBoundsException during

Under certain circumstances, AffinityStrategy#matches is called with -1 in one of the parameters. This causes cpuLayout.socketId to fail.

When this occurred, the native library was missing and I did not have ANY as last strategy. I do not know wether this is of import.

From looking in the code, it seems as if -1 is used to encode "no suitable core found". It is then used as "last assigned core" which is compared with all possible new cores, leading to the error.

Possible fix:

Always assign an arbitrary core if the last was -1
Do not store -1 as last but the last index greater 0 (if any)

core id repeated on Opteron 6100 processors

The Opteron 6100 series are physically two multi-core CPUs on the same socket and the "core id:" field in cpuinfo is reused between CPUs, resulting in VanillaCpuLayout.fromCpuInfo() incorrectly counting them as multiple thread units for the same core.

Contents of /proc/cpuinfo: https://gist.github.com/2209193

Test and document how build this library on Windows

I don't have a Window development environment.

getAffinity: incompatible types: java.util.BitSet cannot be converted to long

I try to set affinity to one thread, however,

If you want to get/set the affinity directly you can do

   long currentAffinity = AffinitySupport.getAffinity();
   AffinitySupport.setAffinity(1L << 5); // lock to CPU 5.

I get some errors about data type.

Error:(21, 52) java: incompatible types: java.util.BitSet cannot be converted to long

Error:(22, 17) java: no suitable method found for setAffinity(long)
    method net.openhft.affinity.Affinity.setAffinity(java.util.BitSet) is not applicable
      (argument mismatch; long cannot be converted to java.util.BitSet)
    method net.openhft.affinity.Affinity.setAffinity(int) is not applicable
      (argument mismatch; possible lossy conversion from long to int)

The code

package net.openhft.affinity;
import static net.openhft.affinity.AffinityStrategies.*;
//import static net.openhft.affinity.Affinity.*;
/**
 * @author peter.lawrey
 */
public final class AffinityLockBindMainTest {
    private AffinityLockBindMainTest() {
        throw new InstantiationError( "Must not instantiate this class" );
    }

    public static void main(String... args) throws InterruptedException {
        AffinityLock al = AffinityLock.acquireLock();

        int threadId = AffinitySupport.getThreadId();
        System.out.println("threadId of main "+ threadId);

        int cpuId = Affinity.getCpu();
        System.out.println("cpuId of main: " + cpuId);

        long currentAffinity = Affinity.getAffinity();
        Affinity.setAffinity(1L << 5); // lock to CPU 5.

        try {
            // find a cpu on a different socket, otherwise a different core.
            AffinityLock readerLock = al.acquireLock(DIFFERENT_SOCKET, DIFFERENT_CORE);
            new Thread(new SleepRunnable(readerLock, false), "reader").start();

            // find a cpu on the same core, or the same socket, or any free cpu.
            //AffinityLock writerLock = readerLock.acquireLock(SAME_CORE, SAME_SOCKET, ANY);
            //new Thread(new SleepRunnable(writerLock, false), "writer").start();

            Thread.sleep(200);
        } finally {
            al.release();
        }

        // allocate a whole core to the engine so it doesn't have to compete for resources.
        //al = AffinityLock.acquireCore(false);
        //new Thread(new SleepRunnable(al, true), "engine").start();

        Thread.sleep(200);
        System.out.println("\nThe assignment of CPUs is\n" + AffinityLock.dumpLocks());
    }

    static class SleepRunnable implements Runnable {
        private final AffinityLock affinityLock;
        private final boolean wholeCore;

        SleepRunnable(AffinityLock affinityLock, boolean wholeCore) {
            this.affinityLock = affinityLock;
            this.wholeCore = wholeCore;
        }

        public void run() {
            affinityLock.bind(wholeCore);

            int threadId = AffinitySupport.getThreadId();
            System.out.println("threadId of one SleepRunnable "+ threadId);

            try {
                Thread.sleep(1000);
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
            } finally {
                affinityLock.release();
            }
        }
    }
}

More info on lock status

Currenty there is little info about what's going on, apart from logging. That one is useful, but, yet again, we use log4j logging, so java.util.logging doesn't cope well.

Info that is currently logged but not accessible in interface:

cpus isolation info (are isolated cpus assignable, or all of them from 1 to N are just assumed to be)
binding info - success/failure, thread-to-cpu relation
binding status (I fugured out that AffinityLock instance fields may be used to deduce this info, but more direct way would be better)

Thank you.

setAffinity seems to not used

I've just checked AffinityLock code, and see, that AffinitySupport.setAffinity() actually not used when we're grab AffinityLock. Actually, acquire lock is as simple as assign it's assignedThread field. I can see no actual thread affinity change. Doesn't we need it?

Incorrect JNA implementation on WINDOWS

It looks like PosixJNAAffinity should have:

public static final CLibrary INSTANCE = (CLibrary) Native.loadLibrary((Platform.isWindows() ? "kernel32" : "c"), CLibrary.class);

public int GetProcessAffinityMask(final int pid, final PointerType lpProcessAffinityMask, final PointerType lpSystemAffinityMask) throws LastErrorException;

public int SetProcessAffinityMask(final int pid, final DWORD lpProcessAffinityMask) throws LastErrorException;

I can provide a working version of it, if you allow me.

André

pom.xml enchancements

Sources/resources encoding are not specified -- which is not portable, since it means "use system default" -> Added project.build.sourceEncoding=UTF-8
Added <license>, <scm>, <distributionManagement> sections
Added maven-antrun plugin to call chmod u+x Makefile
Added maven deployment plugins. For now you should be able to create all standard maven artifacts (jar/javadoc/sources) and install them into your own public available repository <svn-path>/repo by invoking 'mvn deploy'. I can't check it, though, since I haven't write access.

<?xml version="1.0" encoding="UTF-8"?>
<!--
  ~ Copyright 2011 Peter Lawrey
  ~
  ~ Licensed under the Apache License, Version 2.0 (the "License");
  ~ you may not use this file except in compliance with the License.
  ~ You may obtain a copy of the License at
  ~
  ~        http://www.apache.org/licenses/LICENSE-2.0
  ~
  ~ Unless required by applicable law or agreed to in writing, software
  ~ distributed under the License is distributed on an "AS IS" BASIS,
  ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  ~ See the License for the specific language governing permissions and
  ~ limitations under the License.
  -->

<project xmlns = "http://maven.apache.org/POM/4.0.0"
         xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation = "http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>vanilla.java</groupId>
    <artifactId>affinity</artifactId>
    <version>1.1</version>
    <packaging>jar</packaging>

    <licenses>
        <license>
            <name>Apache License Version 2.0</name>
            <url>http://www.apache.org/licenses/LICENSE-2.0</url>
            <comments>A business-friendly OSS license</comments>
            <distribution>repo</distribution>
        </license>
    </licenses>

    <scm>
        <connection>scm:svn:https://github.com/peter-lawrey/Java-Thread-Affinity.git/trunk</connection>
        <developerConnection>scm:svn:https://github.com/peter-lawrey/Java-Thread-Affinity.git/trunk</developerConnection>
        <url>https://github.com/peter-lawrey/Java-Thread-Affinity</url>
    </scm>

    <distributionManagement>
        <repository>
            <uniqueVersion>false</uniqueVersion>
            <id>github</id>
            <url>svn:https://github.com/peter-lawrey/Java-Thread-Affinity.git/repo/</url>
        </repository>
    </distributionManagement>

    <properties>
        <native.source.dir>src/main/c</native.source.dir>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.8.2</version>
            <scope>test</scope>
        </dependency>
        <!-- on linux install with sudo apt-get install libjna-java -->
        <dependency>
            <groupId>net.java.dev.jna</groupId>
            <artifactId>jna</artifactId>
            <version>3.4.0</version>
        </dependency>
    </dependencies>

    <build>
        <extensions>
            <extension>
                <groupId>org.jvnet.wagon-svn</groupId>
                <artifactId>wagon-svn</artifactId>
                <version>1.8</version>
            </extension>
        </extensions>
        <plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.3.2</version>
                <configuration>
                    <source>1.6</source>
                    <target>1.6</target>
                </configuration>
            </plugin>

            <plugin>
                <artifactId>maven-antrun-plugin</artifactId>
                <version>1.7</version>
                <executions>
                    <execution>
                        <id>build-native</id>
                        <phase>process-classes</phase>
                        <goals>
                            <goal>run</goal>
                        </goals>
                        <configuration>
                            <target>
                                <property name="makefile" value="${project.basedir}/${native.source.dir}/Makefile"/>
                                <!-- make it executable -->
                                <chmod file="${makefile}" perm="u+x"/>
                            </target>
                        </configuration>

                    </execution>
                </executions>
            </plugin>

            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>exec-maven-plugin</artifactId>
                <version>1.2.1</version>
                <executions>
                    <execution>
                        <!-- this execution happens just after compiling the java classes, and builds the native code. -->
                        <id>build-native</id>
                        <phase>process-classes</phase>
                        <goals>
                            <goal>exec</goal>
                        </goals>
                        <configuration>
                            <executable>src/main/c/Makefile</executable>
                            <workingDirectory>src/main/c</workingDirectory>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

            <plugin>
                <artifactId>maven-source-plugin</artifactId>
                <version>2.1.2</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>jar</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <artifactId>maven-javadoc-plugin</artifactId>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>javadoc</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

            <plugin>
                <artifactId>maven-jar-plugin</artifactId>
                <version>2.3.1</version>
                <configuration>
                    <archive>
                        <manifest>
                            <addDefaultImplementationEntries>true</addDefaultImplementationEntries>
                        </manifest>
                    </archive>
                </configuration>
            </plugin>

            <plugin>
                <groupId>org.apache.felix</groupId>
                <artifactId>maven-bundle-plugin</artifactId>
                <extensions>true</extensions>
                <configuration>
                    <instructions>
                        <Bundle-RequiredExecutionEnvironment>J2SE-1.5,JavaSE-1.6</Bundle-RequiredExecutionEnvironment>
                        <Bundle-SymbolicName>${project.groupId}.${project.artifactId}</Bundle-SymbolicName>
                        <Export-Package>${project.groupId}.*;version="${project.version}"</Export-Package>
                    </instructions>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

JNA implementation

Hi, can you build this without the native support or just ship a jar from a different build machine and then rely on JNA support ?

Thanks

java.lang.ArrayIndexOutOfBoundsException when AffinityLock.acquireLock()

when invoking AffinityLock.acquireLock(),the exception ArrayIndexOutOfBoundsException occurs.what's the problem?

Eliminate the need to manually deal with LD_library path

Please load native part of the library automatically from packaged in the jar file, with no need for user to do anything(e.g. as it is done in java-snappy: http://code.google.com/p/snappy-java/source/browse/src/main/java/org/xerial/snappy/SnappyLoader.java or sqlite-jdbc etc.)

Ideally - produce ready-to-use cross-platform jar with packaged .so from i386 and amd64, since native platform-specific part is only 7 KB.

INFO: Assigning cpu -1 to Thread

Hi!

I'm still not sure whether I understand the API correctly. But I just hit this:

Aug 08, 2012 4:21:23 PM vanilla.java.affinity.AffinityLock bind
INFO: Assigning cpu 6 to Thread[Renderer0,5,main]
...
Aug 08, 2012 4:21:23 PM vanilla.java.affinity.AffinityLock bind
INFO: Assigning cpu -1 to Thread[Renderer5,5,main]

What is cpu -1 supposed to tell me? The thread is running and doesn't show up in dumpLocks():

0: General use CPU
1: Thread[Renderer4,5,main] alive=true
2: Thread[Renderer3,5,main] alive=true
3: Reserved for this application
4: Thread[Renderer2,5,main] alive=true
5: Thread[Renderer1,5,main] alive=true
6: Thread[Renderer0,5,main] alive=true
7: Reserved for this application

I'd also like to suggest to rephrase "Assign a core(and all its cpus)" to "and all its virtual/logical cores)" or something like that.

Bug: Calling strategies in the wrong way

Disclaimer: I am not entirely positive that this is a bug as I inferred the interface of AffinityStrategy from the code that uses it.

There is at least one instance (AffinityLock#acquireCore) where AffinityStrategy#matches is called with a core id. The way the rest looks, strategies should only be past cpu ids.

-Daffinity.reserved does not work

Having not yet configured and reserved CPUs (and not really wanting to) I tried

mvn install -Daffinity.reserved=cc

and got the test failures and error message.

Include strategy: Same socket, different core

See the title; it is impossible to model this with existing strategies but I think it is quite important for machines with multiple sockets which have multiple cores each. Therefor I think it should be included in AffinityStrategies.

  SAME_SOCKET_DIFFERENT_CORE {
    public boolean matches(int cpuId, int cpuId2) {
      CpuLayout cpuLayout = AffinityLock.cpuLayout();
      return     cpuLayout.socketId(cpuId) == cpuLayout.socketId(cpuId2)
              && cpuLayout.coreId(cpuId) != cpuLayout.coreId(cpuId2);
    }
  }

Simple recipe for a simple 2-thread 2-processor scenario?

Hi Peter,

It would be nice to have some recipes on how to use your library. For example: I have two threads and two physical processors. I want to assign each one to its own physical processor. How do I do that? Now I want to make sure they always run in the same physical processor. How do I do that?

I want to test my 2-threaded (consumer-producer) queue under those two very distinct situations. The queue is here: http://mentaqueue.soliveirajr.com

It would be nice to have the "basic of the basic" as recipes in the wiki. If you show me how to do it I can place it in the wiki to help out. Or you can answer my question by updating the wiki.

Thanks!!!!

-Sergio

Thread affinity can fail on headless linux

Exception in thread "main" java.lang.UnsatisfiedLinkError: /java/jdk1.7.0_13_linux/jre/lib/i386/xawt/libmawt.so: libXtst.so.6: cannot open shared object file: No such file or directory
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1939)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1864)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1825)
at java.lang.Runtime.load0(Runtime.java:792)
at java.lang.System.load(System.java:1059)
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1939)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1864)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1846)
at java.lang.Runtime.loadLibrary0(Runtime.java:845)
at java.lang.System.loadLibrary(System.java:1084)
at sun.security.action.LoadLibraryAction.run(LoadLibraryAction.java:67)
at sun.security.action.LoadLibraryAction.run(LoadLibraryAction.java:47)
at java.security.AccessController.doPrivileged(Native Method)
at java.awt.Toolkit.loadLibraries(Toolkit.java:1648)
at java.awt.Toolkit.(Toolkit.java:1670)
at java.awt.Component.(Component.java:595)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:188)
at com.sun.jna.Platform.(Platform.java:64)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:188)
at com.higherfrequencytrading.affinity.AffinitySupport.isJNAAvailable(AffinitySupport.java:65)
at com.higherfrequencytrading.affinity.AffinitySupport.(AffinitySupport.java:45)

Build fails on OSX

Using mountain lion, testIssue21 fails.

Can supply any details you require or is this a known issue ?

Thanks

Only the first created thread from AffinityThreadFactory gets bound

Only the first thread created by AffinityThreadFactory gets bound to the lock.
For the threads created after the first one, lastAffinityLock.acquireLock(strategies) gets called which calls acquireLock with bind false. When one does a top -H, one can see that only the first thread created with affinityThreadFactory spins on the acquired logical core. The rest don't correspond to the dumpLocks.

//For the first thread this is called with bind true
  public static AffinityLock acquireLock() {
        return acquireLock(true);
    }
// For the rest this method is called with bind false
public AffinityLock acquireLock(AffinityStrategy... strategies) {
        return acquireLock(false, cpuId, strategies);
 }
private static AffinityLock acquireLock(boolean bind, int cpuId, AffinityStrategy... strategies) {
                for (int i = LOCKS.length - 1; i > 0; i--) {
                    AffinityLock al = LOCKS[i];
                    if (al.canReserve() && (cpuId < 0 || strategy.matches(cpuId, al.cpuId))) {
                        al.assignCurrentThread(bind, false);
                        return al;
                    }
                }
}


 private void assignCurrentThread(boolean bind, boolean wholeCore) {
        assignedThread = Thread.currentThread();
        if (bind) // for the threads created through AffinityThreadFactory bind is false and hence no affinity is set.
            bind(wholeCore);
    }

Improve JavaDoc Documentation

Core functionality is not documented sufficiently. In particular, I could answer the following questions only by looking into the code:

What is the exact influence the specified strategies and their order (!) have?
What happens if all strategies fail? Will the thread be prevented from running? Will it run anywhere?

Build failing if JNI lib missed

Currently build is failing on test phrase if JNI 'affinity' lib is not available. It is NativeAffinityTest which fails.

I suppose, such test is meaningless if NativeAffinity is not loaded. So, it should be changed like this:

@BeforeClass
public void checkJniLibraryPresent(){
Assume.assumeTrue( NativeAffinity.LOADED );
}

this will lead to ignoring this testcase in case where NativeAffinity is not available.

Improve behaviour of Mac

I don't have a Mac but everyone who has looked at this has come away with the conclusion that thread affinity is not available.

What the library should do is not fail in such an ugly manner i.e. if its not available just give an appropriate warning and move one.

On 8 March 2012 13:16, William Louth wrote:

figured out I was missing jna.jar but now have. Is there a system property
to resort to a different implementation on this platform?

Mar 8, 2012 2:12:27 PM vanilla.java.affinity.impl.PosixJNAAffinity
WARNING: Unable to load jna library java.lang.UnsatisfiedLinkError: Error
looking up function 'sched_getaffinity': dlsym(0x7fff66593680,
sched_getaffinity): symbol not found
Mar 8, 2012 2:12:27 PM vanilla.java.affinity.AffinitySupport
INFO: Using dummy affinity control implementation!
Mar 8, 2012 2:12:27 PM vanilla.java.affinity.AffinityLock acquireLock
WARNING: No reservable CPU for Thread[main,5,main]
Mar 8, 2012 2:12:27 PM vanilla.java.affinity.AffinityLock acquireLock
WARNING: No reservable CPU for Thread[main,5,main]
Exception in thread "Thread-2" java.lang.IllegalStateException: cpu -1
already bound to Thread[main,5,main]
at vanilla.java.affinity.AffinityLock.bind(AffinityLock.java:278)
at vanilla.java.affinity.AffinityLock.bind(AffinityLock.java:268)
at
vanilla.java.chronicle.impl.IndexedChronicleLatencyMain$1.run(IndexedChronicleLatencyMain.java:63)
at java.lang.Thread.run(Thread.java:680)
Mar 8, 2012 2:12:27 PM vanilla.java.affinity.AffinityLock bind
INFO: Assigning cpu -1 to Thread[main,5,main]

Affinity.getAffinity() returns empty BitSet (zero cores) on Linux with 4 cores.

We would like to use this library on our project so we tried first on windows platform with very nice success.
Then after moving the code to server platform (Red Hat Enterprise Linux Server release 5.11 ) by debugging I found the problem with getting affinity setup:
BitSet bs = Affinity.getAffinity();
In bs is all zero - like there is no processor/core. Even bs.cardinality(); is zero.

The same code running on Win7 with four cores returns exact number of cores (bs.cardinality() is 4)
What am I doing wrong with linux?

i386 support is broken

From the defines it seems that i386 is intended to be supported. But in Makefile you may find hardcoded:
JVM_SHARED_LIB := $(JAVA_HOME)/jre/lib/amd64/server
for i386 it sghould be
JVM_SHARED_LIB := $(JAVA_HOME)/jre/lib/i386/server

Also when compiling it give following warnings:

vanilla_java_affinity_impl_NativeAffinity.c: In function ‘Java_vanilla_java_busywaiting_impl_JNIBusyWaiting_whileEqual0’:
vanilla_java_affinity_impl_NativeAffinity.c:118: warning: cast to pointer from integer of different size
vanilla_java_affinity_impl_NativeAffinity.c: In function ‘Java_vanilla_java_busywaiting_impl_JNIBusyWaiting_whileLessThan0’:
vanilla_java_affinity_impl_NativeAffinity.c:136: warning: cast to pointer from integer of different size

Also (don't know whether it is caused by i386):

Failed tests:
dumpLocks(vanilla.java.affinity.AffinityLockTest)

Tests in error:
assignReleaseThread(vanilla.java.affinity.AffinityLockTest)

JNA dependency in pom is not portable

JNA dependency reference in pom.xml is not portable, since it use "system" scope, which is not portable.

Solution: use com.sun.jna:jna:3.0.9 (older version, which can be resolved through mavencentral), or net.java.dev.jna:jna:3.4.0 (seems like it is newer one).

Different library behaviour when running in cpu shield and when not

I am sorry if my question is more like forum question then bug issue, but I dont know where to write it. I am being slightly confused when i try to bind two threads on the same socket with java affinity library. I have cset shield installed on 2 and 3 processors( i have i5, SUSE 11.1 + RT extensions test system, in production it will be i7), so 0 and 1 are free, When I run my java code not from cpu shield I have this output :

INFO: cpu 2 base= false reservable= true
INFO: cpu 3 base= false reservable= true
It seems ok, but I need to run my program in cpu shield to achieve more determinism and etc. When run it inside shield I have completely opposite output like :
INFO: cpu 2 base= true reservable= false
INFO: cpu 3 base= true reservable= false , which is pretty confusing and I don't know if it is bug or my misbehaviour.
I need to bind threads to exact processors and then bind solarflace NIC interrupt affinity to achive lowest possible network latency and also stay in the shield.

Proprietary API

The compiler/maven kindly informs me:

[WARNING] /.../peter-lawrey-Java-Thread-Affinity-2eb1744/src/main/java/vanilla/java/busywaiting/impl/JavaBusyWaiting.java:19: warning: sun.misc.Unsafe is Sun proprietary API and may be removed in a future release
[WARNING] import sun.misc.Unsafe;
[WARNING] ^
[WARNING] /.../peter-lawrey-Java-Thread-Affinity-2eb1744/src/main/java/vanilla/java/busywaiting/impl/JavaBusyWaiting.java:30: warning: sun.misc.Unsafe is Sun proprietary API and may be removed in a future release
[WARNING] static final Unsafe UNSAFE;
[WARNING] ^

In my experience, such API vanishes already in Java 7 (if for no other reason than that stuff won't be named after Sun anymore) so you should avoid using it.

Error received when running test code and assistance for setting up the library

Hi there,

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

This shows up just before the printing of 'Hello World!', is this normal? I downloaded the 2.1.1 jar affinity and enclosed it in my program.
Also if you can, could you expand on how you assign a thread to a core?
Thanks very much
TS

peter-lawrey / java-thread-affinity Goto Github PK

java-thread-affinity's Introduction

java-thread-affinity's People

Contributors

Stargazers

Watchers

Forkers

java-thread-affinity's Issues

Code part

Output

Recommend Projects

Recommend Topics

Recommend Org

Jobs