GithubHelp home page GithubHelp logo

adoptium / bumblebench Goto Github PK

View Code? Open in Web Editor NEW
22.0 22.0 29.0 151 KB

A microbenchmarking test framework for Eclipse Adoptium

License: Apache License 2.0

Java 96.83% Shell 1.97% C++ 1.20%
benchmark java performance

bumblebench's Introduction

Eclipse Adoptium

This organization provides a home for Git repositories that contain the activities of the Adoptium Working Group, the Eclipse Adoptium Top Level Project and several Eclipse projects that fall under that top level project:

NOTE: The high-level project and issue tracking across all projects is kept in the Adoptium repo issue tracking system.


Please see the Eclipse Adoptium Project description for more information regarding the Adoptium top-level project or its sub-projects (visually depicted in the diagram below).

graph TD

subgraph Eclipse Adoptium
    classDef public fill:#CFE1F3,stroke:#333,stroke-width:4px,color:#000000
    classDef private fill:#FF0000,stroke:#333,stroke-width:4px,color:#000000
    style public fill:#CFE1F3,stroke:#333,stroke-width:4px,color:#000000
    style private fill:#FF0000,stroke:#333,stroke-width:4px,color:#000000
    subgraph Adoptium
        AdoptiumTrigger[adoptium]:::public --- website["adoptium.net"]:::public --- api["api.adoptium.net"]:::public --- blog["blog.adoptium.net"]:::public --- dash["dash.adoptium.net"]:::public
    end
    subgraph Temurin
        subgraph Build
            buildTrigger[temurin-build]:::public --- mirror["mirror-scripts"]:::public --- src["jdk, jdk8u, jdk8u-aarch32, jdk17u"]:::public --- release["github-release-scripts"]:::public --- binaries["temurin8-binaries,<br/>temurin11-binaries,<br/>temurin17-binaries,<br/>temurin19-binaries"]:::public --- installer["installer"]:::public --- build["build-jdk"]:::public
        end
        subgraph Infrastructure
            direction LR
            infraTrigger[infrastructure]:::public --- jenkins["ci-jenkins-pipelines"]:::public --- jenkinshelper["jenkins-helper"]:::public --- support["adoptium-support"]:::public
        end
    end
    subgraph Temurin Compliance
        TCKTrigger[temurin-compliance]:::private --- infra["infrastructure"]:::private --- jck8["JCK8-unzipped"]:::private --- jck11["JCK11-unzipped"]:::private --- jck17["JCK17-unzipped"]:::private --- jck19["JCK19-unzipped"]:::private
    end
    subgraph AQAvit
        AQAvitTrigger[aqa-tests]:::public --- tkg["TKG"]:::public --- test-tools["aqa-test-tools"]:::public --- stf["STF"]:::public --- systemtest["aqa-systemtest"]:::public --- bumblebench["bumblebench"]:::public --- run-aqa["run-aqa"]:::public
    end
    subgraph Incubator
        IncubatorTrigger[jdk11u-fast-startup-incubator]:::public
    end
end
Loading

Eclipse Adoptium Working Group

The Adoptium Working Group promotes and supports high-quality runtimes and associated technology for use across the Java ecosystem. Our vision is to meet the needs of Eclipse and the broader Java community by providing a marketplace for high-quality Java runtimes for Java-based applications. We embrace existing standards and a wide variety of hardware and cloud platforms.

Eclipse Adoptium Top Level Project

The mission of the Eclipse Adoptium Top-Level Project is to distribute high-quality runtimes and associated technology for use within the Java ecosystem. We achieve this through a set of Projects under the Adoptium Project Management Committee (PMC) and a close working partnership with external projects, most notably OpenJDK for providing the Java SE runtime implementation. Our goal is to meet the needs of both the Eclipse community and broader runtime users by providing a comprehensive set of technologies around runtimes for Java applications that operate alongside existing standards, infrastructures, and cloud platforms.

Eclipse AQAvit project

AQAvit is the quality and runtime branding evaluation project for Java SE runtimes and associated technology. During a release it takes a functionally complete Java runtime and ensures that all the additional qualities are present that make it suitable for production use. These quality criteria include good performance, exceptional security, resilience and endurance, and the ability to pass a wide variety of application test suites. In addition to verifying that functionally complete runtimes are release ready, the AQA tests may also serve to verify new functionality during runtime development.

Eclipse Temurin project

The Eclipse Temurin project provides code and processes that support the building of runtime binaries and associated technologies that are high performance, enterprise-caliber, cross-platform, open-source licensed, and Java SE TCK-tested for general use across the Java ecosystem.

Eclipse Temurin Compliance project

The Eclipse Temurin Compliance project is responsible for obtaining, managing, and executing the Oracle Java SE Compatibility Kit (JCK) on Eclipse Temurin binaries. The work is done on private infrastructure and using code managed in closed repositories only available to committers of Temurin Compliance. The public artefacts produced by this project are limited to an indication of whether a particular Eclipse Temurin binary is Java SE compliant or not.

Eclipse Mission Control project

Eclipse Mission Control enables you to monitor and manage Java applications without introducing the performance overhead normally associated with these types of tools. It uses data collected for normal adaptive dynamic optimization of the Java Virtual Machine (JVM). Besides minimizing the performance overhead, this approach eliminates the problem of the observer effect, which occurs when monitoring tools alter the execution characteristics of the system.

bumblebench's People

Contributors

babsingh avatar davehobbs avatar dhong44 avatar gdams avatar karianna avatar nigelyiboyu avatar piyush286 avatar r30shah avatar rmnattas avatar smlambert avatar tellison avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

bumblebench's Issues

InstantiationException while Running KmeansBench GPULambda

Problem Description

Running KmeansBench GPULambda gives java.lang.InstantiationException as shown below. I've saw this issue while using Java 8 OpenJ9 and Java 11 HotSpot but it should happen on all JDKs.

I'm adding this test: bumbleBench-KmeansBench-GPULambda to Adopt Test repo in this playlist for BumbleBench: https://github.com/AdoptOpenJDK/openjdk-tests/blob/master/perf/bumbleBench/playlist.xml.

Relevant Console Output with Error

18:07:39  ===============================================
18:07:39  Running test bumbleBench-KmeansBench-GPULambda_0 ...
18:07:39  ===============================================
18:07:39  bumbleBench-KmeansBench-GPULambda_0 Start Time: Mon Oct 21 22:07:39 2019 Epoch Time (ms): 1571695659641
18:07:39  "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -Xshareclasses:destroyAll; "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -Xshareclasses:groupAccess,destroyAll; echo "cache cleanup done";
18:07:39  JVMSHRC005I No shared class caches available
18:07:39  JVMSHRC005I No shared class caches available
18:07:39  cache cleanup done
18:07:39  variation: NoOptions
18:07:39  JVM_OPTIONS: -Xcompressedrefs 
18:07:39  { itercnt=1; \
18:07:39  mkdir -p "/java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../../TestConfig/test_output_15716952597279/bumbleBench-KmeansBench-GPULambda_0"; \
18:07:39  cd "/java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../../TestConfig/test_output_15716952597279/bumbleBench-KmeansBench-GPULambda_0"; \
18:07:39  "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -jar /java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../../../jvmtest/perf/bumbleBench/bumblebench/BumbleBench.jar KmeansBench.GPULambda; \
18:07:39  	if [ $? -eq 0 ] ; then echo ""; echo "bumbleBench-KmeansBench-GPULambda_0""_PASSED"; echo ""; cd /java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../..;  else echo ""; echo "bumbleBench-KmeansBench-GPULambda_0""_FAILED"; echo ""; fi; } 2>&1 | tee -a "/java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../../TestConfig/test_output_15716952597279/TestTargetResult";
18:07:40  Exception in thread "main" java.lang.InstantiationException: net.adoptopenjdk.bumblebench.gpu.KmeansBench$GPULambda
18:07:40  	at java.lang.J9VMInternals.newInstanceImpl(Native Method)
18:07:40  	at java.lang.Class.newInstance(Class.java:1848)
18:07:40  	at net.adoptopenjdk.bumblebench.core.Launcher.main(Launcher.java:55)
18:07:40  
18:07:40  bumbleBench-KmeansBench-GPULambda_0_FAILED
18:07:40  
18:07:40  "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -Xshareclasses:destroyAll; "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -Xshareclasses:groupAccess,destroyAll; echo "cache cleanup done";
18:07:40  
18:07:40  Attempting to destroy all caches in cacheDir /home/jbench/javasharedresources/
18:07:40  
18:07:40  JVMSHRC806I Compressed references persistent shared cache "sharedcc_jbench" has been destroyed. Use option -Xnocompressedrefs if you want to destroy a non-compressed references cache.
18:07:40  JVMSHRC005I No shared class caches available
18:07:40  cache cleanup done
18:07:40  bumbleBench-KmeansBench-GPULambda_0 Finish Time: Mon Oct 21 22:07:39 2019 Epoch Time (ms): 1571695659956
18:07:40  
18:07:40  ===============================================

To Reproduce

Use the bumbleBench-KmeansBench-GPULambda target on TestKitGen

SSLSocketBench gives java.io.FileNotFoundException

Problem Description

Running SSLSocketBench gives java.io.FileNotFoundException as shown below. I believe Vlad worked on this benchmark since he has his username in the file path.

I'm trying to add BumbleBench targets for testing at Adopt as part of adoptium/aqa-tests#1379.

Error

13:47:11  java.io.FileNotFoundException: /home/vpapro/mySrvKeystore (No such file or directory)

Console Output

13:47:08  ===============================================
13:47:08  Running test bumbleBench-SSLSocketBench_0 ...
13:47:08  ===============================================
13:47:08  bumbleBench-SSLSocketBench_0 Start Time: Fri Nov  1 13:47:08 2019 Epoch Time (ms): 1572630428469
13:47:08  "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -Xshareclasses:destroyAll; "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -Xshareclasses:groupAccess,destroyAll; echo "cache cleanup done";
13:47:08  JVMSHRC005I No shared class caches available
13:47:08  JVMSHRC005I No shared class caches available
13:47:08  cache cleanup done
13:47:08  variation: NoOptions
13:47:08  JVM_OPTIONS: -Xcompressedrefs 
13:47:08  { itercnt=1; \
13:47:08  mkdir -p "/java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/../TestConfig/test_output_15726303782557/bumbleBench-SSLSocketBench_0"; \
13:47:08  cd "/java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/../TestConfig/test_output_15726303782557/bumbleBench-SSLSocketBench_0"; \
13:47:08  "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -jar /java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/../../jvmtest/perf/bumbleBench/bumblebench/BumbleBench.jar SSLSocketBench; \
13:47:08  	if [ $? -eq 0 ] ; then echo ""; echo "bumbleBench-SSLSocketBench_0""_PASSED"; echo ""; cd /java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/..;  else echo ""; echo "bumbleBench-SSLSocketBench_0""_FAILED"; echo ""; fi; } 2>&1 | tee -a "/java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/../TestConfig/test_output_15726303782557/TestTargetResult";
13:47:11  java.io.FileNotFoundException: /home/vpapro/mySrvKeystore (No such file or directory)
13:47:11  	at java.io.FileInputStream.open(FileInputStream.java:195)
13:47:11  	at java.io.FileInputStream.<init>(FileInputStream.java:138)
13:47:11  	at java.io.FileInputStream.<init>(FileInputStream.java:93)
13:47:11  	at net.adoptopenjdk.bumblebench.crypto.SSLSocketBench.<clinit>(SSLSocketBench.java:51)
13:47:11  	at java.lang.Class.forNameImpl(Native Method)
13:47:11  	at java.lang.Class.forName(Class.java:333)
13:47:11  	at net.adoptopenjdk.bumblebench.core.Launcher.loadTestClass(Launcher.java:104)
13:47:11  	at net.adoptopenjdk.bumblebench.core.Launcher.loadTestClass(Launcher.java:85)
13:47:11  	at net.adoptopenjdk.bumblebench.core.Launcher.main(Launcher.java:48)
13:47:11  
13:47:11  -= BumbleBench series 7 version 5.10 running SSLSocketBench  Fri Nov 01 13:47:08 EDT 2019 =-
13:47:11  
13:47:11                Target	Est	Uncert%	MaxPeak	Peak	Peak%	%paused
13:47:11  java.net.ConnectException: Connection refused (Connection refused)
13:47:11  	at java.net.PlainSocketImpl.socketConnect(Native Method)
13:47:11  	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
13:47:11  	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
13:47:11  	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
13:47:11  	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
13:47:11  	at java.net.Socket.connect(Socket.java:607)
13:47:11  	at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:666)
13:47:11  	at sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:173)
13:47:11  	at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
13:47:11  	at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
13:47:11  	at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
13:47:11  	at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
13:47:11  	at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
13:47:11  	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
13:47:11  	at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1162)
13:47:11  	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1056)
13:47:11  	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
13:47:11  	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1570)
13:47:11  	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
13:47:11  	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:268)
13:47:11  	at net.adoptopenjdk.bumblebench.crypto.SSLSocketBench.doBatch(SSLSocketBench.java:88)
13:47:11  	at net.adoptopenjdk.bumblebench.core.MicroBench.attempt(MicroBench.java:66)
13:47:11  	at net.adoptopenjdk.bumblebench.core.BumbleBench.runAttempt(BumbleBench.java:139)
13:47:11  	at net.adoptopenjdk.bumblebench.core.BumbleBench.run(BumbleBench.java:245)
13:47:11  	at net.adoptopenjdk.bumblebench.core.MicroBench.run(MicroBench.java:107)
13:47:11  	at net.adoptopenjdk.bumblebench.core.BumbleBench.bumbleMain(BumbleBench.java:287)
13:47:11  	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
13:47:11  	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
13:47:11  	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
13:47:11  	at java.lang.reflect.Method.invoke(Method.java:498)
13:47:11  	at net.adoptopenjdk.bumblebench.core.Launcher.runBumbleMainOn(Launcher.java:61)
13:47:11  	at net.adoptopenjdk.bumblebench.core.Launcher.main(Launcher.java:55)
13:47:11      0.2s:  >! 110	695.5	 40.0	110	110	470.0
...
...

To Reproduce

NoSuchMethodException: sun.misc.IOUtils.readAllBytes on java 8

Works with jdk 10 and jdk 14, but not java 8

java -jar BumbleBench.jar StringEqualsBench
Exception in thread "main" java.lang.ExceptionInInitializerError
Caused by: java.lang.RuntimeException: java.lang.NoSuchMethodException: sun.misc.IOUtils.readAllBytes(java.io.InputStream)
	at net.adoptopenjdk.bumblebench.core.Util.<clinit>(Util.java:44)
Caused by: java.lang.NoSuchMethodException: sun.misc.IOUtils.readAllBytes(java.io.InputStream)
	at java.lang.Class.getDeclaredMethod(Class.java:2130)
	at net.adoptopenjdk.bumblebench.core.Util.<clinit>(Util.java:36)

java version

openjdk version "1.8.0_222"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_222-b10)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.222-b10, mixed mode)

NullPointerException while Running Humble Benchmarks with JDK 11 Hotspot

Problem Description

Running Humble benchmarks with JDK 11 Hotspot gives NullPointerException as shown below. This benchmark works with JDK 8 Hotpot and JDK 11 and 8 OpenJ9.

I'm trying to add BumbleBench targets for testing at Adopt as part of adoptium/aqa-tests#1379.

Error

15:46:53  Exception in thread "main" java.lang.RuntimeException: java.lang.NullPointerException
15:46:53  	at net.adoptopenjdk.bumblebench.core.Util.freshlyLoadedClass(Util.java:185)
15:46:53  	at net.adoptopenjdk.bumblebench.core.Util.newInstanceOfPossiblyFreshlyLoadedClass(Util.java:221)
15:46:53  	at net.adoptopenjdk.bumblebench.core.Util.newInstanceOfFreshlyLoadedClass(Util.java:208)
15:46:53  	at net.adoptopenjdk.bumblebench.core.HumbleBench$DelayThread.<init>(HumbleBench.java:134)
15:46:53  	at net.adoptopenjdk.bumblebench.core.HumbleBench.<init>(HumbleBench.java:57)
15:46:53  	at net.adoptopenjdk.bumblebench.humble.DistinctStringsEqualsBench.<init>(DistinctStringsEqualsBench.java:35)
15:46:53  	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
15:46:53  	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
15:46:53  	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
15:46:53  	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
15:46:53  	at java.base/java.lang.Class.newInstance(Class.java:584)
15:46:53  	at net.adoptopenjdk.bumblebench.core.Launcher.main(Launcher.java:55)
15:46:53  Caused by: java.lang.NullPointerException
15:46:53  	at java.base/java.lang.reflect.Method.invoke(Method.java:559)
15:46:53  	at net.adoptopenjdk.bumblebench.core.Util.readFully(Util.java:194)
15:46:53  	at net.adoptopenjdk.bumblebench.core.Util.freshlyLoadedClass(Util.java:182)
15:46:53  	... 11 more
15:46:53  
15:46:53  bumbleBench-DistinctStringsEqualsBench_0_FAILED

To Reproduce

  1. Remove the implementation constraint for the tests such as DistinctStringsEqualsBench (https://github.com/AdoptOpenJDK/openjdk-tests/blob/master/perf/bumbleBench/playlist.xml)
  2. Use the bumbleBench-DistinctStringsEqualsBench target on TestKitGen with the relevant SDK

BumbleBench doesn't build with JDK 12

Problem Description

I'm trying to add BumbleBench targets for testing at Adopt as part of adoptium/aqa-tests#1379. BumbleBench compiles and runs successfully with JDK 8 and JDK 11 for both OpenJ9 and HotSpot. It doesn't compile with JDK 12, and hence the targets are just limited to 8 and 11 in the BumbleBench playlist.xml file.

I believe we should see a similar issue with JDK 13 as well even though I haven't tested with it.

Error

16:42:31  compile-all:
16:42:31      [javac] Compiling 8 source files to /java/jenkins/workspace/Piyush-Grinder/openjdk-tests/perf/bumbleBench/bumblebench
16:42:31      [javac] warning: [options] bootstrap class path not set in conjunction with -source 6
16:42:31      [javac] error: Source option 6 is no longer supported. Use 7 or later.
16:42:31      [javac] error: Target option 6 is no longer supported. Use 7 or later.
16:42:31  
16:42:31  BUILD FAILED

Possible Solution

I believe, we'll need to update source and target for javac commands in https://github.com/AdoptOpenJDK/bumblebench/blob/master/build.xml.

To Reproduce

Credit to the original author?

Might I humbly suggest it would be appropriate to give some kind of credit/recognition to the author of Bumblebench (namely me) somewhere in the code or docs?

Missing package statement in net.adoptopenjdk.bumblebench.string.StringTestData

The net.adoptopenjdk.bumblebench.string.StringTestData class is missing a package statement:

package net.adoptopenjdk.bumblebench.string;

Which results in build errors when building the string test classes e.g:

[javac] symbol: variable StringTestData [javac] location: class StringConversionBench [javac] /bumblebench/net/adoptopenjdk/bumblebench/string/StringConversionBench.java:73: error: package StringTestData does not exist [javac] sb.append(StringTestData.POSSIBLE_CHARS[rand.nextInt(StringTestData.POSSIBLE_CHARS.length)]); [javac] ^

DigestBench and HMACBench taking too long to run

Problem Description

DigestBench and HMACBench seem to take forever to run. Benchmarks in BumbleBench usually take less than 2 mins. I had to abort DigestBench run after it kept running for 1 hr 45 min, generating an extremely large output (> 3 GB) as shown in the screenshots below. HMACBench behaves in a similar way.

I'm trying to add these BumbleBench targets for testing at Adopt as part of adoptium/aqa-tests#1379.

image

image

Console Output

13:55:45  ===============================================
13:55:45  Running test bumbleBench-DigestBench_0 ...
13:55:45  ===============================================
13:55:45  bumbleBench-DigestBench_0 Start Time: Fri Nov  1 13:55:43 2019 Epoch Time (ms): 1572630943888
13:55:45  "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -Xshareclasses:destroyAll; "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -Xshareclasses:groupAccess,destroyAll; echo "cache cleanup done";
13:55:45  JVMSHRC005I No shared class caches available
13:55:45  JVMSHRC005I No shared class caches available
13:55:45  cache cleanup done
13:55:45  variation: NoOptions
13:55:45  JVM_OPTIONS: -Xcompressedrefs 
13:55:45  { itercnt=1; \
13:55:45  mkdir -p "/java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/../TestConfig/test_output_15726308899881/bumbleBench-DigestBench_0"; \
13:55:45  cd "/java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/../TestConfig/test_output_15726308899881/bumbleBench-DigestBench_0"; \
13:55:45  "/java/jenkins/workspace/Piyush-Grinder/openjdkbinary/j2sdk-image/bin/java" -jar /java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/../../jvmtest/perf/bumbleBench/bumblebench/BumbleBench.jar DigestBench; \
13:55:45  	if [ $? -eq 0 ] ; then echo ""; echo "bumbleBench-DigestBench_0""_PASSED"; echo ""; cd /java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/..;  else echo ""; echo "bumbleBench-DigestBench_0""_FAILED"; echo ""; fi; } 2>&1 | tee -a "/java/jenkins/workspace/Piyush-Grinder/openjdk-tests/TestConfig/../TestConfig/test_output_15726308899881/TestTargetResult";
13:55:45  Using Provider SUN
13:55:45  Payload size: 4096 bytes
13:55:45  
13:55:45  -= BumbleBench series 7 version 5.10 running DigestBench  Fri Nov 01 13:55:44 EDT 2019 =-
13:55:45  
13:55:45                Target	Est	Uncert%	MaxPeak	Peak	Peak%	%paused
13:55:45      0.0s:  <  110	100	 12.0	-inf	-inf	--
13:55:45      0.0s:  <! 94	88	 14.4	-inf	-inf	--
13:55:45      0.0s:  <! 81.66	75.33	 17.3	-inf	-inf	--
13:55:45      0.0s:  <! 68.82	62.31	 20.7	-inf	-inf	--
13:55:45      0.0s:  <! 55.85	49.39	 24.9	-inf	-inf	--
13:55:45      0.0s:  <! 43.25	37.10	 29.9	-inf	-inf	--
13:55:45      0.0s:  <! 31.56	26.02	 35.8	-inf	-inf	--
13:55:45      0.0s:  <! 21.36	16.70	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 13.36	10.02	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 8.015	6.011	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 4.809	3.607	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 2.885	2.164	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 1.731	1.298	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 1.039	0.7791	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.6233	0.4674	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.3740	0.2805	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.2244	0.1683	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.1346	0.1010	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.08077	0.06058	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.04846	0.03635	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.02908	0.02181	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.01745	0.01309	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.01047	0.007851	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.006281	0.004711	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.003769	0.002826	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.002261	0.001696	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.001357	0.001018	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.0008140	0.0006105	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.0004884	0.0003663	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.0002930	0.0002198	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.0001758	0.0001319	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.0001055	0.00007912	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.00006330	0.00004747	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.00003798	0.00002848	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.00002279	0.00001709	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.00001367	0.00001025	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.000008203	0.000006153	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.000004922	0.000003692	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.000002953	0.000002215	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.000001772	0.000001329	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 0.000001063	797.4n	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 637.9n	478.4n	 40.0	-inf	-inf	--
13:55:45      0.0s:  <! 382.7n	287.1n	 40.0	-inf	-inf	--
...
...

To Reproduce

sun.misc.IOUtils.readFully throws exception due to negative length argument

When trying to run a HumbleBench using the latest adoptopenjdk openjdk8 nightly builds, I got the following exception:

Exception in thread "main" java.lang.AssertionError: java.lang.reflect.InvocationTargetException
	at net.adoptopenjdk.bumblebench.core.Util.readFully(Util.java:201)
	at net.adoptopenjdk.bumblebench.core.Util.freshlyLoadedClass(Util.java:184)
	at net.adoptopenjdk.bumblebench.core.Util.newInstanceOfPossiblyFreshlyLoadedClass(Util.java:223)
	at net.adoptopenjdk.bumblebench.core.Util.newInstanceOfFreshlyLoadedClass(Util.java:210)
	at net.adoptopenjdk.bumblebench.core.HumbleBench$DelayThread.<init>(HumbleBench.java:134)
	at net.adoptopenjdk.bumblebench.core.HumbleBench.<init>(HumbleBench.java:57)
	at CxfMessageHumbleBench.<init>(CxfMessageHumbleBench.java:7)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at java.lang.Class.newInstance(Class.java:442)
	at net.adoptopenjdk.bumblebench.core.Launcher.main(Launcher.java:55)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at net.adoptopenjdk.bumblebench.core.Util.readFully(Util.java:194)
	... 12 more
Caused by: java.io.IOException: length cannot be negative: -1
	at sun.misc.IOUtils.readFully(IOUtils.java:305)
	... 17 more

I think the problem is here: https://github.com/AdoptOpenJDK/bumblebench/blob/master/net/adoptopenjdk/bumblebench/core/Util.java#L194

On that line, method readFully is called with a length of -1.

I also found this PR in openj9 where a similar issue was dealt with: eclipse-openj9/openj9#8312

It looks like the fix would either be to pass Integer.MAX_VALUE to readFully or use the new IOUtils.readAllBytes method.

NullPointerException while Running DispatchBench.InnerClasses with JDK 11 Hotspot

Problem Description

Running bumbleBench-DispatchBench-InnerClasses with JDK 11 Hotspot gives NullPointerException as shown below. This benchmark works with JDK 8 Hotpot and JDK 11 and 8 OpenJ9.

I'm trying to add BumbleBench targets for testing at Adopt as part of adoptium/aqa-tests#1379.

Error

18:12:19  ===============================================
18:12:19  Running test bumbleBench-DispatchBench-InnerClasses_0 ...
18:12:19  ===============================================
18:12:19  bumbleBench-DispatchBench-InnerClasses_0 Start Time: Fri Oct 11 22:12:19 2019 Epoch Time (ms): 1570831939782
18:12:19  Nothing to be done for setup.
18:12:19  variation: NoOptions
18:12:19  JVM_OPTIONS:  
18:12:19  { itercnt=1; \
18:12:19  mkdir -p "/home/jenkins/workspace/Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../../TestConfig/test_output_157083193966/bumbleBench-DispatchBench-InnerClasses_0"; \
18:12:19  cd "/home/jenkins/workspace/Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../../TestConfig/test_output_157083193966/bumbleBench-DispatchBench-InnerClasses_0"; \
18:12:19  "/home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image/bin/java" -jar /home/jenkins/workspace/Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../../../jvmtest/perf/bumbleBench/bumblebench/BumbleBench.jar DispatchBench.InnerClasses; \
18:12:19  	if [ $? -eq 0 ] ; then echo ""; echo "bumbleBench-DispatchBench-InnerClasses_0""_PASSED"; echo ""; cd /home/jenkins/workspace/Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../..;  else echo ""; echo "bumbleBench-DispatchBench-InnerClasses_0""_FAILED"; echo ""; fi; } 2>&1 | tee -a "/home/jenkins/workspace/Grinder/openjdk-tests/TestConfig/scripts/testKitGen/../../../TestConfig/test_output_157083193966/TestTargetResult";
18:12:20  Exception in thread "main" java.lang.RuntimeException: java.lang.NullPointerException
18:12:20  	at net.adoptopenjdk.bumblebench.core.Util.freshlyLoadedClass(Util.java:187)
18:12:20  	at net.adoptopenjdk.bumblebench.core.Util.newInstanceOfPossiblyFreshlyLoadedClass(Util.java:223)
18:12:20  	at net.adoptopenjdk.bumblebench.lambda.DispatchBench$InnerClasses.getCallee(DispatchBench.java:126)
18:12:20  	at net.adoptopenjdk.bumblebench.lambda.DispatchBench.computeCallees(DispatchBench.java:39)
18:12:20  	at net.adoptopenjdk.bumblebench.lambda.DispatchBench.<init>(DispatchBench.java:34)
18:12:20  	at net.adoptopenjdk.bumblebench.lambda.DispatchBench$InnerClasses.<init>(DispatchBench.java:124)
18:12:20  	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
18:12:20  	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
18:12:20  	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
18:12:20  	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
18:12:20  	at java.base/java.lang.Class.newInstance(Class.java:584)
18:12:20  	at net.adoptopenjdk.bumblebench.core.Launcher.main(Launcher.java:55)
18:12:20  Caused by: java.lang.NullPointerException
18:12:20  	at java.base/java.lang.reflect.Method.invoke(Method.java:559)
18:12:20  	at net.adoptopenjdk.bumblebench.core.Util.readFully(Util.java:196)
18:12:20  	at net.adoptopenjdk.bumblebench.core.Util.freshlyLoadedClass(Util.java:184)
18:12:20  	... 11 more
18:12:20  
18:12:20  bumbleBench-DispatchBench-InnerClasses_0_FAILED
18:12:20  
18:12:20  Nothing to be done for teardown.
18:12:20  bumbleBench-DispatchBench-InnerClasses_0 Finish Time: Fri Oct 11 22:12:20 2019 Epoch Time (ms): 1570831940345
18:12:20  
18:12:20  ===============================================

To Reproduce

  1. Remove the implementation constraint for this test: bumbleBench-DispatchBench-InnerClasses (https://github.com/AdoptOpenJDK/openjdk-tests/blob/master/perf/bumbleBench/playlist.xml)
  2. Use the bumbleBench-DispatchBench-InnerClasses target on TestKitGen with the relevant SDK

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.