quick-perf / quickperf

QuickPerf is a testing library for Java to quickly evaluate and improve some performance-related properties

QuickPerf is a testing library for Java to quickly evaluate and improve some performance-related properties

---

            quickperf.io

---

📙 Documentation

Annotations

👉 Core

👉 JVM

👉 SQL

more...

Frameworks and Test Frameworks

👉 Spring

more...

👉 JUnit 4

👉 JUnit 5

👉 TestNG

more...

Frequently Asked Questions

more...

Usage

JVM annotations

    @MeasureHeapAllocation
    @HeapSize(value = 1, unit = AllocationUnit.GIGA_BYTE)
    @Test
    public void execute_batch() {
        ...
    }

📙 All the JVM annotations    🔎 Examples with JUnit4, Junit5, TestNG    🔎 Heap allocation of Apache Maven

SQL annotations

    @ExpectSelect(1)
    @Test
    public void should_find_all_players() {
     ...
    }

[PERF] You may think that <1> select statement was sent to the database
       But there are in fact <10>...

💣 You may have even more select statements with production data.
Be careful with the cost of JDBC roundtrips: https://blog.jooq.org/2017/12/18/the-cost-of-jdbc-server-roundtrips/

Auto-detection of Hibernate and Spring Data JPA:

💡 Perhaps you are facing an N+1 select issue
	* With Hibernate, you may fix it by using JOIN FETCH
	                                       or LEFT JOIN FETCH
	                                       or FetchType.LAZY
	                                       or ...

	* With Spring Data JPA, you may fix it by adding @EntityGraph(attributePaths = { "..." })
      	  on repository method: https://docs.spring.io/spring-data/jpa/docs/current/reference/ht

📙 All the SQL annotations    🔎 Spring Boot & JUnit 4    🔎 Spring Boot & JUnit 5

Talks and videos

English

French

Something to ask us?

📧 [email protected]

💬 Want to chat with us? Join us on gitter

Do you prefer to use a Github issue to ask a question? Create a question issue

Show your support

Please ⭐ this repository or if this project helped you!

Contributing

You are very welcome to contribute to QuickPerf! You can contribute in many ways. Some relatively easy things can be done. Other issues are more challenging. Each contribution is appreciated. Read our contributing guide to learn more.

Contributors

Many thanks to all our contributors!

Jean Bisutti 💻 ⚠ 📖 🎨 💡 👀 📢	Guillaume Raghoumandan 💻 ⚠	Patrice Cavezzan 💻 🚇 📖	Alexandre Blanchard 🐛 💻	Eric McDowell 💻	Jan Krüger 💻	Loïc Mathieu 💻 💡 📖
Daniel Bentley 🚇	Gaurav Deshpande ⚠	rdm100 📖	Artus de Benque 🐛 💻	Minh-Trieu Ha 💻	Bakary Djiba 💻	C Faisal 💻
Thami Inaflas 💻	José Paumard 💻	Edward Rose 💻	Ubaid ur Rehman 💻	Giuseppe B. 💻	Fabrice 💻 📖	Navneet Kumar 💻
Charles Sabourdin 📖	Mohamed Karaga 🐛 💻	Hervé Boutemy 📦	Franck Demeyer 🐛 💻	Oliver Hughes 💻	Hedley Proctor 💻	rsassi 🐛 💻

emoji key

Sponsors

Many thanks Zenika for sponsoring this project!

License

Apache License 2.0

quickperf's Issues

Pretty display of JVM data

Description

@ProfileJvm displays some JVM data in standard output.

In some case, the display is not pretty. Example:

 ALLOCATION (estimations)  |   GARBAGE COLLECTION           |  THROWABLE
 Total:        3,7 GiB    |   Total pause: 1,087 s      | Exception: 0
 Inside TLAB:  3,69 GiB    |   Longest GC pause: 160,438 ms | Error: 36
 Outside TLAB: 10,3 MiB    |                                | Throwable: 36

Some vertical bars are not aligned.

Implementation help

DisplayJvmProfilingValueVerifier class has to be updated.
The text is contained in text local variable.
Write code to automatically align the vertical bars.
Test by adding @ProfileJvm on code_with_scores_of_jmc_rules_greater_than_50 test method contained in JmcJUnit4Tests class (don't commit this modification) or write an object aligning the vertical bars and unit test this object.

Make @ProfileJvm display the young GC count

With @ProfileJvm annotation, the JVM is profiled with the JDK Flight Recorder, an event recorder built into the JVM.

With this annotation, some profiling data are displayed in the standard output.

For example:

 ALLOCATION (estimations)  |   GARBAGE COLLECTION           |  THROWABLE
 Total:        3,7 GiB     |   Total pause: 1,087 s         | Exception: 0
 Inside TLAB:  3,69 GiB    |   Longest GC pause: 160,438 ms | Error: 36
 Outside TLAB: 10,3 MiB    |                                | Throwable: 36

The aim of this issue is to also display the number of young collections.

With two young collections, we expect to display Young: 2 below Longest GC pause..

Implementation

• Class displaying the profiling data: DisplayJvmProfilingValueVerifier

• To get the number of young collections:

      IItemFilter youngGC = ItemFilters.type("jdk.YoungGarbageCollection");
        ItemsAndItemsForDifferenceCollection youngGcItemCollection = jfrEvents.apply(youngGC);

        int youngGcCount = 0;
        for (IItemIterable items : youngGcItemCollection) {
            youngGcCount += items.getItemCount();
        }

If you start to work on this issue, please leave a comment I start to work on this issue, so we can assign it to you. Do not hesitate to ask for information about the issue.

SpringBoot & SQL annotations & JUnit5

Description
Provides integration tests for Spring Boot, SQL annotations and JUnit5

Implementation ideas
Implements the same integration test than for Spring Boot, SQL annotations and JUnit4

@DebugQuickPerf should display the QuickPerf JVM options

Description
QuickPerf add some JVM options when you use some of it's features (for example JMC or some JVM related annotations).
These options should be displayed when the annotation @DebugQuickPerf is added to the test.

Implementation ideas
ConsoleReporter.displayQuickPerfDebugInfos() should logs the jvm options that QuickPerf add to the forked JVM.

Add @ExpectMaxInsert

Description

The aim of this annotation is to check the maximum number of INSERT statements.

For example, a test method having @ExpectMaxInsert(2) annotation will fail if more than two INSERT statements are executed.

Implementation steps

1. Create the @ExpectMaxInsert annotation here

   Example: @ExpectInsert

2. Create a test class in this package and add automatic tests

   Example: ExpectInsertTest.java

3. In this package add a performance measure extractor and a performance issue verifier.

   Examples:

   • InsertCountMeasureExtractor.java
   • InsertNumberPerfIssueVerifier.java
   • MaxOfSelectsPerfIssueVerifier.java

4. Create an AnnotationConfig in SqlAnnotationsConfig. This new AnnotationConfig will use the previously created performance measure extractor, performance issue verifier together with a PersistenceSqlRecorder.

   Example:

       static final AnnotationConfig MAX_SQL_SELECT = new AnnotationConfig.Builder()
           .perfRecorderClass(PersistenceSqlRecorder.class)
           .perfMeasureExtractor(SelectCountMeasureExtractor.INSTANCE)
           .perfIssueVerifier(MaxOfSelectsPerfIssueVerifier.INSTANCE)
           .build(ExpectMaxSelect.class);

5. Use the new AnnotationConfig in loadAnnotationConfigs() method of SqlConfigLoader

6. Complete the measure extractor and the performance issue verifier to make the tests green. Refactor to clean the code.

7. In SqlAnnotationBuilder, add a static method to can build the new annotation. This method can be used to define the annotation with a global scope.

If you start to work on this issue, please leave a comment I start to work on this issue, so we can assign it to you. Do not hesitate to ask for information about the issue.

Instrument dynamic JUnit 5 tests

Description
Great project, I really like the idea and the configuration model.

I'd love to have the alternative to instrument dynamic tests as well, which AFAIK does not respect annotations but would instead need some programmatic API alternative.
Alternatively provide an example on how these could be wired up manually with the current model and some before and after lambdas?

https://junit.org/junit5/docs/current/user-guide/#writing-tests-dynamic-tests

Change Concatenation to String Builder (or String Buffer in Case of Multi-threading)

So I just checked this class org/quickperf/jvm/jmc/value/DisplayJvmProfilingValueVerifier.java and it has a concatenation which is given below:

        String text =
                  LINE
                + " ALLOCATION (estimations)"                      + "   |   " + "GARBAGE COLLECTION           "                              + "|  THROWABLE"  + LINE_SEPARATOR
                + " Total       : " + thirteen.adapt(allocationTotal) + "|   " + twentyNineLength.adapt("Total pause: " + totalGcPause ) + "|  Exception: "  + exceptionsCount +LINE_SEPARATOR
                + " Inside TLAB : " + thirteen.adapt(insideTlabSum)   + "|   " + twentyNineLength.adapt("Longest GC pause: " + gcPause)  + "|  Error: " + errorCount + LINE_SEPARATOR
                + " Outside TLAB: " + thirteen.adapt(outsideTlabSum)  + "|   " + twentyNineLength.adapt("Young: " + youngGcCollection)   + "|  Throwable: " +throwablesCount + LINE_SEPARATOR
                + LINE
                +  twentyEightLength.adapt(" COMPILATION")                    + "|   " + "CODE CACHE" + LINE_SEPARATOR
                +  twentyEightLength.adapt(" Number: " + compilationsCount)   + "|   " +  codeCacheFullCount + LINE_SEPARATOR
                +  twentyEightLength.adapt(" Longest: " + longestCompilation) + "|   " + LINE_SEPARATOR
                + LINE
                + " " + "JVM" + LINE_SEPARATOR
                + " Name: " + jvmName + LINE_SEPARATOR
                + " Version: " + jvmVersion + LINE_SEPARATOR
                + " Arguments: " + jvmArguments + LINE_SEPARATOR
                + LINE
                + " " + "HARDWARE" + LINE_SEPARATOR
                + " Hardware threads: " + minHwThreads + LINE_SEPARATOR
                + " Cores: " + minNumberOfCores + LINE_SEPARATOR
                + " Sockets: " + minNumberOfSockets + LINE_SEPARATOR
                + " CPU: " + LINE_SEPARATOR
                + cpuDescription + LINE_SEPARATOR
                + LINE
                + " OS:" + LINE_SEPARATOR
                + osVersion + LINE_SEPARATOR
                + LINE;

        System.out.println(text);

So, Can I change it to String Builder (or Buffer in case if it is multi-threaded) cause concatenation is not good for performance wise?

Add @ExpectMaxHeap

Description
We can define the heap of the forked VM but not make any assert of the value of it.

Implementation ideas
Add the annotation and the corresponding perf verifier

Add @DisableSelfJoin and @EnableSelfJoin

Description

In many cases, SQL self-joins don't seem to be the best option for performance: Markus Winand 1, Markus Winand 2.

The use of a window function or a pivot may be a better alternative. With window function we have also to pay attention to excessive sorts.

A test with @DisableSelfJoin will fail if at least one SQL self-join is detected. The user may attribute a global scope to this annotation. In some specific cases, @DisableSelfJoin can be disabled at method level with @EnableSelfJoin.

Add @ExpectMaxQueryExecutionTime

Provide an annotation to check the execution time of SQL queries.

QuickPerf TestNG

Description

The goal of this issue is to be able to use QuickPerf with TestNG.

Implementation guidelines

• Create a testng Maven module to make QuickPerf work with TestNG
  You can look at JUnit 4 and JUnit 5 implementations to help you.

• Create a testng-sql-test Maven module verifying that SQL annotations work with TestNG
  You can look at JUnit 5 and JUnit 4 SQL tests to help you.

• Create a testng-jvm-test Maven module verifying that JVM annotations work with TestNG
  You can look at Junit 4 JVM tests to help you.

Don't hesitate to leave a comment if you have a question.

Multiple identical select raised in a JHipster project

Hello,

I'm trying to use QuickPerf within a Jhipster project in which I get an error with QuickPerf stating that I have 2 times the same select statement.

However, I don't think it's the case.

Here is the sample project I'm using to prove my point https://github.com/anthonyrichir/quickperf-sample-test.

For instance, a test on the repository itself:

@Import(QuickPerfPerBeanConfig.class)
@SpringBootTest(classes = BookApiApp.class)
@QuickPerfTest
class BookRepositoryTest {

    @Autowired
    private BookRepository bookRepository;

	@Test
    @ExpectSelect(1)
	void findAllWithEagerRelationships() {
        bookRepository.findAllWithEagerRelationships();
	}

	@Test
    @ExpectSelect(1)
	void findOneWithEagerRelationships() {
	    bookRepository.findOneWithEagerRelationships(1L);
	}
}

Where the repository is:

@Repository
public interface BookRepository extends JpaRepository<Book, Long> {

    @Query(value = "select distinct book from Book book left join fetch book.authors",
        countQuery = "select count(distinct book) from Book book")
    Page<Book> findAllWithEagerRelationships(Pageable pageable);

    @Query("select distinct book from Book book left join fetch book.authors")
    List<Book> findAllWithEagerRelationships();

    @Query("select book from Book book left join fetch book.authors where book.id =:id")
    Optional<Book> findOneWithEagerRelationships(@Param("id") Long id);

}

Will produce this output.

[WARNING] QuickPerf has built several datasource proxies


java.lang.AssertionError: a performance property is not respected

[PERF] You may think that <1> select request was sent to the database
       But in fact <2>...

You may decrease the number of select requests in order to reduce the cost of
server roundtrips: https://blog.jooq.org/2017/12/18/the-cost-of-jdbc-server-roundtrips/

Perhaps you are facing a N+1 select issue
	* With Hibernate, you may fix it by using JOIN FETCH
	                                       or LEFT JOIN FETCH
	                                       or FetchType.LAZY
	                                       or ...
	  Some examples: https://stackoverflow.com/questions/32453989/what-is-the-solution-for-the-n1-issue-in-jpa-and-hibernate";
	                 https://thoughts-on-java.org/jpa-21-entity-graph-part-1-named-entity/

	* With Spring Data JPA, you may fix it by adding
	@EntityGraph(attributePaths = { "..." }) on repository method.
	https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#jpa.entity-graph

[SQL EXECUTIONS]
	Time:0, Success:True, Type:Prepared, Batch:False, QuerySize:1, BatchSize:0, Query:["
    select
        distinct book0_.id as id1_1_0_,
        author2_.id as id1_0_1_,
        book0_.editor_id as editor_i3_1_0_,
        book0_.name as name2_1_0_,
        book0_.owner_id as owner_id4_1_0_,
        author2_.name as name2_0_1_,
        authors1_.book_id as book_id1_2_0__,
        authors1_.author_id as author_i2_2_0__ 
    from
        book book0_ 
    left outer join
        book_author authors1_ 
            on book0_.id=authors1_.book_id 
    left outer join
        author author2_ 
            on authors1_.author_id=author2_.id"], Params:[()]

	Time:1, Success:True, Type:Prepared, Batch:False, QuerySize:1, BatchSize:0, Query:["
    select
        distinct book0_.id as id1_1_0_,
        author2_.id as id1_0_1_,
        book0_.editor_id as editor_i3_1_0_,
        book0_.name as name2_1_0_,
        book0_.owner_id as owner_id4_1_0_,
        author2_.name as name2_0_1_,
        authors1_.book_id as book_id1_2_0__,
        authors1_.author_id as author_i2_2_0__ 
    from
        book book0_ 
    left outer join
        book_author authors1_ 
            on book0_.id=authors1_.book_id 
    left outer join
        author author2_ 
            on authors1_.author_id=author2_.id"], Params:[()]

Is it due to the several datasource proxies message ?

Help the Micronaut Data user to fix N+1 select issues

Description

Several SQL annotations can be used to detect N+1 select issues coming from Micronaut Data.

The aim of this issue is to display a message on console to alert of a potential N+1 select issue and to propose to use Micronaut join queries to fix it.

Implementation ideas

The way this feature is implemented for Hibernate can help you. You can look at the HibernateSuggestion class.

To test your development, execute mnv install in quickerf project and use PlayerServiceTest in the micronaut-data-jdbc module module of quickperf-examples project.

Add @MeasureOffHeapAllocation, @ExpectedOffHeapAllocation, @MaxExpectedOffHeapAllocation

Off-heap allocation has to be understand as direct memory allocation.
It can be measured with the help of a MBean. Undersatand the implemementations of @MeasureHeapAllocation, @ExpectedHeapAllocation and @MaxExpectedHeapAllocation can help you.

Add @MeasureMetaspaceAllocation and @ExpectMetaspaceAllocation annotations

Description
Add @MeasureMetaspaceAllocation to be able to mesure memory allocation on the Metaspace memory pool.

Add @ExpectMetaspaceAllocation to be able to put expectation on the result of @MeasureMetaspaceAllocation and use it to fail the test

Add @DisableQueriesWithoutBindParameters and @EnableQueriesWithoutBindParameters

Why

Using bind parameters is recommanded for performance. Moreover, bind parameters can prevent SQL injections.

References:

• Oracle Performance Survival Guide: A Systematic Approach to Database Optimization
• https://use-the-index-luke.com/sql/where-clause/bind-parameters
• https://dzone.com/articles/why-sql-bind-variables-are-important-for-performan
• https://blogs.oracle.com/sql/improve-sql-query-performance-by-using-bind-variables
• https://www.ibm.com/developerworks/library/se-bindvariables/index.html)
• ...

The role of @DisableQueriesWithoutBindParameters is to prevent the execution of requests without bind parameters. This annotation could be used whith a global scope, that is to say applied on each QuickPerf test.

@EnableQueriesWithoutBindParameters will cancel the behavior of @DisableQueriesWithoutBindParameters. @EnableQueriesWithoutBindParameters may be applied on a specific test method where some values can influence the execution plan (https://use-the-index-luke.com/sql/where-clause/bind-parameters).

Use cases

• With @DisableQueriesWithoutBindParameters, a test sending the following request to database must Not fail:

 SELECT
        * 
    FROM
        book 
    WHERE
        isbn = ? 
        AND title = ?"], Params:[(978-0134685991,Effective Java)]

Java code example generating this request:

EntityManager em = emf.createEntityManager();
String sql = "SELECT * FROM book WHERE isbn = :isbn AND title = :title";
Query nativeQuery = em.createNativeQuery(sql)
                             .setParameter("isbn", "978-0321356680")
                             .setParameter("title", "Effective Java");
nativeQuery.getResultList();

• With @DisableQueriesWithoutBindParameters, a test sending the following request to database must Not fail:

SELECT * FROM book 

• With @DisableQueriesWithoutBindParameters, a test sending the following request to database must fail:

 SELECT
        * 
    FROM
        book 
    WHERE
        isbn = '978-0321356680' 
        AND title = 'Effective Java'

Java code example generating this request:

EntityManager em = emf.createEntityManager();
String sql = "SELECT * FROM book WHERE isbn = '978-0321356680' AND title = 'Effective Java'";
Query nativeQuery = em.createNativeQuery(sql);
nativeQuery.getResultList();

• With @DisableQueriesWithoutBindParameters, a test sending the following request to database must Not fail:

UPDATE
        book 
    SET
        isbn = ?,
        title = ? 
    WHERE
        id = ?"], Params:[(978-0321356680,Effective Java,40)]

Java code example generating this request:

EntityManager em = emf.createEntityManager();
em.getTransaction().begin();
String sql = "UPDATE book SET isbn = :isbn, title = :title WHERE id = :id";
Query nativeQuery = em.createNativeQuery(sql)
                                  .setParameter("isbn", "978-0321356680")
                                  .setParameter("title", "Effective Java")
                                  .setParameter("id", 40);
nativeQuery.executeUpdate();
em.getTransaction().commit();

• With @DisableQueriesWithoutBindParameters, a test sending the following request to database must fail:

 UPDATE
        book 
    SET
        isbn = '978-0321356680',
        title = 'Effective Java' 
    WHERE
        id = '40'

Java code example generating this request:

EntityManager em = emf.createEntityManager();
em.getTransaction().begin();
String sql = "UPDATE book SET isbn = '978-0321356680', title = 'Effective Java' WHERE id = '40'";
Query nativeQuery = em.createNativeQuery(sql);
nativeQuery.executeUpdate();
em.getTransaction().commit();

• With @DisableQueriesWithoutBindParameters, a test sending the following request to database must Not fail:

    DELETE 
    FROM
        book 
    WHERE
        id = ?"], Params:[(40)

Java code example generating this request:

EntityManager em = emf.createEntityManager();
em.getTransaction().begin();
String sql = "DELETE FROM book WHERE id = :id";
Query nativeQuery = em.createNativeQuery(sql)
                                   .setParameter("id", 40);
nativeQuery.executeUpdate();
em.getTransaction().commit();

• With @DisableQueriesWithoutBindParameters, a test sending the following request to database must fail:

 DELETE 
    FROM
        book 
    WHERE
        id = '40'

Java code example generating this request:

EntityManager em = emf.createEntityManager();
em.getTransaction().begin();
String sql = "DELETE FROM book WHERE id = '40'";
Query nativeQuery = em.createNativeQuery(sql);
nativeQuery.executeUpdate();
em.getTransaction().commit();

Implementation

This documentation can help you to implement, in particular this part.

[JUnit 5] Add an automatic test checking that global annotations work

An annotation having a global scope applies on each test. More details here.

It is implemented with JUnit 4. The development of QuickPerf-JUnit 5 has begun.

Global annotations work with JUnit 5. The aim or this issue is to add an automatic test in junit5-sql-test module

You can find an example of JUnit 5 SQL test here.

Below the code (junit4-sql-test module) allowing to check that global annotations work with JUnit 4 :

https://github.com/quick-perf/quickperf/blob/master/junit4-sql-test/src/test/java/org/quickperf/sql/join/GlobalAnnotationTest.java

https://github.com/quick-perf/quickperf/blob/master/junit4-sql-test/src/test/java/org/quickperf/QuickPerfConfiguration.java

We can help you to work on this issue. Don't hesitate to ask questions.

Make RSS annotation work on Mac OS

@MeasureRSS and @ExpectMaxRSS annotations work on Linux thanks to this PR.

The goal of this issue is to make @MeasureRSS and @ExpectMaxRSS annotations work on Mac OS.

@ProfileHeapAllocation

With @ProfileHeapAllocation, heap allocation profiling will be displayed in the console (http://hirt.se/blog/?p=381).

QuickPerf can already profile a test method with Java Flight Recorder. @ProfileHeapAllocation will extract heap allocation profiling data from this recording.

If you would like to have more details on this issue or some help to work on it, please leave a comment.

Add @ExpectMaxDelete

Description

The aim of this annotation is to check the maximum number of DELETE statements.

For example, a test method having @ExpectMaxDelete(2) annotation will fail if more than two DELETE statements are executed.

Implementation steps

1. Create the @ExpectMaxDelete annotation here

   Example: @ExpectDelete

2. Create a test class in this package and add automatic tests

   Example: ExpectDeleteTest.java

3. In this package add a performance measure extractor and a performance issue verifier.

   Examples:

   • DeleteCountMeasureExtractor.java
   • NumberOfSqlDeletePerfIssueVerifier.java
   • MaxOfSelectsPerfIssueVerifier.java

4. Create an AnnotationConfig in SqlAnnotationsConfig. This new AnnotationConfig will use the previously created performance measure extractor, performance issue verifier together with a PersistenceSqlRecorder.

   Example:

       static final AnnotationConfig MAX_SQL_SELECT = new AnnotationConfig.Builder()
           .perfRecorderClass(PersistenceSqlRecorder.class)
           .perfMeasureExtractor(SelectCountMeasureExtractor.INSTANCE)
           .perfIssueVerifier(MaxOfSelectsPerfIssueVerifier.INSTANCE)
           .build(ExpectMaxSelect.class);

5. Use the new AnnotationConfig in loadAnnotationConfigs() method of SqlConfigLoader

6. Complete the measure extractor and the performance issue verifier to make the tests green. Refactor to clean the code.

7. In SqlAnnotationBuilder, add a static method to can build the new annotation. This method can be used to define the annotation with a global scope.

If you start to work on this issue, please leave a comment I start to work on this issue, so we can assign it to you. Do not hesitate to ask for information about the issue.

Add bytes to allocation display

Some annotations, like @MeasureHeapAllocation, can display allocation in console. The allocation is formatted with the help of ByteAllocationMeasureFormatter class.

For example:

Measured heap allocation: 3.1 Giga bytes

Now, we want that the formatted allocation also displays the allocation in bytes between parentheses if the allocation has a magnitude of kilo bytes, mega bytes or giga bytes.

For example:

3.9 Kilo bytes (4 056 bytes)

The test class to update is ByteAllocationMeasureFormatterTest.

If you start to work on this issue, please leave a comment "I start to work on this issue", so we can assign it to you.

Improving design

I analyzed the source code of the quickperf sql subproject with my own 3d analyzer tool.

http://jperf.com/qperf/

Probably it can help you to improve your project, you can navigate into classes/interfaces or packages and see all the relationships between all the components (visibility option) or analyse the complexity from various aspects (dependencies between classes, responsability, or lines of code), or analyze design problems (coupling, cyclic dependencies).

Hope this can help you in the design.

Have the ability to test SQL annotations on several databases with Testcontainers

Today, automatic tests using a memory database (H2) verify that SQL annotations are working as expected.

With the help of Testcontainers, the goal of this issue is to can automatically verify that SQL annotations are working as expected on several non-memory databases (Oracle, PostgreSQL, ...).

Don't hesitate to leave a comment if you have a question.

Add Javadoc to JVM annotations

JVM annotations are documented in QuickPerf wiki.

The goal of this issue is to add Javadoc to JVM annotations that are located in jvm-annotations Maven module.

The documentation contained in QuickPerf wiki can be used to add Javadoc to the JVM annotations. Don't hesitate to improve it.

Micronaut & SQL annotations & JUnit 5

SQL annotations can be used with JUnit 5.

We have to:

• use @QuickPerfTest
• build a QuickPerf proxy of javax.sql.DataSource

QuickPerfSqlDataSourceBuilder.aDataSourceBuilder().buildProxy(dataSource)

Given the JUnit 5 extension mechanism, SQL annotations should work with Micronaut + JUnit 5.

The aim of this issue is to check this and give an example using together @QuickPerfTest and @MicronautTest. Don't forget to build a QuickPerf proxy of javax.sql.DataSource (see above).

The Micronaut example could be given by creating Github project or by adding a non regression test in QuickPerf code (for example by adding a junit5-micronaut-sql-test Maven module).

Don't hesitate to leave a comment if you have a question.

Add Maven wrapper

Tutorial: https://www.baeldung.com/maven-wrapper

If you start to work on this issue, please leave a comment "I start to work on this issue", so we can assign it to you.

@ExpectMaxQueryExecutionTime alert message always displays "0 ms" as max

Hello Jean,

I noticed the following, I'd be happy to have your opinion on this:

Describe the bug
When using @ExpectMaxQueryExecutionTime, the console alert message always displays 0 ms in "[PERF] At least one request exceeds the max expected query execution time <0 ms>." and does not remind the value and unit set on the annotation.
Example :
@ExpectMaxQueryExecutionTime(value = 2, unit = TimeUnit.NANOSECONDS)

Expected behavior
"[PERF] At least one request exceeds the max expected query execution time < 2 ns >." where value and unit are the ones set in the annotation (see example above).

Actual behavior
"[PERF] At least one request exceeds the max expected query execution time <0 ms>."

To Reproduce
Here is an example using the @ExpectMaxInsertTest.
CAREFUL ! You should ONLY launch the AClassHavingAMethodAnnotatedWithExpectMaxInsert test (first of the three tests in the class), it should fail. When launching all the tests on the class, the test passes.
I used my IDE in order to launch only the first test (Intellij).

Versions

• QuickPerf: v1.0
• JDK: openjdk version "11.0.7"
• OS: Linux / Ubuntu 18.04
• Database (if SQL annotation bug): JPA

Additional context (Add any other context about the problem here.)
From my understanding, the issue is linked to the corresponding verifier class. Other verifier classes (such as the one in the core time package) often use a dedicated formatter class to help format and display this kind of informations. Time package is restricted to core annotations and the sql package formatter doesn't (and is not supposed to) format execution time.

Kind regards,
Fabrice

SQL annotations didn't work anymore with Quarkus starting with 1.3

Describe the bug
SQL annotations didn't work with Quarkus starting with 1.3.
No issues are detected.

I chased down the issue to SqlRecorderRegistry that uses an InheritableThreadLocal to stores the SqlRecorder.
Starting with Quarkus 1.3, when inside the DataSourceQuickPerfListener the list of sql recorders is empty?

I'm not sure if it's due to the changes in Classloader inside Quarkus or the fact that it is now run on Vert.X for all wordload (so possibly on a separate thread).

To Reproduce
Update the Quarkus example application of Quickperf to use Quarkus 1.3.0.Final.

Versions

• QuickPerf: master
• JDK: 11
• OS: Unbuntu
• Database (if SQL annotation bug): All

Additional context (Add any other context about the problem here.)

Issue with non-serializable classes in exceptions with fork VM

Describe the bug
When the VM is forked, all the test results from the forked VM are serialized to disk to be sent to the origin VM.

This uses Java Serialization.

If some non-serializable class is used inside the exception code of an exception thrown by the test, this leads to a crash of QuickPerf due to java.lang.IllegalStateException.

Expected behavior
(Describe the expected behavior clearly and concisely.)

Actual behavior

An error on Micronaut 2.0.0.M3 will generate the following stacktraces as it uses a non-serializable class

Unable to run test in a new JMV: Exception in thread "main" java.lang.IllegalStateException: Unable to save failures.
	at org.quickperf.repository.ObjectFileRepository.buildSerializationException(ObjectFileRepository.java:49)
	at org.quickperf.repository.ObjectFileRepository.save(ObjectFileRepository.java:44)
	at org.quickperf.repository.BusinessOrTechnicalIssueRepository.save(BusinessOrTechnicalIssueRepository.java:35)
	at org.quickperf.testlauncher.TestRunnerFromMain.executeTestMethod(TestRunnerFromMain.java:40)
	at org.quickperf.junit5.QuickPerfJunit5Core.main(QuickPerfJunit5Core.java:24)
Caused by: java.io.NotSerializableException: io.micronaut.http.client.exceptions.HttpClientErrorDecoder$1
	at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1185)
	at java.base/java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1553)
	at java.base/java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1510)
	at java.base/java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1433)
	at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1179)
	at java.base/java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1553)
	at java.base/java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1510)
	at java.base/java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1433)
	at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1179)
	at java.base/java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1553)
	at java.base/java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1510)
	at java.base/java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1433)
	at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1179)
	at java.base/java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:349)
	at org.quickperf.repository.ObjectFileRepository.save(ObjectFileRepository.java:41)
	... 3 more

Versions

• QuickPerf: master
• JDK: 11
• OS: Unbuntu 20.04
• Database (if SQL annotation bug): H2

Provides a JUnit 5 extension

Description
Currentlly, only JUnit 4 is supported.

Implementation ideas
There should be a JUnit 5 extension to be able to use quickperf with JUnit 5.
I can try to work on it.

Add `@MeasureRSS` and `@ExpectRSS` annotations

Description
RSS : Resident Set Size, is the total memory usage of a process on the OS it runs. This is the memory that the JVM uses for all it's memory pools (including direct memory if any).

Having a way to measure it at the begining of the test and at the end would be a good addition to QuickPerf

Implementation ideas
@MeasureRSS will measure RSS at the before and after the test and logged it to the console.

@ExpectRSS will make expectation of the size of the RSS before/after the test and fail the test if needed.

Test NG / Spring Boot

A QuickPerf Test NG listener is developed (see #32 and 3932345 ).

A Spring Boot application is tested with QuickPerf+JUnit4 and QuickPerf+Junit5.

The aim of this issue is to provide a Github repository containing this Spring Boot application with tests using QuickPerf and Test NG.

Add JDK 13 on CI (Travis)

The aim of this issue is to add two builds on Travis:

• one with OpenJDK JDK 13 and Linux
• one with Oracle JDK 13 and Linux

Add QuickPerf examples

You can look at these issues.

Add Oracle JDK 10 on CI (Travis)

The aim of this issue is to add a build on Travis with Oracle JDK 10 and Linux.

Make @ProfileJvm display the old GC count

Description

With @ProfileJvm annotation, the JVM is profiled with the JDK Flight Recorder, an event recorder built into the JVM.

With this annotation, some profiling data are displayed in the standard output.

For example:

 ALLOCATION (estimations)  |   GARBAGE COLLECTION           |  THROWABLE
 Total:        3,7 GiB     |   Total pause: 1,087 s         | Exception: 0
 Inside TLAB:  3,69 GiB    |   Longest GC pause: 160,438 ms | Error: 36
 Outside TLAB: 10,3 MiB    |                                | Throwable: 36

The aim of this issue is to also display the number of old collections.

With two old collections, we expect to display Old: 2 below Longest GC pause.

Implementation

• Class displaying the profiling data: DisplayJvmProfilingValueVerifier

• To get the number of old collections:

        IItemFilter oldGC = ItemFilters.type("jdk.OldGarbageCollection");
        ItemsAndItemsForDifferenceCollection oldGcItemCollection = jfrEvents.apply(oldGC);

        int oldGcCount = 0;
        for (IItemIterable items : oldGcItemCollection) {
            oldGcCount += items.getItemCount();
        }

If you start to work on this issue, please leave a comment I start to work on this issue, so we can assign it to you. Do not hesitate to ask for information about the issue.

Have configurable verbosity on none passing test

Description
Some Feedback on non-working test (for instance N+1 select) are very long.
It's useful but it could be drawing (overwhelming) sometime.
It also could fill the CI system

It could be nice have its verbosity configurable
Implementation ideas
(If you have any implementation ideas, they can go here.)
It would be great to works with -Dquickperf.hints=0

Add @ExpectMaxUpdate

Description

The aim of this annotation is to check the maximum number of UPDATE statements.

For example, a test method having @ExpectMaxUpdate(2) annotation will fail if more than two UPDATE statements are executed.

Implementation steps

1. Create the @ExpectMaxUpdate annotation here

   Example: @ExpectUpdate

2. Create a test class in this package and add automatic tests

   Example: ExpectUpdateTest.java

3. In this package add a performance measure extractor and a performance issue verifier.

   Examples:

   • UpdateCountMeasureExtractor.java
   • UpdateNumberPerfIssueVerifier.java
   • MaxOfSelectsPerfIssueVerifier.java

4. Create an AnnotationConfig in SqlAnnotationsConfig. This new AnnotationConfig will use the previously created performance measure extractor, performance issue verifier together with a PersistenceSqlRecorder.

   Example:

       static final AnnotationConfig MAX_SQL_SELECT = new AnnotationConfig.Builder()
           .perfRecorderClass(PersistenceSqlRecorder.class)
           .perfMeasureExtractor(SelectCountMeasureExtractor.INSTANCE)
           .perfIssueVerifier(MaxOfSelectsPerfIssueVerifier.INSTANCE)
           .build(ExpectMaxSelect.class);

5. Use the new AnnotationConfig in loadAnnotationConfigs() method of SqlConfigLoader

6. Complete the measure extractor and the performance issue verifier to make the tests green. Refactor to clean the code.

7. In SqlAnnotationBuilder, add a static method to can build the new annotation. This method can be used to define the annotation with a global scope.

If you start to work on this issue, please leave a comment I start to work on this issue, so we can assign it to you. Do not hesitate to ask for information about the issue.

Improve SQL annotation issue due to the framework debugging

Description
Improve SQL annotation issue due to the framework debugging by generating a specific exception when no query has been intercepted.

Implementation ideas

1. When one of the SQL annotation is used but no query has been intercepted, throw a specific exception with possible causes. This is specifically important for annotation like @ExpectMax[...] as it will validate no queries.
2. Provide @ExceptNoSQLQuery annotation for a user to specifically check for no queries.
3. Provide "framework hints":
   3.1 Add the capabulity to detect known framework at runtime: SpringBoot, Micornaut, Quarkus
   3.2. Hint of what could possibly be wrong and link to the documentation site (or the example repository).

The point 3 can be provided later.

Quarkus & SQL annotations & JUnit5

Description
Provides integration tests for Quarkus, SQL annotations and JUnit5

Implementation ideas
Implements the same integration test than for Spring Boot, SQL annotations and JUnit4

Make @ProfileJvm display the allocation rate

Description

With @ProfileJvm annotation, the JVM is profiled with the JDK Flight Recorder, an event recorder built into the JVM.

With this annotation, some JVM data are displayed in the standard output.

Example:

 ALLOCATION (estimations)  |   GARBAGE COLLECTION           |  THROWABLE
 Total:        3,7 GiB     |   Total pause: 1,087 s         | Exception: 0
 Inside TLAB:  3,69 GiB    |   Longest GC pause: 160,438 ms | Error: 36
 Outside TLAB: 10,3 MiB    |                                | Throwable: 36

The aim of this issue is to add the allocation rate.

Implementation ideas

• Class displaying JVM data: DisplayJvmProfilingValueVerifier
• Total allocation can be computed by using JdkAggregators.ALLOCATION_TOTAL.

   IQuantity totatlAlloc =(IQuantity) itemCollection.getAggregate(JdkAggregators.ALLOCATION_TOTAL);
   long allocationInBytes = totatlAlloc.longValue();

• The duration can be computed in the following way:

  • Filter IItemCollection by inside and outside TLAB events:

     itemCollection.apply(JdkFilters.ALLOC_INSIDE_TLAB)       
     itemCollection.apply(JdkFilters.ALLOC_OUTSIDE_TLAB)

  • Retrieve the min and max timestamps
    Code to get the timestamp of a JFR event below

    IType<IItem> type= (IType<IItem>) item.getType();
    IMemberAccessor<IQuantity, IItem> endTimeAccessor = JfrAttributes.END_TIME.getAccessor(type);
    IQuantity quantityEndTime = endTimeAccessor.getMember(jfrEvent);
    long timeStampInMs = quantityEndTime.longValueIn(UnitLookup.EPOCH_MS);

  • Compute the difference between the min and max time stamps

• Compute (total allocation)/duration

QuickPerf should have a minimum version of Java 8

Description
Today, the Maven Enforce plugin configuration of QuickPerf is configure for Java 7 compatiility !
Java 7 is EOL since April 2015 and a lot of libraries are compatible with Java 8 only.

Implementation ideas
Upgrade the Maven Enforce plugin configuration to target Java 8.

Create a QuickPerf web site

The website will contain the documentation, the release notes, ...

Add @ExpectUpdatedColumn

Description
The aim of this annotation is to check the number of columns updated by a SQL UPDATE statement.

Implementation steps
The implementation steps are described (here).
The code added to implement @ExpectUpdatedColumn will help you.

If you start to work on this issue, please leave a comment "I start to work on this issue", so we can assign it to you. Do not hesitate to ask for information about the issue.

Support for Spock Framework

Support for Spock Framework Test Cases
Can support be added for Spock Framework Tests ?
http://spockframework.org

Add Javadoc to core annotations

core annotations are documented in QuickPerf wiki .

The goal of this issue is to add Javadoc to core annotations that are located in core Maven module.

The documention contained in QuickPerf wiki can be used to add Javadoc to the core annotations. Don't hesitate to improve it.

Add @ExpectNoInflaterMemoryLeak

java.util.zip.Inflater can cause non-heap memory leaks, as explained in this presentation. They can be detected during the application execution thanks to the Leakchecker project. This issue aims to provide an @ExpectNoInflaterMemoryLeak annotation in QuickPerf to detect this memory leak as soon as possible. The @ExpectNoInflaterMemoryLeak will be based on the Leakchecker project. This project has an Apache 2.0 license so that it is ok to detect inflater memory leak in the same way in QuickPerf project. Both in QuickPerf source code and QuickPerf documentation, we will take care of mentioning the Leakchecker project.

This issue is focussed on memory leaks that could come from Inflater. Later, we could detect other kinds of potential memory leaks, such as those that could be raised from ObjectInputStream.

Add Javadoc to SQL annotations

SQL annotations are documented in QuickPerf wiki.

The goal of this issue is to add Javadoc to SQL annotations that are located in sql-annotations Maven module.

The documentation contained in QuickPerf wiki can be used to add Javadoc to the SQL annotations. Don't hesitate to improve it.

Update contributing with Maven wrappers

The goal of this issue is to update the Contributing.md file to explain why we provide Maven wrappers and how to use them.

Cannot resolve dependency org.openjdk.jmc:flightrecorder.rules.jdk:7.0.0-SNAPSHOT

I have created a Spring Boot 2.2.1 project from https://start.spring.io/ and added testImplementation 'org.quickperf:quick-perf-junit4-spring5:1.0.0-RC4' as Gradle dependency.

Gradle cannot resolve org.openjdk.jmc:flightrecorder.rules.jdk:7.0.0-SNAPSHOT which seems to be required by org.quickperf:quick-perf-jvm-annotations:1.0.0-RC4.

The project uses OpenJDK11.

What am I missing?