albertattard / comparing-the-performance-of-some-list-implementations

Java Creed - Comparing the Performance of some List Implementations

A question that developers ask is: with many List (Java Doc) implementations, which implementation should a developer uses? The correct answer to this question is: it depends. All implementations serve a different purpose and one cannot simply say that a particular implementation is always better than the others, in all cases. One can say that one implementation may be more suitable in a particular situation than others or that an implementation will perform slower than the others in certain circumstances.

In this article we measure the time take required by various implementations to perform a set of specific actions. We will consider the following list implementations in the experiment:

• Vector (Java Doc)
• Vector (with size set at construction time)
• ArrayList (Java Doc)
• ArrayList (with size set at construction time)
• LinkedList (Java Doc)
• Stack (Java Doc)
• CopyOnWriteArrayList (Java Doc)

During the tests we will perform the following actions on each instance:

• List.add(Object)
• List.get(int)
• List.iterator()
• List.size()

All tests are carried on a single thread and concurrency is not taken into account. The test is designed to be extendable and we can add as many List implementations as we want in order to increase the test coverage. Furthermore, we can also include new tests to cover more List actions.

All code listed below is available at: https://github.com/javacreed/comparing-the-performance-of-some-list-implementations. Most of the examples will not contain the whole code and may omit fragments which are not relevant to the example being discussed. The readers can download or view all code from the above link.

Results

The output of the test is formatted as shown in the following matrix.

List Type	add()	get()	iterate()	size()
Vector	12.691	0.143	0.286	0.047
Vector with init size	10.134	0.045	0.042	0.009
ArrayList	9.873	0.051	0.037	0.013
ArrayList with init size	9.845	0.036	0.003	0.005
LinkedList	9.913	172.824	0.538	0.030
Stack	9.843	0.105	0.129	0.060
CopyOnWriteArrayList	36.909	0.092	0.099	0.051

The above results are the average performance of 100 runs for a list with size of: 10000 and performing 10000 operations. Therefore each operation was performed 10000 times on each list instance. All time figures listed in this article are in milliseconds.

Each test is described in further details in the following sections.

Inserting Elements to a List

The first test that was performed involved populating the list with some generated strings as shown the following code fragment.

  private final String pattern = "Element %d";

  @Override
  public long timeAction(List<String> list, int limit) {
    final long start = System.nanoTime();
    for (int i = 0; i < limit; i++) {
      list.add(String.format(pattern, i));
    }
    return System.nanoTime() - start;
  }

Most of the tested implementations obtained similar performance time with the exception of the CopyOnWriteArrayList implementation. As documented in the class Java Doc, a new array is created when a new item is added to the list.

A thread-safe variant of ArrayList in which all mutative operations (add, set, and so on) are implemented by making a fresh copy of the underlying array.

Therefore, a new copy of the underlying array was created for every insert made during the test. This explains why the insertions were considerably slower when compared with the other lists implementations. Note that this implementation was designed to work with many threads and where the number reads are far greater than the number of writes.

The following graph provides a visual comparison of the performance of all tested implementations.

The Vector is slightly slower than the others (excluding the CopyOnWriteArrayList). This is because the Vector is thread-safe and the insert operation made use of locks. These lock were not contented but nevertheless, synchronisation will always reduce the performance. With that said, the difference is hardly measured and this is insignificant for most applications. Should you refactor all your code and replace the Vector with another implementation? The answer is simple: NO.

What we learn from this test is that it is important to know the API and how the implementations we are using behave. The CopyOnWriteArrayList is expected to perform slower than the others when manipulated, thus unless required, prefer the other List implementations, if the list will be altered (elements are added or removed). On the other hand, the Iterator (Java Doc) returned by the CopyOnWriteArrayList implementation, does not require any further synchronisation and will not fail if the list is modified.

Retrieving Elements from a List

The second test that was performed was the retrieving of the elements from the list at a given index. The following code fragment shows the operations performed on each list implementation.

  private final String pattern = "Element %d";

  @Override
  public long timeAction(final List<String> list, final int limit) {
    for (int i = 0; i < limit; i++) {
      list.add(String.format(pattern, i));
    }

    final long start = System.nanoTime();
    for (int i = 0, size = list.size(); i < limit; i++) {
      list.get(i % size);
    }
    return System.nanoTime() - start;
  }

Note that the list implementation was first populated and the population process was not measured. Only the retrieval operation was measured during this test.

This test uncovered the weakness of the LinkedList as illustrated by the following graph.

The time taken for all other lists to execute this test was negligible and their differences are hardly noticeable. On the other hand, a LinkedList is expected to perform poorly in this test as it has to iterate through all previous elements in order to get to the required one. While the other lists implementation will take almost no time to access the last element, the LinkedList will have to perform n operations (where n is the index from where the element is retrieved). If we need to retrieve the 100th elements, then the LinkedList needs to iterate 100 times, before it can reach this element.

If the list will be accessed in a random manner, as tested here, one should avoid the LinkedList implementation as this performs slower than the other implementations. If this is not possible for various reasons, try to perform the retrieval operations through another list instance as shown in the following example.

int index = ...
LinkedList<String> ll = ...
List<String> list = new ArrayList<>(ll);
list.get(index);

The example shown above has its limitations too. While this example would perform well in a convert-once-use-many-times scenario, it will perform poorly when we have to convert the LinkedList several times (such as once for every retrieval operation).

Iterate through all Elements in a List

The third test that was conducted iterated through the list content. As shown in the table above, all implementations performed more or less the same. One can argue that the LinkedList was the slowest of them all, as shown below.

The difference is not very significant, even though the graph makes this looks worse than it seems.

Get the list size

The final test measured the time it takes to retrieve the size of the list as shown in the following figure.

Similar to the previous test, most implementation took the same time.

Conclusion

In this article we compared various lists implementations and measured their performance when executing some common actions. Some implementations performed poorly when compared to the others are highlighted in the first two tests. This does not mean that those that did not do well should never be used. On the contrary, all implementations have their place and here we only considered single threading environment. Before using any given list, stop and think how this list is going to be used and run some tests like the ones we saw here before jumping into any hasty conclusions.