openjdk team mailing list archive

[Erich Schubert <erich@xxxxxxxxxx>]

Subject: openjdk-11-jre: Performance of JDK11/12 is 10%-15% worse than JDK8
Package: openjdk-11-jre
Version: 11.0.4+11-1
Severity: normal
Tags: upstream

In my benchmarks, JDK 11 and 12 scores around 10%-20% worse runtime than 
JDK 8.

I also tried Oracle Graal CE and Graal EE, and they perform even worse. 
I also
tried AdoptOpenJDK packages, and these perform similar to the Debian 
versions.

The difference is substantial; this is not a toy microbenchmark that 
only tests
a single tiny code fragment. Instead, I run ELKI to benchmark DBSCAN 
clustering
on a dataset with over 100k points; a typical workload that I am 
interested in.
This code is well optimized, and requires very little garbage collection (5
young gen runs each, about 80 ms total GC time). Memory limits are high 
enough
to let this run smoothly.
I have measured both in-Java time of two steps, and wall-clock time of the
entire process until completion.

Loading 110250 vectors from a .csv.gz file with

Java   8: 1837 ms
Java  11: 1955 ms
Java  12: 1946 ms
Graal EE: 4834 ms (ouch)

The difference between 8 and 11/12 is significant (this is within Java, 
so it
is not just a startup difference). The difference between 11 and 12 is 
random.
Graal is really bad here, but loading the data is just 2-5 seconds total 
here
anyway.

Running DBSCAN clustering on this data, using linear scans (I also have 
results
with index acceleration):

Java   8: 231677 ms
Java  11: 255017 ms
Java  12: 255266 ms
Graal EE: 261761 ms

Again, Java 11 and 12 are similar, within the usual measurement differences.
But they are both 10% worse than Java 8, which is significant. Graal is even
13% worse. In other experiments, I have seen differences of 15% (J11/12) and
20% (Graal).

For larger data sets, many iterations, etc. this runtime difference of 30
seconds quickly adds up to a lot of time...

Because of this, I currently stick to Java 8.



-- System Information:
Debian Release: bullseye/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 4.19.0-5-amd64 (SMP w/8 CPU cores)
Locale: LANG=en_US.utf8, LC_CTYPE=en_US.utf8 (charmap=UTF-8) (ignored: 
LC_ALL set to de_DE.utf-8), LANGUAGE=en_US.utf8 (charmap=UTF-8) 
(ignored: LC_ALL set to de_DE.utf-8)
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages openjdk-11-jre depends on:
ii  libc6                    2.28-10
ii  libgif7                  5.1.4-3
ii  libgl1                   1.1.0-1
ii  libjpeg62-turbo          1:1.5.2-2+b1
ii  libpng16-16              1.6.37-1
ii  libx11-6                 2:1.6.7-1
ii  libxext6                 2:1.3.3-1+b2
ii  openjdk-11-jre-headless  11.0.4+11-1
ii  zlib1g                   1:1.2.11.dfsg-1+b1

Versions of packages openjdk-11-jre recommends:
ii  fonts-dejavu-extra       2.37-1
ii  libatk-wrapper-java-jni  0.35.0-2

openjdk-11-jre suggests no packages.

-- no debconf information