Java/Spring application cannot start up on cloudfoundry: OutOfMemoryError Compressed class space

Question

After deploying a new version of our Java/Spring Boot software to the Swisscom Developer Cloud, running on CloudFoundry, the startup suddenly failed with the following error: OutOfMemoryError: Compressed class space. So, we decided to deploy a previous version of the software, the version which was running actually just before: The same error occurred. We did not switch from Java7 to Java8, nor did we change any configuration. This leads to the question: Is this really an error on our side or rather on the server's side?

---

We then tried to increase the MaxMetaspaceSize by setting the variable JBP_CONFIG_OPEN_JDK_JRE to one of the following lines:

[jre: {version: 1.8.0_+}, memory_calculator: {memory_sizes: {metaspace: 128m}}]
{memory_calculator: {memory_sizes: {metaspace: 128m}}}
{memory_sizes: {metaspace: 128m}}

The application always warned, that the value of memory_sizes was invalid. What is the correct format of this YAML variable?

[ConfigurationUtils] WARN User config value for 'memory_sizes' is not valid, existing property not present

---

We then deleted the Java app and the database service on the Swisscom Developer Console and recreated it. It had no effect, the same error occured.

And finally, do you know why this error does suddenly occur, even with a version which was just running fine a few minutes ago?

---

EDIT:

This is the manifest ([database-service-name] and [application-name] were replaced):

---
path: .
instances: 1
buildpack: https://github.com/cloudfoundry/java-buildpack
services:
- [database-service-name]
applications:
- name: [application-name]
  domain: scapp.io
  host: [application-name]
  memory: 1024M
  disk_quota: 1024M
  env:
    SPRING_PROFILES_ACTIVE: stage, cloudfoundry

The Java buildpack version is (according to the logs):

2017-03-03 11:47:02 [STG/0] OUT -----> Java Buildpack Version: b08a692 | https://github.com/cloudfoundry/java-buildpack#b08a692

This command seems to be executed (in the logs after the crash):

2017-03-03 11:46:25 [APP/PROC/WEB/0] OUT vcap 8 0 99 10:46 ? 00:01:09 /home/vcap/app/.java-buildpack/open_jdk_jre/bin/java -Djava.io.tmpdir=/home/vcap/tmp -XX:OnOutOfMemoryError=/home/vcap/app/.java-buildpack/open_jdk_jre/bin/killjava.sh -XX:MaxDirectMemorySize=10M -XX:MaxMetaspaceSize=68540K -XX:ReservedCodeCacheSize=240M -XX:CompressedClassSpaceSize=8731K -Xmx408104K -Djavax.net.ssl.trustStore=/home/vcap/app/.java-buildpack/container_certificate_trust_store/truststore.jks -Djavax.net.ssl.trustStorePassword=java-buildpack-trust-store-password -cp /home/vcap/app/. org.springframework.boot.loader.WarLauncher

Answer 1

The OutOfMemory error occurred because the Java buildpack was changed to use version 3.x of the memory calculator. Similar problems arising from this change are under discussion in GitHub issue 390. Please refer to this issue for details.

In general, v3.x of the memory calculator chooses values for various JVM memory settings based on the number of class files in the application and some default values which depend on the version of Java. It then sets the maximum heap size to the remaining amount of memory.

The previous version of the memory calculator was configured by setting JBP_CONFIG_OPEN_JDK_JRE. However, v3.x can be configured simply by setting the corresponding Java memory settings in JAVA_OPTS. For example, you could set the maximum metaspace size to 100 Mb as follows:

cf set-env app-name JAVA_OPTS '-XX:MaxMetaspaceSize=100m'

If you simply want a workaround, you can use the version of the Java buildpack released before the memory calculator change:

cf push -b https://github.com/cloudfoundry/java-buildpack.git\#v3.14 ...

Answer 2

comment from Swisscom Java developer:

For sure they will be reverting the memory heuristics to what they were in 3.13 or at least refine the calculations. The current recommendation to all customers is either

• to use 3.13 or
• to use some explicit options with the environment variable JAVA_OPTS.