When running a native-image of a Micronaut application retrieving Prometheus metrics fails because it cannot get jvm metrics.
Of course native-image
pre-compiles the application and includes only what's actually being used, but what kind of JVM is used when running a native-image? I assume it's also a greatly reduced version of the standard hotspot vm.
So is it even possible to get jvm metrics when running a native image or should other means be used to retrieve cpu and memory stats of the running application?
In this case I used Micronaut but I recon it is common for getting metrics from any native-image application with prometheus metrics.
To reproduce:
- install Micronaut cli (e.g. via Homebrew on Mac)
- install GraalVM 19.2.0 (e.g. via Sdkman on Mac)
- install GraalVM native-image tool: gu install native-image
- create application: mn create-app --features=management,micrometer-prometheus,graal-native-image --build maven metrics-graalvm-image
- update src/main/resources/application.yml to allow anonymous access to prometheus endpoint:
endpoints:
all:
enabled: true
sensitive: false
- run application: man exec:exec
- check prometheus metrics can be retrieved: open http://localhost:8080/prometheus
- build application: mvn clean verify
- create native image: native-image --no-server -jar target/metrics-graalvm-image-0.1.jar
- run native image: ./metrics-graalvm-image
- get prometheus metrics: open http://localhost:8080/prometheus
No with the native image, this exception is thrown:
11:06:51.186 [pool-2-thread-4] ERROR i.m.h.s.netty.RoutingInBoundHandler - Unexpected error occurred: ThreadMXBean methods
com.oracle.svm.core.jdk.UnsupportedFeatureError: ThreadMXBean methods
at com.oracle.svm.core.util.VMError.unsupportedFeature(VMError.java:102)
at com.oracle.svm.core.jdk.SubstrateThreadMXBean.getAllThreadIds(ManagementSupport.java:448)
at io.micrometer.core.instrument.binder.jvm.JvmThreadMetrics.getThreadStateCount(JvmThreadMetrics.java:85)
at io.micrometer.core.instrument.binder.jvm.JvmThreadMetrics.lambda$bindTo$0(JvmThreadMetrics.java:75)
at io.micrometer.core.instrument.internal.DefaultGauge.value(DefaultGauge.java:40)
at io.micrometer.prometheus.PrometheusMeterRegistry.lambda$newGauge$3(PrometheusMeterRegistry.java:245)
at io.micrometer.prometheus.MicrometerCollector.collect(MicrometerCollector.java:69)
at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.findNextElement(CollectorRegistry.java:183)
at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:216)
at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:137)
at io.prometheus.client.exporter.common.TextFormat.write004(TextFormat.java:22)
at io.micrometer.prometheus.PrometheusMeterRegistry.scrape(PrometheusMeterRegistry.java:95)
at io.micrometer.prometheus.PrometheusMeterRegistry.scrape(PrometheusMeterRegistry.java:80)
at io.micronaut.configuration.metrics.micrometer.prometheus.management.PrometheusEndpoint.scrape(PrometheusEndpoint.java:52)
at io.micronaut.configuration.metrics.micrometer.prometheus.management.$PrometheusEndpointDefinition$$exec1.invokeInternal(Unknown Source)
at io.micronaut.context.AbstractExecutableMethod.invoke(AbstractExecutableMethod.java:144)
at io.micronaut.context.DefaultBeanContext$BeanExecutionHandle.invoke(DefaultBeanContext.java:2792)
at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:235)
at io.micronaut.web.router.RouteMatch.execute(RouteMatch.java:122)
at io.micronaut.http.server.netty.RoutingInBoundHandler.lambda$buildResultEmitter$19(RoutingInBoundHandler.java:1408)
at io.reactivex.internal.operators.flowable.FlowableCreate.subscribeActual(FlowableCreate.java:71)
at io.reactivex.Flowable.subscribe(Flowable.java:14918)
at io.reactivex.Flowable.subscribe(Flowable.java:14865)
at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:68)
at io.reactivex.Flowable.subscribe(Flowable.java:14918)
at io.reactivex.internal.operators.flowable.FlowableMap.subscribeActual(FlowableMap.java:37)
at io.reactivex.Flowable.subscribe(Flowable.java:14918)
at io.reactivex.Flowable.subscribe(Flowable.java:14865)
at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:68)
at io.reactivex.Flowable.subscribe(Flowable.java:14918)
at io.reactivex.internal.operators.flowable.FlowableSwitchIfEmpty.subscribeActual(FlowableSwitchIfEmpty.java:32)
at io.reactivex.Flowable.subscribe(Flowable.java:14918)
at io.reactivex.Flowable.subscribe(Flowable.java:14865)
at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:68)
at io.reactivex.Flowable.subscribe(Flowable.java:14918)
at io.reactivex.Flowable.subscribe(Flowable.java:14868)
at io.micronaut.http.context.ServerRequestTracingPublisher.lambda$subscribe$0(ServerRequestTracingPublisher.java:52)
at io.micronaut.http.context.ServerRequestContext.with(ServerRequestContext.java:52)
at io.micronaut.http.context.ServerRequestTracingPublisher.subscribe(ServerRequestTracingPublisher.java:52)
at io.micronaut.configuration.metrics.binder.web.WebMetricsPublisher.subscribe(WebMetricsPublisher.java:122)
at io.reactivex.internal.operators.flowable.FlowableFromPublisher.subscribeActual(FlowableFromPublisher.java:29)
at io.reactivex.Flowable.subscribe(Flowable.java:14918)
at io.reactivex.Flowable.subscribe(Flowable.java:14865)
at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:68)
at io.reactivex.Flowable.subscribe(Flowable.java:14918)
at io.reactivex.Flowable.subscribe(Flowable.java:14865)
at io.reactivex.internal.operators.flowable.FlowableSubscribeOn$SubscribeOnSubscriber.run(FlowableSubscribeOn.java:82)
at io.reactivex.internal.schedulers.ExecutorScheduler$ExecutorWorker$BooleanRunnable.run(ExecutorScheduler.java:288)
at io.reactivex.internal.schedulers.ExecutorScheduler$ExecutorWorker.run(ExecutorScheduler.java:253)
at io.micrometer.core.instrument.composite.CompositeTimer.record(CompositeTimer.java:79)
at io.micrometer.core.instrument.Timer.lambda$wrap$0(Timer.java:144)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:460)
at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:193)
In two small tests I ran where GraalVM was able to create a native executable, the native executable ran significantly faster than the equivalent Scala/Java code running with the Java 8 JVM, and also reduced RAM consumption by a whopping 98% in a long-running example.
Native Image is a technology to ahead-of-time compile Java code to a standalone executable, called a native image. This executable includes the application classes, classes from its dependencies, runtime library classes, and statically linked native code from JDK.
After the analysis is complete, Graal compiles all the reachable code into a platform-specific native executable. That executable is fully functional on its own and doesn't need the JVM to run.
It supports additional programming languages and execution modes, like ahead-of-time compilation of Java applications for fast startup and low memory footprint. The first production-ready version, GraalVM 19.0, was released in May 2019. The most recent version is GraalVM 22.1. 0, made available in April 2022.
The GraalVM native-image
in the standalone executables uses SubstrateVM - a partial VM.
You can find more details about it on its official GitHub repo here.
Additionally according to the SubstrateVM limitations document, the standard management and debugging interfaces that Java offers are not supported. This is because they require access to Java bytecode, which is not longer available at runtime, in case of the native image.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With