Wildfly java.lang.NoClassDefFoundError: org/jboss/modules/ModuleLoader when using custom logging formatter

Question

I am using the default jboss-logging (LogManager) in Wildfly 10 and am attempting to create my own logging formatter.

I am configuring it like this in the standalone.xml:

    <subsystem xmlns="urn:jboss:domain:logging:3.0">
        <use-deployment-logging-config value="false"/>
        <console-handler name="CONSOLE">
            <level name="INFO"/>
            <formatter>
                <named-formatter name="JSON_PATTERN"/>
            </formatter>
        </console-handler>
        <formatter name="JSON_PATTERN">
            <custom-formatter
                    class="com.mycompany.JsonLogFormatter"
                    module="com.mycompany.logging"/>
        </formatter>
    </subsystem>

I created a jboss module like this:

<?xml version="1.0" encoding="UTF-8"?>
<module xmlns="urn:jboss:module:1.3" name="com.mycompany.logging">
    <resources>
        <resource-root path="log-utils-1.0.0.jar" />
    </resources>
    <dependencies>
        <module name="javax.json.api"/>
        <module name="org.jboss.logmanager" />
        <module name="org.jboss.logging" />
        <module name="org.jboss.modules"/>
    </dependencies>
</module>

where the log-utils-1.0.0.jar contains my JsonLogFormatter class and is located in the same directory as the module.xml (located in $JBOSS_HOME/modules/com/mycompany/logging/main)

When firing up the wildfly container, it fails to start and I am seeing this exception:

java.lang.IllegalArgumentException: Failed to load module "com.mycompany.logging" for formatter "JSON_PATTERN"
    at org.jboss.logmanager.config.AbstractPropertyConfiguration.<init>(AbstractPropertyConfiguration.java:64) [jboss-logmanager-2.0.4.Final.jar:2.0.4.Final]
    at org.jboss.logmanager.config.FormatterConfigurationImpl.<init>(FormatterConfigurationImpl.java:30) [jboss-logmanager-2.0.4.Final.jar:2.0.4.Final]
    at org.jboss.logmanager.config.LogContextConfigurationImpl.addFormatterConfiguration(LogContextConfigurationImpl.java:171) [jboss-logmanager-2.0.4.Final.jar:2.0.4.Final]
    at org.jboss.as.logging.logmanager.ConfigurationPersistence.addFormatterConfiguration(ConfigurationPersistence.java:218)
    at org.jboss.as.logging.CustomFormatterResourceDefinition$2.performRuntime(CustomFormatterResourceDefinition.java:117)
    at org.jboss.as.logging.LoggingOperations$LoggingAddOperationStepHandler$1.execute(LoggingOperations.java:204)
    at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:890) [wildfly-controller-2.2.0.Final.jar:2.2.0.Final]
    at org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:659) [wildfly-controller-2.2.0.Final.jar:2.2.0.Final]
    at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:370) [wildfly-controller-2.2.0.Final.jar:2.2.0.Final]
    at org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1329) [wildfly-controller-2.2.0.Final.jar:2.2.0.Final]
    at org.jboss.as.controller.ModelControllerImpl.boot(ModelControllerImpl.java:493) [wildfly-controller-2.2.0.Final.jar:2.2.0.Final]
    at org.jboss.as.controller.AbstractControllerService.boot(AbstractControllerService.java:387) [wildfly-controller-2.2.0.Final.jar:2.2.0.Final]
    at org.jboss.as.controller.AbstractControllerService.boot(AbstractControllerService.java:349) [wildfly-controller-2.2.0.Final.jar:2.2.0.Final]
    at org.jboss.as.server.ServerService.boot(ServerService.java:397) [wildfly-server-2.2.0.Final.jar:2.2.0.Final]
    at org.jboss.as.server.ServerService.boot(ServerService.java:366) [wildfly-server-2.2.0.Final.jar:2.2.0.Final]
    at org.jboss.as.controller.AbstractControllerService$1.run(AbstractControllerService.java:299) [wildfly-controller-2.2.0.Final.jar:2.2.0.Final]
    at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_151]
 Caused by: java.lang.NoClassDefFoundError: org/jboss/modules/ModuleLoader
    at org.jboss.logmanager.config.AbstractPropertyConfiguration$ModuleFinder.getClassLoader(AbstractPropertyConfiguration.java:463) [jboss-logmanager-2.0.4.Final.jar:2.0.4.Final]
    at org.jboss.logmanager.config.AbstractPropertyConfiguration.<init>(AbstractPropertyConfiguration.java:62) [jboss-logmanager-2.0.4.Final.jar:2.0.4.Final]
    ... 16 more

Answer 1

I was able to get a clean start up, w/o the exception, when I added the following parameter to the JAVA_OPTS environment:

JAVA_OPTS="$JAVA_OPTS -Xbootclasspath/p:$JBOSS_HOME/jboss-modules.jar"

I got the idea from this page: https://inspectit-performance.atlassian.net/wiki/spaces/DOC17/pages/54657166/JBoss+6.x

Updated Information:

My full set of JAVA_OPTS are these:

-server -Xms2048m -Xmx2048m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m 
-Djava.net.preferIPv4Stack=true 
-Djava.awt.headless=true 
-Djboss.https.port=8443 
-Djboss.http.port=8080 
-Djboss.bind.address=0.0.0.0 
-Djboss.server.log.dir=/home/app/logs 
-agentlib:jdwp=transport=dt_socket,address=8787,server=y,suspend=n 
-DconfigurationLocationStrategy=filesystem 
-Djava.util.logging.manager=org.jboss.logmanager.LogManager 
-Djboss.modules.system.pkgs=org.jboss.byteman,org.jboss.logmanager 
-Xbootclasspath/p:/home/app/wildfly/modules/system/layers/base/org/jboss/logmanager/main/jboss-logmanager-2.0.4.Final.jar 
-Xbootclasspath/p:/home/app/wildfly/jboss-modules.jar 
-javaagent:/opt/prometheus/jmx_prometheus_java_agent.jar=31500:/opt/prometheus/prometheus.yaml

It appears the need for doing all of the logmanager properties arose from the fact that I have deployed a prometheus agent with my container.

If I don't deploy the prometheus agent, I don't need to add any of the logmanager parameters to the JAVA_OPTS. Unfortunately, not deploying this agent is not an option.