Hazelcast Logging 70% threshold Message with out adding anything to map

Question

I am using hazel cast in a 2 node cluster following is my configuration:

<hz:hazelcast id="hazelcast_instance">
        <hz:config>
            <hz:instance-name>hazelcastinstance</hz:instance-name>
            <hz:group name="cluster" password="asdfg" />
            <hz:properties>
                <hz:property name="hazelcast.health.monitoring.level">SILENT</hz:property>

            </hz:properties>
            <hz:network port="5710" port-auto-increment="false">
                <hz:join>
                    <hz:multicast enabled="false" multicast-group="224.2.2.5"
                        multicast-port="54327" />
                    <hz:tcp-ip enabled="true">
                        <hz:members>${cluster.hzmembers}</hz:members>
                    </hz:tcp-ip>
                </hz:join>
                <hz:interfaces enabled="false">
                    <hz:interface>192.168.1.*</hz:interface>
                </hz:interfaces>
            </hz:network>


            <hz:map name="cluster.map" max-size="100" max-idle-seconds="55"
                time-to-live-seconds="55"  eviction-policy="LRU" eviction-percentage="50"/>
        </hz:config>

    </hz:hazelcast>

I am just using the map to check the partition , so that i can run only one instance of batch in a cluster:

hz.getPartitionService().getPartition("cluster.map").getOwner().localMember()

But time and again in the log i see messages like

[cluster] memory.used=1.8G, memory.free=583.9M, memory.total=2.4G, memory.max=2.4G, memory.used/total=76.07%, memory.used/max=76.07%, load.process=-1.00%, load.system=-1.00%, load.systemAverage=13.00%, thread.count=147, thread.peakCount=158, event.q.size=0, executor.q.async.size=0, executor.q.client.size=0, executor.q.operation.size=0, executor.q.query.size=0, executor.q.scheduled.size=0,

Not sure what is filling up the hazelcast cluster memory.

Any dvice will be highly appreciated.

I am using Hazelcasr version 3.1.7

Answer 1

Just turn OFF the healthmonitor thread by passing the configuration property hazelcast.health.monitoring.level=OFF

com.hazelcast.internal.monitors.HealthMonitor holds a hard reference to com.hazelcast.spi.impl.proxyservice.impl.ProxyServiceImpl thereby causing a'proxies' Map object from being collected and eventually allowing it to grow out of bounds.

Answer 2

Please upgrade to newest version of Hazelcast; 3.3.x. In 3.1/3.2 there were some issues that could lead to a memory leakage; perhaps you are running into one of these issues.

I don't know what is consuming your memory. I normally take a heapdump and analyse it. But first try upgrading.

Answer 3

Use the below configuration to disable the log. Config config = new Config(); then,

config.setProperty("hazelcast.logging.type", "log4j2");
config.setProperty("hazelcast.health.monitoring.level", "OFF");

It works for me.