"Could not add subscription" error when WSO2 API Manager starts

Question

I have a WSO2 API Manager 2.5.0 installation, deployed in two nodes as active-active. It also has WSO2 Analytics and WSO2 Identity Server as Key Manager.

When each node starts, it leaves three times the same error in logs:

2018-11-14 07:56:50,989 [-] [DisruptorInboundEventThread-8] ERROR AndesSubscriptionManager Could not add subscription: subscriptionId=3091b9ba-867b-4539-bf25-fee511d1813d,storageQueue=AMQP_Topic_throttledata_NODE:OpenDataApiM1/10.0.0.101,protocolType=AMQP,isActive=true,subscriberConnection=Y29ubmVjdGVkSVA9LzEwLjAuMC40OjMzMTgyLzEsY29ubmVjdGVkTm9kZT1OT0RFOk9wZW5EYXRhQXBpTTEvMTAuMC4wLjQscHJvdG9jb2xDaGFubmVsSUQ9Y2Y3NDI4MDgtZGY3MS00NzJhLWFiMTEtOTY1Nzc2ZTBkNTZl
org.wso2.andes.kernel.subscription.SubscriptionException: StorageQueue: AMQP_Topic_throttledata_NODE:OpenDataApiM1/10.0.0.101 is not registered while creating subscription id=3091b9ba-867b-4539-bf25-fee511d1813d
    at org.wso2.andes.kernel.subscription.AndesSubscription.<init>(AndesSubscription.java:136)
    at org.wso2.andes.kernel.subscription.AndesSubscriptionManager.reloadSubscriptionsFromStorage(AndesSubscriptionManager.java:921)
    at org.wso2.andes.kernel.disruptor.inbound.InboundDBSyncRequestEvent.reloadSubscriptions(InboundDBSyncRequestEvent.java:208)
    at org.wso2.andes.kernel.disruptor.inbound.InboundDBSyncRequestEvent.updateState(InboundDBSyncRequestEvent.java:76)
    at org.wso2.andes.kernel.disruptor.inbound.InboundEventContainer.updateState(InboundEventContainer.java:268)
    at org.wso2.andes.kernel.disruptor.inbound.StateEventHandler.onEvent(StateEventHandler.java:70)
    at org.wso2.andes.kernel.disruptor.inbound.StateEventHandler.onEvent(StateEventHandler.java:40)
    at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
2018-11-14 07:56:50,991 [-] [DisruptorInboundEventThread-8] ERROR AndesSubscriptionManager Could not add subscription: subscriptionId=c9a279eb-6c1f-447c-84bc-c077d33a06e1,storageQueue=AMQP_Topic_throttledata_NODE:OpenDataApiM1/10.0.0.101,protocolType=AMQP,isActive=true,subscriberConnection=Y29ubmVjdGVkSVA9LzEwLjAuMC40OjM0NTA0LzEsY29ubmVjdGVkTm9kZT1OT0RFOk9wZW5EYXRhQXBpTTEvMTAuMC4wLjQscHJvdG9jb2xDaGFubmVsSUQ9NGI5YmVmZmUtNjk1OC00M2Q4LTg3NGYtMzA5YmE5M2IyNzMw
org.wso2.andes.kernel.subscription.SubscriptionException: StorageQueue: AMQP_Topic_throttledata_NODE:OpenDataApiM1/10.0.0.101 is not registered while creating subscription id=c9a279eb-6c1f-447c-84bc-c077d33a06e1
    at org.wso2.andes.kernel.subscription.AndesSubscription.<init>(AndesSubscription.java:136)
    at org.wso2.andes.kernel.subscription.AndesSubscriptionManager.reloadSubscriptionsFromStorage(AndesSubscriptionManager.java:921)
    at org.wso2.andes.kernel.disruptor.inbound.InboundDBSyncRequestEvent.reloadSubscriptions(InboundDBSyncRequestEvent.java:208)
    at org.wso2.andes.kernel.disruptor.inbound.InboundDBSyncRequestEvent.updateState(InboundDBSyncRequestEvent.java:76)
    at org.wso2.andes.kernel.disruptor.inbound.InboundEventContainer.updateState(InboundEventContainer.java:268)
    at org.wso2.andes.kernel.disruptor.inbound.StateEventHandler.onEvent(StateEventHandler.java:70)
    at org.wso2.andes.kernel.disruptor.inbound.StateEventHandler.onEvent(StateEventHandler.java:40)
    at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
2018-11-14 07:56:50,992 [-] [DisruptorInboundEventThread-8] ERROR AndesSubscriptionManager Could not add subscription: subscriptionId=3091b9ba-867b-4539-bf25-fee511d1813d,storageQueue=AMQP_Topic_throttledata_NODE:OpenDataApiM1/10.0.0.101,protocolType=AMQP,isActive=true,subscriberConnection=Y29ubmVjdGVkSVA9LzEwLjAuMC40OjMzMTgyLzEsY29ubmVjdGVkTm9kZT1OT0RFOk9wZW5EYXRhQXBpTTEvMTAuMC4wLjQscHJvdG9jb2xDaGFubmVsSUQ9Y2Y3NDI4MDgtZGY3MS00NzJhLWFiMTEtOTY1Nzc2ZTBkNTZl
org.wso2.andes.kernel.subscription.SubscriptionException: StorageQueue: AMQP_Topic_throttledata_NODE:OpenDataApiM1/10.0.0.101 is not registered while creating subscription id=3091b9ba-867b-4539-bf25-fee511d1813d
    at org.wso2.andes.kernel.subscription.AndesSubscription.<init>(AndesSubscription.java:136)
    at org.wso2.andes.kernel.subscription.AndesSubscriptionManager.reloadSubscriptionsFromStorage(AndesSubscriptionManager.java:972)
    at org.wso2.andes.kernel.disruptor.inbound.InboundDBSyncRequestEvent.reloadSubscriptions(InboundDBSyncRequestEvent.java:208)
    at org.wso2.andes.kernel.disruptor.inbound.InboundDBSyncRequestEvent.updateState(InboundDBSyncRequestEvent.java:76)
    at org.wso2.andes.kernel.disruptor.inbound.InboundEventContainer.updateState(InboundEventContainer.java:268)
    at org.wso2.andes.kernel.disruptor.inbound.StateEventHandler.onEvent(StateEventHandler.java:70)
    at org.wso2.andes.kernel.disruptor.inbound.StateEventHandler.onEvent(StateEventHandler.java:40)
    at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

I have no idea about the problem. I have looked for the error on the internet, but there are no similar cases. Does anyone have any ideas?

Answer 1

2018-11-14 07:56:50,989 [-] [DisruptorInboundEventThread-8] ERROR AndesSubscriptionManager Could not add subscription: subscriptionId=3091b9ba-867b-4539-bf25-fee511d1813d,storageQueue=AMQP_Topic_throttledata_NODE:OpenDataApiM1/10.0.0.101,protocolType=AMQP,isActive=true,subscriberConnection=Y29ubmVjdGVkSVA9LzEwLjAuMC40OjMzMTgyLzEsY29ubmVjdGVkTm9kZT1OT0RFOk9wZW5EYXRhQXBpTTEvMTAuMC4wLjQscHJvdG9jb2xDaGFubmVsSUQ9Y2Y3NDI4MDgtZGY3MS00NzJhLWFiMTEtOTY1Nzc2ZTBkNTZl
org.wso2.andes.kernel.subscription.SubscriptionException: StorageQueue: AMQP_Topic_throttledata_NODE:OpenDataApiM1/10.0.0.101 is not registered while creating subscription id=3091b9ba-867b-4539-bf25-fee511d1813d

You might have probably found out the answer for this error. But the possible cause for the issue is when the APIM is unable to identify the correct topic connection factory related to its own node due to the sharing of the 'WSO2_MB_STORE_DB' between the nodes.

If you refer to their Installing and configuring databases documentation, it has mentioned that each Traffic Manager node must have its own Message Broker database (WSO2_MB_STORE_DB).

Therefore, please check the database configurations under the <APIM_Home>/repository/conf/master-datasource.xml file in both nodes whether you have shared the 'WSO2_MB_STORE_DB' between the nodes. If that is the case, you can point them to two separate fresh databases to get rid of this error.