Avoiding javax.management.InstanceAlreadyExistsException when starting WSO2 Message Broker in Clustering Mode

WSO2 Message Broker can be set up in a clustered environment according to 5 different patterns. The broker nodes coordination happens through Apache ZooKeeper and when in clustered mode and we can configure WSO2 MB to use either an external ZooKeeper server or the built-in ZooKeeper server which is shipped with MB by default.

When starting a broker node in a cluster with using built-in ZooKeeper server as described in this pattern you might be facing with the following error sometimes.

TID: [0] [MB] [2013-07-17 20:43:55,894]  INFO {org.wso2.carbon.coordination.

server.CoordinationServer} -  Starting Coordination server in clustered mode... {org.wso2.carbon.coordination.server.CoordinationServer}
TID: [0] [MB] [2013-07-17 20:43:55,916] ERROR {org.apache.log4j.jmx.AppenderDynamicMBean} -  Could not add DynamicLayoutMBean for [CARBON_LOGFILE,layout=org.wso2.carbon.utils.logging.TenantAwarePatternLayout]. {org.apache.log4j.jmx.AppenderDynamicMBean}
javax.management.InstanceAlreadyExistsException: log4j:appender=CARBON_LOGFILE,layout=org.wso2.carbon.utils.logging.TenantAwarePatternLayout
    at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:453)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.internal_addObject(DefaultMBeanServerInterceptor.java:1484)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:963)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
    at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:483)
    at org.apache.log4j.jmx.AppenderDynamicMBean.registerLayoutMBean(Unknown Source)
    at org.apache.log4j.jmx.AppenderDynamicMBean.preRegister(Unknown Source)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.preRegisterInvoke(DefaultMBeanServerInterceptor.java:1010)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:938)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
    at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:483)
    at org.apache.log4j.jmx.LoggerDynamicMBean.registerAppenderMBean(Unknown Source)
    at org.apache.log4j.jmx.LoggerDynamicMBean.appenderMBeanRegistration(Unknown Source)
    at org.apache.log4j.jmx.LoggerDynamicMBean.postRegister(Unknown Source)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.postRegisterInvoke(DefaultMBeanServerInterceptor.java:1035)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:974)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
    at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:483)
    at org.apache.log4j.jmx.HierarchyDynamicMBean.addLoggerMBean(Unknown Source)
    at org.apache.log4j.jmx.HierarchyDynamicMBean.addLoggerMBean(Unknown Source)
    at org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:67)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:122)
    at org.wso2.carbon.coordination.server.CoordinationServer.run(CoordinationServer.java:78)
TID: [0] [MB] [2013-07-17 20:43:55,916] ERROR {org.apache.log4j.jmx.AppenderDynamicMBean} -  Could not add DynamicLayoutMBean for [CARBON_LOGFILE,layout=org.wso2.carbon.utils.logging.TenantAwarePatternLayout]. {org.apache.log4j.jmx.AppenderDynamicMBean}
javax.management.InstanceAlreadyExistsException: log4j:appender=CARBON_LOGFILE,layout=org.wso2.carbon.utils.logging.TenantAwarePatternLayout
    at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:453)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.internal_addObject(DefaultMBeanServerInterceptor.java:1484)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:963)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
    at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:483)
    at org.apache.log4j.jmx.AppenderDynamicMBean.registerLayoutMBean(Unknown Source)
    at org.apache.log4j.jmx.AppenderDynamicMBean.preRegister(Unknown Source)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.preRegisterInvoke(DefaultMBeanServerInterceptor.java:1010)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:938)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
    at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:483)
    at org.apache.log4j.jmx.LoggerDynamicMBean.registerAppenderMBean(Unknown Source)
    at org.apache.log4j.jmx.LoggerDynamicMBean.appenderMBeanRegistration(Unknown Source)
    at org.apache.log4j.jmx.LoggerDynamicMBean.postRegister(Unknown Source)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.postRegisterInvoke(DefaultMBeanServerInterceptor.java:1035)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:974)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
    at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:483)
    at org.apache.log4j.jmx.HierarchyDynamicMBean.addLoggerMBean(Unknown Source)
    at org.apache.log4j.jmx.HierarchyDynamicMBean.addLoggerMBean(Unknown Source)
    at org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:67)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:122)
    at org.wso2.carbon.coordination.server.CoordinationServer.run(CoordinationServer.java:78)

This doesn't break any functionality of Message Broker however it would be annoying to see a long error log when the server starts up. The reason for this happening is Apache ZooKeeper enables JMX by default during the ZooKeeper server start up. With that it also tries to register new Log4j MBeans to manage log4j through JMX which causes the error.

To overcome this and start the server normally we can simply disable Log4j MBean registration of Zookeeper at the Message Broker start up by starting MB with using following command.

sh wso2server.sh -Dzookeeper.jmx.log4j.disable=true

This will pass the message to JVM that disable Log4j MBeans when starting ZooKeeper. 

You can learn more on all the five different clustering patterns of WSO2 MB by visiting the Message Broker clustering documentation here.