Isha's Writings: 2013

Monday, September 16, 2013

Securing a Proxy Service in WSO2 ESB using a Username and passwordDigest (Hashpassword)

In this post i am going to explain how to secure a simple proxy service in WSO2 ESB 4.7.0 to be authenticated with UsernameToken method with using the username and hashpassword.

The HashPassword or NoPassword options are defined with WS-Policy 1.2 specification onwards. Therefore in order to have this feature we need to write a policy file using WS-Policy 1.2 spec.

First let's create a simple pass through proxy and make it secured.

1. Add a new pass through proxy and specify an endpoint of a running service there. I have used SimpleStockQuoteService which can be run in <ESB_Home>/samples/axis2Server/src/SimpleStockQuoteService. Build the service using 'ant' and start the Axis2 server by,

cd ESB_home/samples/axis2Server $ sh axis2Server.sh

2. After this go to the list of service in the UI. You will see that 'StockQuoteSecure' service is displayed as "Unsecured". Click on this link and it will redirect to a page where you can enable security to this service. Enable security there and select UsernameToken as the basic authentication mechanism as shown in the image below. After that goto next page and select the user groups who can access this service.

3. Once you finish this, WSO2 ESB admin console will display the service as secured, and we can only invoke this service using https:// endpoint now.

4. Next we need to change a default UsernameToken policy of the service and make it able to validate Hashpasswords. Goto the service dashboard of the secured service by clicking on it. Under the 'Quality of Service Configuration' section 'Policies' will be defined.

5. When you click on 'Policies' link it will be redirected to edit the current policy. Find 'StockQuoteSecureSoap11Binding' tab in here and click on 'Edit Policy' button as shown below.

6. Now let's define our new policy here. Copy the Following policy configuration and replace the existing policy definition with this one.

Save the policy back.



    
        
            
                
                    
                        
                    
                
                
                    
                        
                    
                
                
                    
                        
                    
                
                
            
        
        
            
                
                    
                        
                        
                    
                
            
        
        
            useReqSigCert
	    admin
	    org.wso2.carbon.digestpwd.PWCBHandler

7. The StockQuoteSecure service is secured now and is configured to use Username and Hashpassword for authentication.

8. As defined in the policy configuration, it uses org.wso2.carbon.digestpwd.PWCBHandler class to validate the user. Here i have written PWCBHandler.java class which can validate the default 'admin' user of WSO2 ESB. Before trying to invoke the proxy service we need to add this PWCBHandler-1.0.jar client library into <ESB_Home>/repository/components/lib directory. You can download the jar from here.

[ The above sample PWCBHandler class is written to validate the 'admin' user only. According to your requirements you can write a customized PasswordCallBackHandler class which validates a set of registered users etc. in similar manner. ]

9. Now we can invoke the secured service using a Client, and i have used SOAP UI as the sample client here. Create a new SoapUI project by using SimpleStockQuoteService.wsdl file attached here. Then use any of the operations defined in the SimpleStockQuoteService and send a request to oure secured proxy service using SoapUI. Before sending the request enter the following values as Request Properties.

Username: admin

Password: admin

WSS-Password Type: PasswordDigest

WSS Time to Live: 2000

10. Once sent the request there will be a response message returned from SimpleStockQuoteService which implies that the Usename/Hashpassword combination is authenticated successfully.

Saturday, August 17, 2013

Using WSO2 Message Broker with RabbitMQ C# .Net Client

As WSO2 Message Broker supports AMQP (Advanced Message Queuing Protocol) v 0.91 standard, this can be used to provide interoperability between different platforms via AMQP clients written on different languages. This blog post is about how to connect with WSO2 MB 2.1.0 using a C# .Net client and perform Queue/Topic based operations on Message Broker.

In order to connect with MB first we need to find a C# .Net client library which supports AMQP standards. I have used RabbitMQ .Net client library here for this purpose.

1. How to Publish/Subscribe to Queues via .Net client

The following tutorial in WSO2 Library provides a complete guide on how to perform Queue operations in by connecting to WSO2 MB via the .Net client. Therefore i am not going to repeat it here and let's see how to use manipulate Topics with a C# .Net client.

2. How to Publish/Subscribe to Topics via .Net client

The main difference between connecting to a Topic and a Queue is, by default in WSO2 MB they use two different exchanges in order to connect with Queue and Topics. Therefore if you need to connect to a Queue/Topic, first it needs to be bound to the 'Exchange' in the broker by specifying the exchange name. There are two default exchanges used in WSO2 Message Broker.

'amq.direct' : The pre-defined direct exchange, used with Queues
'amq.topic' : The pre-defined topic exchange, used with Topics

You can still define and create new exchanges if needed, however they need to be first added into <MB_Home>/Repository/conf/advanced/qpid-virtualhosts.xml file under the <exchanges> section as the example below. However it is usually recommended to use the default direct/topic exchanges.

       
    
        
           direct
           carbon.direct
           true
       
       
            topic
            carbon.topic

Now let's get back to writing our sample code.

I. The following is a sample client code to publish into a Topic in the broker. The code itself includes self-explanatory comments which describe its functions.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using RabbitMQ.Client;
 
 
namespace MB_Topic_Publisher
 
{
 
    class TopicPublisher
 
    {
        static void Main(string[] args)
 
        {
            TopicPublisher topicPublisher = new TopicPublisher();
            topicPublisher.PublishMessage("Test Message");
            Console.WriteLine("Message Sent..");
            Console.ReadLine();
        }
 
        public void PublishMessage(string message)
 
        {
 
//Setup the connection with the message broker
            ConnectionFactory factory = new ConnectionFactory();
            IProtocol protocol = Protocols.AMQP_0_8_QPID;
            factory.VirtualHost = "/carbon";
            factory.UserName = "admin";
            factory.Password = "admin";
            factory.HostName = "localhost";
            factory.Port = 5672;
            factory.Protocol = protocol;
 
            using (IConnection conn = factory.CreateConnection())
 
            {
                using (IModel ch = conn.CreateModel())
                {
 
// Declare a topic exchange to publish messages, here we have used the default topic exchange of WSO2 MB
                 ch.ExchangeDeclare("amq.topic", "topic");
 
// Publish the message to the exchange, it will send it to the routing key which is our name 'myTopic'.
// The syntax is ch.BasicPublish(exchange_name, topic_name, message_properties,message_body)
                 ch.BasicPublish("amq.topic", "test-topic", null, Encoding.UTF8.GetBytes(message));
 
                }
            }
        }
    }
}

II. Then let's write the sample client code to consume messages from a Topic in the broker.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using RabbitMQ.Client;
 
namespace MB_TopicClient
 
{
    class TopicConsumer
    {
        static void Main(string[] args)
       {
            TopicConsumer topicConsumer = new TopicConsumer();
            topicConsumer.getMessages();
        }
 
        public void getMessages()
        {
 
//Setup the connection with the message broker
            ConnectionFactory factory = new ConnectionFactory(); 
            IProtocol protocol = Protocols.AMQP_0_8_QPID;
            factory.VirtualHost = "/carbon";
            factory.UserName = "admin";
            factory.Password = "admin";
            factory.HostName = "localhost";
            factory.Port = 5672;
            factory.Protocol = protocol;
 
            using (IConnection conn = factory.CreateConnection())
            {
                using (IModel ch = conn.CreateModel())
 
                {
// Declare a topic exchange to be bound to retrieve messages, here we have used the default topic exchange of WSO2 MB
                    ch.ExchangeDeclare("amq.topic", "topic");
             
// Declare a topic name, here we use a non-durable topic. To make it durable use the 2nd parameter as 'true' 
                    ch.QueueDeclare("test-topic", false, false, false, null);
  
// Bind the Topic in to the exchange
                    ch.QueueBind("test-topic", "amq.topic", "test-topic");
                     
// Declare a consumer which listens on the messages published to 'test-topic' topic, we need to declare an exclusive subscriber, in order to get this work.
// The syntax is BasicConsume(queuename, noAck,consumerTag, noLocal, exclusive, arguments, Consumer)                      
                     
                    QueueingBasicConsumer consumer = new QueueingBasicConsumer(ch);
 
                    ch.BasicConsume("test-topic", false, "1", false, true, null, consumer);
                    while (true)
                    {
                        try
                        {
                            RabbitMQ.Client.Events.BasicDeliverEventArgs e =(RabbitMQ.Client.Events.BasicDeliverEventArgs)consumer.Queue.Dequeue();
                            byte[] body = e.Body;
                            string message = Encoding.UTF8.GetString(body);
                            Console.WriteLine(message);
                            ch.BasicAck(e.DeliveryTag, false);
 
                        }
                        catch (OperationCanceledException e)
                        {
                            Console.WriteLine(e);
                            break;
                        }
                     }
                }
            }
        }
    }
}

3. Download and Running the sample

To run the clients first run the sample Topic Consumer where it will create a new topic subscription for the 'myTopic'. After the when you run the sample Topic Publisher client, the consumer will receive the message published and will display it in the console as "Test Message".

The complete source code along with the RabbitMQ.Client.dll library can be downloaded from here too.

4. A guide to RabbitMQ .Net library API

If you need to learn on how to use the API to customize the sample Publisher and Consumer before using it with WSO2 Message Broker, refer the guides given below.

RabbitMQ .Net client v2.8.4 API Guide : find here
RabbitMQ .Net client v2.8.4 User Guide : find here

Sunday, July 28, 2013

Writing a Node.js Client to Send/Receive messages with WSO2 Messge Broker

WSO2 Message Broker supports AMQP protocol 0-9-1. If you need to use Node.js client to publish/subscribe to Message Broker this can be done with using any compatible Node.js AMQP 0-9-1 Client Library. Some of the examples i found are,

amqp.node : https://github.com/squaremo/amqp.node
node-amqp : https://github.com/postwait/node-amqp

First of all we need to install Node.js if we haven't done it yet. The following page explains how to install Node.js in Ubuntu.

$ sudo apt-get install python-software-properties python g++ make

$ sudo add-apt-repository ppa:chris-lea/node.js

$ sudo apt-get update $ sudo apt-get install nodejs

Then add 'amqplib' module to get enable the functions in 'node.amqp' library.

$ npm install amqplib

The following sample code, written using amqp.node library can be used now as a NodeJS client to publish or receive messages from WSO2 Message Broker. You have to use the format amqp://{username}:{password}@{hostname}:{port} to establish a connection with Message Broker. All messages will be sent as byte messages but can be received as text.

'amqp.node' library provide a rich API which can be used to other Queue operations MB too.

A Sample Node.js Queue Publisher for WSO2 MB

    
    var queuename = 'MyQueue'; 

    var openConn = require('amqplib').connect('amqp://admin:admin@localhost:5672'); 

    // amqp://{username}:{password}@{hostname}:{port} is default AMQP connection URL of WSO2 MB 

    openConn.then(function(conn) { 

    var ok = conn.createChannel(); 

    ok = ok.then(function(channel) 

    { channel.assertQueue(queuename); 

    channel.sendToQueue(queuename, new Buffer('New Message')); }); 

    return ok; 

    }).then(null, console.warn);

A Sample Node.js Queue Consumer for WSO2 MB

var queuename = 'MyQueue'; 

    var openConn = require('amqplib').connect('amqp://admin:admin@localhost:5672'); 

    // amqp://{username}:{password}@{hostname}:{port} is default AMQP connection URL of WSO2 MB 

    openConn.then(function(conn) { 

    var ok = conn.createChannel(); 

    ok = ok.then(function(channel) { 

           channel.assertQueue(queuename); 

    channel.consume(queuename, function(msg) { 

    console.log(msg.content.toString()); 

    channel.ack(msg); }); 

    }); 

    return ok; 

    }).then(null, console.warn);

Thursday, July 25, 2013

Providing Dynamic Queue Support when Intergating WSO2 Message Broker with Other Products

WSO2 Message Broker can be integrated with many other WSO2 products like WSO2 Enterprise Service Bus, Application Server and Data Services Server etc. via JMS transaport. Previously, when starting the integrated product (say WSO2 Application Server) after connecting with the Message Broker, the following exception might happen continuously.

ERROR {org.apache.axis2.transport.base.threads.NativeWorkerPool} - Uncaught exception
java.lang.UnsupportedOperationException: The new addressing based sytanx is not supported for AMQP 0-8/0-9 versions
at org.wso2.andes.client.AMQSession_0_8.handleAddressBasedDestination(AMQSession_0_8.java:572)
at org.wso2.andes.client.AMQSession.registerConsumer(AMQSession.java:2838)
at org.wso2.andes.client.AMQSession.access$500(AMQSession.java:117)
at org.wso2.andes.client.AMQSession$4.execute(AMQSession.java:2031)
at org.wso2.andes.client.AMQSession$4.execute(AMQSession.java:1997)
at org.wso2.andes.client.AMQConnectionDelegate_8_0.executeRetrySupport(AMQConnectionDelegate_8_0.java:305)
at org.wso2.andes.client.AMQConnection.executeRetrySupport(AMQConnection.java:621)
at org.wso2.andes.client.failover.FailoverRetrySupport.execute(FailoverRetrySupport.java:102)
at org.wso2.andes.client.AMQSession.createConsumerImpl(AMQSession.java:1995)
at org.wso2.andes.client.AMQSession.createConsumer(AMQSession.java:993)
at org.apache.axis2.transport.jms.JMSUtils.createConsumer(JMSUtils.java:642)
at org.apache.axis2.transport.jms.ServiceTaskManager$MessageListenerTask.createConsumer(ServiceTaskManager.java:871)
at org.apache.axis2.transport.jms.ServiceTaskManager$MessageListenerTask.getMessageConsumer(ServiceTaskManager.java:741)
at org.apache.axis2.transport.jms.ServiceTaskManager$MessageListenerTask.receiveMessage(ServiceTaskManager.java:498)
at org.apache.axis2.transport.jms.ServiceTaskManager$MessageListenerTask.run(ServiceTaskManager.java:420)
at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

The reason for this annoying error log is, when WSO2 MB is integrated with some other product, at the server start up it tries to create a Queue in MB for each deployed service in other product (WSO2 AS in this case), with the corresponding service name. However since there was no dynamic queue support previously, it refers the 'jndi.properties' file in <Carbon_Home>/Repository/conf folder and see whether a queue with given service name is defined. When it is not defined there, the server will be throwing following exception.

With the current dynamic queue support there is no need to define the queue/topic names as it will be not limited to a 'jndi.properties' file and all the queues for available services will be created on the fly. This feature is supported via Axis2-JMS-Transport and WSO2 ESB 4.6.0, AS 5.1.0 and DSS 3.0.1 uses axis2-transport-jms_1.1.0.wso2v7 version. Since the jms transport version, axis2-transport-jms_1.1.0.wso2v8 which facilitates dynamic queues has not been released yet, to have these changes you need to replace the axis2-transport-jms library in <Carbon_Home>/Repository/components/plugins with axis2-transport-jms_1.1.0.wso2v7.jar file added in this location. However the new versions of WSO2 products will be shipped with axis2-transport-jms_1.1.0.wso2v8 version and there afterwards there is no need to do the above replacement.

The following are the new versions which will facilitate dynamic queue support with WSO2 MB by default.

Note: According to the Message Broker roadmap, 'Dynamic Queue Support' will be officially provided with MB 3.0.0 version which will be released in early next year. Therefore this is not the final verified implementation of this feature, however

WSO2 ESB 4.7.0 : Already Released, You can go to this doc to see the integration guide.

WSO2 AS 5.2.0 : Will be released soon, find the integration guide for AS 5.1.0 here.

WSO2 DSS 3.0.2 : Will be released soon, find the integration guide for DSS 3.0.1 here.

Saturday, July 20, 2013

Avoiding javax.management.InstanceAlreadyExistsException when starting WSO2 Message Broker in Clustering Mode

WSO2 Message Broker can be set up in a clustered environment according to 5 different patterns. The broker nodes coordination happens through Apache ZooKeeper and when in clustered mode and we can configure WSO2 MB to use either an external ZooKeeper server or the built-in ZooKeeper server which is shipped with MB by default.

When starting a broker node in a cluster with using built-in ZooKeeper server as described in this pattern you might be facing with the following error sometimes.

TID: [0] [MB] [2013-07-17 20:43:55,894] INFO {org.wso2.carbon.coordination.

server.CoordinationServer} - Starting Coordination server in clustered mode... {org.wso2.carbon.coordination.server.CoordinationServer}
TID: [0] [MB] [2013-07-17 20:43:55,916] ERROR {org.apache.log4j.jmx.AppenderDynamicMBean} - Could not add DynamicLayoutMBean for [CARBON_LOGFILE,layout=org.wso2.carbon.utils.logging.TenantAwarePatternLayout]. {org.apache.log4j.jmx.AppenderDynamicMBean}
javax.management.InstanceAlreadyExistsException: log4j:appender=CARBON_LOGFILE,layout=org.wso2.carbon.utils.logging.TenantAwarePatternLayout
    at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:453)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.internal_addObject(DefaultMBeanServerInterceptor.java:1484)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:963)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
    at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:483)
    at org.apache.log4j.jmx.AppenderDynamicMBean.registerLayoutMBean(Unknown Source)
    at org.apache.log4j.jmx.AppenderDynamicMBean.preRegister(Unknown Source)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.preRegisterInvoke(DefaultMBeanServerInterceptor.java:1010)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:938)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
    at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:483)
    at org.apache.log4j.jmx.LoggerDynamicMBean.registerAppenderMBean(Unknown Source)
    at org.apache.log4j.jmx.LoggerDynamicMBean.appenderMBeanRegistration(Unknown Source)
    at org.apache.log4j.jmx.LoggerDynamicMBean.postRegister(Unknown Source)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.postRegisterInvoke(DefaultMBeanServerInterceptor.java:1035)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:974)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
    at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:483)
    at org.apache.log4j.jmx.HierarchyDynamicMBean.addLoggerMBean(Unknown Source)
    at org.apache.log4j.jmx.HierarchyDynamicMBean.addLoggerMBean(Unknown Source)
    at org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:67)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:122)
    at org.wso2.carbon.coordination.server.CoordinationServer.run(CoordinationServer.java:78)
TID: [0] [MB] [2013-07-17 20:43:55,916] ERROR {org.apache.log4j.jmx.AppenderDynamicMBean} - Could not add DynamicLayoutMBean for [CARBON_LOGFILE,layout=org.wso2.carbon.utils.logging.TenantAwarePatternLayout]. {org.apache.log4j.jmx.AppenderDynamicMBean}
javax.management.InstanceAlreadyExistsException: log4j:appender=CARBON_LOGFILE,layout=org.wso2.carbon.utils.logging.TenantAwarePatternLayout
    at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:453)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.internal_addObject(DefaultMBeanServerInterceptor.java:1484)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:963)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
    at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:483)
    at org.apache.log4j.jmx.AppenderDynamicMBean.registerLayoutMBean(Unknown Source)
    at org.apache.log4j.jmx.AppenderDynamicMBean.preRegister(Unknown Source)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.preRegisterInvoke(DefaultMBeanServerInterceptor.java:1010)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:938)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
    at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:483)
    at org.apache.log4j.jmx.LoggerDynamicMBean.registerAppenderMBean(Unknown Source)
    at org.apache.log4j.jmx.LoggerDynamicMBean.appenderMBeanRegistration(Unknown Source)
    at org.apache.log4j.jmx.LoggerDynamicMBean.postRegister(Unknown Source)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.postRegisterInvoke(DefaultMBeanServerInterceptor.java:1035)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:974)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
    at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:483)
    at org.apache.log4j.jmx.HierarchyDynamicMBean.addLoggerMBean(Unknown Source)
    at org.apache.log4j.jmx.HierarchyDynamicMBean.addLoggerMBean(Unknown Source)
    at org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:67)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:122)
    at org.wso2.carbon.coordination.server.CoordinationServer.run(CoordinationServer.java:78)

This doesn't break any functionality of Message Broker however it would be annoying to see a long error log when the server starts up. The reason for this happening is Apache ZooKeeper enables JMX by default during the ZooKeeper server start up. With that it also tries to register new Log4j MBeans to manage log4j through JMX which causes the error.

To overcome this and start the server normally we can simply disable Log4j MBean registration of Zookeeper at the Message Broker start up by starting MB with using following command.

sh wso2server.sh -Dzookeeper.jmx.log4j.disable=true

This will pass the message to JVM that disable Log4j MBeans when starting ZooKeeper.

You can learn more on all the five different clustering patterns of WSO2 MB by visiting the Message Broker clustering documentation here.

Monday, July 15, 2013

Creating an OSGi Bundle out of a Third Party Library for Using as WSO2 Product Dependency

WSO2 platform supports a modular based architecture where third party libraries which are used in WSO2 products are exposed into the environment as OSGi bundles. This blog post is about how to OSGify a third party library, to be used as a dependency inside WSO2 MB.

We have recently used 'Disruptor', a high performance inter-thread messaging library as a dependency in WSO2 Message Broker for improving the performance by using disruptor based message writing operations into Cassandra storage. When doing this it is first needed to create an OSGi bundle out of 'Disruptor' library, so it can be referred in the runtime when Message Broker is running.

As common to all OSGi bundle generations, first you need to have a manifest.mf file which describes the bundle information, version info, which packages needed to be imported/exported etc. However as it is not easy to write this file correctly by hand we use maven-bundle-plugin in order to get this done. This process is common to any third party dependency and you can follow the same process in building an osgi bundle for any of them.

1. First let's create a directory named 'disruptor' inside <WSO2_Carbon_Source>/platform/dependencies/orbit/ directory. As we have used Disruptor 2.10.4 version i created a new package called 2.10.4-wso2v1 inside disruptor directory. (This is the common notation across the platform <LibraryVersion>-wso2v<VersionNumber>when naming the versions of the dependencies.)

2. Add a new maven build file (pom.xml) inside package 2.10.4-wso2v1.

3. Now let's add the required details which needs to generate the OSGi bundle into this file. This pom file can be used as a sample template by replacing the required entries when adding another dependency.

<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 ">http://maven.apache.org/maven-v4_0_0.xsd">

<parent>
   <groupId>org.wso2.carbon</groupId>
   <artifactId>carbon-dependents</artifactId>
   <version>4.1.0</version>
   <relativePath>../../../pom.xml</relativePath>
</parent>

<modelVersion>4.0.0</modelVersion>
<groupId>com.googlecode.disruptor.wso2</groupId>
<artifactId>disruptor</artifactId>
<packaging>bundle</packaging>
<name>WSO2 Carbon - Orbit - disruptor</name>
<version>2.10.4-wso2v1</version>
<description>This bundle exports packages from disruptor jar files</description>
<url>http://wso2.org</url>

<dependencies>
   <dependency>
      <groupId>com.googlecode.disruptor</groupId>
      <artifactId>disruptor</artifactId>
      <version>2.10.4</version>
      <optional>true</optional>
   </dependency>
</dependencies>

<build>
     <plugins>
            <plugin>
                 <groupId>org.apache.felix</groupId>
                 <artifactId>maven-bundle-plugin</artifactId>
                 <version>1.4.0</version>
                 <extensions>true</extensions>
                 <configuration>
                      <instructions>
                        <Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName>
                        <Bundle-Name>${project.artifactId}</Bundle-Name>
                        <Export-Package>
                                com.lmax.disruptor.*;version=2.10.4,
                         </Export-Package>
                        <Import-Package>
                                !sun.misc,
                                *
                        </Import-Package>
                      </instructions>
                </configuration>
            </plugin>
     </plugins>
</build>



<properties>
     <disruptor.build.version>2.10.4</disruptor.build.version>
     <disruptor.version>${disruptor.build.version}-wso2v1</disruptor.version>
<disruptor.orbit.version>${disruptor.build.version}.wso2v1</disruptor.orbit.version>
</properties>
</project>

Let's go trough the important entries in the file.

<dependencies>
<dependency>
<groupId>com.googlecode.disruptor</groupId>
<artifactId>disruptor</artifactId>
<version>2.10.4</version>
<optional>true</optional>
</dependency>
</dependencies>

As we are using external library in the OSGi bundle we need to add this as a maven dependency entry for the bundle. You can find maven dependency entries of lot of libraries in here.

<build>
<plugins>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<version>1.4.0</version>
<extensions>true</extensions>
<configuration>
<instructions>
<Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName>
<Bundle-Name>${project.artifactId}</Bundle-Name>
<Export-Package>
             com.lmax.disruptor.*;version=2.10.4,
</Export-Package>
<Import-Package>
             !sun.misc,
             *
</Import-Package>
</instructions>
</configuration>
</plugin>
</plugins>
</build>

This is how we use maven-bundle-plugin to generate the disruptor osgi bundle. The Export-Package, Import-Package and DynamicImport-Package tags are used to control the exposure of certain packages to other bundles. With <Export-Package> tag it will find the defined project classes or dependencies, and they will be copied to the osgi bundle. With using <Import-Package> here, we can import the extra classes that have been referred from our osgi bundle. By keeping * as the value here by default it imports all the reference classes that needs for OSGi bundle generation. However when using default value here for 'disruptor bundle' i encountered the following error.

org.osgi.framework.BundleException: The bundle could not be resolved. Reason: Missing Constraint: Import-Package: sun.misc; version="0.0.0"

So I added the entry '!sun.misc' to resolve this issue and it seems as the sun.misc package is already there to be referred by the osgi framework, we need to tell that it is not needed to explicitly import it in this case. This is a 'Disruptor' bundle specific entry and you don't have to add the same configuration in each bundle you create. However if the bundle's classes use sun.misc package, this setting might be useful in resolving similar errors.

4. That's it! Save the pom file and build it with maven ( WSO2 projects use Maven3). You will see that in the <WSO2_Carbon_Source>/platform/dependencies/orbit/disruptor/2.10.4-wso2v1/target folder the newly created osgi bundle 'disruptor-2.10.4-wso2v1.jar' is present.

If you see the manifest.mf file of the new bundle it will be as follows.

Manifest-Version: 1.0
Export-Package: com.lmax.disruptor.collections;version="2.10.4",com.lm
ax.disruptor;uses:="com.lmax.disruptor.util";version="2.10.4",com.lma
x.disruptor.dsl;uses:="com.lmax.disruptor,com.lmax.disruptor.util";ve
rsion="2.10.4",com.lmax.disruptor.util;uses:="com.lmax.disruptor";ver
sion="2.10.4"
Ignore-Package: sun.misc
Built-By: <your_host_name>
Tool: Bnd-0.0.238
Bundle-Name: disruptor
Created-By: Apache Maven Bundle Plugin
Bundle-Version: 2.10.4.wso2v1
Build-Jdk: 1.6.0_30
Bnd-LastModified: 1373904351577
Bundle-ManifestVersion: 2
Bundle-Description: This bundle exports packages from disruptor jar fi
les
Bundle-SymbolicName: disruptor
Import-Package: com.lmax.disruptor;version="2.10.4",com.lmax.disruptor
.collections;version="2.10.4",com.lmax.disruptor.dsl;version="2.10.4"
,com.lmax.disruptor.util;version="2.10.4"

To know more about creating OSGi bundles with maven-bundle-plugin, you can read the following article in WSO2 Library.

[1]. http://wso2.com/library/tutorials/develop-osgi-bundles-using-maven-bundle-plugin

Wednesday, May 29, 2013

Configuring SSO session timeout in WSO2 Identity Server 4.5.0

One of the key features in WSO2 Identity Server is SAML2 based single sign on (SSO) feature. The default time period for a SSO session is 10 hours in the Identity Server versions so far and this default SSO Session Expire time value can not be changed according to the user needs. But from WSO2 IS 4.5.0 onwards the SSO session timeout can be configured at the server start time.

This configuration lies on <IS_HOME>/repository/conf/identity.xml file under the SSOService element. Add the following entry under that in order to explicitly add the SSO timeout.

<SSOService>
<SessionTimeout>120</SessionTimeout> // Add the expecting timeout value in seconds here.
</SSOService>

Save the configuration and restart the server. Now the SSO session will be timed out as the newly configured value and a user may need to re-authenticate once he logs back after this timeout. It is not a must to have this entry in 'identity.xml' and if a timeout is not configured, the SSO session will use the '10 hours (36000s)' default time out value.

After configuring the SSO Session Expire time you can use this article in order to setup Single Sign On with WSO2 Identity Server.

Adding a new function into WSO2 MB FE using JavaToWsdl approach - My notes

In most of the WSO2 products JavaToWsdl approach is used in adding new functionality. In here we first add the necessary new methods into the relevant service class in /platform/components and then generates the service wsdl for that. After that the service wsdl is added into the corresponding wsdl file in the /platform/service-stubs this stub is used in the front end UI components, in order to call the new method.

As an example let's add a new method to obtain the currently logged in user's name from the UI components.

WSO2 Message Broker's source code is mainly written in the /platform/components/andes component. The admin functionalities are coded in 'org.wso2.carbon.andes.admin' sub component under the AndesAdminService.java class.

First add the new method into the java class.

public String getCurrentUser(){

// ADD METHOD BODY HERE
        return userName.trim();
}

Due to authentication requirements in the WSO2 MB, you need to add the following entry into the relevant 'services.xml' file in the 'Resouces' package in the component.

<operation name="getCurrentUser">
     <parameter name="AuthorizationAction" locked="true">/permission/admin/configure</parameter>
</operation>

Now save the two files. Once we build the 'org.wso2.carbon.andes.admin' component back these changes will be available in the back end. But to use them in front end we need to edit the corresponding service wsdl file in the /platform/service-stubs.

For this goto <MB_HOME/Repository/conf/carbon.xml> file. Find the <HideAdminServiceWSDLs> entry. This is set 'true' by default, as we don't want to expose admin service details, but now we need to see the service wsdl file to see the new functions we added there. Hence make this into 'false'.

<HideAdminServiceWSDLs>false</HideAdminServiceWSDLs>

Build 'org.wso2.carbon.andes.admin' component using maven and replace org.wso2.carbon.andes.admin_4.1.2.jar file in the <MB_HOME/Repository/Components/Plugins> directory, with this newly built jar file found in /target folder.
(Note: Remember to rename the jar file according to notation)

Start the MB server back and goto ,
https://localhost:9443/services/AndesAdminService?wsdl

You will see the AndesAdminService wsdl file with the newly added changes. Now let's add these to the service-stub to be used in Front End.

The corresponding service-stub for 'org.wso2.carbon.andes.admin' component is, /platform/service-stubs/org.wso2.carbon.andes.stub/4.1.0

In here you will find the /resources/AndesAdminService.wsdl file. Copy the new changes from the wsdl file that is viewed in the browser, into this wsdl file. Some of the changes would be like,

<wsdl:message name="getCurrentUserRequest"><wsdl:part name="parameters" element="ns:getCurrentUser"/></wsdl:message>
<wsdl:message name="getCurrentUserResponse"><wsdl:part name="parameters" element="ns:getCurrentUserResponse"/></wsdl:message>   and more ....

After adding all the changes build the service-stub back using maven. Replace org.wso2.carbon.andes.stub_4.1.0.jar file in the <MB_HOME/Repository/Components/Plugins> directory, with this newly built jar file found in /target folder.

Now you can call this stub class from andes ui components and use the newly added 'getCurrentUser()' method as given below. The same procedure is to be followed whenever we add new functions into the code base.

AndesAdminServiceStub stub = UIUtils.getAndesAdminServiceStub(config, session, request);
String username = stub.getCurrentUser();

Providing Queue browing, message sending and purging features in WSO2 MB 2.1.0

WSO2 MB is a Message Broker which enables applications to exchange communications asynchronously or publish messages for timely access by many subscribers. For the past versions of WSO2 MB there was no support for sending sample messages from the admin console or viewing the messages in a queue. hence in WSO2 MB 2.1.0 onwards, we have added this functionality into the broker.

Therefore it is now possible to,

Send sample text messages to a queue in WSO2 MB
Browse the content in a queue using MB admin console
Purge a queue via admin console and make it empty in WSO2 MB as of the expectation of many users.

The following are some of the snapshots from the new view of the MB admin console. The 2.1.0 version is not yet released for public but will be releasing by end of this month.

1. New view of the 'Queue Menu'

2. New Queue Browser in WSO2 MB

3. New Message Sending UI in MB

4. New Queue purging option in MB

More details on the new features can be found in MB 2.1.0 documentation once it is released.

Monday, April 29, 2013

Deploying and Running WSO2 API Manager 1.3.1 in an Amazon EC2 instanace

This post is a quick guide on how to deploy WSO2 API Manager 1.3.1 in an AWS EC2 instance and running operations on it. You can also run API Manager behind an Elastic Load Balancer (ELB) in EC2 instance. The configuration procedure is same as described below, except the public DNS address to be used is the DNS of the ELB in that case.

Requirements: Amazon Web Services account
WSO2 API Manager 1.3.1

1. If you still don't have an AWS account create one as given here. In the account creation process you will need to specify a password and you will be given a Access Key ID and a Secret Access Key which will be needed later for you to connect to your AWS EC2 instance.

2. Using your credentials log in to AWS Management Console and open Amazon EC2 console.

https://console.aws.amazon.com/ec2/

3. Create a new Linux instance by selecting "Launch Instance". Create your new instance by following the steps given in this guide.
In doing this choose ' Ubuntu Server 12.04' as the Amazon Machine Image (AMI) type and 'M1 Medium' as the instance type. Specify relevant key-pair or create a new one. If you create a new key-pair, download this into your computer and make sure you remember where it is located in your machine. (You need this later in this post!)

Goto the location where you key-pair is located. Now we are going to connect to our Ubuntu server instance in EC2 using SSH.

Find the Public DNS address of your instance. In my case this is 'ec2-00-000-000-00.compute-1.amazonaws.com' (Note that i have changed the digits in my EC2 instance DNS here).

Enter the following command in the terminal.

ssh -i ishara.pem ubuntu@ec2-00-000-000-000.

compute-1.amazonaws.com

In here 'isarap.pem' is the keypair file that was downloaded when creating the EC2 instance.
ec2-00-000-000-000.compute-1.amazonaws.com is the public DNS address of your instance.

Most probably there will be 'permission denied' error when you try this the first time. This is because there is more permission given for your keypair file when it is created. We need to make to more access restricted. For that use,

chmod 400 ishara.pem

After that when you try to SSH you will be able to successfully connect to your instance in AWS EC2.You will now see the following message which means you are now connected.

Welcome to Ubuntu 12.04.1 LTS (GNU/Linux 3.2.0-36-virtual x86_64)

* Documentation: https://help.ubuntu.com/

System information as of Mon Apr 29 17:48:41 UTC 2013
System load: 0.08 Processes: 59
Usage of /: 32.9% of 7.87GB Users logged in: 0
Memory usage: 32% IP address for eth0: 10.202.151.238
Swap usage: 0%

Graph this data and manage this system at https://landscape.canonical.com/ Get cloud support with Ubuntu Advantage Cloud Guest http://www.ubuntu.com/business/services/cloud

Use Juju to deploy your cloud instances and workloads. https://juju.ubuntu.com/#cloud-precise

Last login: Mon Apr 29 07:03:15 2013 from 203.143.18.194
ubuntu@ip-10-202-151-238:~$
.........................................................................................................

5. As now you are logged into ubuntu instance, consider this as a normal scenario where you work on your local computer. Make a new directory in EC2 instance as apim/ and copy API Manager 1.3.1.zip pack into this EC2 instance.

ubuntu@ip-10-202-151-238:~$ pwd
/home/ubuntu

ubuntu@ip-10-202-151-238:~$ mkdir apim/

Open another terminal and enter command,
scp -i ishara.pem wso2am-1.3.1.zip ubuntu@ec2-00-000-000-00.compute-1.amazonaws.com:/home/ubuntu/apim

This will be copying Am 1.3.1 distribution files into the apim/ directory in EC2 instance. Similarly copy JDK 1.6 pack as you needs it to run the API Manager in EC2.

6. Unzip both distributions now. If 'zip' is not installed you will first need to install it. Edit the .bashrc in /home/ubuntu directory and add the path to your JDK ad JAVA_HOME in it.

7. Unzip the API Manager 1.3.1 pack too. In order to start the API Manager server in an outside location we need to configure carbon.xml and api-manager.xml files in AM_HOME/repository/conf directory . The following guide explains how to do this in detail.

API Manager User Guide: http://dist2.wso2.org/products/api-manager/1.3.0/APIManager-GettingStarted-v1.3.0.pdf

In addition find the following entries in api-manager.xml and specify the values given below.

<KeyValidatorClientType>WSClient</KeyValidatorClientType>
<EnableThriftServer>false</EnableThriftServer>

..........................................................................................................
Note: You need to install 'curl' in EC2 instance to follow the steps in this guide. If there are issues occurs when trying with,

pubname=$(curl http://169.254.169.254/latest/metadata/publichostname 2>/dev/null);
export JAVA_OPTS=Damazon.pub.hostname=$pubname

In that case remove the above lines from .bashrc file and manually enter the 'EC2 instance public DNS url' into the places described in the configuration.

For example instead of adding,

<HostName>${amazon.pub.hostname}</HostName> in carbom.xml file you can enter as
<HostName>ec2-00-000-000-00.compute-1.amazonaws.com</HostName>

...............................................................................................................................

8. When all these configurations are over, start API Manager from /home/ubuntu/apim/wso2am-1.3.1/bin. Point your browser to,

https://ec2-00-000-00-00.compute-1.amazonaws.com:9443/carbon/

9. The WSO2 Admin Console for API Manager will be started! You are running AM 1.3.1 in AWS EC2 now :)

Isha's Writings