Saturday, 12 September 2015

Deploying Mule to Websphere

This post describes the necessary stops to deploy a Mule application to WebSphere Application Server(In this case 7.x and Mule 3.3.x, both older version of each products so may need less/additional changes for newer versions) and some additional steps if you are using certain modules such as CXF.

Enable parent last class loading

Only do this at the MODULE level. Navigate to:

Application Types > WebSphere enterprise applications > swaggerProject_war > Modules > Manage Modules > myApp.war

Classes loaded with local class loader first (parent last) Apply and Save directly to the master configuration.

Exclude Servlet APIs and Geronimo Stax lib

Disable IBM JAXWS Engine

Modify the configuration of the maven-war-plugin to add a manifest entry to shut off activation of IBM WSAS' JAX-WS engine. (Add the following manifestEntries element to the maven-war-plugin configuration:

Add additional JVM/Endorsed dependencies

Thursday, 12 June 2014

Synchronizing data across Mule applications with MongoDB

One of the many things that Mule does extremely well, is transparently store state between messages. Take the idempotent-message-filter for example; The idempotent-message-filter keeps track of what messages have been processed previously to ensure no duplicate messages are processed. This is all achieved with one single xml element. Or take OAuth enabled cloud connectors that need to keep track of access tokens so you're not redirected to some authorize page each time. All this is handled behind the scenes without you having to worry about it.

Mule’s ability to store this state is implemented via object-stores. Object-stores provide a simple generic way to store data in Mule. Many message processors such as the idempotent-meesage-filter, OAuth message processors, until-successful routers and many more all transparently use object-stores behind the scenes to store state between messages.

By default Mule uses in-memory object-stores behind the scenes. Things get more interesting however, when your Mule application is distributed across multiple Mule nodes. At some point you are going to need to synchronise object-stores across multiple applications. If you are fortunate to be using the Enterprise Edition of Mule you can take advantage of its out of the box clustering support for synchronizing state across an entire cluster. But even then as mentioned in this blog post by John Demic: Synchronizing Mule Applications Across Data Centers with Apache Cassandra, this may not be ideal if your Mule nodes are geographically distributed across multiple data centres.

In the aforementioned post, John gives a great example of how to use Apache Cassandra as a shared object-store. But many other traditonal and NoSql data stores can be used as well. In this blog post I'm going to show a simple example of how you can achieve the same using MongoDB. If it's good enough for the Hipster Hacker then it's good enough for me!

Configuring the MongoDB object-store

MongoDB is an open-source document database, and the leading NoSQL database. The MongoDB module can be used to read/write/query and perform a whole load of other operation a MongoDB database as part of your Mule flows. It also exposes an additional submodule for using your MondogDB database as an object-store implementation in Mule. Here is a sample configuration from one of the tests:

As you can see, it's classed as it's own module with it's own namespace and is configured simply using the mos:config name="mos" database="mongo-connector-test" element.

However, this doesn't work for the current version of the connector. Due to the following bug. You can simply apply the fix in the aforementioned pull request, or you can instantiate the object store yourself using Spring - similar to that in the Apache Cassandra configuration.

The object-store class can be found here: As you can see, it has various annotations for default values and initialising the class. However, as we are not using this as a fully fledged Mule Devkit module, those annotations have no effect. So we have to pass in the default values ourselves and call the initialise method when instantiating our object-store.

This is quite simple with Spring. Here is a full working example using the Spring instantiated object-store as the store for the idempotent-message-filter:

If you run this configuration, Mule will poll every 10 seconds with the exact same message: "1". The first time it polls you should see "Passed Filter" in the log output. Subsequent polls should not log anything as the idempotent message filter will filter them out. This is usually performed with the default in-memory object-store. To prove this is persisted you should be able to check the value in the Mongo database collection: db.getCollection('mule.objectstore._default').find(). Also you can stop your Mule app and restart it to prove that it works between restarts and failovers etc.

Thursday, 29 May 2014

Getting Started with Mule Cloud Connect v1.2 is published

The O'Reilly book "Getting Started with Mule Cloud Connect"( has been out for just over a year now and has received some great feedback. But there's always room for improvement and with Mule 3.5 ready, I have updated the book to include some new 3.5 functionality and added some extra content.

The latest revision covers new Mule 3.5 features including new OAuth functionality and data synchronization. There is also a new chapter on orchestration - something that the original was missing for people new to Mule.

Already got it, and want the updates for free?

Good news is that if you already purchased the ebook edition then these updates are free and you can grab the new edition from O'Reilly.

Want a free copy?

If you would like to review the book on Amazon, O'Reilly or Google books let me know and I can send you a free review copy.

Want to buy a copy?

Buy it direct from O'Reilly and get free updates for the lifetime of the book:

Friday, 23 August 2013

Running Mule on Openshift - Part 2

A little while back I wrote a post on "Running Mule Natively on OpenShift" - This post showed some workarounds for a few conflicting "features" of both products and showed how to run a simple Hello World app. Since then both products have moved up major versions with both fixing the need for some of the previous work-arounds and also introducing some new ones. I have also had time to expand on a simple Hello World app and start trying out some other Mule components which also require some hacks to get up and running. In this article I'll do a quick recap of getting the app up and running and then show how to work with these new features.

HTTP Server Binding

In the previous post we tried to deploy the following app to Mule 3.3.1:

The only thing to take note of here is that we are using the ${OPENSHIFT_DIY_IP} environment variable which has changed from the previous ${OPENSHIFT_INTERNAL_IP}. This is the suggested IP address for creating socket connections on localhost. Typical values like "localhost", "" and "" are all locked down.

However, if you try using this environment variable as your host you will get an error similar to the following:

Permission denied ( (null) 2. Failed to bind to uri ""

After digging through the source, there is a slight issue with Mules' TCP transport for versions < 3.4.

Here, the internal IP is a loopback address, so Mule forces it down the path of creating a TCP socket that listens on all interfaces for that port. Fortunately there is already af fix in the 3.4 release which is now GA. If you want to use Mule versions < 3.4 then you will need to modify the TCP transport as detailed in the previous post.

HTTP Client Binding

The previous app just focused on exposing an inbound endpoint over HTTP and didn't look at using outbound endpoints. Take the following modified Hello World application:

Feeling a bit european, this modified app simply contacts an external HTTP API to request "Hello World" in Italian. Here, the http:outbound-endpoint shouldn't need to bind to any internal port as it's not necessary for client TCP connections. However, when running the application you will still get: Permission denied ( Under the hood, Mule uses the Apache commons-httpclient3 library for client connections which in turn uses uses it's DefaultSocketFactory class to create objects. The class has several different constructors, and commons-httpclient by default will call:

public Socket(String host, int port, InetAddress localAddr, int localPort) throws IOException...

Here, for the "localAddr" argument, null is passed. When null is passed for the "localAddr" argument, the will use InetAddress.anyLocalAddress(); This will perform a bind on the localAddr/port which is not necessary for client TCP connections, although under most circumstances is harmless. Except on OpenShift where this is specifically locked down and results in the being thrown.

This is rectified in later versions of the library which are now refactored into the Apache HttpComponents library and the HttpClient Inerfaces have changed signigicantly resulting in a non-trivial upgrade path. There is a solution however, with thanks to mikebennettjackbe - to create a custom SocketFactory to override the creation of the Socket and not bind on any local address:

To make commons-httpclient use the new OSProtocolSocketFactory instead of its own, we need to regiser it when he app starts up. One approach to this, is we can implement the MuleNotificationListener interface:

This class allows us to check when the Mule context is initialised and register the new Socket factory for the http and/or the https protocols. After creating the above class, you just need to define it as a bean within your Mule config, like so:

And that's it. You should now be able to use http outbound endpoints and use connectors that rely on the Apache commons-httpclient3 library such as the Twitter Cloud Connector. There are more than likely other transports or connectors that have issues around this area so feel free to comment or raise an issue or pull request on Github. Full project can be found here:

Monday, 4 March 2013

Natively Running Mule on OpenShift

In this article I'll show how you can run Mule natively on OpenShift wihout using a Servlet container and show you how I got over a few implementation hurdles.

If you are familiar with Mule you know it gives you many deployment options including both standalone deployment or embedding itself within a Java application or Webapp. The recommended approach is to run Mule ESB standalone from the command prompt, as a service or daemon, or from a script. This is the simplest architecture, so it reduces the number of points where errors can occur. It's typically best for performance as well, since it reduces the number of layers and eliminates the inherent performance impact of an application server on the overall solution. With Mule 3.x, you can also now run multiple applications side by side in a Mule instance using the new deployment model can support live deployment and hot redeployment of applications. Lastly, standalone mode has full support for the Mule High Availability module and the Mule management console.

OpenShift gives you many choices for developing and deploying applications in the cloud. You can pick among PHP, Ruby, Perl, Python, Node.js or Java. As Mule is Java based, we are pretty much covered. OpenShift provides an end to end Java application stack including: Java EE6, CDI/WeldSpring and Spring. You can choose between multiple application server's for Webapps including JBoss AS7, JBoss EAP6, Tomcat, and GlassFish. But if you want to run Mule natively in standalone mode for the aforementioned benefits, you will need to create a "DIY" cartridge/application.

A "DIY" application is just a barebones app, with no server preloaded, ready to be tailored to your needs. With this app type, OpenShift is begining to blur the line between an IaaS and a PaaS, providing you with a controlled and scalable environment, and at the same time giving you the freedom to implement the technology that best suits your needs.

Getting Started

Before attempting to create a DIY application on OpenShift, you should familiarize yourself with the technology you are about to use. You should have a clear understanding of the steps needed to set it all up on your workstation and then reproduce it on OpenShift.

For a Mule application we won't need JBoss, or any application sever, not any servlet container at all. We just have to install Mule and start it up.

Doing this on your own workstation is as easy as downloading Mule, unzipping it, and then running:

And to stop Mule:

Now we'll have to do the same on our server at OpenShift. First, let's create a new application named "mule":

Now let's see what we created. Running the following script:

Should output similar to he following:

You can browse to to see the default index page running. It's just the same static page you can find at raw/index.html

Now let's see what we have in our repo:

It's a pretty barebone app, but there's a folder that's quite interesting for us - .openshift/action_hooks:

These are the scripts that OpenShift uses for building, deploying, starting and stopping our app. These scripts are executed on the remote OpenShift server. These are the scripts we will need to ammend to download Mule and perform any configuration as well as starting and stopping our Mule Server. Let's take a look at a simplified version of the script that we used to install Mule.

Installing Mule


The pre_build script is used for downloading the required Mule installation and unzipping it.


Then to start the Mule server:


And to stop the Mule server:

Upgrading the Java Service Wrapper

When you run the mule command, it launches the mule.bat or script in your MULE_HOME/bin directory. These scripts invoke the Java Service Wrapper. The Java Service Wrapper from Tanuki Software is s fancy, little tool which helps with managing your application and JVM it is running in. By default it uses sockets to communicate back and forth with the JVM. But OpenShift is very restrcitive on what IP's and ports you are allowed to listen on.

By default the current Mule 3.3.1 release uses version 3.5.7 of the Java Service Wrapper. If you try running the default Mule instalation on OpenShift, you will get the following error:

"unable to bind listener to any port in the range 32000-32999. (Permission denied)"

The Java Service Wrapper is controlled by a wrapper.conf file that can be found in you MULE_HOME/conf directory and has a host of configuration of options, including setting the rang eof ports that the wrapper can listen on. Ports aside, OpenShift only allows applications to bind on a specific IP address via the environment variable OPENSHIFT_INTERNAL_IP. Unfortunately there is no configuration option to override this IP address. Game Over!

Extra Life! In a later version of the wrapper, there is a new configuration option: wrapper.backend.type=PIPE to allow you to avoid using sockets and use pipes instead to get around this problem.

To upgrade the wrapper we simply download the later wrapper libraries and replace them within the MULE_HOME/lib directory.


To update the wrapper.conf file with the new configuration. We take a copy of the original wrapper.conf file, ammended to contain the wrapper.backend.type=PIPE option and includ it within our git repo so that we can replace the original when building the instalation.


Deploying a Mule application

Deploying the application is as simple as copying a Mule application archive to the required appsdirectory:


Where is a simple Mule applicaion exposed over HTTP that returns "Hello World".

HTTP Binding

The only thing to take note of here is that we are using the ${OPENSHIFT_INTERNAL_IP} environment variable. This is the suggested IP address for creating socket connections on localhost. Typical values like "localhost", "" and "" are all locked down.

However, if you try using this environment variable as your host you will get an error similar to the following:

Permission denied ( (null) 2. Failed to bind to uri ""

As you can see; the internal IP resolves fine and we are using 8080 which is the suggested port for HTTP connections, but still no dice.

Hacking the TCP transport

After digging through the source, there is a slight issue with Mules' TCP transport.

Here, the internal IP is a loopback address, so Mule forces it down the path of creating a Socket that listens on all interfaces for that port. Fortunately there is already af fix in the upcoming 3.4 release - MULE-6584: HTTP/ TCP bound to listens on all interfaces.

Unfortunately, it's only upcoming at the moment. So instead I have ammended the source of this transport myself for the same functionality and included the resulting jar as part of my diy project to replace the orignal transport jar.


And that's it!

If you now take a look at your app at: you should now see "Hello World"! The full diy project for this with instructions can be found on GitHub:

Sunday, 3 February 2013

Cross-Origin Resource Sharing with Mule, AJAX and JavaScript

The Same-Origin Policy

The Same-Origin policy is a security policy enforced on client-side web apps to prevent interactions between resources from different origins. While useful for preventing malicious behaviour such as XSS(Cross Site Scripting) attacks, this security measure also prevents useful and legitimate interactions between known origins.

For example, your new awesome JavaScript mashup hosted at might want to use a REST API hosted at However, because these are two different origins from the perspective of the browser, the browser won't allow a script from to fetch resources from because the resource being fetched is from a different origin.

Cross-Origin Resource Sharing

Fortunately, there is a solution via Cross-Origin Resource Sharing(CORS). The CORS spec was developed by the World Wide Web Consortium (W3C) to support this very case. It's a working draft but is already supported by the majority of web browsers, probably including the very browser you are using to view this page. The full specification can be found at: and supported browsers can be found here:

How CORS works

CORS works via a set of HTTP headers in the request from the client app and the response from the requested resource. In it's simplest form; the requesting application specifies a Origin header in the request which describes the origin of the request and the requested resource will reply intern with an Access-Contol-Allow-Origin header indicating specific origins that are allowed to access a particular resource.

Request headers: Response headers:

There are more complicated scenarios that require additional HTTP headers when using non-simple HTTP headers. More information on this can be found here: For the purposes of this post we will just be using simple headers.

Using CORS with the Mule HTTP transport

To demonstrate CORS in action, I'll show a simple JavsScript client app using JQuery to access a simple HTTP service in Mule.

Simple JQuery Client

Simple HTTP Mule Flow

This is just a simple JQuery client and a simple HTTP Mule flow returning some plain text: "Hello World". The most important part here is the set-property element. Here we are setting the HTTP header to be returned in the response. Simple right? We have just set the value to "*" indicating that any origin is allowed. This can be configured as needed to include specific origins if you so desire.

Using CORS with the Mule AJAX transport

On top of the HTTP transport in Mule, there is also a specific AJAX transport. The Mule AJAX transport allows Mule events to be sent and received asynchronously to and from the web browser.

You might think that you would be able to se this property the same way. Unfortunately, no. Under the hood; the AJAX transport uses Jetty and the CometD libraries to provide the long-polling functionality and currently do not propagate HTTP headers set in Mule and instead set their own.

Never fear, there is a solution. It's a little more long winded, but still simple none the less. The solution relies on Jetty's configuration, which is used by the AJAX transport when running in embedded mode. This configuration can be overrided within your Mule application by provided a custom Jetty XML configuration file and creating custom Handler to add new HTTP headers.

Simple JQuery CometD client

To start let's amend the original client application to use CometD to subscribe to a channel in Mule.

Mule AJAX Flow

The Mule flow just polls every ten seconds and publishes a message to an AJAX outbound endpoint.

In addition to the standard AJAX connector configuration, we are injecting a reference to a custom jetty configuration file to register our CORS handler.

Jetty Configuration

This is just a simple jetty configuration file that we referenced in the previous Mule configuration to register our new custom Handler. The most important part here is the class reference that will be our new Handler to add the required headers: org.oreilly.mulecloudconnect.CORSHandler

Custom CORS Handler

And finally the last part to our CORS puzzle is the custom Handler itself. This class is an extension of the org.mortbay.jetty.handler.AbstractHandler class that gives us access to the Servlet request and response. In this example we are simply adding the Access-Control-Allow-Origin header to the HttpServletResponse. But again, you can customize this to add specific origins and so on.

And that's it. Happy mashing!

Friday, 1 February 2013

Getting Started with Mule Cloud Connect: Accelerating Integration with SaaS, Social Media, and Open APIs - Sample Chapters

Print Edition Now Available!

With the number of open APIs reaching over 13,000 this year according to APIhub, 2013 will all be about how developers orchestrate APIs to create applications. Mule Cloud Connect is here to help. For those looking to get started with Mule and Mule Cloud Connect or even just working with APIs, my latest O'Reilly book will get you up and running: If you're not already convinced, here are the first 2 chapters to get you started