Developing in WebSphere Message Broker Best practices guide Part 1

In this post I will provide with some guidelines you need to keep in mind when developing message flows in WMB (WebSphere Message Broker).

WMB is a good integration software but you need to know some internals for developing suit solution.

1. Avoid using hard wired looping in the message flows, Use PROPAGATE statement instead.

When using hard wired looping in message flow All the resources of the loop participant are stored on the stack storage. Nodes , message trees, variables copied and held in memory.
Therefore while message processing the stack storage of the Execution Group (process) can ran out.
You also probably will get abends and core dumps.

By using the PROPAGATE statement the data won't be copied and stored during the loop iterations. After each iteration all the iteration data will be freed and re-used.

2. Work with reference variable and don't navigate through message tree directly.

SET OutputRoot.XML.Root.Child.Field1 = 1;
SET OutputRoot.XML.Root.Child.Field2 = 2;


For those simple assigning statements about 20 navigation action will be made. And it's only if the OutputRoot element is empty.
If not it can be much more.


If you have complex message transformation and performance is your concern work with Reference variable and Create statement.

DECLARE ref REFERENCE TO OutputRoot;
CREATE LASTCHILD OF ref as ref DOMAIN('XML') NAME 'XML';
CREATE LASTCHILD OF ref as ref NAME 'Root';
CREATE LASTCHILD OF ref as ref NAME 'Child';
CREATE LASTCHILD OF ref NAME 'Field1' VALUE '1';
CREATE LASTCHILD OF ref NAME 'Field2' VALUE '2';


We no using SET statement and since all CREATES specify relative keywords no navigation are required. But we got more code :-(.
You can use the ROW statement to get more code visibility and simplicity combined with SET statement.

remember to work with REFERENCE.

3. Avoid using Environment tree to handle local variables.

All data which stored in the Environment tree are stored in Memory during the whole message flow and not freed after it's out of scope.
so if you need space to store local data for node work with the LocalEnvironment tree. Thus avoiding high memory consumption.
Even if you delete the fields from the Environment tree it still won't free the memory till message flow end processing the message. The broker use pool allocation and reuse an pointed resources of the Environment Tree.

more will come soon.

Memory Leak using Biztalk mapper

In one of our ESB projects that I worked on with my team we encountered on a huge memory leak issue with biztalk projects. In our project we had a couple orchestrations and maps (simple, only text functoids).
The problem was that after running the map the first time the memory usage jumped for 200-300 MB. After working on 4/5 messages the biztalk host service took about 2 GB of memory. And with no load at all messages began to stuck. Messages run into message box but no dequeue was made at all (Tracking DB).
The messages was about 1-2 kb size and the orchestration was standart work flow, without any looping.

After some resarch made by us and my work colegue Matanya Weinshtock that found the next kb :
http://support.microsoft.com/kb/918643

summary of it :

The System.Policy.Security.Evidence object is often used in transforms and can consume a lot of memory. Whenever a map contains a scripting functoid that uses inline C# (or any other inline language), the assembly is created in memory. The System.Policy.Security.Evidence object uses the object of the actual calling assembly. This situation creates a rooted object that is not deleted until the BizTalk service is restarted.Most of the default BizTalk functoids are implemented as inline script. These items can cause System.Byte[] objects to collect in memory.

Suggested Solution by Microsoft

To minimize memory consumption, we recommend that you put any map that uses these functoids into a small assembly. Then, reference that assembly.

The Solution

We decided not split each map to different assemblybecause as you can see at kb almost all of the functoids are implemented with inline scripts.
I suggest the next develop metodology with biztalk project :

Alwayes split the project to at least three projects :

1. Schema project.
2. Maps project.
3. Orchestration project.
   

with this develop metodology we succeded to almost stop the leaking at all.

Again don't take biztalk as It is simple and supposed to work, always check it.