MQ 6 on Windows 64 bit operating systems

So, is there such thing as WMQ 64 bit on 64 bit Windows platform?

I discovered that there is no such thing.
WMQ 6 on windows is great 32 bit queueing system that supports 64 bit windows platforms.
This issue doesn't exist on Linux/unix systems.

If you developing .Net application for 64 bit systems you should compile it as an x86 program and not 64 bit, because WMQ 6.0 doesn't support 64 bit and you will get several unexpected errors while compilation/ running of the program.

go to this link http://publib.boulder.ibm.com/infocenter/wmqv6/v6r0/index.jsp and search for "installing on 64-bit windows operating systems" for more details.

SOA & BPM what is the connection

In the past few years the words SOA and BPM became common thing.

I won't explain about SOA or BPM in this post, I will explain the connection between those aspects.

Why do organization want to enter the SOA world?

Those are some of the reasons:

• Business Agility - react fast to changes with Orchestration solutions.
• Greater visibility of organization assets,services.
• Business empowerment by IT abstration.
• Reduction in integration expense, services reuse.

Why BPM, I already have SOA?
SOA is the enabler of BPM, with SOA you can simplify the BPM solution and gain the next things:

• Transparency of Business processes.
• Business Analyst empowerment.
• With BPMN you can model your business process.
• Business process optimization.
  

For Conclusion

You making SOA for the BPM. Your goal is to be ready for the changes to come and in our word changes happing all the time. If your business will be inflexible than in a short time it won't be relevant.

For BPM to succeed you need a good service infrastructure in your organization.
If not your BPM projects will soon become integration projects.
Your mind will be busy with services issues and not with the process.

optimize Biztalk projects by using outer mapping tools

I suggest everyone who work with biztalk to look at the xslt the mapper makes. You will be amezed with the poor xslt that it generates.

There is a good feature in the map properties which allows you to insert xslt for the map to run.
My recomendation is to use biztalk map only as a host and to make the xslt with other tool like Altova MapForce.

By this you will gain better performance of the mapping process.

Memory Leak using Biztalk mapper

In one of our ESB projects that I worked on with my team we encountered on a huge memory leak issue with biztalk projects. In our project we had a couple orchestrations and maps (simple, only text functoids).
The problem was that after running the map the first time the memory usage jumped for 200-300 MB. After working on 4/5 messages the biztalk host service took about 2 GB of memory. And with no load at all messages began to stuck. Messages run into message box but no dequeue was made at all (Tracking DB).
The messages was about 1-2 kb size and the orchestration was standart work flow, without any looping.

After some resarch made by us and my work colegue Matanya Weinshtock that found the next kb :
http://support.microsoft.com/kb/918643

summary of it :

The System.Policy.Security.Evidence object is often used in transforms and can consume a lot of memory. Whenever a map contains a scripting functoid that uses inline C# (or any other inline language), the assembly is created in memory. The System.Policy.Security.Evidence object uses the object of the actual calling assembly. This situation creates a rooted object that is not deleted until the BizTalk service is restarted.Most of the default BizTalk functoids are implemented as inline script. These items can cause System.Byte[] objects to collect in memory.

Suggested Solution by Microsoft

To minimize memory consumption, we recommend that you put any map that uses these functoids into a small assembly. Then, reference that assembly.

The Solution

We decided not split each map to different assemblybecause as you can see at kb almost all of the functoids are implemented with inline scripts.
I suggest the next develop metodology with biztalk project :

Alwayes split the project to at least three projects :

1. Schema project.
2. Maps project.
3. Orchestration project.
   

with this develop metodology we succeded to almost stop the leaking at all.

Again don't take biztalk as It is simple and supposed to work, always check it.

MQ correlation set initialization in biztalk

WMQ manager can generate a unique message id (MQMD_MsgID) for you if you put message to queue with MQMD_MsgID set to null.
In MQ/MQC biztalk adapters there is a great feature to get the MQMD_MsgID after putting the message to WMQ queue using solicit response port. After putting the message to queue you get an xml which includes the MQMD_MsgID.

So if you want to send a request and get the response by WMQ you need to do the next steps :

1. Create send solicit-response port, configure it to Mqseries/MQC adapters.
2. Create Correlation Set based on MQSeries.Biztalk_CorrelationID
   
1. Add Mqseries.dll to your project, from Biztalk bin directory.
2. Create the correlation type,You can make the correlation only by the MQSeries.Biztalk_CorrelationID because the adapters update it automatically. On send to queue they copy the MQMD_MsgID to MQSeries.Biztalk_CorrelationID and on the receive the adapter copies the value of MQSeries.Biztalk_CorrelationID to MQMD_CorrelationID.
3. Initialize the Correlation set using "dummy send" port (Null/Nope adapters)
   You cannot initialize it on the receive shape like in the following orchestration , because you will get this error when compile : "in a sequential convoy the ports must be identical" .
   
   
   
   you get this error because biztalk cannot determine which message it will get first, the message of the solicit-response (receive shape which initialize the correlation set) or the message of the response followed up the Correlation set.
4. Set the follow correlation set attribute on the needed receive shape.
   
   enjoy
   
   

<

Biztalk and MQ Correlation Set with NOPE/ NULL adapter causing Race Condition.

In my integration scenario I needed to implement request/response interface between application running on Mainframe (Supplier) platform and windows (Biztalk ) . The communication between the platforms done with WMQ.

General MQ Stuff

In WMQ you have great feature for Correlation. When putting a message to a queue with MQMD_MsgID set to Null, WMQ Server generates a unique ID for this message and thus you can correlate the transaction with this MQMD_MsgID. The supplier get the message from queue, do it's stuff, sets the Correlation_ID of the response message to the MQMD_MsgID of the received message and puts it to queue.
All the client need to do is to listen on the queue for a message with Correlation_ID equal to it request MSG_ID.

Biztalk implementation

Biztalk support this way of work with it's MQSeries/MQC adapters using the solicit response port type. When issue the put to queue command with the adapter, you get the MQMD_MsgID which was assigned to your message by the WMQ Manger. The adapter copies it's also to the MQSeries.BizTalk_CorrelationId, thus you can work only with it and not using the MQMD_MsgID/MQMD_CorrelationID (on receive).

To implement this you need to do the next steps, by best practices :

1. Create solicit-response port for the destination wmq queue.
2. Create Correlation Set based on MQSeries.BizTalk_CorrelationId. this context property will be initiate automatically by the adapter with the MQMD_MsgID created by the WMQ Manager after issue the send to queue port.
   
3. Create "dummy send" port for dev null/ NOPE adapter for the Correlation Set initialization. (*)
   
4. Setting the Correlation set on the receive shape, waiting for the responce. The adapter will set the MQSeries.BizTalk_CorrelationId with the value of the MQMD_CorrelationID.
   
   (*) see my previous post on this subject.
   

The Problem

seems simple yes, but there is a huge problem in step 3 and 4 because they are not an atomic transaction.
What will happen if biztalk will get a response message prior the setting of the Correlation Set, you will get the no subscribers error, because the Orchestration did not succeeded yet to subscribe it self in the message engine for this particular message.

We got a Race Condition problem.
Yep I got it , the roundtrip of the message from Biztalk to Mainframe , suppliers work, and back took less time than the Correlation Set initialization in biztalk. Surprised I WAS too.


Solution

So to solve it I needed to write my own code for unique message id generation and set the correlation set before sending the request message with it.

Summery

Be aware of biztalk publish/subscribe latency.
On my next post I will give some tips for it optimization by administration table configuration. In this scenario it didn't help.

Biztalk SQL Server management and optimization

I found two great posts about biztalk server 2004/2006 DB optimization, deadlock solutions.
I suggest everyone who work with biztalk to read this posts.

1. http://support.microsoft.com/kb/917845
2. Biztalk core engine's WebLog - What you can and can't do with the Messagebox Database server.
   

Summery :

• Don't enable/use statistics.
• Shutdown the parallelism.
• Don't built maintenance plan for indexes, use only bts_RebuildIndexes stored procedure, shutdown biztalk engine first.
  
• Don't change the stored procedure code.
  

enjoy,

InfoPath 2007 Be aware of Its traps

I worked on an ordinary form for my customer in Infopath 2007. I updated the data source connection (Web Service) by saving the changes and closed the form, so infopath will be updated with the changes.
When I tried to open my form I get the next Error :

The form definition (.xsf) file contains elements, attributes, or structures that are not valid.

Error details:
The keyref 'RepeatingData_15' does not resolve to a key for the Identity Constraint '{http://schemas.microsoft.com/office/infopath/2003/solutionDefinition}xmlToEdit_name_key'.

Line 689, Position 5

----^

It took me one hour but I managed to reopen my form.
I solve it by:

• extracting the XSN file ( just change the xsn to cab, it's a regularly cab file)
  
• deleted any keref in the manifest.xsf file. (each time I reopen the form and get the same error but different keyref)
  

Infopath saves a temp refs in the form which after a while becomes junk that avoiding you to work on your form.
I suggest to save the form for each change and reopen it after the save.

I know it's sounds not so good but it's better than seating for a hour and editing xsf file by hand.

Expose Stored Procedure with SQL 2005

I needed to expose a SQL 2005 stored procedure as Web Service. I searched the net and found that it can be done straight forward by the SQL 2005 tools , the End Points.
It is very simple to do it follow this link

http://msdn2.microsoft.com/en-us/library/ms345123.aspx#sql2005websvc_topic4

really three steps and you done:

1. Create the stored procedure.
2. Create the End Point that exposes the stored procedure.
3. Grant the user access to the end point.

IIS 6.0 - HTTP Error 401.1

Hello,

On this week I had a strange problem when working with Asp.Net site on IIS 6.0, Win 2003.
I defined an application pool to run under domain user account and Web site is configured to use Integrated Windows authentication only.
Just for the record the I gave the domain account administrators permissions on the server.
I tried to consume the web site with the same user from the server but get the following error :

HTTP Error 401.1 - Unauthorized: Access is denied due to invalid credentials.

I managed to workaround this issue by setting the web site to work with digest authentication.
Please consider the security issues before doing it, I worked on a test environment on the organization intranet.

Another issue was when I tried to consume the web site from different pc with other domain account which was defined as a member of the user group on the server.
I get the user/password dialog box.
I manage to fix the problem by this Microsoft KB article http://support.microsoft.com/kb/871179/

Because I worked on a test environment and need a fast solution I went on the workaround section and It do the job.

WORKAROUND

To work around this behavior if you have multiple application pools that run under different domain user accounts, you must force IIS to use NTLM as your authentication mechanism if you want to use Integrated Windows authentication only. To do this, follow these steps on the server that is running IIS:

1.	Start a command prompt.
2.	Locate and then change to the directory that contains the Adsutil.vbs file. By default, this directory is C:\Inetpub\Adminscripts.
3.	Type the following command, and then press ENTER: cscript adsutil.vbs set w3svc/NTAuthenticationProviders "NTLM"
4.	To verify that the NtAuthenticationProviders metabase property is set to NTLM, type the following command, and then press ENTER: cscript adsutil.vbs get w3svc/NTAuthenticationProviders The following text should be returned: NTAuthenticationProviders : (STRING) "NTLM"

Biztalk 2006 MQSC adapter

Hi,

In this post I will explain the benefit of the MQSC (WMQ native api client adapter) over MQS (COM+ server adapter).

When working with wmq and Biztalk on multi server environment I suggest to work with the MQSC adapter and not with MQS adapter.

Because of the following arguments:

• MQSC Support clustered MQSeries Queue Managers.
• The MQS adapter make subscriber for each clustered queue, even if it is the same
  queue, but on different qmgr. Creates duplication of messages.
  
• MQSC Support clustered BizTalk servers.
• You can't difine with the MQS adapter a dynamic recieve loaction. Works only with local Qmgr on the biztalk application server.
• The COM+ level overhead of the MQS adapter.
• Can define by WMQ channel table file a simple fail over mechanism, using the wild char "*" in the qmgr name.
  

You can get the inforamtion about the MQSC adapter here :

http://msdn2.microsoft.com/en-us/library/aa772116.aspx

When using the MQSC adapter it is very important to set the transaction mode on (default off ) when integrating with WMQ queus. Otherwise duplication of messages can occur.

WMQ CSD Install problem- File in use

Hi,

When you try to install csd for wmq on an existing running WMQ qmgrs almost in every case the installation will fail because of dlls being in use.
No matter if you already stopped the qmgr and the wmq service.
So if it is your problem then do the following :

1. extract the csd.
2. open cmd window.
   
3. run in the csd directory amqicsdn -a MQPINUSEOK=1.
   

enjoy
Alex Linder.