Mar 13, 2011

Handling binary data in SOAP - WMB V7 and MTOM/XOP

If you need to send large binary files over SOAP I suggest to use MTOM/XOP and not the common base64 encoding. ( One of the common types to transfer binary data over SOAP).
The benefit of using MTOM is that it not increasing the size of the original message. base64 converts each 6 bits to 8. if working with UTF-8 the encoded data will be larger in about 33%.
The data will be transfered as binary over soap. no conversion.
When setting type of an element as base46binary the Axis2 framework ( Web services in WMB handled by Axis2).

From Alex Linder's Blog - C&SI

Archive Service - Soap --> File

I made a sample project to test this technology with WMB V7.0.2.
I implemented an archive service, using SOAP Input node and FileOutput node.
The consumer invoke the service with file name and file data attached to soap using MTOM.
for the testing I worked with SoapUI, great testing tool for Web Services. use the following link to configure SoapUI work with MTOM : Adding Headers and Attachments

I exposed the following WSDL

The method get to input parameters, fileName and fileBuffer ( base64binary type).
here the image of soapUi console with the request and mtom configuration

From Alex Linder's Blog - C&SI
.

When invoking the WS the binary data sent in binary mode ( MIME).

From Alex Linder's Blog - C&SI


On the server side, the base64binary is encoded as BLOB, you can see this in the following debug screen shot :

From Alex Linder's Blog - C&SI


to save the file buffer to disk I create an output message of BLOB domain in the compute node.

From Alex Linder's Blog - C&SI


don't forget to built the soap reply message.

During my testing I notice that when handling files over 1 MB during debug mode the message flow use about 50% CPU and crash.
When not in debug mode I sent files over 6 MB with no problem.

links for the tester projects:
message flow project
message set project

4 comments:

Anonymous said...

Hi,

Just wanted to check, have you enabled Validation and checked "Build tree using XML schema data types" at the SOAP Input node? We have observed only when you enable the above properties, WMB will retain the actual message size. In case if you have not enabled above properties, broker uses CHARACTER data type to parse all the elements while parsing and this will increase the message size by around 33%.

Hari

Anonymous said...

Thanks for this, Alex, you have saved me from a lot of investigation and headaches.

Jacques

Anonymous said...

Thank you for the post, what if I have an array of files, how I can call the FIleOutput multiple times from one invoke.

Thanks in advance

Unknown said...

Hey
I had the same issue and this post helped to work attachments with WMB

Another thing, I wanted to deal the BLOB element in the next flow that using MQInput so I had to use BASE64ENCODE function so the data will be correct and it worked