Aspera Software – High-Speed File Transfer - Boost the file transfer

Used Aspera's solution in one of my latest integration projects.
We needed the ability to upload large files over geographical spread sites.
By Aspera solution we were able to use all the bandwidth of network for the upload and download.
We were able to transfer files ( various sizes 0.5 - 9 GB ) over 10 / 15 Mbps networks , over the internet with the maximum provided bandwidth provided ( 10/15 Mbps) by using the Aspera's "fasp" solution.
Aspera supports much much larger rate , 1 GBbit and more.
Aspera's algorithm can use all the bandwidth that is provided by the WAN / LAN.
Depends on the license.

Before FTP was used and the max rate we saw was only round 500 kbps over the same 10/15 Mbps lines.

We got great service by the support , I am really recommend to use this solution.
it saved us a lot of time and created a great solution to our customers.

Biztalk FTP adapter caused ACK Zombies messages in Orchestration

We handled a strange behavior of logical FTP port in Orchestration which was set to Transmitted true.
In the Orchestration we had several send ports in a single none transcriptional scope. this scope also included call and start orchestration shapes.
The invocation of the orchestration left after each invocation two zombie messages , those was the ACK messages of the FTP adapter with the known error :

0xC0C01B4C The instance completed without consuming all of its messages. The
instance and its unconsumed messages have been suspended.

I tried to tweak the propertices of the scope to Synchronized true and also switch to different type of transaction. I know that such changes influence on the orchestration behavior , this I won't describe here.
Those changes didn't solve the issue.

Solution :
Finally I created another Atomic scope that included only the send ports , this configuration solved the issue.
That seemed to work fine but the error returned.
The problem was caused by using Send Port Groups, we have one group with two ports FILE and FTP. when we removed the FILE port from group we stopped getting the error.

In general you should always use Atomic scope if you have several send ports in a row , this save the Dehydration and Rehydration of the orchestration between each send.

Strange behavior of the adapter and orchestration mediation in Biztalk 2009.

AIM - Application infrastructure and middleware - Cloud and Appliance

AIM

Application infrastructure and middleware you can right it or buy it.

The AIM solutions provided by most major vendors like Oracle , IBM and Microsoft.
The products that out in the market contains many features that suppose to make the AIM solutions easy to handle.
You must not forget that the purpose of those products not only to create the solutions but also to provide an easy maintenance for the solutions.

The SMB market preferred not so far to do the integration by custom code because the AIM products were not so cost affective. If you need to do several integration processes a year why to spend tens of thousands on it? This was right in the past.

Today you can find a good products out there that will not only speed the integration process but also will provide good platform for maintenance.

If you don't have the man power or the resources you can get an appliance solution or a cloud.
appliance solution can be very easy to use , but keep in mind that it is an appliance , the update process can hard and you can have problems to scale your solution.

Cloud is the world that everyone use , and you can do integration as a service , INaaS.

some INaas vendors that you should check when seeking for integration solution for your business :

- Dell boomi
- WebSphere Cast Iron
- Mule Cloud connect
- Progress

Handling binary data in SOAP - WMB V7 and MTOM/XOP

If you need to send large binary files over SOAP I suggest to use MTOM/XOP and not the common base64 encoding. ( One of the common types to transfer binary data over SOAP).
The benefit of using MTOM is that it not increasing the size of the original message. base64 converts each 6 bits to 8. if working with UTF-8 the encoded data will be larger in about 33%.
The data will be transfered as binary over soap. no conversion.
When setting type of an element as base46binary the Axis2 framework ( Web services in WMB handled by Axis2).

From Alex Linder's Blog - C&SI

Archive Service - Soap --> File

I made a sample project to test this technology with WMB V7.0.2.
I implemented an archive service, using SOAP Input node and FileOutput node.
The consumer invoke the service with file name and file data attached to soap using MTOM.
for the testing I worked with SoapUI, great testing tool for Web Services. use the following link to configure SoapUI work with MTOM : Adding Headers and Attachments

I exposed the following WSDL

The method get to input parameters, fileName and fileBuffer ( base64binary type).
here the image of soapUi console with the request and mtom configuration

From Alex Linder's Blog - C&SI

.

When invoking the WS the binary data sent in binary mode ( MIME).

From Alex Linder's Blog - C&SI

On the server side, the base64binary is encoded as BLOB, you can see this in the following debug screen shot :

From Alex Linder's Blog - C&SI

to save the file buffer to disk I create an output message of BLOB domain in the compute node.

From Alex Linder's Blog - C&SI

don't forget to built the soap reply message.

During my testing I notice that when handling files over 1 MB during debug mode the message flow use about 50% CPU and crash.
When not in debug mode I sent files over 6 MB with no problem.

links for the tester projects:
message flow project
message set project