Mohsin Kalam’s Blog

November 7, 2007

Batching

Filed under: R2 EDI — Mohsin Kalam @ 8:11 pm

Goal:

To provide an in depth analysis of how R2 Batching subsystem is designed to provide out of box batching capability of EDI documents.

Overview:

Batching subsystem is implemented as a combination of BizTalk pipeline component Batch Marker in the Resolve Party stage in the EDI Receive pipeline out of box and a couple of orchestrations namely the Batching Orchestration (Batching ODX) and the Routing Orchestration (Routing ODX). Together these 3 components provide the ability to batch EDI documents out of box.

Batch Marker:

The Batch Marker component is responsible for stamping a message to be included in the Batching process. More specifically it checks the filter criteria (explained later on) on parties with active batching to see if a match has happened. 

If the filter criteria match happens for a single party with active batching, it stamps the following to the context of that message:

  • ToBeBatched = True
  • DestinationPartyID (same as BizTalk Server Party ID and is unique among all parties)
  • EncodingType (if known)

If the filter criteria match happens for more than one party with active batching, it stamps the following to the context of that message:

  • ToBeRouted = True
  • DestinationParties (comma separated values of the DestinationPartyIDs)
  • EncodingType (if known)

If the filter criteria match happens and the encoding type is not known, it stamps the following to the context of that message:

  • ToBeRouted = True
  • DestinationParties (in case of a single match, this field contains only one value)

Batching ODX:

Everytime batching is activated on a party, an instance of the batching orchestration is instantiated. When batching is stopped this instance is taken away. Batching orchestration subscribes to the following three properties:

  • ToBeBatched = True
  • DestinationPartyID
  • EncodingType

Everytime a document matches this subscription, it gets picked up by the Batching ODX and added to the active batch. Once a release criteria on a batch is met, that batch is released in the MessageBox. This batch can then be subscribed by a send port by using the expression (which is set by the Batching ODX)

DestinationPartyName = “PartyName” And
ToBeBatched = “False”

As you can see, the Batch Marker stamps any document meeting the filter criteria with the properties that a Batching ODX subscribes to for this magic to happen. 

Routing ODX:

The Routing ODX subscribes to the following property

  • ToBeRouted = True

Everytime a document matches this subscription, it gets picked up by the Routing ODX. Routing ODX then makes N number of copies of the original message depending on the list of DestinationParties and stamps each message with the properties that the Batching ODX subscribes to (namely ToBeBatched = True, DestinationPartyID and EncodingType).

In the case where the encoding type is not known, the Batch Marker stamps just the ToBeRouted and DestinationParties property on a message. By the time the message arrives at the Routing ODX, the encoding type has to be set. It then follows the similar strategy of making N number of copies and stamping each message with the properties that the Batching ODX will subscribe to. If the encoding type can not be determined at that point, the message gets suspended.

Summary: 

If the encoding type is known:

1. If filter criteria matches for one party, the batch marker will stamp the properties that the Batching ODX subscribes to

2. If match happens for more than one party, the batch marker will stamp the properties that the Routing ODX subscribes to. The Routing ODX will make N number of copies and stamp each message with properties that the Batching ODX subscribes to.

If the encoding type is not known:

1. The batch marker will stamp the properties that the Routing ODX subscribes to. By the time the document hits the Routing ODX, the encoding type should be set. The Routing ODX then makes N copies and stamp each message with the properties that the Batching ODX will subscribe to.

Batching Configuration:

Batching is available only at the party level and is configurable through PAM under EDI properties. For each party, there is a node for both X12 and EDIFACT standards labelled “Interchange Batch Creation Settings” that controls all the settings related to batching. The following snapshot shows this view

Batching Screen Shot 

Filter Criteria:

Batching follows the exact same Pub-Sub model inherent to BizTalk Server. So naturally the filter criteria is similar to the send port subscription model and works in the exact same way as a send port subscription would work. For example users can filter based on the DocType or the ReceivePort etc.

Release Criteria:

The release criteria is compared against the active set of batches to determine the release of a batch. R2 supports the following release criterias

  • Time Based
    • Hourly
    •  Daily
    • Weekly
  • Number Based
    • Max number of transaction sets in a group
    • Max number of transaction sets in an interchange
    • Max number of characters in an interchange
  • External release trigger

Activation Range:

Users can select the start date and end date of a batch. Once a batch is started, users have the option of specifying no end date, end after a number of occurrences or end by a certain date. The batch is forced released if the end date or the number of occurrence option is met before the release criteria is met. For example if a user has set a release criteria to have 3 transaction sets in a group and currently the batch holds only 2, that incomplete batch will be released when the activation range is over.  

Control Buttons:

These constiture the Start, Override and Stop buttons. Start is used to start an instance of the Batching ODX. Once a batch is started, the Batching ODX id appears in the read only box and the screen indicates at the bottom that batching is activated if the refresh button is pressed or the window is closed and reopened.

The override and stop button are inactive while a batch is in the stopped state. Starting the batch disables the start button and enables the override and stop button. Override button lets you release the batch without stopping the Batching ODX. The stop button releases the batch (even if incomplete) and kills the Batching ODX instance. 

End Note: 

I hope this post has been useful in understanding Batching components in R2 EDI used during baching of an EDI document. Your comments/questions are always welcome.

Thanks

Mohsin Kalam

Advertisements

6 Comments »

  1. Very well written and great piece. Thank you for the effort.

    Been around Biztalk for a while, but am new to this company and brand new to EDI (like two weeks new). Ran into a problem with batching that has me stumped. It concerns the really poor performance on our test server (BizTalk).

    Eg. all processes disabled except for batching. Yet we added 60,000 rows in our MsgBox tracking tables in a 15 minute period. We’ve been trying to track down what’s going on for several hours now. Any ideas would be greatly appreciated.

    Thanks.

    Comment by John Lingenfelder — December 18, 2007 @ 5:47 pm

  2. Can you specify the tracking table that is being filled up? Also do you have EDI status reporting turned on? Do you have any other sort of tracking turned on?

    Thanks
    Mohsin

    Comment by Mohsin Kalam — December 19, 2007 @ 12:12 pm

  3. I do not know the source for the following, but it looks like the problem is solved.

    When the activation range start date is older than the current, this causes the batching orchestration to scan the BizTalkMsgBox database for data that meets the filter criteria that has not been batched yet. This could cause high CPU utilization on the BizTalk and/or SQL Server. To avoid, update the Activation Range start date to the existing day a few minutes before the current time. Also update the batch schedule “First release at” to be in the future time, or you will receive an error when trying to start the batching orchestration.

    Like I said I don’t know the source for this but it solved the problem.

    Comment by John Lingenfelder — December 19, 2007 @ 1:29 pm

  4. Mohsin,
    Glad to see atleast there is one blog dedicated to BT2K6 R2 EDI related stuff.

    How can we get the 837 schema with ISA/GS/GE/IEA information?

    I need to preserve interchange and when i set the property in receive EDI pipeline it requires the schema which has Interchange information. But the Microsoft provided schemas does not have it.

    I would apreciate your help.

    Thanks
    G

    Comment by Ganesh — March 7, 2008 @ 8:06 am

  5. Hi,
    I having problem in batching.
    I want to send EDI 810 in batch file to partner. I have created receive port with EDIRecive pipe and send port with EDISend pipe. Send port filter is DestinationPartyID = party name and ToBeBatched = fales. I set these same as interchange batch creation setting in the party. I set the ReceiveportName = my received port in the interchange batch filter. I have started all the batching orchestration.
    when i drop the edi file in the infolder for batching , I am getting 3 out put files. two are mixed of 810 and 997 with TA1 segment. Why do I get this TA1 and 997 in the out put batch?
    I am also getting batched 997 files in the same out folder. Why do i get this 997?
    Am i missing any set up? how can I set process to batch and send edi files to destination?
    Thank you in advance,
    Siva

    Comment by Anonymous — April 27, 2008 @ 8:04 pm

  6. How do you call batch marker in an orchestration?

    Comment by Eric Stott — May 2, 2008 @ 1:36 pm


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: