Mohsin Kalam’s Blog

October 2, 2007

R2 Engine Receive

Filed under: R2 EDI — Mohsin Kalam @ 5:06 pm

Goal:

To provide an in depth analysis of how R2 Engine Receive disassembles an EDI document. 

Overview:

Engine Receive is implemented as a BizTalk pipeline with EDI Disassembler component in the Disassemble stage and Batch Marker component in the Resolve Party stage. Batch Marker component is connected to the Batching sub-system and will be explained in another post in great detail.

Engine Receive:

Below is a diagram of the EDI Receive Pipeline showing the individual components

EDI Receive

EDI Receive pipeline does the following when parsing a document

  • Delimiter Discovery
  • Party Resolution
  • Performs Authentication
  • Schema Discovery
  • Document Processing
  • Apply Party Specific Properties (or Global)
  • EDI Validation
  • Character Set Validation
  • Status Reporting

Delimiter Discovery:

EDI files are flat files with delimiters separating the data so the first step in parsing is to determine these delimiters. Delimiter discovery in R2 is done on the fly and is explained in detail here.

Party Resolution:

Once the delimtiers are discovered, R2 engine parses the document based on the delimiters read to determine the party resolving to this document. This is needed to determine the source of settings to apply during parsing. Party resolution is explained in detail here.

Performs Authentication:

EDI Receive performs authentication on an incoming message at the Party level based on values specified in PAM. For X12, it matches the fields in ISA1-4 of the instance to the values in PAM and on mismatch, the whole interchange is suspended. For EDIFACT, it matches the fields in UNB6.1 and UNB6.2.

Authentication is only performed at the party level. If a party cannot be resolved, no authentication is performed.

Schema Discovery:

Once the source of the settings is determined, Engine Receive tries to discover the schema. Schema discovery process is defined here.

Document Processing:

With the delimiters discovered, the party resolved and the schema discovered, the Engine is ready to Parse the interchange to XML. In case the party is resolved, Engine Receive can be configured to supports 3 different modes of processing. These can be set under Party as Interchange Sender/Ack Generation and Validation Settings/Inbound Batch Processing option.

1. Split Interchange as Transaction Sets:

This is the default mode. As it suggests, the whole interchange is splitted into individual XML documents during parsing. For example if an interchange has 1 group with 3 transaction sets within that group, after parsing, the receive pipeline will produce 3 XML documents from the 3 transaction sets and promote certain properties on each of those documents. In case of validation errors in the envelope, the whole interchange will be suspended and an entry written in the event viewer. In case of validation errors in the individual transaction sets, only those transaction sets will be suspended.

2. Preserve Interchange – suspend Interchange on Error:

This is described as Batch In Batch Out (BIBO) scenario and is primarily used to validate the interchange as a whole. The whole interchange along with all the envelope information is transformed into XML. In case of validation errors in either the envelope of the individual transaction sets, the whole interchange will be suspended and an entry written in the event viewer.

3. Preserve Interchange – suspend Transaction Sets on Error:

This is BIBO scenario with the added clean up functionality. The whole interchange along with all the envelope information is transformed into XML. In case of validation errors in the envelope, the whole interchange will be suspended and an entry written in the event viewer. However in case of validation errors in the individual transaction sets, those transaction sets will be removed from the interchange and the rest of the document is preserved. So this is option gives the customer the option of preserving the whole interchange while cleaning up the faulty transaction sets.

Note: If a party cannot be resolved, then the choice of selecting the processing mode is not available and the default mode is implicitly applied!

Apply Party Specific Properties (Or Global):

During parsing, R2 provides the ability to apply certain settings on the document being processed. These include generating both technical and functional acknowledgements, specifying the mode of processing and turning validation on or off. If a party cannot be resolved, these settings are applied from a combination of global and pipeline component level settings. If a document resolves to a party, the settings are taken from that party. In other words, the global and pipeline settings are enough to perform processing. Party specific properties are just an addition to the process that allows fine grain customization. 

A separate post will follow that will explain all the properties in PAM, Global and Pipeline component settings.

EDI Validation:

EDI validation includes Min/Max length validation, Min/Max Occurs validation and validation of specific data types against the standard (e.g validating X12_Nn datatype only contains numeric type etc). A document failing any of these validation will be suspended.

In case of a party not being resolved, validation is always performed. At the party level, the default setting is to enable validation. The validation can be made lax by unchecking this option in PAM under Party as Interchange Sender/ACK Generation and Validation Settings.

Please note that the envelope validation will always happen against the control schemas regardless of this option configured in PAM.

Character Set Validation:

EDI documents allow certain encoding types as defined by the specific standard. Each encoding type allow only certain characters as the valid characters. The engine performs this character set validation to ensure the characters fall within the allows range as the standard defines them. The detailed information can be found here.

Status Reporting:

EDI engine provided detailed status reporting for the documents flowing within the EDI system. The reporting is available at both the Party and Global level. Status reporting is turned off by default. When turned on, the status reporting provides details about Interchange and correlated ACK status, Batch status, Interchange Aggregation status for each party and Transaction Set Aggregation status for each Transaction set. The report is available within the group hub page under EDI Status Reports.

End Note:

I hope this post has been useful in understanding the EDI Engine Receive. Your comments/questions are always welcome.

Thanks

Mohsin Kalam

Advertisements

8 Comments »

  1. mohsin,

    this is a great site for biztalk EDI information.

    Is it possible to to use biztalk 2006 R2 to just perform compliance checking without having to map from edi to xml.

    This is what i am looking in to doing.

    Lets say i have 4 folders.

    EDI_IN
    GOOD
    BAD
    ACK

    i want to drop an edi file in to EDI_IN directory and have biztalk perform validation.

    if the validation is successful i need the file to go to GOOD directory and create 997 file in ACK directory.

    if the validation fails i need the file to go to BAD directory and create 997 file in ACK directory.

    How would i acheive this using biztalk.

    my situation is that we may build some edi files using .net programming and want to use biztalk to validate and route the edi files before sending them to trading partners.

    thanks for your time.

    sasi

    Comment by sasi — November 30, 2007 @ 8:48 pm

  2. This is a classic Batch In Batch Out (BIBO) scenario as described in the document processing mode section. You can have the following setup.

    1 Receive Port with 1 Receive Location – The Receive Location will have EDI Receive Pipeline in it and it will point to EDI_IN. The Receive Port will have “Enable routing for failed messages” checked
    1 Send Port with EDI Send Pipeline in it that will filter on specific properties from the message and route the message to GOOD
    1 Send port with Pass through Send Pipeline that will filter on ErrorReport.ErrorType == “Failed Message” and route the message to BAD
    1 Send port with EDI Send Pipeline that filters on 997 message type and route the message to 997
    1 party with 997 enabled and option 2 selected as mode of processing

    If you drop in an EDI file in the EDI_In directory, the EDI message will be resolved to the Party in your setup, a 997 will be generated and the message will be validated. If the validation succeeds the whole message will be persisted to the database which will then be pulled out by the send pipeline and routed to the GOOD directory. If the message has EDI Validation errors, it will be suspended and the message will be routed to the BAD directory.

    I hope this is clear to you.

    Thanks
    Mohsin Kalam

    Comment by Mohsin Kalam — December 3, 2007 @ 2:21 pm

  3. mohsin,

    Really good and helpfull site for BizTalk 2006 R2 EDI release.

    I have a question about validation done in this R2.

    Will validation process able to validate pattern specify in X12 schema?

    Comment by Rocky — January 10, 2008 @ 2:44 pm

  4. Hi Rocky,

    Thanks for your comments. I am not sure I understand your question. Could you please elaborate a bit more and I will try to help out the best I can?

    Comment by Mohsin Kalam — January 11, 2008 @ 11:06 am

  5. Here is my scenario

    In X12_4010_301 schema,
    Segment Y308 with EDI Specification:-
    337 time with Type =TM Min length =4 Max length =8
    To validate above specification, we are using pattern =”(([01]\d)|2[0123])[0-5]\d([0-5]\d\d{0,2}){0,1}” in schema definition.
    My question is
    Will BizTalk R2 EDI validation be able to validate this kind of pattern?

    Thanks

    Comment by Rocky — January 14, 2008 @ 8:54 am

  6. This is not supported in the schema definition as far as my knowledge goes. What is the behavior that you are observing?

    Comment by Mohsin Kalam — January 14, 2008 @ 11:34 am

  7. Hi Mohsin,
    Good article. Since I am new to Biztalk, I am learning alot from your articles.

    I have an issue in receving EDI file. When EDI message have “å” or similer to these
    characters EDI message is susspend by receive pipe line. How can i allow all
    characters?
    I tried the following set up.
    EDI receive pipe line’s Character set : UTF8, no luck
    character set: Extended, still no luck.
    I tried with and without EDI TYPE – Extended Validation in party properties. still no luck.

    could you help me to solve this issue please?
    Siva

    Comment by Siva — May 1, 2008 @ 7:32 pm

  8. Hi Mohsin,

    very good site, thanks for all these infos.
    I have to build an in-out interface (EDI>XML and XML>EDI) but I can´t find the xsd in bts2006R2 repository for my EDI format (EDIFACT/ LOGMES document). What is the simpliest way to implement this stuff?
    Thanks

    Comment by Hernani — May 14, 2008 @ 5:40 am


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: