Mohsin Kalam’s Blog

October 9, 2007

PAM

Filed under: R2 EDI — Mohsin Kalam @ 2:00 pm

Goal:

To describe various settings used by the BizTalk EDI runtime components stored in PAM.

Overview:

Partner Agreement Manager (PAM) is the central repository for the configuration settings accessed by the EDI runtime components during document processing. It is a UI driven model which plugs in the BizTalk Administration Console.

PAM defines 2 levels: Global and Party. Settings from the Global level are used during processing if no party is resolved for a document. R2 gives the users the ability to customize processing of individual documents by resolving them to a specific party. In other words, the global settings are enough to perform processing. Party specific properties are just an addition to the process that allows fine grain customization.

In addition to the levels, PAM defines 2 roles under Party settings: Party as Interchange Sender and Party as Interchange Receiver. Settings from Interchange Sender role are used during parsing and settings from Receiver role are used during serialization.

The following diagram explains the above

Roles

Global Settings:

The global properties are defined in a detailed post here.

Party Settings:

A party can be created by selecting the party node under BizTalk Server 2006 Administration Console and right clicking on the right hand view and selecting New->Party. Once a party is created, the EDI Party properties are accessed by right clicking on that party and selecting “EDI Properties“. The following snapshot shows the EDI properties view

EDI Properties

Party properties are divided into a General section used by all EDI components, a contact information tab and a separate EDIFACT and X12 section that is subdivided for Engine Receive, Send and Batching.

General Properties:

General properties include Activating Status Reporting which is turned off by default. It also has an option of copying settings from an already configured party. It also specifies certain agreement details like start and end date and a text field to add notes related to this agreement. General settings are applied for all EDI runtime components (except AS2 that has a separate page dedicated to it).

Contact Information:

Allows you to enter the contact information for the party. It includes fields like Name, Company and email etc.

EDIFACT Properties:

EDIFACT properties are used by the EDI runtime to process an EDIFACT encoded interchange. It is described in great detail here.

X12 Properties:

X12 properties are used by the EDI runtime to process X12 encoded interchange. It is described in great detail here.

Validation:

Validation of certain fields specified in PAM are done according to the standard defined by X12 and EDIFACT. The following rules are followed:

  • Validation of ISA, GS, UNB and UNG fields are done against the control schema deployed under the BizTalk EDI Application. These include checking for min/max length and performing data type validation.
  • Validation of delimiters is done to ensure the delimiters do not collide with other delimiters and they are under the valid range of 0-127 ASCII
  • For EDIFACT, UNB2.1, UNB2.2, UNB3.1 and UNB3.2 are validated as a quartet to ensure they are unique among all parties. The same applies for ISA5-8
  • For control numbers (ISA13, GS6, ST2, UNB5, UNG5 and UNH1), validation is performed to ensure their length is within the allowed range. Invalid values are not saved. If during processing of a doc the control numbers exceed maximum length, an error is raised in the event viewer and the document is suspended. A user has to manually reset the control number to allow further processing

End Note:

I hope this post has been useful in understanding PAM and how it is used by the runtime components during processing of an EDI document. Your comments/questions are always welcome.

Thanks

Mohsin Kalam

October 5, 2007

R2 Engine Send

Filed under: R2 EDI — Mohsin Kalam @ 11:59 am

Goal:

To provide in depth analysis of how R2 Engine Send serializes an EDI document. 

Overview:

Engine Send is implemented as a BizTalk pipeline with EDI Assembler component in the Assemble stage.

Engine Send:

Below is a diagram of the EDI Send Pipeline showing the individual component

EDI Send

EDI Send pipeline does the following when serializing a document

  • Party Resolution
  • Schema Discovery
  • Document Processing
  • Envelope Application
  • Apply Party Specific Properties (or Global)
  • EDI Validation
  • Character Set Validation
  • Status Reporting

Party Resolution:

Like Engine Receive, Engine Send requires party or global settings to apply during serialization. Party resolution is explained in detail here.

Schema Discovery:

Once the source of the settings is determined, Engine Send tries to discover the Schema. Schema discovery process is defined here.

Document Processing:

Like Receive, Engine Send also supports 3 different mode of processing but unlike receive where you have to configure the mode of processing, the processing mode in Engine Send is dynamic. They are as follows:

1. Serializing an Individual Transaction Set:

In this scenario, XML documents without the envelope segments are fed to the send pipeline and an EDI document is produced. The envelope information comes from Party as Interchange Receiver settings of either the party the document resolves to or global settings.

2. Serializing a BIBO Message:

In this scenario, the whole interchange with all the envelopes in XML format is fed to the send pipeline and the it is serialized. Since the envelope information and the delimiters are already present in the XML file so no additional settings are required.

3. Serializing a Batch Message:

This is essentially the same as BIBO but Send Pipeline updates selected fields in the envelopes. Send pipeline differentiates between the mode 2 and 3 by the presence of the promoted property “PopulateInterchangeValues” in a batch message. If this property is promoted and set to true, Send Pipeline updates selected fields (explained in the next section).

The difference between Batch and BIBO message is that a Batch message is Party dependent. Meaning if a party cannot be resolved to fetch the settings, the message will be suspended.

Envelope Application:

Envelope application is at the heart of Send Pipeline and the functionality varies for the 3 modes of processing. It is explained in great detail here.

Apply Party Specific Properties (or Global):

During serialization, R2 provides the ability to apply certain settings on the document being processed. These include generating trailing separators and turning validation on or off. If a party cannot be resolved, these settings are applied from a combination of global and pipeline component level settings. If a document resolves to a party, the settings are taken from that party. In other words, the global and pipeline settings are enough to perform processing. Party specific properties are just an addition to the process that allows fine grain customization. 

A separate post will follow that will explain all the properties in PAM, Global and Pipeline component settings.

EDI Validation:

EDI validation includes Min/Max length validation, Min/Max Occurs validation and validation of specific data types against the standard (e.g validating X12_Nn datatype only contains numeric type etc). A document failing any of these validation will be suspended.

In case of a party not being resolved, validation is always performed. At the party level, the default setting is to enable validation. The validation can be made lax by unchecking this option in PAM under Party as Interchange Receiver/ACK Processing and Validation Settings.

Please note that in case of BIBO or Batch processing mode, the envelope validation will always happen against the control schemas regardless of this option configured in PAM.

Character Set Validation:

EDI documents allow certain encoding types as defined by the specific standard. Each encoding type allow only certain characters as the valid characters. The engine performs this character set validation to ensure the characters fall within the allows range as the standard defines them. The detailed information can be found here.

Status Reporting:

EDI engine provided detailed status reporting for the documents flowing within the EDI system. The reporting is available at both the Party and Global level. Status reporting is turned off by default. When turned on, the status reporting provides details about Interchange and correlated ACK status, Batch status, Interchange Aggregation status for each party and Transaction Set Aggregation status for each Transaction set. The report is available within the group hub page under EDI Status Reports.

End Note:

I hope this post has been useful in understanding the EDI Engine Send. Your comments/questions are always welcome.

Thanks

Mohsin Kalam

October 2, 2007

R2 Engine Receive

Filed under: R2 EDI — Mohsin Kalam @ 5:06 pm

Goal:

To provide an in depth analysis of how R2 Engine Receive disassembles an EDI document. 

Overview:

Engine Receive is implemented as a BizTalk pipeline with EDI Disassembler component in the Disassemble stage and Batch Marker component in the Resolve Party stage. Batch Marker component is connected to the Batching sub-system and will be explained in another post in great detail.

Engine Receive:

Below is a diagram of the EDI Receive Pipeline showing the individual components

EDI Receive

EDI Receive pipeline does the following when parsing a document

  • Delimiter Discovery
  • Party Resolution
  • Performs Authentication
  • Schema Discovery
  • Document Processing
  • Apply Party Specific Properties (or Global)
  • EDI Validation
  • Character Set Validation
  • Status Reporting

Delimiter Discovery:

EDI files are flat files with delimiters separating the data so the first step in parsing is to determine these delimiters. Delimiter discovery in R2 is done on the fly and is explained in detail here.

Party Resolution:

Once the delimtiers are discovered, R2 engine parses the document based on the delimiters read to determine the party resolving to this document. This is needed to determine the source of settings to apply during parsing. Party resolution is explained in detail here.

Performs Authentication:

EDI Receive performs authentication on an incoming message at the Party level based on values specified in PAM. For X12, it matches the fields in ISA1-4 of the instance to the values in PAM and on mismatch, the whole interchange is suspended. For EDIFACT, it matches the fields in UNB6.1 and UNB6.2.

Authentication is only performed at the party level. If a party cannot be resolved, no authentication is performed.

Schema Discovery:

Once the source of the settings is determined, Engine Receive tries to discover the schema. Schema discovery process is defined here.

Document Processing:

With the delimiters discovered, the party resolved and the schema discovered, the Engine is ready to Parse the interchange to XML. In case the party is resolved, Engine Receive can be configured to supports 3 different modes of processing. These can be set under Party as Interchange Sender/Ack Generation and Validation Settings/Inbound Batch Processing option.

1. Split Interchange as Transaction Sets:

This is the default mode. As it suggests, the whole interchange is splitted into individual XML documents during parsing. For example if an interchange has 1 group with 3 transaction sets within that group, after parsing, the receive pipeline will produce 3 XML documents from the 3 transaction sets and promote certain properties on each of those documents. In case of validation errors in the envelope, the whole interchange will be suspended and an entry written in the event viewer. In case of validation errors in the individual transaction sets, only those transaction sets will be suspended.

2. Preserve Interchange – suspend Interchange on Error:

This is described as Batch In Batch Out (BIBO) scenario and is primarily used to validate the interchange as a whole. The whole interchange along with all the envelope information is transformed into XML. In case of validation errors in either the envelope of the individual transaction sets, the whole interchange will be suspended and an entry written in the event viewer.

3. Preserve Interchange – suspend Transaction Sets on Error:

This is BIBO scenario with the added clean up functionality. The whole interchange along with all the envelope information is transformed into XML. In case of validation errors in the envelope, the whole interchange will be suspended and an entry written in the event viewer. However in case of validation errors in the individual transaction sets, those transaction sets will be removed from the interchange and the rest of the document is preserved. So this is option gives the customer the option of preserving the whole interchange while cleaning up the faulty transaction sets.

Note: If a party cannot be resolved, then the choice of selecting the processing mode is not available and the default mode is implicitly applied!

Apply Party Specific Properties (Or Global):

During parsing, R2 provides the ability to apply certain settings on the document being processed. These include generating both technical and functional acknowledgements, specifying the mode of processing and turning validation on or off. If a party cannot be resolved, these settings are applied from a combination of global and pipeline component level settings. If a document resolves to a party, the settings are taken from that party. In other words, the global and pipeline settings are enough to perform processing. Party specific properties are just an addition to the process that allows fine grain customization. 

A separate post will follow that will explain all the properties in PAM, Global and Pipeline component settings.

EDI Validation:

EDI validation includes Min/Max length validation, Min/Max Occurs validation and validation of specific data types against the standard (e.g validating X12_Nn datatype only contains numeric type etc). A document failing any of these validation will be suspended.

In case of a party not being resolved, validation is always performed. At the party level, the default setting is to enable validation. The validation can be made lax by unchecking this option in PAM under Party as Interchange Sender/ACK Generation and Validation Settings.

Please note that the envelope validation will always happen against the control schemas regardless of this option configured in PAM.

Character Set Validation:

EDI documents allow certain encoding types as defined by the specific standard. Each encoding type allow only certain characters as the valid characters. The engine performs this character set validation to ensure the characters fall within the allows range as the standard defines them. The detailed information can be found here.

Status Reporting:

EDI engine provided detailed status reporting for the documents flowing within the EDI system. The reporting is available at both the Party and Global level. Status reporting is turned off by default. When turned on, the status reporting provides details about Interchange and correlated ACK status, Batch status, Interchange Aggregation status for each party and Transaction Set Aggregation status for each Transaction set. The report is available within the group hub page under EDI Status Reports.

End Note:

I hope this post has been useful in understanding the EDI Engine Receive. Your comments/questions are always welcome.

Thanks

Mohsin Kalam

Create a free website or blog at WordPress.com.