BizTalk design document
Content
BizTalk Technical Document
BizTalk Technical Document
Prepared for
NALCO
Thursday, 1 December 2011
Version 1.0
Prepared by
MPS Partners
Contributors
Ashish Talati
Prepared for NALCO
The information contained in this document represents the current view of MPS Partners, LLC. on the issues discussed as of the date of publication. Because
MPS Partners must respond to changing market conditions, it should not be interpreted to be a commitment on the part of MPS Partners, and MPS Partners
cannot guarantee the accuracy of any information presented after the date of publication.
MPS PARTNERS MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT.
Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be
reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or
otherwise), or for any purpose, without the express written permission of MPS Partners, LLC..
MPS Partners may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document.
Except as expressly provided in any written license agreement from MPS Partners, the furnishing of this document does not give you any license to these
patents, trademarks, copyrights, or other intellectual property.
The descriptions of other companies’ products in this document, if any, are provided only as a convenience to you. Any such references should not be considered
an endorsement or support by MPS Partners. MPS Partners cannot guarantee their accuracy, and the products may change over time. Also, the descriptions are
intended as brief highlights to aid understanding, rather than as thorough coverage. For authoritative descriptions of these products, please consult their
respective manufacturers.
© 2010 MPS Partners, LLC.. All rights reserved. Any use or distribution of these materials without express authorization of MPS Partners is strictly prohibited.
MPS Partners and Windows are either registered trademarks of MPS Partners, LLC. in the United States and/or other countries.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
Page ii
BizTalk Technical Document
Revision and Signoff Sheet
Change Record
Date
Author
Version
Change reference
10/18/201
1
Ashish Talati
1.0
Created
11/30/201
1
Ashish Talati
1.1
Completed Cooling Controller
Reviewers
Name
Version
approved
Position
Date
Page
BizTalk Technical Document
Table of Contents
1
Environment.....................................................................................1
1.1
Overview............................................................................................................ 1
1.2
Installation Summary......................................................................................... 1
2
Implementation................................................................................2
2.1
Overview............................................................................................................ 2
2.2
TFS..................................................................................................................... 3
2.3
Cooling Controller Process.................................................................................3
2.4
Schemas – Cooling Controller............................................................................ 4
2.4.1 Cooling Schema.......................................................................................... 4
2.4.1 Cooling Property Schema............................................................................6
2.4.1 Cooling Alarm Schema................................................................................7
2.4.1 Cooling Apache Schema............................................................................. 7
2.4.1 Record Schema........................................................................................... 8
2.5
Scenario Components...................................................................................... 10
2.5.1 Cooling Data Receipt................................................................................ 10
2.5.2 Helper Components.................................................................................. 13
2.5.3 Cooling Processor..................................................................................... 14
2.5.4 Testability.................................................................................................. 16
3
3.1
Monitoring and Exception Handling..................................................19
Tracking & Message Monitoring.......................................................................19
3.1.1 BizTalk....................................................................................................... 19
Page
BizTalk Technical Document
1
Environment
1.1 Overview
The NALCO BizTalk proof of concept did not plan for performance testing as a
success criteria, so a highly available environment was not created. Two
environments were created for the POC:
Development Workstation: Used by developers to create and test the
solution.
Developer Integration Server: Solution was deployed to this server for testing
purposes. This environment is typically were multiple developers’ code is
integrated and tested before moving on to a QA process.
The development workstation had the following software installed:
Windows 7
IIS 7
SQL Server 2008 R2
Visual Studio 2010
BizTalk Server 2010
The development integration server had the following software installed:
Windows 2008 R2 Enterprise
IIS 7
SQL Server 2008 R2
Visual Studio 2010
BizTalk Server 2010
1.2 Installation Summary
Because each of the environments were installed with single server envionrments,
the installations were very basic and followed the installation guides from Microsoft.
The following guides were used:
Development Workstation:
Installing BizTalk Server 2010 on Windows 7 and Windows Vista.docx
Development Integration Server:
Installing BizTalk Server 2010 on Windows Server 2008 R2 and 2008.docx
Page 1
BizTalk Technical Document
2
Implementation
2.1 Overview
The following is the high level process when a Boiler or Cooling controller Data file
( either in email format or an XML Format ) is received:
File Receipt:
o
File is Archived
o
Data is parsed and Converted to Schema
o
Optionally if file is compressed, uncompress it
o
Component adds context information into message Header
o
Mapper component: transforms file to canonical
o
Schema Validator
Validation & Enrichment
o
Business validation
Business Process
o
Cooling Data Process
o
Cooling Alarms Process
o
Boiler Data, Event, Alarms Process
o
Web for All Data and Configuration Process
Monitoring
o
Monitoring Orchestration Performance
This document will now describe each component of the scenario in a little more
detail to show how this was implemented using BizTalk Server.
Page 2
BizTalk Technical Document
2.2 TFS
All source code are stored in TFS @ $/3DT Web/WPS Biztalk Projects
A separate BizTalk application and solution are created for each Parser. Every
solution is following same naming convention based on type of artifacts for BizTalk.
The default naming convention is that the name of a folder containing a BizTalk
Project is identical to project’s output DLL name ( without extension ). For example,
if BizTalk project resides in Cooling.Controller.Orchestrations folder, then that project
must output a DLL named Cooling.Controller.Orchestrations.dll
Typical example of project naming structure is
Orchestration = Project.Orchestraions
Pipelines = Project.Pipelines
Pipeline Components = Project.PipelineComponents
Helper Components = Project.Components
Schemas = Project.Schema ( or Project.Generated.Schemas )
Maps/Transformaton = Project.Transform
Unit Tests = Project.Tests
2.3 Cooling Controller Process
High Level Process
Cooling Alarms, Cooling Data, Cooling Apache email messages are picked up
by BizTalk Receive Location
Receive Pipeline component
o
Parses email Header and store that information into context properties
o
Parses email body and using BizTalk flat file disassembler component
will convert Flat file into XML ( for apache and data ). For alarm parser
XML file is created with appropriate email body as received
Page 3
BizTalk Technical Document
o
Filters out SPAM message ( by looking into email header information as
well as content in email body )
o
Invoke “Archive” Process to archive data based on business
requirement
o
Few Business processes are done in pipeline component ( Message
logging, saving email Subject Line, inserting blacklisted alarms )
Pipeline component will drop either apache, alarm or data XML message to
BizTalk MessageBox
Alarms Data
o
Alarms Orchestration/Workflow will pickup the message and perform
business processes ( Parse Alarms - .net SP Call ) and saving debug
data ( Debug2 )
o
For Noflow, Interlock or NDM alarms, it will call Alarms recurring
Workflow
Email Data
o
Workflow will find out ControllerId based on controller header data and
insert data into ParameterValuesN or ParameterValuesT for each
section data received for controller
Apache Data
o
Workflow will execute business process by calling parseApache stored
procedure
2.4 Schemas – Cooling Controller
Before we define the scenarios itself, it makes sense to look at each of the message
types being used in the scenario.
2.4.1 Cooling Schema
The Cooling Data schema is a flat file schema created using BizTalk Flat File
Schema wizard feature. Each section within the file are created as a separate
schema and all those schemas are then imported into main CoolingFF Schema.
Page 4
BizTalk Technical Document
Since not all the groups can come in data file, “Choice Group” option with max
Occurs of “0” is set for all sections.
The following diagram is a representation of this schema:
Page 5
BizTalk Technical Document
The schema was modified to relax the way input data was received. This was done
to allow looser checking of data in the pipeline ( for e.g. not always all data section
are received in input message )
1
Cooling Property Schema
The Cooling property schema was created to store the most common data field in
message context. When message is received in BizTalk, it contains the actual data
as well as the message context. The message context is same concept as Message
Header, SOAP Header or WCF Header as in a container to store various properties
that can be used throught BizTalk Process.
There are two types of message context properties. Property fields are context
properties used by BizTalk Messaging enging mainly for message routing and some
evaluations within BizTalk Orchestrations. One limitation is they can be upto 255
characters.
It provides centralized mechanism for pulling key pieces of information that is then
easily accessible to BizTalk Server components that are handling the message as it
passes through BizTalk Server.
Page 6
BizTalk Technical Document
Email header related data are used throughout the process to subscribe and/or
route message along the process.
The way to identify if it’s property schema is by looking into it’s Properties windows
with a Schema Type as “Property”
1
Cooling Alarm Schema
The Cooling Alarm schema will transform Cooling Alarm data from email body in Flat
file to message schema
1
Cooling Apache Schema
The Cooling Apache schema will transform Cooling Apache data from email body in
Flat file to message schema
Page 7
BizTalk Technical Document
1
Record Schema
The data that are received for Cooling controller has multiple sections and each
section has multiple rows. For every single rows that is received in data, we need
to update “Parameter” Table for all parameters for that row. To improve the
performance instead of calling stored procedure to update parameter values for
every single column in a row, the whole row is send to SQL server as “Composite”
Operation. Following is the sample of DATA2 Section received with multiple rows
with specific parameter values for that timeframe.
[DATA2]|
02/23/2011 04:15
-999.00
0.00 0.00
128
0
104.22 49.84
0.00 0.00
0.00 0.00
128
0
1151.34
0.00 0.00
128
0
0
0
0.00 -532.00
1.06
-14.51 11.16 94.42 0.00
0
0
128
0
|
0.01
1
128
107.91
1
0
02/23/2011 04:30
-999.00
0.00 0.00
128
0
105.99 50.80
0.00 0.00
0.00 0.00
128
0
1149.05
0.00 0.00
128
0
0
0
0.00 -532.00
1.06
-14.44 11.17 95.27 0.00
0
0
128
0
|
0.01
1
128
109.49
1
1
02/23/2011 04:45
-999.00
0.00 0.00
128
0
105.54 50.84
0.00 0.00
0.00 0.00
128
0
1181.72
0.00 0.00
128
0
0
0
0.00 -532.00
1.00
-14.28 11.14 95.34 0.00
0
0
128
0
|
0.01
1
128
109.51
1
2
02/23/2011 05:00
-999.00
0.00 0.00
128
0
105.84 51.24
0.00 0.00
0.00 0.00
128
0
1190.01
0.00 0.00
128
0
0
0
0.00 -532.00
1.00
-14.28 11.14 95.44 0.00
0
0
128
0
|
0.01
1
128
110.25
1
2
[END_DATA2]|
To process and perform Composite operation on a single row, for every single
section a new Schema is created for simpler mapping process. Looping and parsing
single record from a source data is done within BizTalk Orchestration using xPath.
Page 8
BizTalk Technical Document
Page 9
BizTalk Technical Document
2
Scenario Components
1
Cooling Data Receipt
The purpose of the cooling receipt process is to archive the message that is
received ( with an ability to disable at runtime ) and bring the cooling data into
BizTalk and transform it to a format that can be used to process the message. This
process is represented by the BizTalk receive pipeline as shown below:
Cooling Receive Pipeline Component
The following sections describe each portion of the process.
Page 10
BizTalk Technical Document
1
File Archive
The file archive component was implemented as a BizTalk custom pipeline
component. BizTalk tracing is an alternate way to achieve that as well, but since we
already have an existing process to archive the data to a file the same approach is
taken here. This gives us an exact replica of the file before any processing
commences. Below is a screenshot of the configuration of that pipeline component:
2
Disassemble Component
BizTalk Disassembler pipeline Component is implement to
Single Component to process Cooling water Data, Alarms and Apache Data
Parse the incoming email message ( as *.eml ) and create a strongly typed
.Net Object
Page 11
BizTalk Technical Document
Determine validity of message and filter out Spam
Business Processing like verifying blacklisted alarms, Verify PI data archiving
Promote Context Properties
Archive Data
Disassemble Flat file email Body data and create XML representation using
BizTalk Standard Flat file disassembler.
Page 12
BizTalk Technical Document
2
Helper Components
Helper Components are .Net Class Library that are used throughout the Business
Processing. They are used from within Pipeline Components or from within
Orchestrations.
EmailObject : A strongly typed .net Object to represent email Object
ParseEmailObject : A .net Class to parse email message ( *.eml ) and create a
email Object identifying business specific data like email subject, soldTo,
username, message type
CoolingDynamicTransform : A .net class ( serializable ) used from within
BizTalk Orchestration to perform Dynamic Transformation using XSLT.
Page 13
BizTalk Technical Document
3
1
Cooling Processor
Composite Update
The out of box WCF-SQL adapter enables adapter clients to perform composite operations on
the SQL Server database. A composite operation can include:
Insert, Update, and Delete operations. A Select operation is not supported as part of
a composite operation.
Stored procedures executed as operations.
A single composite operation can have any number of these operations, in any order.
Composite operation pattern is implemented to improve performance and send multiple
calls to Stored procedure as a single composite operation.
The following orchestration represents this process:
2
Data Transformation and Translation
BizTalk mapper is used to create maps, which are used for translating and
transforming xml messages. Depending on the type and complexity of
transformation, following different transformation features in within BizTalk Mapper
are used
Using Inline XSL
Inline XSLT is used in the map below to transform many to one relationship
Page 14
BizTalk Technical Document
Using BizTalk Functoids
Usage of Inbuilt BizTalk Functoids to perform and check if source element is
null or empty and based on the conditional apply the mapping to destination
schema
Using XSL
BizTalk mapper by default uses XSL to perform transformation and
translation. If out of the box functoid provided by BizTalk doesn’t satisfy
business requirement, they can be extended with using custom XSL.
Page 15
BizTalk Technical Document
4
Testability
All BizTalk Artifacts had unit Testing enabled and depending on type of BizTalk
Artifact Object mocking was performed if needed.
1
Schema Testing
BizTalk Projects were enable to allow Schema Testing
Page 16
BizTalk Technical Document
Once the project is enabled, testing can be performed using TestTools provided by
Microsoft BizTalk Assemblies.
2
Maps Testing
Maps can be tested as Schema, alternatively BizUnit ( open source ) was used to
perform BizTalk Map testing as well as validating the output data via XPath
Page 17
BizTalk Technical Document
3
PipelineComponent Testing
Pipelinetesting unit tests are written to use Winterdom Tools
Page 18
BizTalk Technical Document
2
Monitoring and Exception Handling
1
Tracking & Message Monitoring
1
BizTalk
BizTalk has its own tracking and debugging capabilities built into the administration
tool. We can use these abilities to get information about the processes being run as
well as view message contents and processes. Please see screenshots below:
We can also use the administration tool to debug the orchestration processes or
view a process that occurred in the past.
Page 19
BizTalk Technical Document
BizTalk Technical Document
Prepared for
NALCO
Thursday, 1 December 2011
Version 1.0
Prepared by
MPS Partners
Contributors
Ashish Talati
Prepared for NALCO
The information contained in this document represents the current view of MPS Partners, LLC. on the issues discussed as of the date of publication. Because
MPS Partners must respond to changing market conditions, it should not be interpreted to be a commitment on the part of MPS Partners, and MPS Partners
cannot guarantee the accuracy of any information presented after the date of publication.
MPS PARTNERS MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT.
Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be
reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or
otherwise), or for any purpose, without the express written permission of MPS Partners, LLC..
MPS Partners may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document.
Except as expressly provided in any written license agreement from MPS Partners, the furnishing of this document does not give you any license to these
patents, trademarks, copyrights, or other intellectual property.
The descriptions of other companies’ products in this document, if any, are provided only as a convenience to you. Any such references should not be considered
an endorsement or support by MPS Partners. MPS Partners cannot guarantee their accuracy, and the products may change over time. Also, the descriptions are
intended as brief highlights to aid understanding, rather than as thorough coverage. For authoritative descriptions of these products, please consult their
respective manufacturers.
© 2010 MPS Partners, LLC.. All rights reserved. Any use or distribution of these materials without express authorization of MPS Partners is strictly prohibited.
MPS Partners and Windows are either registered trademarks of MPS Partners, LLC. in the United States and/or other countries.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
Page ii
BizTalk Technical Document
Revision and Signoff Sheet
Change Record
Date
Author
Version
Change reference
10/18/201
1
Ashish Talati
1.0
Created
11/30/201
1
Ashish Talati
1.1
Completed Cooling Controller
Reviewers
Name
Version
approved
Position
Date
Page
BizTalk Technical Document
Table of Contents
1
Environment.....................................................................................1
1.1
Overview............................................................................................................ 1
1.2
Installation Summary......................................................................................... 1
2
Implementation................................................................................2
2.1
Overview............................................................................................................ 2
2.2
TFS..................................................................................................................... 3
2.3
Cooling Controller Process.................................................................................3
2.4
Schemas – Cooling Controller............................................................................ 4
2.4.1 Cooling Schema.......................................................................................... 4
2.4.1 Cooling Property Schema............................................................................6
2.4.1 Cooling Alarm Schema................................................................................7
2.4.1 Cooling Apache Schema............................................................................. 7
2.4.1 Record Schema........................................................................................... 8
2.5
Scenario Components...................................................................................... 10
2.5.1 Cooling Data Receipt................................................................................ 10
2.5.2 Helper Components.................................................................................. 13
2.5.3 Cooling Processor..................................................................................... 14
2.5.4 Testability.................................................................................................. 16
3
3.1
Monitoring and Exception Handling..................................................19
Tracking & Message Monitoring.......................................................................19
3.1.1 BizTalk....................................................................................................... 19
Page
BizTalk Technical Document
1
Environment
1.1 Overview
The NALCO BizTalk proof of concept did not plan for performance testing as a
success criteria, so a highly available environment was not created. Two
environments were created for the POC:
Development Workstation: Used by developers to create and test the
solution.
Developer Integration Server: Solution was deployed to this server for testing
purposes. This environment is typically were multiple developers’ code is
integrated and tested before moving on to a QA process.
The development workstation had the following software installed:
Windows 7
IIS 7
SQL Server 2008 R2
Visual Studio 2010
BizTalk Server 2010
The development integration server had the following software installed:
Windows 2008 R2 Enterprise
IIS 7
SQL Server 2008 R2
Visual Studio 2010
BizTalk Server 2010
1.2 Installation Summary
Because each of the environments were installed with single server envionrments,
the installations were very basic and followed the installation guides from Microsoft.
The following guides were used:
Development Workstation:
Installing BizTalk Server 2010 on Windows 7 and Windows Vista.docx
Development Integration Server:
Installing BizTalk Server 2010 on Windows Server 2008 R2 and 2008.docx
Page 1
BizTalk Technical Document
2
Implementation
2.1 Overview
The following is the high level process when a Boiler or Cooling controller Data file
( either in email format or an XML Format ) is received:
File Receipt:
o
File is Archived
o
Data is parsed and Converted to Schema
o
Optionally if file is compressed, uncompress it
o
Component adds context information into message Header
o
Mapper component: transforms file to canonical
o
Schema Validator
Validation & Enrichment
o
Business validation
Business Process
o
Cooling Data Process
o
Cooling Alarms Process
o
Boiler Data, Event, Alarms Process
o
Web for All Data and Configuration Process
Monitoring
o
Monitoring Orchestration Performance
This document will now describe each component of the scenario in a little more
detail to show how this was implemented using BizTalk Server.
Page 2
BizTalk Technical Document
2.2 TFS
All source code are stored in TFS @ $/3DT Web/WPS Biztalk Projects
A separate BizTalk application and solution are created for each Parser. Every
solution is following same naming convention based on type of artifacts for BizTalk.
The default naming convention is that the name of a folder containing a BizTalk
Project is identical to project’s output DLL name ( without extension ). For example,
if BizTalk project resides in Cooling.Controller.Orchestrations folder, then that project
must output a DLL named Cooling.Controller.Orchestrations.dll
Typical example of project naming structure is
Orchestration = Project.Orchestraions
Pipelines = Project.Pipelines
Pipeline Components = Project.PipelineComponents
Helper Components = Project.Components
Schemas = Project.Schema ( or Project.Generated.Schemas )
Maps/Transformaton = Project.Transform
Unit Tests = Project.Tests
2.3 Cooling Controller Process
High Level Process
Cooling Alarms, Cooling Data, Cooling Apache email messages are picked up
by BizTalk Receive Location
Receive Pipeline component
o
Parses email Header and store that information into context properties
o
Parses email body and using BizTalk flat file disassembler component
will convert Flat file into XML ( for apache and data ). For alarm parser
XML file is created with appropriate email body as received
Page 3
BizTalk Technical Document
o
Filters out SPAM message ( by looking into email header information as
well as content in email body )
o
Invoke “Archive” Process to archive data based on business
requirement
o
Few Business processes are done in pipeline component ( Message
logging, saving email Subject Line, inserting blacklisted alarms )
Pipeline component will drop either apache, alarm or data XML message to
BizTalk MessageBox
Alarms Data
o
Alarms Orchestration/Workflow will pickup the message and perform
business processes ( Parse Alarms - .net SP Call ) and saving debug
data ( Debug2 )
o
For Noflow, Interlock or NDM alarms, it will call Alarms recurring
Workflow
Email Data
o
Workflow will find out ControllerId based on controller header data and
insert data into ParameterValuesN or ParameterValuesT for each
section data received for controller
Apache Data
o
Workflow will execute business process by calling parseApache stored
procedure
2.4 Schemas – Cooling Controller
Before we define the scenarios itself, it makes sense to look at each of the message
types being used in the scenario.
2.4.1 Cooling Schema
The Cooling Data schema is a flat file schema created using BizTalk Flat File
Schema wizard feature. Each section within the file are created as a separate
schema and all those schemas are then imported into main CoolingFF Schema.
Page 4
BizTalk Technical Document
Since not all the groups can come in data file, “Choice Group” option with max
Occurs of “0” is set for all sections.
The following diagram is a representation of this schema:
Page 5
BizTalk Technical Document
The schema was modified to relax the way input data was received. This was done
to allow looser checking of data in the pipeline ( for e.g. not always all data section
are received in input message )
1
Cooling Property Schema
The Cooling property schema was created to store the most common data field in
message context. When message is received in BizTalk, it contains the actual data
as well as the message context. The message context is same concept as Message
Header, SOAP Header or WCF Header as in a container to store various properties
that can be used throught BizTalk Process.
There are two types of message context properties. Property fields are context
properties used by BizTalk Messaging enging mainly for message routing and some
evaluations within BizTalk Orchestrations. One limitation is they can be upto 255
characters.
It provides centralized mechanism for pulling key pieces of information that is then
easily accessible to BizTalk Server components that are handling the message as it
passes through BizTalk Server.
Page 6
BizTalk Technical Document
Email header related data are used throughout the process to subscribe and/or
route message along the process.
The way to identify if it’s property schema is by looking into it’s Properties windows
with a Schema Type as “Property”
1
Cooling Alarm Schema
The Cooling Alarm schema will transform Cooling Alarm data from email body in Flat
file to message schema
1
Cooling Apache Schema
The Cooling Apache schema will transform Cooling Apache data from email body in
Flat file to message schema
Page 7
BizTalk Technical Document
1
Record Schema
The data that are received for Cooling controller has multiple sections and each
section has multiple rows. For every single rows that is received in data, we need
to update “Parameter” Table for all parameters for that row. To improve the
performance instead of calling stored procedure to update parameter values for
every single column in a row, the whole row is send to SQL server as “Composite”
Operation. Following is the sample of DATA2 Section received with multiple rows
with specific parameter values for that timeframe.
[DATA2]|
02/23/2011 04:15
-999.00
0.00 0.00
128
0
104.22 49.84
0.00 0.00
0.00 0.00
128
0
1151.34
0.00 0.00
128
0
0
0
0.00 -532.00
1.06
-14.51 11.16 94.42 0.00
0
0
128
0
|
0.01
1
128
107.91
1
0
02/23/2011 04:30
-999.00
0.00 0.00
128
0
105.99 50.80
0.00 0.00
0.00 0.00
128
0
1149.05
0.00 0.00
128
0
0
0
0.00 -532.00
1.06
-14.44 11.17 95.27 0.00
0
0
128
0
|
0.01
1
128
109.49
1
1
02/23/2011 04:45
-999.00
0.00 0.00
128
0
105.54 50.84
0.00 0.00
0.00 0.00
128
0
1181.72
0.00 0.00
128
0
0
0
0.00 -532.00
1.00
-14.28 11.14 95.34 0.00
0
0
128
0
|
0.01
1
128
109.51
1
2
02/23/2011 05:00
-999.00
0.00 0.00
128
0
105.84 51.24
0.00 0.00
0.00 0.00
128
0
1190.01
0.00 0.00
128
0
0
0
0.00 -532.00
1.00
-14.28 11.14 95.44 0.00
0
0
128
0
|
0.01
1
128
110.25
1
2
[END_DATA2]|
To process and perform Composite operation on a single row, for every single
section a new Schema is created for simpler mapping process. Looping and parsing
single record from a source data is done within BizTalk Orchestration using xPath.
Page 8
BizTalk Technical Document
Page 9
BizTalk Technical Document
2
Scenario Components
1
Cooling Data Receipt
The purpose of the cooling receipt process is to archive the message that is
received ( with an ability to disable at runtime ) and bring the cooling data into
BizTalk and transform it to a format that can be used to process the message. This
process is represented by the BizTalk receive pipeline as shown below:
Cooling Receive Pipeline Component
The following sections describe each portion of the process.
Page 10
BizTalk Technical Document
1
File Archive
The file archive component was implemented as a BizTalk custom pipeline
component. BizTalk tracing is an alternate way to achieve that as well, but since we
already have an existing process to archive the data to a file the same approach is
taken here. This gives us an exact replica of the file before any processing
commences. Below is a screenshot of the configuration of that pipeline component:
2
Disassemble Component
BizTalk Disassembler pipeline Component is implement to
Single Component to process Cooling water Data, Alarms and Apache Data
Parse the incoming email message ( as *.eml ) and create a strongly typed
.Net Object
Page 11
BizTalk Technical Document
Determine validity of message and filter out Spam
Business Processing like verifying blacklisted alarms, Verify PI data archiving
Promote Context Properties
Archive Data
Disassemble Flat file email Body data and create XML representation using
BizTalk Standard Flat file disassembler.
Page 12
BizTalk Technical Document
2
Helper Components
Helper Components are .Net Class Library that are used throughout the Business
Processing. They are used from within Pipeline Components or from within
Orchestrations.
EmailObject : A strongly typed .net Object to represent email Object
ParseEmailObject : A .net Class to parse email message ( *.eml ) and create a
email Object identifying business specific data like email subject, soldTo,
username, message type
CoolingDynamicTransform : A .net class ( serializable ) used from within
BizTalk Orchestration to perform Dynamic Transformation using XSLT.
Page 13
BizTalk Technical Document
3
1
Cooling Processor
Composite Update
The out of box WCF-SQL adapter enables adapter clients to perform composite operations on
the SQL Server database. A composite operation can include:
Insert, Update, and Delete operations. A Select operation is not supported as part of
a composite operation.
Stored procedures executed as operations.
A single composite operation can have any number of these operations, in any order.
Composite operation pattern is implemented to improve performance and send multiple
calls to Stored procedure as a single composite operation.
The following orchestration represents this process:
2
Data Transformation and Translation
BizTalk mapper is used to create maps, which are used for translating and
transforming xml messages. Depending on the type and complexity of
transformation, following different transformation features in within BizTalk Mapper
are used
Using Inline XSL
Inline XSLT is used in the map below to transform many to one relationship
Page 14
BizTalk Technical Document
Using BizTalk Functoids
Usage of Inbuilt BizTalk Functoids to perform and check if source element is
null or empty and based on the conditional apply the mapping to destination
schema
Using XSL
BizTalk mapper by default uses XSL to perform transformation and
translation. If out of the box functoid provided by BizTalk doesn’t satisfy
business requirement, they can be extended with using custom XSL.
Page 15
BizTalk Technical Document
4
Testability
All BizTalk Artifacts had unit Testing enabled and depending on type of BizTalk
Artifact Object mocking was performed if needed.
1
Schema Testing
BizTalk Projects were enable to allow Schema Testing
Page 16
BizTalk Technical Document
Once the project is enabled, testing can be performed using TestTools provided by
Microsoft BizTalk Assemblies.
2
Maps Testing
Maps can be tested as Schema, alternatively BizUnit ( open source ) was used to
perform BizTalk Map testing as well as validating the output data via XPath
Page 17
BizTalk Technical Document
3
PipelineComponent Testing
Pipelinetesting unit tests are written to use Winterdom Tools
Page 18
BizTalk Technical Document
2
Monitoring and Exception Handling
1
Tracking & Message Monitoring
1
BizTalk
BizTalk has its own tracking and debugging capabilities built into the administration
tool. We can use these abilities to get information about the processes being run as
well as view message contents and processes. Please see screenshots below:
We can also use the administration tool to debug the orchestration processes or
view a process that occurred in the past.
Page 19
BizTalk Technical Document
