Business Process extension with Integration B2B

Introduction

Already for a few years Enterprises are aware of the need of Application Integration. The days we build large integrated Custom Development Application are behind us (although I know of some exceptions). Enterprises tend more and more to choose for standard application suites. But it is an illusion to think a company could do with only one integrated standard application suite. A company certainly has special business needs it has to full fill with other applications, that can be standard commercial ones or custom developed applications. So there is a need to application integration.

The integration discussion is also grown to the business level with Process Integration and Service Oriented Architecture (SOA). It is not only about synchronizing data between applications (data integration) or exchanging business events (as in Enterprise Application Integration). A company has a business process that has to be executed and preferably in an agile way controlled by a business process manager that merely call services of applications instead of control applications or be controlled by an application. Like in SOA. This way the Business Process can be modeled in a way that suites best with the companies business model but can be adapted according to changed insights of the business model or changes in the marketplace. It then can support business process optimization. But also it opens the possibility to extend the business process over the boundaries of the organization.

Conventionally companies exchange data by ftp-ing large datafiles to each other once in a while. Batch processes have to read and process the file and execute different functions for each record in the file. These functions can be seen as services that should be called on message driven basis. These messages could be received from a trading partner on ad-hoc basis and should lead to an invocation of certain parts of the Business Process as a Business Event.

However, a company does not want to open up his business process for just every individual in the evil kosmos. Only the trading partners that it does business with and probably not every business partner may assert every service, raise every possible business event. Also not every business partner support every protocol. Some do ftp, some do http. Some do EDI-documents, some do Custom documents. Some even do some highlevel business protocols like Rosettanet.

Another thing a company wants abstract of is the guarantee of the delivering. When companies exchange files of multiples of data-records, they have to decide how to handle faults in the processing of individual records. Does that lead to the withdrawal of the complete datafile and thus rolling back the correctly processed data records? Or does the delivering partner receives a list of only disapproved records? With a message driven solution a company has to be sure that a message has been delivered correctly. When it has an acknowledgment of the delivery of the message it should not care any more about the processing of the message. Only when the message turns out to be faulty it should get a functional notification of disapproval of the message. It should not bother about the delivery.

A B2B product like Integration B2B takes care of the certainty of delivery of messages. A backoffice raises a business action for a trading partner to Integration B2B and Integration B2B will deliver the message to the partner using the right protocol. It handles the acknowledgments and if needed the resending of a message at time-outs for a predefined maximum number of times. Also it signs and/or encrypts the messages using certificates. But everything provided that it is according to the agreement that is defined between the two partners. So it disapproves business messages send by or send to trading partners when they are not defined in the agreement or when the agreement is not valid on the current date. For the backoffice a business message can be considered as a Business Event, in B2B terms called a business action. Doing so, Integration B2B provide a means of delivering configurable services that can be called within a Business Process or that call or invoke a Business Process.

When to use Integration B2B

When to use Integration B2B, ESB or Webservices? Why use Integration B2B when one can also create a Webservice that is secured by authentication? Or why create webservices while it is also possible to define an ESB service that calls an application interface using an adapter?

All three approaches provide a means to extend the business process to interact with a trading partner. Although there is overlap between the approaches in terms of functionality, they're not the same and certainly have their functions.

Webservices

In a Service Oriented Architecture it is about services. But a service is not per definition a Webservice. A webservice is in fact a manner of exposing a service based on industry standards. It enables the service to be called regardless of technology. The technology that calls the service can be totally different and also on a geographically very different location as the called service. I would dare to state that a webservice should be no more than a wrapper around a service. I think it is a best practice to separate the implementation of the service and the exposure of the service as a webservice, even if the implementation and the exposure are using the same technology. But that is a different discussion.

Webservices can be secured using industry standards on authentication and authorization. Also the data exchange can be encrypted for example using SSL.

But webservices have to be created per exchange. Ok, a webservice can support multiple operations, described in the WSDL (Webservice Description Language) document. But every operation have to be programmed and made available through the WSDL.

Another disadvantage is that a Webservice in nature is a request/response operation. Although a webservice can be synchronous or asynchronous, the request and the response is closely related. With a synchronous webservice the response comes as a result of the request in one operation. With an asynchronous webservice the webservice calls the requesting client back with the response. But as described in the WSDL the client expects one certain response and has to trust that the webservice will come with that response. There are no means of guaranteeing message delivery and also no means of doing a retry if the response is not received within a certain amount of time. These mechanisms have to be build around the webservice exchange. The same counts for juridical protocols that state that a request should be followed by an acknowledgment of receipt as well as a response message.

Also there are no means of Trading partner management, controlling what actions a certain partner may call during a certain period. Of course there are tools that fill that gap (like Oracle Webservice Manager).

ESB

The ESB is an integration tool that can pick up events that are raised in various ways. Events can be raised by placing a file in a directory, enqueueing a message on a queue, updating data in a database or a webservice call. Adapters exist for picking up these technical events but also for events in specific standard Applications. On these events various actions can be subscribed that can be actually one or more of the counterparts of the events named above. So interfaces in packaged applications can be called, database updates can be done, messages enqueued on queues (AQ, JMS, MQ Series, etc.), files can be created or webservices can be called. Also BPEL Processes can be called to perform more complex processing. The flexibility and the publish subscribe features of ESB make it possible to integrate with trading partners or expose webservices on specific services in the applications of a trading partner, even without programming a line of code.

But it lacks of more complex protocol handling. It is aimed for internal application to application integration (A2A), but for doing authentication and authorization other means have to be put in place (like Oracle Webservice Manager, OWSM). And complexer business protocol handling have to be coded.

Integration B2B

With Integration B2B it is possible to send messages over multiple protocols and define which protocol to be used for a certain business action for a trading partner. A trading partner can be defined with its capabilities, that define which business actions it can receive or initiate. Also the particular communication channels can be defined for each trading partner. A communication channel can be file, SMTP or HTTP(S). The corresponding servers, authentication, Signing and Encryption certificates and parameters can be defined.

An agreement defines which supported business action a tradingpartner may initiate or receive within the validity-period of the agreement. Integration B2B will not except a business action that is initiated by a tradingpartner that is not defined within a valid agreement. Integration B2B takes care of the guaranteed delivery of messages over a predefined communication channel for a certain trading partner. Provided that the protocol supports the guaranteed delivery of messages (like in ebXML). Also Integration B2B will sign and encrypt messages using certificates if defined.

This can be defined using a UI in a declarative way. No line of code have to be programmed. Of course the backoffice have to be coupled to Integration B2B. The backoffice applications should be able to deliver messages to Integration B2B using its internal delivery channel. For Integration B2B the preferred internal delivery channel is AQ. So when an application (this can be ESB or BPEL PM, but also custom developed functionality) can enqueue a message on the queue, Integration B2B can send it. Using BPEL PM a message can be send without coding anything.

Supported Protocols, Standards and certification

Integration B2B supports an extensive list of protocols, standards. Below the protocols that I could find in a datasheet of B2B.

Industry Standards	Documents	Message formats	Transports	Exchanges
EDI UCCnet RosettaNet CIDX PIDX VICS ebXML UBL	UBL UN/EDIFACT X12 X12 Property & Casualty X12 HIPAA X12 HIPAA External code list X12N Life & Annuity X12N Healthcare NCPDP SCRIPT HL7 OAG cXML xCBL DTD W3C XML Schema Custom	MIME SMIME XMLDSig XMLEncrypt SOAP	File FTP FTPS HTTP HTTPS SMTP TCP / IP	AS2 ebMS RNIF MLLP

Integration B2B is certified on several protocols by accredited instances. For example the Drummond group certified Integration B2B on ebXML.

The ebXML standard is a very useful protocol that supports encryption and signing and uses SOAP over Http to exchange messages. It supports reliable messaging using acknowledgments and an configurable number of retries with configurable time to acknowledge. Since it supports any kind of message the specific transformation of the message can be handled within Integration B2B or outside using the transformation capabilities of ESB or BPEL PM.

Where to put in the Integration Architecture

How to couple Integration B2B with the backoffice? What topologies to use? Since a tool as Integration B2B can be considered as just another application in the business's IT-landscape there are several topologies that can be used to integrate a B2B tool with the backoffice.

Tightly coupled

The preferred internal delivery channel of Integration B2B is AQ. So a database oriented application could directly enqueue out-going messages in the out-queue of B2B and consume in-comming messages from the In-queue. A java-application could use the Oracle AQJms interfaces to produce and/or consume messages to B2B. Also a file based coupling is supported. However such a tight coupling is not recommended. It will lead to problems in routing the messages internally between the message handling business processes and/or the different applications a business uses.

When each application that uses the gateway has it's own set of businessactions with the remote tradingpartners it is possible to couple each application in this way with Integration B2B. But there are minor capabilities to route messages over the applications. Actually the only way is define each application as a partner in the agreemeents with the remote tradingpartners. Each application should then have it's own Routing ID, an application specific code that the remote tradingpartner have to denote on sending a message. Then the remote tradingpartner must have “knowledge” about the internal applications of the business. When the business decides to change the business process for example by migrating applications to a standard application suite, the remote tradingpartner is needed to change his process too.

Recommended is to have a setup that no matter how messages are processed internally, which applciation is handling certain messages, the remote trading partner should have only one Routing ID. To the outside world the company should be exposed as one tradingpartner.

Using EAI/SOA

This is the recommended topology. An EAI environment Integration B2B is considered as a spoke of the EAI tool. In a SOA environment the different Business Actions defined in Integration B2B are considered as seperate services. By this means an abstraction layer is build between Integration B2B and the backoffice. Actually Integration B2B is an extra application that supports the extension of the business processes over the boundaries of the business. So that each other application or each separate business process can make use of it. Routing over the different applications can be done within the business processes of a Process Manager like BPEL PM or using publish/subscribe mechanisms of an ESB. Since different business processes and applications send and receive messages over the same infrastructure, project-collisions should be considered. When there is a clear separation of business actions over the different processes and applications problems are little. But when there is a overlap in used businessactions then the scenarios have to be thought over carefully and tested thoroughly. Also if one project is responsible for implementing the Integration B2B infrastructure and is also on the overall critical path of all running projects, other projects can be stressed. The other projects can't get their interactions implementated.

Routing Service

In spite of my recommendations above there are several reasons to separate the different applications within an enterprise and connect them to Integration B2B independly. It can be that the internal processes are to separate. Or for projectmanagement reasons it is not wise to incorporate them into one. For example for planning or scoping reasons. In that case you should provide a means of routing incoming messages over the different applications and/or processes. Sometimes that is easy when there is no overlap (see the section about tightly coupling). But it might be that different applications or processes have the same interactions. A routing service can then solve the problem. A routing service is a piece of functionality that receives incoming messages and route them over the different applications. For a product like Integration B2B that uses AQ as an internal Delivery channel this service can be a quite easy Pl/Sql service in the database, that is subscribed (see documentation on Pl/Sql Callback notifications in Oracle Streams AQ) on the in-queue of Integration B2B. To do the job properly the service should also be responsible of receiving out going messages of the internal applications to be send via Integration B2B. The reason for this is that the main algorithm for doing the routing is to try to couple an incoming message to an earlier send outgoing message. For the routing four types of routing rules can be defined:

• On In-reply-to message id: when a message is a response on an earlier outgoing message this is denoted by supplying the id of the request message as an in-reply-to-message-id. When the Routing Service processes the outgoing request message it can store that in a message-table related to the application that send the message. When an incoming messages refers to that message using the id, the routing service can query on the message table with that id. When the request message is found the response message can be routed to the corresponding application. This should be a 100% match for respons messages: when a request-message cannot be found the trading partner probably provided an invalid in-reply-to-message-id.
  An additional requirement for this kind of routing is that the different applications should use mutually exclusive message id ranges to get enterprise wide unique message-id's.

• On Business Action/Document type: when no in-reply-to-message-id is provided the routing service can check the Business Action or Document Type definitions. The Document Type definitions describe for ebXML messages the ebMS header attributes (the envelop). When a Business action/Document Type combination only exist for one application a routing rule of this type can denote the application to route to for that particular Business Action/Document type.

• On content based routing: for outgoing messages rules can be defined that have xpath-expressions to extract identification-information from the outgoing message and store that in the Routing Service's repository referring to the outgoing message. For incoming messages the same mechanism can provide an identification code that can be looked up in the repository. Finding an outgoing message this way lead to the application to route to. This is a type of routing that is especially useful for incoming messages on ad-hoc-basis, provided that they relate to particular cases (for example order-id's) in the content. An additional requirement (like in-reply-to-routing) is the need for enterprise wide unique identification expressions.

Multi-hop

Integration B2B contains functionality to function as a hop. This is useful in ebXML based infrastructures. A tradingpartner sends messages to other tradingpartners. But instead of communicating directly to the transport-server of the tradingpartner it connects to the Hop. The Hop (a central B2B implementation) reads the envelop and concludes that the message is not adressed to him. So it will forward the message to the transport server that it considers to be the transportserver of the addressed tradingpartner. This can in fact be another Hop. The chain of hops can be any number, provided that each hop has a correct configuration routing-definitions.

This configuration is especially useful in those wide-area networks where each trading partner has its own set of firewalls, etc. Using a central hop centralizes the complexity. Only the hop has to know the addresses of each firewalled transportserver of the tradingpartners. Each tradingpartner only need to know how to connect to the hop. If a ping to the hop succeed, each other tradingpartner can be reached through the hop.

This topology is not useful within an enterprise to connect applications using ebXML. That does not make sense. Doing that is comparable of having UPS deliver your reports from your desk to the desk of your colleague in the room next door. It is still needed to create interfaces to Integration B2b and to write the messages and produce them on a queue and consume from another queue. This has to be done for each application within the enterprise. Also n+1 implementations of Integration B2B is needed, one for each application and one extra for the hop. This is very expensive, where one (quite simple) implementation of AQ or ESB would suffice.

Oracle EBS's XML Gateway

Integration B2B has an internal delivery channel especially for use with XML Gateway of EBS. This is exist for backward compatibility. This can be useful for those EBS implementations where EBS is the core application of the company and no other applications have to be coupled to B2B. Mark that this is actually an implementation of the first topology mentioned: tightly coupled. This can suffice for a company but keep in mind that there is nothing as permanent as a temporary solution! And nothing changes as much as requirements, not even the weather. So requirements may read that in no hundred years another application have to interact with that same tradingpartner using the same interactions. And within a year after going live the first exception is defined. But once an application is coupled tightly to Integration B2B and interfaces are build, it is not very likely that the program management chooses to break it up and introduce a ESB or EAI tool in between. That's why I don't recommend this and see a combination with SOASuite (or comparable middleware solution) as the preferred topology.

Conclusion

Business to Business integration is growing in importance. And there are several ways to integrate with Trading partners. But it is very important to think about how to integrate the B2B-gateway within the internal business's integration. Consider that a company's partners are part of his business process. So it's best practice to consider a B2B-Business action as a Business Service.

Martien van den Akker
Integration Specialist
Oracle Netherlands

How to set up an SSL test environment with OC4J and Oracle BPEL

Recently, I needed to configure a SSL test environment configured with a self-signed certificate, a Oracle BPEL process for the SSL client and a EJB3 HelloWorld service secured with SSL. As an Oracle employee the choice for a (standalone) OC4J as the server container should be evident.

This post is hopefully a step by step tutorial to configure this SSL test environment.

OC4J container
- Download and install a standalone OC4J container version 10.1.3.3 from OTN

Developing a HelloWord EJB3 service
- Use the EJB3 session bean wizard in JDeveloper 10.1.3.x to generate the skeleton code for you HelloWorld service. Make sure that you also generate a webservice interface.
- Add a simple HelloWorld method to your bean implemention and add this method signature to the webservice interface as well. The method will now be exposed as a webservice operation when
deployed to the OC4J container.
- Add JSR-181 annotation to webservice interface to configure the webservice wsdl. Here's an example that I used:

package nl.oracle.com.ssl;

import java.rmi.Remote;

import javax.jws.WebMethod;
import javax.jws.WebParam;
import javax.jws.WebService;

import oracle.webservices.annotations.Deployment;


@WebService(name="HelloWorldWS",
            targetNamespace="http://ejb.oracle.com/targetNamespace",
            serviceName="HelloWorldWSService")
@Deployment(contextPath="ejb-ws",
            uriPath="HelloWorldWS",
            portName="HelloWorldService")

public interface HelloWorldEJBWebService extends Remote {
    
    @WebMethod
    public void helloWorld(@WebParam(name="input") String in);
}

After finishing the implementation cycle you generate a deployement descriptor and deploy it to your just installed OC4J container. Verify that you can access the webservice and test it with the Test Webservice functionality.

Configure OC4J for SSL
(look here for more info)

First, we need to generate a self-signed certificate and put it in a new keystore. Use the keytool utility from java SDK.
Generate a certificate using the following commando:

keytool -genkey -alias oc4j-server -keyalg RSA -keypass welcome -storepass welcome -keystore c:\oc4jserver.jks

You will be prompted for specific certificate information. YOU HAVE TO MAKE SURE THAT THE "FIRSTNAME/LASTNAME" question is answered with the hostname of the server on which you have installed the OC4J container (I used localhost, because the container is running on my laptop). This will overcome "wrong hostname" exceptions later on.

Now you have a RSA encrypted certificate in the oc4jserver.jks keystore located in the root of your C drive.

As a second step, we need to configure which web application we want to secure with SSL. Open the $ORACLE_HOME/j2ee/home/config/default-web-site.xml file
and copy all the contents to $ORACLE_HOME/j2ee/home/config/secure-web-site.xml (create the file when it does not exist yet). Change the element and make
sure that it looks like the example below. Change where needed.

    <web-site   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                xsi:noNamespaceSchemaLocation="http://xmlns.oracle.com/oracleas/schema/web-site-10_0.xsd"
                port="4443"
                secure="true"
                display-name="OC4J 10g (10.1.3) Default Web Site"
                schema-major-version="10"
                schema-minor-version="0" >
            ...
            <ssl-config keystore="c:\oc4jserver.jks" keystore-password="welcome" />
            ...
    </web-site> 

Delete all the web-app entries except the helloworld entry and the default web application entry. Here is the one that I am using:

<?xml version="1.0"?>

<web-site xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:noNamespaceSchemaLocation="http://xmlns.oracle.com/oracleas/schema/web-site-10_0.xsd"  
port="4443" secure="true" display-name="OC4J 10g (10.1.3) Default Web Site" schema-major-version="10" schema-minor-version="0" > 
 <default-web-app application="default" name="defaultWebApp" />
  <web-app application="helloworld" name="helloworld" load-on-startup="true" root="/ejb-ws" />
 <access-log path="../log/default-web-access.log" split="day" />
 <ssl-config factory="com.evermind.ssl.JSSESSLServerSocketFactory" keystore="d:/java-keystores/mykeystore.jks" keystore-password="welcome" truststore-password=""/>
</web-site>

As a final step we need to tell the OC4J instance that we have some SSL secured web applications add the following line to the server.xml (located under: $ORACLE_HOME/j2ee/home/config/):

<web-site path="./secure-web-site.xml" />

Basically, we now have specified that the OC4J container should secure the HelloWorld webservice with SSL and using the certificate in the specified keystore. You can test this by going to the HTTPS url of the HelloWorld test service (change http to https and change the port to 4443 of the URL you used before).

Configuring JDeveloper and BPEL for the SSL HelloWorld service
To be able to invoke the HelloWorld service over HTTPS we need to make sure that JDeveloper and the BPEL container trust the certificate. I assume that you have installed JDeveloper 10.1.3.x and a BPEL container (I have used a complete SOA Suite installation).

First we have to export the certificate from the oc4j keystore:

keytool -keystore c:\oc4jserver.jks -export -alias oc4jserver-server -file c:\oc4jserver.cer

Now we have a trusted certificate oc4jserver.cer that need to import in the keystore of JDeveloper and BPEL. The command examples will use the standard Java SDK keystore cacerts and
the default keystore password changeit:

  keytool -keystore <JDEV_HOME>\jdk\jre\lib\security\cacerts -import -file c:\oc4jserver.cer

Do the same for your BPEL installation.

The SSL configuration is finished now. I leave the implementation of the BPEL process that invokes the HelloWorld service over HTTPS as an exercise for the reader :)

In Action: Adapter Rejection Handlers in Oracle BPEL

In Oracle BPEL you can use several types (e.g. file, JMS or Oracle AQ) of technology adapters to activate a bpel process. This is done by using a BPEL receive task on a adapter partner link that is that actually is polling a file directory for specific files or is listening on a JMS or Oracle AQ queue for inbound messages. In most cases the inbound message or file contains XML structured data or data that needs to be translated to a custom native XML structure. In both cases the data must cohere to a defined structure otherwise the data cannot be translated to the XML type of the variable that the bpel process will use the store the message for later use. But what if the data doesn't cohere to the structure?

The adapter will reject malformed messaged and will log a message and will store the rejected message in a default folder. In the BPM console you don't see any indication that a message has been rejected by the inbound adapter. So this is not very convenient for a production environment. The use of rejection handlers makes it possible to handle rejected messages in a proper way. A rejection handler must defined in bpel.xml with a adapter activation agent and only works for inbound bpel receive operations. Rejection handlers come in for different flavors:

• File based


• Oracle AQ based


• BPEL process rejection handler


• WSIF based

In this posting I will only discuss the BPEL process rejection handler. Please look here here for more information about the other three flavors.
To use a bpel process rejection handler you have to define a bpel process that takes an input message of type RejectedMessage. Like:

<message name="RejectionMessage">

<part name="message" element="err:RejectedMessage">

</part>
</message>

The schema for this type is in the rejectionmessage.wsdl file in the xmllib directory. In the wsdl file of your BPEL process you can just import this wsdl file and the appropriate partnerlink. E.g.

<definitions xmlns="http://schemas.xmlsoap.org/wsdl/" bpws="http://schemas.xmlsoap.org/ws/2003/03/business-process/" plnk="http://schemas.xmlsoap.org/ws/2003/05/partner-link/" tns="http://xmlns.oracle.com/pcbpel/rejectionHandler" xsd="http://www.w3.org/2001/XMLSchema" name="RejectionMessage" targetnamespace="http://xmlns.oracle.com/pcbpel/rejectionHandler">
  <import location="http://genomw01:7777/Artifacts/RejectionMessage.wsdl" namespace="http://xmlns.oracle.com/pcbpel/rejectionHandler">
  <plnk:partnerlinktype name="RejectionHandlerPortType_PL">
      <plnk:role name="RejectionHandlerPortType_Role">
          <plnk:porttype name="tns:RejectionHandlerPortType">
      </plnk:porttype>
  </plnk:role>
</plnk:partnerlinktype>

In this example BPEL wsdl definition file you can see that the wsdl is imported (a project copy is used here) and that RejectionHandlerPortType is referenced here.

In the RejectionHandler process you have now access to the rejection reason, payload of the reject message and many more useful attributes to show a proper error message to an administrator. For example, you can use your custom Error Hospital for further processing.

The final things you have to do to make things work is adding an rejection handler property to the adapter activation agent in the bpel.xml file. E.g:

<activationagents>
    <activationagent classname="oracle.tip.adapter.fw.agent.jca.JCAActivationAgent" partnerlink="DEQUEUE_DCS731_B2B_IP_IN_QUEUE">
      <property name="portType">Dequeue_ptt</property>
      <property name="rejectedMessageHandlers">bpel://default|JCARejectionHandler|handleRejection|message</property>
    </activationagent>
</activationagents>

Just applying some small steps you are able to add more robustness to you inbound adapters so that malformed rejected message will eventually reach the system administrator that can decide how to deal with them. This sounds more convenient than scanning log files for possible rejected messages :)

XML Debatching in File / FTP adapter

With the release of patchset 10.1.3.3 for Oracle SOA Suite 10.1.3, some cool new features have been introduced that I would like to bring to your attention. Besides the long-awaited error-policy framework, we now also get the option to de-batch large XML files when polling them from a file or FTP location. Debatching has been an option for the File and FTP adapters for some time now, but was always restricted to native (flat) files (typically record-based layouts). Large XML files have always been a challenge to BPEL, because internally XML is loaded in a DOM tree, which takes a considerable amount of memory compared to the original size of the document in its text form.

What does de-batching do?

Debatching polls for incoming files, and when it finds a file and starts reading it, it doesn't start a single new instance of a BPEL process and delivers the file contents, but instead it chops up the file in "batches" of configurable size (typically 1), and starts a new instance for each batch. So for a file of 1000 records (lines) and a batch size of 1, you will end up with 1000 new instances. There is no guarantee that these 1000 instances will be executed in order of the records in the original file, mind you. These instances will be processed by the pool of engine threads just like any other active instance. For XML, as part of the patch install instructions for the 10.1.3.3 patchset, you install a "stream-based" XML parser called STaX, which is well-known in the Java community. Instead of reading the whole file into main memory, the STaX parser just "browses" like a lexical scanner through the XML file and does not build large trees of objects in memory.

How it works

To activate the de-batching behaviour of the adapter, you have to add the "PublishSize" attribute to the jca:operation tag in the adapter's wsdl:

<jca:operation

PhysicalDirectory="D:\"

ActivationSpec="oracle.tip.adapter.file.inbound.FileActivationSpec"
DeleteFile="true"

IncludeFiles=".*\.xml"
PublishSize="1"
PollingFrequency="10"
MinimumAge="5"
OpaqueSchema="false" >

What it will do at runtime is take your XML file, preserve the root tag, and for each child of the root, generate a separate inbound document containing the child with the root tag again wrapped around it:

<dcs320 xmlns="http://www.customer.com/schemas/dcs320/1.0">
<salesline>
<messagetype>DCS320</messagetype>
<messageid>487235</messageid>
<messagelinenumber>1</messagelinenumber>
<datamutationtype>0</datamutationtype>
<dcaddressnumber>0000410</dcaddressnumber>
</salesline>
<salesline>
<messagetype>DCS320</messagetype>
<messageid>487235</messageid>
<messagelinenumber>2</messagelinenumber>
<datamutationtype>0</datamutationtype>
<dcaddressnumber>0000411</dcaddressnumber>
</salesline>
<salesline>
<messagetype>DCS320</messagetype>
<messageid>487235</messageid>
<messagelinenumber>3</messagelinenumber>
<datamutationtype>0</datamutationtype>
<dcaddressnumber>0000412</dcaddressnumber>
</salesline>
</dcs320>

The above document will launch instances that get documents as input that look like:

<dcs320 xmlns="http://www.customer.com/schemas/dcs320/1.0">
<salesline>
<messagetype>DCS320</messagetype>
<messageid>487235</messageid>
<messagelinenumber>1</messagelinenumber>
<datamutationtype>0</datamutationtype>
<dcaddressnumber>0000410</dcaddressnumber>
</salesline>
</dcs320>

And so on, one for each <salesline> child. The ability to de-batch large XML files gives us the option to process large XML documents (that have a repeating structure) in BPEL, one piece at a time.

Process Control Challenges

The main challenge I've always had with debatching or other "parallel processing" patterns is to stay in control regarding error handling and monitoring progress & completion. Especially for non-transactional protocols like File and FTP, this can be tricky. Another neat feature that has been built into the 10.1.3.3 adapter is a batch notification mechanism. This means that at certain points in its processing, the adapter will post meta-data events about its progress to an event handler, typically another BPEL process. You configure this notification behaviour in the activationAgent section of the bpel.xml in the debatching process. Below an example of such a bpel.xml:

<?xml version = '1.0' encoding = 'UTF-8'?>
<BPELSuitcase>
<BPELProcess id="XMLDebatcher" src="XMLDebatcher.bpel">
<partnerLinkBindings>
<partnerLinkBinding name="debatch">
<property name="wsdlLocation">debatch.wsdl</property>
</partnerLinkBinding>
</partnerLinkBindings>
<activationAgents>
<activationAgent className="oracle.tip.adapter.fw.agent.jca.JCAActivationAgent" partnerLink="debatch">
<property name="batchNotificationHandler">bpel://default|BatchMgmtProcess</property>
<property name="portType">Read_ptt</property>
</activationAgent>
</activationAgents>
</BPELProcess>
</BPELSuitcase>

Above, the batch notification events are sent to a BPEL process called "BatchMgmtProcess", deployed in the "default" domain on the same BPEL server. The wsdl of this BatchMgmtProcess must define a number of messageTypes and operations. This is what it should look like (never mind the correlation stuff for now):

<definitions
name="BatchManagerInterface"
targetNamespace="http://xmlns.oracle.com/pcbpel/batching"
xmlns="http://schemas.xmlsoap.org/wsdl/"
xmlns:bpws="http://schemas.xmlsoap.org/ws/2003/03/business-process/"
xmlns:tns="http://xmlns.oracle.com/pcbpel/batching"
xmlns:plnk="http://schemas.xmlsoap.org/ws/2003/05/partner-link/"
xmlns:pns1="http://xmlns.oracle.com/BatchMgmtProcess/correlationset"
xmlns:batch="http://xmlns.oracle.com/pcbpel/batching/types"
>
<import namespace="http://xmlns.oracle.com/BatchMgmtProcess/correlationset" location="BatchMgmtProcess_Properties.wsdl"/>
<types>
<schema attributeFormDefault="qualified" elementFormDefault="qualified" targetNamespace="http://xmlns.oracle.com/pcbpel/batching/types"
xmlns:tns="http://xmlns.oracle.com/pcbpel/batching/types" xmlns="http://www.w3.org/2001/XMLSchema">
<element name="batchReadInitiateElement" type="tns:batchReadInitiateType"/>
<complexType name="batchReadInitiateType">
<sequence>
<element name="batchId" type="string"/>
<element name="batchMetaData" type="string"/>
<element name="batchDescription" type="string"/>
<element name="process" type="string"/>
<element name="domain" type="string"/>
</sequence>
</complexType>
<element name="batchReadCompleteElement" type="tns:batchReadCompleteType"/>
<complexType name="batchReadCompleteType">
<sequence>
<element name="batchId" type="string"/>
<element name="batchMetaData" type="string"/>
<element name="batchDescription" type="string"/>
<element name="batchExpectedSize" type="long"/>
<element name="process" type="string"/>
<element name="domain" type="string"/>
</sequence>
</complexType>
<element name="batchProcessCompleteElement" type="tns:batchProcessCompleteType"/>
<complexType name="batchProcessCompleteType">
<sequence>
<element name="batchId" type="string"/>
<element name="batchMetaData" type="string"/>
<element name="batchDescription" type="string"/>
<element name="batchFinalSize" type="long"/>
<element name="process" type="string"/>
<element name="domain" type="string"/>
</sequence>
</complexType>
<element name="batchReadFailureElement" type="tns:batchReadFailureType"/>
<complexType name="batchReadFailureType">
<sequence>
<element name="batchId" type="string"/>
<element name="batchMetaData" type="string"/>
<element name="batchDescription" type="string"/>
<element name="batchPartialSize" type="long"/>
<element name="process" type="string"/>
<element name="domain" type="string"/>
</sequence>
</complexType>
</schema>
</types>
<message name="batchReadInitiateMessage">
<part name="event" element="batch:batchReadInitiateElement"/>
</message>
<message name="batchReadCompleteMessage">
<part name="event" element="batch:batchReadCompleteElement"/>
</message>
<message name="batchProcessCompleteMessage">
<part name="event" element="batch:batchProcessCompleteElement"/>
</message>
<message name="batchReadFailureMessage">
<part name="event" element="batch:batchReadFailureElement"/>
</message>
<portType name="BatchManagerInterface">
<operation name="onBatchReadStart">
<input message="tns:batchReadInitiateMessage"/>
</operation>
<operation name="onBatchReadComplete">
<input message="tns:batchReadCompleteMessage"/>
</operation>
<operation name="onBatchProcessComplete">
<input message="tns:batchProcessCompleteMessage"/>
</operation>
<operation name="onBatchReadFailure">
<input message="tns:batchReadFailureMessage"/>
</operation>
</portType>
<plnk:partnerLinkType name="BatchManagerInterfacePartnerLinkType">
<plnk:role name="BatchManagerInterfaceRole">
<plnk:portType name="tns:BatchManagerInterface"/>
</plnk:role>
</plnk:partnerLinkType>
<bpws:propertyAlias propertyName="pns1:batchId" messageType="tns:batchReadCompleteMessage" part="event"
query="/batch:batchReadCompleteElement/batch:batchId"/>
<bpws:propertyAlias propertyName="pns1:batchId" messageType="tns:batchReadInitiateMessage" part="event"
query="/batch:batchReadInitiateElement/batch:batchId"/>
<bpws:propertyAlias propertyName="pns1:batchId" messageType="tns:batchReadFailureMessage" part="event"
query="/batch:batchReadFailureElement/batch:batchId"/>
</definitions>

The value of the batchId field is generated at runtime by the adapter, and can for example be used to initiate a correlationSet. You can then later on receive (e.g. using a Pick with correlation) a batchReadComplete or batchReadFailure message for the same file in the same monitoring instance. Here’s a fragment of what it could look like:

<correlationSets>

<correlationSet name="CorrelationSet_1" properties="ns3:batchId"/>

</correlationSets>

<sequence name="main">

<pick name="Pick_1" createInstance="yes">

<onMessage portType="ns1:BatchManagerInterface"

operation="onBatchReadStart"

variable="OnMessage_onBatchReadStart_InputVariable"

partnerLink="BatchManagerInterface">

<correlations>

<correlation initiate="yes" set="CorrelationSet_1"/>

</correlations>

Happy Debatching!

How to approach an SOA project

Service Oriented Architecture is all about business processing. How is the current/future business process is defined? What are the rules within this process? What is the input and the output of the process. How are will the business process act when in some particular stage a failure occurs. For example, the customer does not pay, the partner does not deliver the goods, the product is out of stock or some technical issues occurs; the netwrok is down, database crashes and many many more.

To implement a business process into a Service Oriented Architecture is not different before SOA did not exists. All the information flows, business flows , exceptions should be described in detail. This is what we still call the functional design. While we normally describe this in one or more documents, we should now focus more on process diagrams and data flows. An image says more than thousand words.

Drawing business process with their information flows make the process more transparent and the communication to the customer more clear. Drawings are easily to change, in case new functionality is added or flows should be combined to make the process more efficient.

A functional process flow diagram should the describe the whole process including the exceptions. After the functional process flow diagram a design will be made of the process. This means that the business process could be split in one ore more other processes. In practice it will result in many processes.

These small processes can be split up again in two different flavors, processes that are related to functional business process and processes that are more technical. Examples of functional processes are "collect products for shipping", "select customers who did not pay their order". Technical processes are for example; "send customer an notification", "update the order status for a customer".

Technical processes should not be exposed to the outside world. They are not meant to be called as a reusable service. These processes will not cover the business rules from their parent processes.

The functional processes can marked for being reused in the business process itself, but can also be exposed to other processes in the organization or third parties.

To summarize:

• Define the functional process with all the incoming and outgoing information flows.
• Define all internal information flows.
• Described the content of the flows.
• Describe the type of flows.
• Split the functional processes into smaller processes.
• For each smaller process, described also the previous steps.
• Define which of the smaller processes can be reused.
• Define for each smaller process the technical processes needed.

Regards,

Marc Kelderman
Technical Architect
http://orasoa.blogspot.com