Tuesday, October 19, 2004

Date-dependent convoys

The project I'm currently working is very heavily driven by various external system dates, and we've been having exhaustive discussions on how to implement the schedule - using BizTalk itself, using an external "scheduler" application, or relying on the source data systems to provide data at the requisite time. (My argument has always been the BTS is the wrong place to have any scheduling, as BTS is message-driven, and should simply react to what its been given, however I'm fighting a one-man war on that front!)

There are two types of scheduled process involved:
1. Data must be pulled (itself something I don't like) from systems at a given date.
2. Data will be pushed into BTS when it's made available, but cannot then be sent on to the consumers until some nominal "start date" has been passed. Furthermore, if a nominal "end date" has also been passed then the message should be killed off entirely. The final issue is that of updates. If a message is submitted before the start date, updates to the data contained in the original message could also be submitted via the same channel, in which case the final update is the one that the consumer is interested in (i.e. managing duplicates internally).

I may well post more about this scheduling business, as it's something I haven't directly come across before, and it raises some fairly fundamental issues re. message-driven "real-time" architectures (and how appropriate they are in such circumstances)

In the meantime I've done a simple demo that might be of interest. I've modelled the second scenario, where messages are submitted and then held, using a sequential uniform convoy to suck up all messages and correlate them; since I've done a quick demo I thought I might as well post it here in case anyone else finds it useful.

The zip file contains the following BTS artefacts:
1. A message schema, containing the aforementioned start and end dates, together with a field to use for correlation, and a data field that can be used to verify that the latest message is the one that comes out of the orchestration.
2. A property schema used to promote the correlation data.
3. The orchestration. This has three possible outputs:
  • If the end date of the initial message has passed, the message is sent to a port marked as "timedout".
  • If the start date has passed already, the message will simply shoot out the end, unchanged.
  • If the start date has not yet passed the orchestration will sit and listen for further messages ofthe same type (and correlation set). New messages are used to overwrite the original data. When the start date arrives the last message to be received will be the one that is sent.
In reality it looks as if our messages won't contain the start and end dates, and that the orchestration will have to call a web service to get these dates (the dates are held in a separate
system), but this demo works pretty well, and gives a quick insight to convoys.


STOP PRESS As Blogger doesn't host files, I'll have to host it from home, which I can't set up from here. I'll sort it asap.

1 comment:

Unknown said...

Hey Hugo,

I've just signed up for free image hosting with these guys, http://www.villagephotos.com and they had a link to a place with free file hosting, http://www.ripway.com/index.asp.

Not sure if there is a catch to any of this but could solve your hosting problem