Why the WCF Workflow Application is no good as an Orchestration layer

Why the WCF Workflow Application is no good as an Orchestration layer

I've been architecting and implementing a Service Oriented Architecture where I decided on using the WCF Workflow application as the middle Orchestration tier.  This enabled us to connect together the many backend services as single business processes for the UI.  It all looked good on paper and prototyping it raised some small issues but nothing too serious that we couldn't overcome. 

So it was all hands on deck as we developed using this as our Orchestration.  However much to my annoyance, and my teams, we started coming across issues with it, and after this last week of research, I have taking the decision to replace all our middle Orchestration with simple WCF Services and coding the Orchestration process in C#.

Here's a list of reasons why I had to ditch the nice GUI orchestration via Workflow and had to manually do the coding:

• Transaction handling - although the WCF workflow service can handle transactions via the TransactedReceiveScope Activity I was always frustrated about this as my original design allowed the Orchestration to maintain the transaction, thus removing the need for the UI to know about or keep longish (> 5 sec) running processes waiting for responses.  Ideally I wanted the Orchestration to start a transaction and roll it back or commit it based on its underlying method calls.  This would have enabled me to wrap up multiple requests from the UI in to a single request and make the Orchestration commit or rollback internal processes as required.  Due to the WCF service not being able to start a transaction  I instead had to pass this down from the UI and keep it in the UI layer, which I found a bit of a pain.  By re-writing the workflow as a WCF service I instead have the ability to keep transactions around many different services as and where I need them.
• XAML Development - the ability to visually see the processes while developing is a very cool idea and I used it to even present back to the client Business Processes and identify issues.  For example I had a Workflow that calculated taxation which was very complex.  The user could visually see all the coding steps and could quite easily point out problems with the business process.  This in itself will be missed with the transition to the code centric version.  However we noticed that once the XAML got to a reasonable size, which it can do when acting as a facade, Visual Studio became steadily unusable.  For example my development machine has 16GB of RAM with an Intel Core i7 processor and a Solid State Hard drive, yet to open a reasonably size XAML file took over 5 minutes.  Then to make any change to the business process would result in Visual Studio going in to hibernation for close to a minute...and I mean any change.  The benefits of the visual representation of the business process soon became something close to a nightmare.  Obviously the XAML editor was not meant for editing medium sized workflows.
• Non thread safe - this became the killer for the WCF workflow service.  I started noticing issues where the WCF Workflow service threw some very weird errors like "item is already in dictionary" upon initialisation of a service call.  This raised its head due to us wanting to wrap up multiple requests and execute them in parallel thus making use of multi-core CPU's.  I found that running a single unit of work by itself was no issue however when I parallelled 5 together the first one would work and the rest would fail.  This was probably due to the orchestration having to accept transactions and trying to do 5 at the same time.  I would always get errors on at least 1 of the packets of work.  I tried a few workarounds like initialising the ChannelFactory first and then firing off the requests, and also delaying each request by 100ms.  These all improved the chance of getting a packet through the orchestration but it was not full proof.  Also I was left wondering what would happen if 50 users were on the site and hit the same request at the same time.  I am sure errors would be aplenty!  This left me feeling like the WCF workflow service was a ticking timebomb ready for failure.

So unfortunately we have taken the hit and started replicating the functionality in the orchestration as WCF services, which after initial testing is working fine with non of the above issues.  I do miss the UI that comes with XAML but I am a lot more comfortable in proceeding without WCF Workflow Services.

Hope this helps someone deciding on solutions.