Over the years I’ve done a bit of work with Workflow or Business Process products. Not quite enough to learn the difference between the two but enough to know what I like and what I don’t like. Most of my recent work has been on jBPM so I am heavily influenced by that product, when I look at others I compare them to it, not because I like it but because I know it. Most are similar with respect to the points I raise here. Anyway, I’m going to call everything ‘workflow’ and ignore the pedants who might attempt to correct me.

Workflow is essentially about handling long running transactions, things that might take weeks or months to work through like a complex insurance claim that involve coordinating different participants such as assessors, surveyors and so on, all through some process that has been actually designed by someone to deliver what is needed. This is in contrast to people figuring things out as they go along, missing steps, going back, screwing up etc.

You need to store the current state of things in a database, have queues of tasks that are directed at different people, ways they can access those queues and do the work (surveyor must enter her assessment of the damage to the building) and ways things can escalate automatically if they take too long (your support request has not been addressed within 2 hours, bump it up to a supervisor).

This is all really worth doing.

How you go about it is the question. There is a belief that the people who should write these processes should be business analysts rather than programmers. Consequently the tools to write them come with point-and-click graphical editors. Some of these are really impressive and a lot of work has gone into them. I, however, am always suspicious of point-and-click graphical editors. This stems from a time in the 80s when I worked with a rules based product that had a truly wonderful graphical UI (especially for the time) and underneath it was a rather shoddy inference engine. It demoed very, very well. Actually building stuff was a real pain. I have to contrast that with any modern spreadsheet application which are in daily use by accountants (not programmers) who manage to get them to do great work. So it is possible to get this right, but I’m wary.

The point-and-click graphical editors let you design processes and they save them into a format called variously BPMN, BPEL etc. There seems to be a tension here between adhering to a standard and coping with a standard that doesn’t cover all the needs, so there are local extensions, and that means processes developed in one system aren’t easy to port to another system. I just went through the exercise of migrating a demo process (ie a small one) from jBPM to Activiti and I ended up rewriting it rather than importing it. The pretty diagram imported just fine, but everything underneath it was different, for example how to define forms, how to define calls to external systems, how to define conditional branching. Activiti started out as a clone of an earlier version of jBPM so you might expect them to be similar in some respects but no.

And, while you will get business analysts making up the diagrams, that underlying part of the process definition (which is an integral part of the process) is quite fiddly. More fiddly than, say, spreadsheet formulae. Is this just a limitation of the tools? Maybe. But it is real. And it looks a lot more like programming than business analysis.

These two things, the lack of portability of process design and the underlying detail of it lead me to conclude that the people who actually use these tools are programmers rather than BAs. And this is also my experience. The BA will do a fine job specifying in detail what the process must do, a programmer will actually go write it.

That leads me to the obvious conclusion: the tools need to be programmer-friendly rather than BA friendly. So I went and did that. I cooked up a product called Madura-Workflow where the process definitions look enough like Java for a programmer to read them immediately and be able to write them in half an hour or so. The point-and-click tools are much more complicated than that, there really is a lot of new stuff to know such as different kinds of conditional gateways and so on. Java already has conditional statements that do much the same thing so I use that syntax. Here’s a sample:

queue: name=”Q1″ permission=”ORDERCLERK”;  
queue: name=”Q2″ permission=”STOCKCLERK” read-permission=”ORDERCLERK”;  
queue: name=”Q3″ permission=”SUPERVISOR” read-permission=”ORDERCLERK”;  
   
process: Order “AcceptOrder” “Accept an order”   
    launchForm=InitialForm queue=”Q1″ {  
    if (decisionField) {  
         try {  
             form=SecondForm queue=”Q2″;  
             compute=orderCompute log=”some message”;  
         }          
         catch (timeout=10 HOURS) {  
             form=SupervisorForm queue=”Q3″;  
             retry;  
         }  
         if (rejected) {   
            abort “Rejected this order”  
         } else {  
             message=stockManagementSender;   
        }  
     }  
     message=orderProvisioningSender;  
}

If you already know Java you probably know what is going on there, though you would have a some questions. I’ll answer those now:

• The process named AcceptOrder is focused on an object named Order, which is a bean (with getters and setters) and all fields referenced are on the Order object eg Order.decisionField.
• There are forms defined somewhere (else) named InitialForm, SecondForm and SupervisorForm. We don’t care too much what they look like, but the process waits when they are invoked and they gather data and set fields in the Order object.
• We can call Java classes with ‘compute’ statements and these can perform whatever Java we choose to write. Normally these are smallish computational things.
• When we want to send a message somewhere we use a ‘message’ statement. These connect to Spring Integration (or similar) to orchestrate sending/retrying messages to external servers, getting responses back etc.
• The launchForm is presented just before we start the process and it populates the Order object with initial information.
• The queue definitions determine which users see the task generated. The ORDERCLERK can launch this process, can see the STOCKCLERK’s tasks (eg SecondForm) but cannot see the SUPERVISOR tasks (SupervisorForm).

Sure there is still defining the data entry forms and the messages, but those are (quite rightly) outside the process definition and do not clutter it. In practice the message definitions consist of some XSL and some XML in a Spring context file. The data entry forms depend on your choice of technology.

My technology preference for defining forms is to use a mix of Vaadin, Madura Objects and Madura Rules. That way my forms are automatically validated and monitored by rules. For example the decisionField is a boolean that might be set by something on the InitialForm. It could be directly input as, say, a checkbox, but it might be invisible to the user and set by a rule triggered by the user entering some other field like a particular stock code. I supply a generic Vaadin form which generates itself from the object (eg the Order) and all it needs is an implementation class that extends it. You can leave the implementation empty for a generic form, or you can add code to customise the form if you need to.

The product does some other things too. You can add attachments to processes and this is fully supported by the UI, and by attachments I mean as many as you like and to any and every process instance. This is strangely missing from some workflow products I’ve used and seems to be needed by every workflow process I write.

However the core workflow is not dependent on a specific UI so people can choose what they like best.

Other features

• On-the-fly deployment of processes, including their rules and UI details.
• Field security on the forms, some users can see some fields but not others etc. This is a consequence of using Madura Objects.
• Security based on Spring Security, so endlessly configurable.
• Multiple servers for added throughput
• A full workbench UI for launching and operating processes but, unlike other workflow offering it doesn’t let you define processes in it, you do that from Eclipse.
• An Eclipse plugin to assist programmers writing their processes (syntax checking etc)

None of which is so very different from other workflow products.

But my focus here is really about the process definitions. I do believe we are going down the wrong path with clever diagrams that end up being a real pain for the people who end up having to implement them.

If you are interested in more tedious detail there are documents on the core workflow (PDF) and the workbench (PDF) which includes a demo script. The whole thing is in github and releases are published to maven.