I got thinking today that rule induction systems for ESP/ CEP systems are a likely candidate for near future work. I have worked on rule induction in the past (specifically fuzzy rule induction, guided by a training evolutionary algorithm). It seems that using real time optimization techniques, you could do the same – induce rules.
Establish goals:

Maximize Individual Trade Profit
Minimize Average Drawdown
Maximize total profit

Set basic inputs to the induction/ evaluation system:

Market data from instrument stream X
Real time portfolio information
Indicators derived from the above (technical analysis, quant analysis, etc)

Generate candidate ESP rule sets

Store success/ failure of rule sets in their execution environments for each goal
Breed/ Mutate ESP rule sets

Loop

The core algorithm should recombine/ change ESP rule sets to meet goals. This would induce rules (in whatever runtime language) for Event Driven Systems. Rule induction has been done in many many data mining applications, I am sure it is just a matter of time until these paradigms are being applied to large event based systems and the rules/ processing that happens to those events.

Thoughts in progress

Chris

You can leave a response, or trackback from your own site.

The 1st ever ..
Distributed Event-Based Systems Conference

The area is getting bigger and bigger.

Click to read more …

You can leave a response, or trackback from your own site.

For a current project – we have .net clients and java servers – seems to be the wall street standard – at least for my past several clients/ employers. That said – another common scenario is to standardize message formats via XSDs. If we standardize XSDs we now have a contract for message formats. This is the way it is usually – nothing new here. That said – XSDs are not so great at extended validation. You can do some amount of simple validation at the element level (regex, must exist, enumerations, etc) but to do more complex validations between elements of more complex types – it gets increasingly expressive and unwieldy to do in XSD.

Another complicated scenario is partial validation. UIs that have pages of forms to fill out are one example of where partial and/ or contextual validation must be considered. You cannot simply consider the whole thing valid or invalid – you must consider where in the scheme of things you are – and given that context -are you OK.

I am considering as an alternative using a dynamic shareable language like Ruby or Python. I would prefer Ruby – mostly because I think it is a nicer language. That said – JRuby and Ruby CLR could certainly do the trick on either side of the fence. If we were to have object schemas defined as XSDs and defer the complete validation to another set of constructs – we might be in business.

Say for example we have a ‘person’ defined in our message XSDs. We could then have java and .net objects that are ‘person’ class instances. We could then have a ruby routine that validates a person. This Ruby code could run in java and .net and thus – to a large extend be shared. The only tricky bit that comes in is the ‘includes’ or ‘imports’ into the Ruby in it’s respective runtime. In other words – if we have JRuby as the runtime – we would need to let JRuby load in the package for the correct java class ‘person’ in this example. Similarly – the .net runtime would need to reference the correct assembly for the .net object equivlent of ‘person’. Aside from that – it seems that Ruby should work to lay out the validation with arbitrary complexity.

I will soon see if I can get some buy on to try out this idea and I will let the blog world know how goes it.

-Chris

You can leave a response, or trackback from your own site.