Congrats to John Lam (RubyClr’s dad) – as he goes off to MS to work on dynamic languages with the CLR team

With the Sun folks taking the JRuby people in house and the MS folks now taking more of an interest (as if iron python was not enough) in the dynamic languages – it seems there is some momentum with the big boys (finally). Boy was that a run on sentence.

-Chris

You can leave a response, or trackback from your own site.

I got thinking today that rule induction systems for ESP/ CEP systems are a likely candidate for near future work. I have worked on rule induction in the past (specifically fuzzy rule induction, guided by a training evolutionary algorithm). It seems that using real time optimization techniques, you could do the same – induce rules.
Establish goals:

Maximize Individual Trade Profit
Minimize Average Drawdown
Maximize total profit

Set basic inputs to the induction/ evaluation system:

Market data from instrument stream X
Real time portfolio information
Indicators derived from the above (technical analysis, quant analysis, etc)

Generate candidate ESP rule sets

Store success/ failure of rule sets in their execution environments for each goal
Breed/ Mutate ESP rule sets

Loop

The core algorithm should recombine/ change ESP rule sets to meet goals. This would induce rules (in whatever runtime language) for Event Driven Systems. Rule induction has been done in many many data mining applications, I am sure it is just a matter of time until these paradigms are being applied to large event based systems and the rules/ processing that happens to those events.

Thoughts in progress

Chris

You can leave a response, or trackback from your own site.

The 1st ever ..
Distributed Event-Based Systems Conference

The area is getting bigger and bigger.

Click to read more …

You can leave a response, or trackback from your own site.

For a current project – we have .net clients and java servers – seems to be the wall street standard – at least for my past several clients/ employers. That said – another common scenario is to standardize message formats via XSDs. If we standardize XSDs we now have a contract for message formats. This is the way it is usually – nothing new here. That said – XSDs are not so great at extended validation. You can do some amount of simple validation at the element level (regex, must exist, enumerations, etc) but to do more complex validations between elements of more complex types – it gets increasingly expressive and unwieldy to do in XSD.

Another complicated scenario is partial validation. UIs that have pages of forms to fill out are one example of where partial and/ or contextual validation must be considered. You cannot simply consider the whole thing valid or invalid – you must consider where in the scheme of things you are – and given that context -are you OK.

I am considering as an alternative using a dynamic shareable language like Ruby or Python. I would prefer Ruby – mostly because I think it is a nicer language. That said – JRuby and Ruby CLR could certainly do the trick on either side of the fence. If we were to have object schemas defined as XSDs and defer the complete validation to another set of constructs – we might be in business.

Say for example we have a ‘person’ defined in our message XSDs. We could then have java and .net objects that are ‘person’ class instances. We could then have a ruby routine that validates a person. This Ruby code could run in java and .net and thus – to a large extend be shared. The only tricky bit that comes in is the ‘includes’ or ‘imports’ into the Ruby in it’s respective runtime. In other words – if we have JRuby as the runtime – we would need to let JRuby load in the package for the correct java class ‘person’ in this example. Similarly – the .net runtime would need to reference the correct assembly for the .net object equivlent of ‘person’. Aside from that – it seems that Ruby should work to lay out the validation with arbitrary complexity.

I will soon see if I can get some buy on to try out this idea and I will let the blog world know how goes it.

-Chris

You can leave a response, or trackback from your own site.

Recently – my current employer has been looking for a solution for what it has been calling ‘continuous queries’. The CEP groups @ Yahoo have been debating about the most appropriate language for processing real time events. It winds up being quite an interesting topic. Depending on what your ‘query-able’ target is – you may want different semantics.

If you pre-suppose that you have an ‘in memory database’ with real time events when an insert, update or delete happens, the SQL-like languages seem to make sense. Since classic sql is bad at the temporal -  these SQL like languages basically add temporal functionality.

What if you want to mostly query XML messages passing in the wind?? Then some sort of XPath predicate query semantics seem most desirable.

What if you want to query state changes of in memory java or .net objects?? Do you use an OGNL like query language? JXPath?? Some db4o like Soda query, query by example or native query?? What makes most sense for registering interest in real time distributed events for object based updates? How does a delta update work in this world? Do I REALLY have to turn my nice objects into a relational database-like structure to use real time eventing???

These are just a few of the questions running through my heads and the heads of several others in my area. I can never say that I do not get to play with interesting problems

-Chris

You can leave a response, or trackback from your own site.

  Google Code Search

You can leave a response, or trackback from your own site.