• Use a binary transaction protocol to eliminate data conversions and complex parsing.

• Use Remote Direct Memory Access (RDMA) verbs and zero copy mechanisms to eliminate network protocol stack processing.

• Use preallocated data structures to completely eliminate all memory turnover and associated garbage collections.

• Tune the hardware and OS for low latency

Part of me says “duh” but part of me says that maybe this is not necessarily common application yet?

Another “worlds fastest matching engine” – here

Tibco FTL messaging – seems interesting…

Fixnetix Raises the Bar: World’s Fastest Nanosecond Trading – more FPGA trading…

I just had to mention this one- Rabbit MQ:

Built on the Erlang OTP
Implementation of AMQP
Java and .Net implementations
Open Source (similar to qpid)

Recently – my current employer has been looking for a solution for what it has been calling ‘continuous queries’. The CEP groups @ Yahoo have been debating about the most appropriate language for processing real time events. It winds up being quite an interesting topic. Depending on what your ‘query-able’ target is – you may want different semantics.

If you pre-suppose that you have an ‘in memory database’ with real time events when an insert, update or delete happens, the SQL-like languages seem to make sense. Since classic sql is bad at the temporal -  these SQL like languages basically add temporal functionality.

What if you want to mostly query XML messages passing in the wind?? Then some sort of XPath predicate query semantics seem most desirable.

What if you want to query state changes of in memory java or .net objects?? Do you use an OGNL like query language? JXPath?? Some db4o like Soda query, query by example or native query?? What makes most sense for registering interest in real time distributed events for object based updates? How does a delta update work in this world? Do I REALLY have to turn my nice objects into a relational database-like structure to use real time eventing???

These are just a few of the questions running through my heads and the heads of several others in my area. I can never say that I do not get to play with interesting problems

-Chris