Fantastic Ted Talk about how algorithms are changing our world.

How Algorithms Shape Our World

• Use a binary transaction protocol to eliminate data conversions and complex parsing.

• Use Remote Direct Memory Access (RDMA) verbs and zero copy mechanisms to eliminate network protocol stack processing.

• Use preallocated data structures to completely eliminate all memory turnover and associated garbage collections.

• Tune the hardware and OS for low latency

Part of me says “duh” but part of me says that maybe this is not necessarily common application yet?

Another “worlds fastest matching engine” – here

Tibco FTL messaging – seems interesting…

Fixnetix Raises the Bar: World’s Fastest Nanosecond Trading – more FPGA trading…

CME Study Claims HFT Reduces Price Swings

Original Story

Phantom Indexes

If so much volume trades off-primary – how valid are our index values??? Good food for thought.

Smart Order Routing: Multi Agent System for Real Time Adaptive SOR in Dark Pools

More SOR Reading

Enhyper – always a good read:

http://enhyper.blogspot.com/2011/02/what-to-do-with-your-fpga-enabled.html

I saw the original article after the flash crash when it came out here, great graphics and an interesting explanation. Zerohedge had another article that linked back to another Nanex.net article – here. All worth the read…  Quote Stuffing (jamming a ton of data down your competitor’s throats), Crop Circles (the resultant patterns in the data visualized), and more fun stuff inside!

In essence, there is a growing consensus that the HFT boys are causing more trouble than they are worth… let’s see how the regulators cope!

Several weeks back I got a message from Barry Johnson marketing his book on algo trading. Since I have a book buying problem and I generally have all books related to trading, especially automated trading – I went and ordered it.

The book; Algorithmic Trading and DMA is a good overview of what the title says it is about. Since people are often asking me for reading material – I have lately been referring people to this book. All in all – if you are at all interested in these things; get it.

The the January 2007 issue of Automated Trader Magazine I wrote an article about optimising trading systems. One concept I discussed was firms offering enterprise optimisation algorithms for client use (and fees of course). I found a reference this morning:

FlexTrade FlexPTS Offers Portfolio Optimisation Algos Using IBM Ilog Cplex

The addition of IBM’s Ilog Cplex software to FlexPTS allows the FlexTrade platform to optimise its trading schedule every 15 minutes

..

many firms use “rudimentary algorithms they’ve created in-house to schedule transactions. But in our view, anyone who’s serious about optimisation uses a powerful optimization engine like CPLEX from IBM.

..

the ability to define a trading strategy that can adapt dynamically not only to changes in the market, but also to the impact of other firms’ trading strategies.

All of this is in line with my own continued personal beliefs. IBM is offering optimisation as a service. Trading systems are consuming optimisation as a service. The intent is to make trading algorithms more adaptive to continuously changing conditions and to optimise the intended objectives correctly, robustly etc.

There are several papers (and authors) that are referenced again and again in the algorithmic trading literature. 1st – the ‘mother’ of most papers in algo trading:

• Robert Almgren and Neil Chriss. Optimal execution of portfolio transactions. J. Risk, 3(2):5–39, 2000.

This paper in particular is referenced by just about *every* paper on algorithmic trading. In this paper the generalized model for arrival price algorithms is related to the reader. This paper itself also does reference an earlier and oft referenced paper worth mentioning:

• Bertsimas and Lo (1998). Optimal control of liquidation costs. J. Financial Markets

The second most important paper referenced constantly is:

• Optimal Trading Strategy and Supply/Demand Dynamics Anna Obizhaeva and Jiang Wang

This paper is also fantastic, referencing the work of Almgren and Chriss and talking more about limit order books and such.

Next there are several worth getting to – listed in my preferred order:

• Bayesian Adaptive Trading with a Daily Cycle Robert Almgren? and Julian Lorenz
• Understanding the Profit and Loss Distribution of Trading Algorithms Robert Kissell 2005
• The Expanded Implementation Shortfall: “Understanding Transaction Cost Components” Robert Kissell May 2006

There are dozens more, and I would point the interested reader to these ‘less than seminal’ yet educational papers in the area of algo trading:

• Statistical properties of stock order books: empirical results and models Jean-Philippe Bouchaud, Marc M ?ezard, Marc Potters February 6, 2008
• Order Aggressiveness in Limit Order Book Markets Angelo Ranaldo* UBS Global Asset Management
• Optimal Trading in a Dynamic Market Robert Almgren? June 30, 2009
• Optimal Execution with Nonlinear Impact Functions and Trading-Enhanced Risk Robert F. Almgren? October 2001
• Optimal execution strategies in limit order books with general shape functions Aur ?elien Alfonsi?, Alexander Schied? 2007
• Algorithmic Trade Execution and Market Impact Richard Coggins†, Marcus Lim‡, Kevin Lo 2006

I have been reading, re-reading and re-reading these (and more) over and over – all great stuff for anyone interested in algo trading.

Hyde Park Global Bets on Adaptive Models to Trade Arbitrage Strategies in Milliseconds

Because no formula is going to work all the time, Hyde Park Global develops adaptive models, using a genetic algorithm (i.e, such as mutations and crossover), which are designed to respond to changing market conditions in real time, Afshar explains. While he refers to this as machine-based learning, he points out that the machines don’t actually learn. Rather, “They recalibrate themselves within the parameters that you have identified,” Afshar says, adding that they rely on data and quotes from previous trades to recalibrate.

Sounds just like what we began doing in ~2004.  It is *very* easy to do this poorly and we put in *years* of working on #1) a process to enable us to use these techniques properly, #2) An incorporation and understanding of the state of the art technologies (multi-objective optimization, boosting/ bagging, SVMs, fuzzy rule induction, etc). #3) specific implementations of the core machine learning techniques specialized for automated trading.

My basic belief is that these patterns of machine learning will continue to drive the state of the art of extracting money from the global markets.