I saw the original article after the flash crash when it came out here, great graphics and an interesting explanation. Zerohedge had another article that linked back to another Nanex.net article – here. All worth the read…  Quote Stuffing (jamming a ton of data down your competitor’s throats), Crop Circles (the resultant patterns in the data visualized), and more fun stuff inside!

In essence, there is a growing consensus that the HFT boys are causing more trouble than they are worth… let’s see how the regulators cope!

I was reading an article about how Google is going to start releasing versions of it’s browser every 6 weeks – much, much faster than it’s competitors (Firefox, IE, Safari). They have the same goals I do in my day to day job – deliver faster with fixed quality. HOW !??

Google is serious about test automation:

• Runs 120M individual tests / day
• 1,400 continuous integration builds
• 65K builds per day
• 6,000 developer / 1,500+ projects

That is serious scale and commitment to test automation. This is the only way to go fast at scale – full stop

I have several software developers coming from a Windows C++ background. I have been trying to sort out a good IDE for them. Eclipse CDT C++ plugin is OK. It is useful, better than nothing for someone not happy using Emacs or VI. I used it for ~1 month for ‘toy project work’ in order to see how it was for them.

From there – I was somewhat dissatisfied so I kept looking. Netbeans had some C++ support so I gave it a go. All in all – I preferred Netbeans to Eclipse out of the box. Then – I found my killer feature. Remote development and remote GDB debugging…

Remote Development

You can run a Netbeans IDE on a Windows machine, or a Mac and point it to a Linux (or Solaris) machine. It will SSH/ SCP files back and forth to the box behind the scenes. It will also give you a completely Visual-Studio-like visual step-through debugging interface into GDB. Impressive. After considering running an X Server and getting Eclipse running via VNC – it seems a no brainer.

Anyhow – I am curious to see how it scales to large codebases. I am very optimistic that Netbeans provides a real, viable solution for some devs that have been developing C++ on Windows; to work on Linux…

Several weeks back I got a message from Barry Johnson marketing his book on algo trading. Since I have a book buying problem and I generally have all books related to trading, especially automated trading – I went and ordered it.

The book; Algorithmic Trading and DMA is a good overview of what the title says it is about. Since people are often asking me for reading material – I have lately been referring people to this book. All in all – if you are at all interested in these things; get it.

(Compiled from Yelp here)

As a native New Yorker – I found this great chat series on Yelp comparing NY and London neighborhoods. IMO – VERY accurate!

LONDON WEST
Notting Hill = New York Soho/West Village
Holland Park = Sutton Place
Mayfair = Fifth Avenue on the Upper East Side
Knightsbridge = Central Park South, the very lower portion of the Upper East Side

Chelsea = Upper East side east of Lexington
LONDON CENTER
West End/Picadilly Circus/Lecester Square = Times Square
London’s Soho = the Village (sort of the hectic bustling tourist parts) + Chelsea (gay scene)

LONDON NORTH
Camden town = sort of also like the east village
the nice parts of Islington (barnsbury, highbury fields) = Brooklyn Heights
Clerkenwell = Fort Greene
the rest of Islington = miscellaneous brooklyn
Hampstead = Park Slope
West Hampstead = bo-co-ca brooklyn (god I hate that word)
[you get the picture - basically, North London is generally brooklyn]

LONDON EAST
The City = Wall Street
Hoxton and Shoreditch = Lower East Side
East London = the bronx

LONDON SOUTH OF THE RIVER
the vast unknown of South London = Queens

Essex (the suburban region where everyone from the east end and east london settled) = New Jersey
Surrey = Connecticut around Greenwich

(Complete aside – a friend of mine; Dave asked me to give a small signal of notification to another ex-colleauge – so consider this it !)

The the January 2007 issue of Automated Trader Magazine I wrote an article about optimising trading systems. One concept I discussed was firms offering enterprise optimisation algorithms for client use (and fees of course). I found a reference this morning:

FlexTrade FlexPTS Offers Portfolio Optimisation Algos Using IBM Ilog Cplex

The addition of IBM’s Ilog Cplex software to FlexPTS allows the FlexTrade platform to optimise its trading schedule every 15 minutes

..

many firms use “rudimentary algorithms they’ve created in-house to schedule transactions. But in our view, anyone who’s serious about optimisation uses a powerful optimization engine like CPLEX from IBM.

..

the ability to define a trading strategy that can adapt dynamically not only to changes in the market, but also to the impact of other firms’ trading strategies.

All of this is in line with my own continued personal beliefs. IBM is offering optimisation as a service. Trading systems are consuming optimisation as a service. The intent is to make trading algorithms more adaptive to continuously changing conditions and to optimise the intended objectives correctly, robustly etc.

So – 2009 was a great year for my continued sci-fi “reading”. I actually read perpetually – and the actual reading that I do is usually of the technical sort (quant finance, trading, software, teams, leadership, etc.). In the walking commute and during my daily running – I listen to audio books – quite a lot of them actually. I have been listening to audio books for maybe 5 or 6 years now and it is something that I really enjoy.

So – here are the audio books that I “read” this year in my order of preference:

Scott Lynch – The Lies of Locke Lamora (amazing – I love this book – can’t wait for the next ones – best book of the year!)

Robert Jordan, Brandon Sanderson - The Gathering Storm (Love it – can’t wait for the rest)

Patrick Rothfuss – The Name of the Wind (great 1st effort – I hope the rest is as good when it comes)

Brent Weeks – Night Angel Trilogy (fantastic series – fantastic magic thieves/ heros run amuck )

• The Way of Shadows
• Shadow’s Edge
• Beyond the Shadows

Brandon Sanderson – Mistborn (amazing series, sci-fi at its best)

• The Hero of Ages
• The Well of Ascension
• The Final Empire

Jim Butcher - Codex Alera (great series)

• Furies of Calderon
• Academ’s Fury
• Captain’s Fury
• Cursor’s Fury
• Princeps’ Fury
• First Lord’s Fury

David Anthony Durham -The Other Land (great continuation)

David Weber – By Heresies Distressed (a great continuation)

Brandon Sanderson - Elantris (good – less than Mistborn by far)

Malcolm Gladwell - Outliers (good – non-fiction!)

Brandon Sanderson -Warbreaker

Richard K. Morgan - The Steel Remains (mediocre – I prefer Kovacs)

Neal Stephenson – Anathem (too long and self indulgent – but OK)

John Steakley - Armor – (just mediocre semi-interesting)

Suzanne Collins – The Hunger Games (finished it – but it was an error – too teen aged girlie!)

David Weber – On Basilisk Station (could not finish it – too mediocre)

David Eddings – Guardians of the West (worst book – I could not stomach it and quit it)

There are several papers (and authors) that are referenced again and again in the algorithmic trading literature. 1st – the ‘mother’ of most papers in algo trading:

• Robert Almgren and Neil Chriss. Optimal execution of portfolio transactions. J. Risk, 3(2):5–39, 2000.

This paper in particular is referenced by just about *every* paper on algorithmic trading. In this paper the generalized model for arrival price algorithms is related to the reader. This paper itself also does reference an earlier and oft referenced paper worth mentioning:

• Bertsimas and Lo (1998). Optimal control of liquidation costs. J. Financial Markets

The second most important paper referenced constantly is:

• Optimal Trading Strategy and Supply/Demand Dynamics Anna Obizhaeva and Jiang Wang

This paper is also fantastic, referencing the work of Almgren and Chriss and talking more about limit order books and such.

Next there are several worth getting to – listed in my preferred order:

• Bayesian Adaptive Trading with a Daily Cycle Robert Almgren? and Julian Lorenz
• Understanding the Profit and Loss Distribution of Trading Algorithms Robert Kissell 2005
• The Expanded Implementation Shortfall: “Understanding Transaction Cost Components” Robert Kissell May 2006

There are dozens more, and I would point the interested reader to these ‘less than seminal’ yet educational papers in the area of algo trading:

• Statistical properties of stock order books: empirical results and models Jean-Philippe Bouchaud, Marc M ?ezard, Marc Potters February 6, 2008
• Order Aggressiveness in Limit Order Book Markets Angelo Ranaldo* UBS Global Asset Management
• Optimal Trading in a Dynamic Market Robert Almgren? June 30, 2009
• Optimal Execution with Nonlinear Impact Functions and Trading-Enhanced Risk Robert F. Almgren? October 2001
• Optimal execution strategies in limit order books with general shape functions Aur ?elien Alfonsi?, Alexander Schied? 2007
• Algorithmic Trade Execution and Market Impact Richard Coggins†, Marcus Lim‡, Kevin Lo 2006

I have been reading, re-reading and re-reading these (and more) over and over – all great stuff for anyone interested in algo trading.

Hyde Park Global Bets on Adaptive Models to Trade Arbitrage Strategies in Milliseconds

Because no formula is going to work all the time, Hyde Park Global develops adaptive models, using a genetic algorithm (i.e, such as mutations and crossover), which are designed to respond to changing market conditions in real time, Afshar explains. While he refers to this as machine-based learning, he points out that the machines don’t actually learn. Rather, “They recalibrate themselves within the parameters that you have identified,” Afshar says, adding that they rely on data and quotes from previous trades to recalibrate.

Sounds just like what we began doing in ~2004.  It is *very* easy to do this poorly and we put in *years* of working on #1) a process to enable us to use these techniques properly, #2) An incorporation and understanding of the state of the art technologies (multi-objective optimization, boosting/ bagging, SVMs, fuzzy rule induction, etc). #3) specific implementations of the core machine learning techniques specialized for automated trading.

My basic belief is that these patterns of machine learning will continue to drive the state of the art of extracting money from the global markets.

I have been reading a few books that are related:

• Empirical Market Microstructure
• An Introduction to High Frequency Finance

This paper gives a pretty good overview and meaningful conclusions from the space…

Characterization of Financial Time Series

Google’s Go: A New Programming Language That’s Python Meets C++

Go site

1st blush looks nice. Types good, C like performance good. Built for multi-core etc. I will admit I wish it looked more functionally inspired. I am loving Erlang these days (and to a lesser extent Haskell) so going more imperative again feels a step back…

I will give this one a *go* I think…

Why blog about Erlang now?

For me writing blog posts is often a therapeutic way to work out puzzles and clarify my own understanding of a concept. I have been writing Erlang on and off for around 1.5 years now. As of late I have also been typing a lot more Haskell and some OCaml. This has all been great and I love these languages. I have decided to commit a bit more to Erlang lately and I thought I would start blogging again in earnest – mostly with small-ish Erlang examples just to force myself to be able to explain the concepts. I can program in Erlang (to some level at least) – now I want to be able to be good enough to explain Erlang software to people.

Essentially – I think that erlang the language is OK, I think that Erlang the runtime is amazing and I think that OTP – the common set of Erlang behaviors is very powerful. The entire ecosystem puts together a very compelling offering. You can achieve massively parallel, manageable bits of software in small enough bits of code that it is ownable. So – Erlang it is…

So in that spirit – I will start here with some very simple Erlang concepts.

Erlang is Dynamically Typed

Erlang is a dynamically typed language that to be honest – I found pretty ugly (syntactically) at first. I have done plenty of programming in dynamic languages – especially in Ruby (and to a slightly lesser extent – Python). I really do like dynamic languages but to be honest; I do believe that in larger software projects today that static typing can really by you a generally higher degree of assertable ‘correctness’. I prefer Haskell’s type system to many others – but my basic point is that I think types are powerful. In any case – I still think Erlang’s total offering is powerful enough to make a concede this point.

While Erlang is dynamically typed it does have a simple way to organize primitive data elements into composite types – this is Erlang records. You can also of course use tuples (a sort of anonymous type). In addition, Erlang code is organized into modules. So – while we may organize our code differently than in other languages – we have some good organizational concepts.

Modules

-module(exchange).

Records and Tuples

As I mentioned – erlang lets us organize primative elements into composite data types. The anonymous way of doing this is tuples. The ‘named’ way of doing this is records.

-record(order,{market, price, size, side}).

This is the declaration of a record in Erlang. You can declare an instance of a record like this:

#order{market=vodln,price=127.95,size=350,side=bid}

It is essentially a nice dress on top of a tuple which would look like this:

{vodln,127.95, 350, bid}

So – records are really just nice ways to put together composite types with names. Tuples are ways to assembly composite types anonymously. Both have their usages.

Object Orientation vs Functional Programming

Notice that these records/ tuple things have no methods ‘on’ them. Typically when you declare a type in OO languages – you put methods on them. This whole bundling up of methods and data IS OO programming. Functional Programming lets you have data and describe functionality. What this means is that it is usually easier to add more functions on a given data type in a functional language – but harder to change the data structures. This is because in an OO language – the data is WITH the functionality. In a functional language the data is apart from the functionality. It is all a tradeoff, but I think the functional paradigm is often much closer to the domains I have been working in (trading specifically). So – FP takes away the general encapsulation of data/ methods together. Note that you can still hide private functions inside a module. Eg; you implement a module and export 1 of 10 methods. This is still a good way to expose a minimal surface area of your software to the outside consumer of it.

Functions

Functions are – as you may have guessed at the heart of Erlang.

levelFor(Order, Exch) when Order#order.side == bid ->

levelForSide(Order, Exch, bid, fun(E) -> E#exch.bids end);

levelFor(Order, Exch) when Order#order.side == offer ->

levelForSide(Order, Exch, offer, fun(E) -> E#exch.offers end).

Here we have an example function that I will use as a demonstration for several points.

• You declare functions in erlang as a top level item. You do not put them inside of records, you put them in a module – but that is it.
• Values/ Parameters all start with upper case letters (I found this very odd at 1st).
• You can declare the same function N times each with different ‘guard clauses’ and the order of the declrations is the order in which the runtime attempts to bind to them.
• This example shows how to extract values from records.
• This example shows  simple funs- anonymous functions.

Declaring functions, Guard Clauses, Extracting data from records

“levelFor” is the name of our function. This function takes 2 arguments named Order and Exch. The arguments come after the name in the (…).

After that is a guard clause – this ‘when’ is evaluated by the runtime to see if the method should be executed. So – we can see in case 1 – if the side == bid – then we will execute the method. The actual method body comes after the ->.

You can extract values from records using this syntax ValueName#recordName.fieldName. So our example above has several of those.

Funs and multiple function Declarations

The bit after the -> is invoking another method and passing in as the last argument an anonymous function. That bit that starts with fun(E) and ends with “end”. This is simply an unnamed function (delegate, function ptr, etc) passed to the next function.

Also notice that I declared the function 2x – at the end of the 1st declaration I put a ; and at the end of the 2nd I put a “.” ( a dot). This is a convenient way to get switch statements out of our code nesting. We use the guard statements to see which of the functions get bound to at runtime. We could have also used if or case statements – but for this example’s sake – we are not

OK – that is all the time I have for today… more next weekend

-Chris