(Compiled from Yelp here)

As a native New Yorker – I found this great chat series on Yelp comparing NY and London neighborhoods. IMO – VERY accurate!

LONDON WEST
Notting Hill = New York Soho/West Village
Holland Park = Sutton Place
Mayfair = Fifth Avenue on the Upper East Side
Knightsbridge = Central Park South, the very lower portion of the Upper East Side

Chelsea = Upper East side east of Lexington
LONDON CENTER
West End/Picadilly Circus/Lecester Square = Times Square
London’s Soho = the Village (sort of the hectic bustling tourist parts) + Chelsea (gay scene)

LONDON NORTH
Camden town = sort of also like the east village
the nice parts of Islington (barnsbury, highbury fields) = Brooklyn Heights
Clerkenwell = Fort Greene
the rest of Islington = miscellaneous brooklyn
Hampstead = Park Slope
West Hampstead = bo-co-ca brooklyn (god I hate that word)
[you get the picture - basically, North London is generally brooklyn]

LONDON EAST
The City = Wall Street
Hoxton and Shoreditch = Lower East Side
East London = the bronx

LONDON SOUTH OF THE RIVER
the vast unknown of South London = Queens

Essex (the suburban region where everyone from the east end and east london settled) = New Jersey
Surrey = Connecticut around Greenwich

(Complete aside – a friend of mine; Dave asked me to give a small signal of notification to another ex-colleauge – so consider this it !)

The the January 2007 issue of Automated Trader Magazine I wrote an article about optimising trading systems. One concept I discussed was firms offering enterprise optimisation algorithms for client use (and fees of course). I found a reference this morning:

FlexTrade FlexPTS Offers Portfolio Optimisation Algos Using IBM Ilog Cplex

The addition of IBM’s Ilog Cplex software to FlexPTS allows the FlexTrade platform to optimise its trading schedule every 15 minutes

..

many firms use “rudimentary algorithms they’ve created in-house to schedule transactions. But in our view, anyone who’s serious about optimisation uses a powerful optimization engine like CPLEX from IBM.

..

the ability to define a trading strategy that can adapt dynamically not only to changes in the market, but also to the impact of other firms’ trading strategies.

All of this is in line with my own continued personal beliefs. IBM is offering optimisation as a service. Trading systems are consuming optimisation as a service. The intent is to make trading algorithms more adaptive to continuously changing conditions and to optimise the intended objectives correctly, robustly etc.

So – 2009 was a great year for my continued sci-fi “reading”. I actually read perpetually – and the actual reading that I do is usually of the technical sort (quant finance, trading, software, teams, leadership, etc.). In the walking commute and during my daily running – I listen to audio books – quite a lot of them actually. I have been listening to audio books for maybe 5 or 6 years now and it is something that I really enjoy.

So – here are the audio books that I “read” this year in my order of preference:

Scott Lynch – The Lies of Locke Lamora (amazing – I love this book – can’t wait for the next ones – best book of the year!)

Robert Jordan, Brandon Sanderson - The Gathering Storm (Love it – can’t wait for the rest)

Patrick Rothfuss – The Name of the Wind (great 1st effort – I hope the rest is as good when it comes)

Brent Weeks – Night Angel Trilogy (fantastic series – fantastic magic thieves/ heros run amuck )

• The Way of Shadows
• Shadow’s Edge
• Beyond the Shadows

Brandon Sanderson – Mistborn (amazing series, sci-fi at its best)

• The Hero of Ages
• The Well of Ascension
• The Final Empire

Jim Butcher - Codex Alera (great series)

• Furies of Calderon
• Academ’s Fury
• Captain’s Fury
• Cursor’s Fury
• Princeps’ Fury
• First Lord’s Fury

David Anthony Durham -The Other Land (great continuation)

David Weber – By Heresies Distressed (a great continuation)

Brandon Sanderson - Elantris (good – less than Mistborn by far)

Malcolm Gladwell - Outliers (good – non-fiction!)

Brandon Sanderson -Warbreaker

Richard K. Morgan - The Steel Remains (mediocre – I prefer Kovacs)

Neal Stephenson – Anathem (too long and self indulgent – but OK)

John Steakley - Armor – (just mediocre semi-interesting)

Suzanne Collins – The Hunger Games (finished it – but it was an error – too teen aged girlie!)

David Weber – On Basilisk Station (could not finish it – too mediocre)

David Eddings – Guardians of the West (worst book – I could not stomach it and quit it)

There are several papers (and authors) that are referenced again and again in the algorithmic trading literature. 1st – the ‘mother’ of most papers in algo trading:

• Robert Almgren and Neil Chriss. Optimal execution of portfolio transactions. J. Risk, 3(2):5–39, 2000.

This paper in particular is referenced by just about *every* paper on algorithmic trading. In this paper the generalized model for arrival price algorithms is related to the reader. This paper itself also does reference an earlier and oft referenced paper worth mentioning:

• Bertsimas and Lo (1998). Optimal control of liquidation costs. J. Financial Markets

The second most important paper referenced constantly is:

• Optimal Trading Strategy and Supply/Demand Dynamics Anna Obizhaeva and Jiang Wang

This paper is also fantastic, referencing the work of Almgren and Chriss and talking more about limit order books and such.

Next there are several worth getting to – listed in my preferred order:

• Bayesian Adaptive Trading with a Daily Cycle Robert Almgren? and Julian Lorenz
• Understanding the Profit and Loss Distribution of Trading Algorithms Robert Kissell 2005
• The Expanded Implementation Shortfall: “Understanding Transaction Cost Components” Robert Kissell May 2006

There are dozens more, and I would point the interested reader to these ‘less than seminal’ yet educational papers in the area of algo trading:

• Statistical properties of stock order books: empirical results and models Jean-Philippe Bouchaud, Marc M ?ezard, Marc Potters February 6, 2008
• Order Aggressiveness in Limit Order Book Markets Angelo Ranaldo* UBS Global Asset Management
• Optimal Trading in a Dynamic Market Robert Almgren? June 30, 2009
• Optimal Execution with Nonlinear Impact Functions and Trading-Enhanced Risk Robert F. Almgren? October 2001
• Optimal execution strategies in limit order books with general shape functions Aur ?elien Alfonsi?, Alexander Schied? 2007
• Algorithmic Trade Execution and Market Impact Richard Coggins†, Marcus Lim‡, Kevin Lo 2006

I have been reading, re-reading and re-reading these (and more) over and over – all great stuff for anyone interested in algo trading.

Hyde Park Global Bets on Adaptive Models to Trade Arbitrage Strategies in Milliseconds

Because no formula is going to work all the time, Hyde Park Global develops adaptive models, using a genetic algorithm (i.e, such as mutations and crossover), which are designed to respond to changing market conditions in real time, Afshar explains. While he refers to this as machine-based learning, he points out that the machines don’t actually learn. Rather, “They recalibrate themselves within the parameters that you have identified,” Afshar says, adding that they rely on data and quotes from previous trades to recalibrate.

Sounds just like what we began doing in ~2004.  It is *very* easy to do this poorly and we put in *years* of working on #1) a process to enable us to use these techniques properly, #2) An incorporation and understanding of the state of the art technologies (multi-objective optimization, boosting/ bagging, SVMs, fuzzy rule induction, etc). #3) specific implementations of the core machine learning techniques specialized for automated trading.

My basic belief is that these patterns of machine learning will continue to drive the state of the art of extracting money from the global markets.

I have been reading a few books that are related:

• Empirical Market Microstructure
• An Introduction to High Frequency Finance

This paper gives a pretty good overview and meaningful conclusions from the space…

Characterization of Financial Time Series

Google’s Go: A New Programming Language That’s Python Meets C++

Go site

1st blush looks nice. Types good, C like performance good. Built for multi-core etc. I will admit I wish it looked more functionally inspired. I am loving Erlang these days (and to a lesser extent Haskell) so going more imperative again feels a step back…

I will give this one a *go* I think…

Why blog about Erlang now?

For me writing blog posts is often a therapeutic way to work out puzzles and clarify my own understanding of a concept. I have been writing Erlang on and off for around 1.5 years now. As of late I have also been typing a lot more Haskell and some OCaml. This has all been great and I love these languages. I have decided to commit a bit more to Erlang lately and I thought I would start blogging again in earnest – mostly with small-ish Erlang examples just to force myself to be able to explain the concepts. I can program in Erlang (to some level at least) – now I want to be able to be good enough to explain Erlang software to people.

Essentially – I think that erlang the language is OK, I think that Erlang the runtime is amazing and I think that OTP – the common set of Erlang behaviors is very powerful. The entire ecosystem puts together a very compelling offering. You can achieve massively parallel, manageable bits of software in small enough bits of code that it is ownable. So – Erlang it is…

So in that spirit – I will start here with some very simple Erlang concepts.

Erlang is Dynamically Typed

Erlang is a dynamically typed language that to be honest – I found pretty ugly (syntactically) at first. I have done plenty of programming in dynamic languages – especially in Ruby (and to a slightly lesser extent – Python). I really do like dynamic languages but to be honest; I do believe that in larger software projects today that static typing can really by you a generally higher degree of assertable ‘correctness’. I prefer Haskell’s type system to many others – but my basic point is that I think types are powerful. In any case – I still think Erlang’s total offering is powerful enough to make a concede this point.

While Erlang is dynamically typed it does have a simple way to organize primitive data elements into composite types – this is Erlang records. You can also of course use tuples (a sort of anonymous type). In addition, Erlang code is organized into modules. So – while we may organize our code differently than in other languages – we have some good organizational concepts.

Modules

-module(exchange).

Records and Tuples

As I mentioned – erlang lets us organize primative elements into composite data types. The anonymous way of doing this is tuples. The ‘named’ way of doing this is records.

-record(order,{market, price, size, side}).

This is the declaration of a record in Erlang. You can declare an instance of a record like this:

#order{market=vodln,price=127.95,size=350,side=bid}

It is essentially a nice dress on top of a tuple which would look like this:

{vodln,127.95, 350, bid}

So – records are really just nice ways to put together composite types with names. Tuples are ways to assembly composite types anonymously. Both have their usages.

Object Orientation vs Functional Programming

Notice that these records/ tuple things have no methods ‘on’ them. Typically when you declare a type in OO languages – you put methods on them. This whole bundling up of methods and data IS OO programming. Functional Programming lets you have data and describe functionality. What this means is that it is usually easier to add more functions on a given data type in a functional language – but harder to change the data structures. This is because in an OO language – the data is WITH the functionality. In a functional language the data is apart from the functionality. It is all a tradeoff, but I think the functional paradigm is often much closer to the domains I have been working in (trading specifically). So – FP takes away the general encapsulation of data/ methods together. Note that you can still hide private functions inside a module. Eg; you implement a module and export 1 of 10 methods. This is still a good way to expose a minimal surface area of your software to the outside consumer of it.

Functions

Functions are – as you may have guessed at the heart of Erlang.

levelFor(Order, Exch) when Order#order.side == bid ->

levelForSide(Order, Exch, bid, fun(E) -> E#exch.bids end);

levelFor(Order, Exch) when Order#order.side == offer ->

levelForSide(Order, Exch, offer, fun(E) -> E#exch.offers end).

Here we have an example function that I will use as a demonstration for several points.

• You declare functions in erlang as a top level item. You do not put them inside of records, you put them in a module – but that is it.
• Values/ Parameters all start with upper case letters (I found this very odd at 1st).
• You can declare the same function N times each with different ‘guard clauses’ and the order of the declrations is the order in which the runtime attempts to bind to them.
• This example shows how to extract values from records.
• This example shows  simple funs- anonymous functions.

Declaring functions, Guard Clauses, Extracting data from records

“levelFor” is the name of our function. This function takes 2 arguments named Order and Exch. The arguments come after the name in the (…).

After that is a guard clause – this ‘when’ is evaluated by the runtime to see if the method should be executed. So – we can see in case 1 – if the side == bid – then we will execute the method. The actual method body comes after the ->.

You can extract values from records using this syntax ValueName#recordName.fieldName. So our example above has several of those.

Funs and multiple function Declarations

The bit after the -> is invoking another method and passing in as the last argument an anonymous function. That bit that starts with fun(E) and ends with “end”. This is simply an unnamed function (delegate, function ptr, etc) passed to the next function.

Also notice that I declared the function 2x – at the end of the 1st declaration I put a ; and at the end of the 2nd I put a “.” ( a dot). This is a convenient way to get switch statements out of our code nesting. We use the guard statements to see which of the functions get bound to at runtime. We could have also used if or case statements – but for this example’s sake – we are not

OK – that is all the time I have for today… more next weekend

-Chris

• 15 Scalability and Performance Best Practices
• The Joy of Rx: Building an Asynchronous API with Rx and IQueryable
• We are not doing this to beat the other guys (amazing Honda inspiration)
• Japan Lean Study Mission Day 4: Toyota Home, Norman Bodek, and Takeshi Kawabe (More amazing Honda inspiration)

So – I am deep into Erlang (still learning for > 1.5 years), Getting into Clojure (new) and getting into Haskell (a few months now). I have decided to spurn the OCaml/ F# branch of the world at least for a little while…

Anyhow – Erlang is the functional language I have the most experience with, yet next to Haskell – the LANGUAGE feels weak. Erlang the language + OTP + the Erlang VM runtime is amazing but the Language itself is not as lovely as Haskell. When I am typing C# code, I am finding myself more and more functionally minded. For a few years now I have felt the move to functional programming on me. C# has had great improvements – but it still feels way too brittle and verbose next to Haskell.

I recently read these articles:

Haskell for C# Programmers Part 2: Understanding IO.

Haskell for C# Programmers Part 3: Visualizing Monads

I found them helpful understanding monads as someone with a large C# experience. I also found this one:

Monads as containers very helpful. I will admit to having read dozens of articles on monads and just now really getting to a point where I think I can actually perceive the need for writing one of my own.

Functional programming style has made great differences in my personal style in several languages. I like programming languages and I am literate in many. Seek the monad, but 1st seek to understand the primitive bits of functional programming and it will aide your programming in any language.

I have been thinking quite a lot about a few programming topics:

• Correctness – how do we make it easy to write correct software
• Size – how do we curb growth in size of our software (characters – not just lines – it all counts!)
• Understandability – how do we keep our software simple looking
• Parallelism – how do we really scale out across CPUs, machines from a language perspective
• Performance – how do we keep all of these other things AND have performance

All in all – I am after a functional/ functional hybrid language in order to achieve improved correctness, reduce size, increase understandability and promote parallelism. I have been working on a ‘pet project’ – mini-exchange for matching orders – a limit order book system. This has got me thinking about extreme performance.

Recently a former colleague went to work for the LSE, while another went to go work for a low latency trading group @ one of the banks. In light of recent events (the LSE is going to abandon windows) and when considering the experiences of reality – I think windows is simply not the tool for exchanges/ low latency trading – at least not now. This is much debated and I do not want to debate it – just to state my opinion.

Windows is not the solution for high frequency trading ’nuff said

Now – I have classically spent most of my professional development career working on windows machines developing software to be used by traders. I have done Java programming, some C++ programming, ruby and a zillion other languages for play.

Here is a good pointer to s set of writing on windows vs *nix performance.

What functional languages offer what performance?

Functional languages will win in the end for all the goals I have  - especially the languages that allow some OO-isms. I am really a fan of several functional languages. I have been doing small to mid size experimental projects with Clojure, OCaml, F#, Clean, Haskell and a few others. I have done some work on my mac, some on windows, some on Ubuntu. At the end of the day, this is all well and good – and with my 1st caveat (not just windows), the next question is what language on what *nix?

Here are a few links to the great language/ performance shootout

• Ubuntu 64 Bit Quad Core
• Gentoo P4
• Ubuntu 64 Bit Quad Core OCaml vs C++ Intel
• Ubuntu 64 Bit Quad Core OCaml vs Clean

You can see a few basic things here. Currently - C++ GNU g++ is the best perf across the board- not surprising at all.

Essentially we see that OCaml is fast – from 1-3x slower than C++ – max, similar for memory – certain tests OCaml wins, but mostly 1-3 multiplier for memory size. OCaml – unsurprisingly wins – code typically 1/2 the size of the C/C++ code. Clean is also fast in many cases. In certain cases it outperfs OCaml, but mostly OCaml wins.

In many – less trivial cases I bet that OCaml/ functional solutions will compare with C/C++ because it will be easier to make them small, understandable, less redundant etc. For ‘benchmarks’ where you can optimize the code in a small area – you can be as obtuse/ obscure as you need to in order to optimize for the benchmark. It becomes increasingly difficult to scale your software and maintain its performance if it is in some obscure form. Program size and understandability become very important – even to performance as the software scales up.

We need a higher performance functional language in order to use it to compete in the arms race vs uber low-latency trading systems. If we can write low latency trading systems that are competitive with C based systems on *nix machines. 3x slower worst case is still way to slow. It is not 30x slower like F#/ mono – but it is still too slow at that level of competition. Haskell is doing better these days – raking 6th after c/ c++ and java servers. OCaml and clean are a bit behind – but doing OK relative to the bunch.

Abstraction costs performance

Ultimately c/c++ abstract the computer for the most part – they abstract the computers memory and cpu resources. Functional languages are more abstract – they try to hide away the hardware-ness of the computer. This has a cost. The cost is still performance. The value for cost is size, understandability etc. Hopefully we can get the perf more in line and then we will be onto something…

OCaml Trading

Old news, but an excellent watch – Jane Street does trading using OCaml, here is a video…

ATS

Interestingly ATS is a programming language I never heard of. According to Wikipedia:

The performance of ATS has been demonstrated to be comparable to that of the C and C++ programming languages.

and

ATS is derived mostly from the ML and Objective Caml programming languages.

Now this is interesting…. According to the most relevant current shootout mentioned above, ATS is at worst 1.83x worse than C/C++. This is not a bad start. I bet that we could improve that number and we would have a functional – OCaml like programming language that is nearer to competitive with C/C++….

LAgent : an agent framework in F# – Part I – Workers and ParallelWorkers

LAgent : an agent framework in F# – Part II – Agents and control messages

LAgent: an agent framework in F# – Part III – Default error management

LAgent: an agent framework in F# – Part IV – Custom error management

LAgent: an agent framework in F# – Part V – Timeout management

LAgent: an agent framework in F# – Part VI – Hot swapping of code (and something silly)