I have been working on building some very interesting global derivative trading applications - so how do python and ruby factor in?? Well - I could have ‘just used them’ in several places. That of course is not valuable, but I have found a few ways to use these tools to really add value to my projects.

Scripting for your existing .net application

I have been making an effort to make our plugin application suite scriptable. I have been able to get IronPython 2 running along with a syntax hi-lighting code editor. It has proved extremely useful. I have also tried to do the same for IronRuby, but with less useful success. Why you may ask - predominantly for 2 reasons:

IronPython vs IronRuby for scripting your existing applications

Basically, IronRuby does not let you do some things that are a must! You CANNOT unsub from events yet . The other big issue is calling existing .net APIs that need generic args. You can see Jon Lam’s response to my queries here. Jon also said:

Our .NET interop is fairly weak today. Because of our current push to get the language running in time for RailsConf, all .NET interop work is being pushed back into June on our schedule.

I understand, but this does make it virtually unusable for using as an existing scripting engine against my current .net applications.

IronPython 2 does a MUCH better job of .net interop. It basically just works as expected in 99% of the cases I have needed it for. This is just a joy I must say. Especially trying to use the early cuts of IronRuby on the DLR, it was painful to get anything interop based between existing .net and IronRuby really working.

Another thing I have done with IronPython is to support ‘calculated formula columns’ in a UI Grid control. We basically implemented this and it is currently ‘not supported’ in production, but in a future iteration we have planned (and requested from users), excel like formula summaries/ aggregations. We will continue this work with IronPython.

So - as you can see - IronPython made itself able to add value.

So how have I fit in Ruby in a meaningful way?

Essentially, we have a plugin framework so that several teams can concurrently develop ‘independent’ applications that can integrate. This means that there are several global teams developing plugins (think eclipse) that can integrate, use and extend one another. This is great, but it means there are all kinds of build, assembly versioning and other application packaging, deployment issues.

In any big bank, or other corporation I expect, there will be different ways of doing things. The continuous builds for differing teams may be different. Some are nant based, some are simple, some are complex, some are msbuild, some do a better job of CruiseControl setup than others, etc, etc. Now if you want to take the work from several teams like this, and build, package, test, deploy in a sane way, there is some orchestration needed. This is where I have used ruby to add real value.

Ruby, Rake/ Python, BuildBot

Rake (and plain old ruby, not j, iron, or another variety) have enabled me to have a single homogenous view of the otherwise complex world. It is basically my tool for integrating multiple teams that are doing things as is appropriate in their own area, but not necessarily what is the ‘best’ for use by everyone else in the world. Ruby has REALLY shined here. In stead of having dozens of XML, Nant, MSBuild scripts, MSBuild extensions, Nant extensions, cruisecontrol configs, bat files, etc - I have ‘owned’ them all with some rake build scripts. I have managed to do this with some ~200 lines of terse, easy to read ruby.

Interestingly, a smart colleague of mine has done something very similar with plain-old-python (and BuildBot). He has managed to take some 50 projects that can be put together in varying ways, built, deployed, etc. He has taken a very difficult problem (managing a large enterprise suite of .net of branched, versioned and VERY WIDELY UTILIZED software) and owned it with some 400-500 lines of python.

What did we get for our efforts with python and ruby?

Number one is just the joy of really getting into daily use of other languages. I have programmed several languages over the years from C/C++, to Java, to SQL, Javascript, Tradestation EL, to Python and Ruby. Working with new languages is a good experience - bottom line. I have had a lot more time with ruby than python. I always love ruby, and I have really enjoyed more real time with python in the past year. I think these languages have each made me a better programmer.

With regards to scripting applications, these languages, in particular IronPython have just made it too easy to add value fast. I have done the work with ANTLR, regex garbage languages, compiling C#, etc. These approaches ALL take much more effort to get similar rewards. I can see myself working in a mode of ‘core plumbing’ in C# and/ java and the higher level stuff in IronRuby JRuby.

That reminds me, a small note. Since we are java server/ .net client apps, I have used JRuby on several occasions to quickly do some send messages around the network, and otherwise mimic stuff coming from the server. I have had more effort on the client/ .net side of things, but I am sure if I was doing more server work, I would be using JRuby all the time.

Regarding build and integration - the big thing that we got IMO was a homogenous solution. Whereas before, there were the xml files, bat files, etc. - we each now have small bits of easy to manage code that does all we need. This is coupled to a small degree of ‘convention over configuration’, but essentially, we have gotten a lot of value for a little code and effort. We have also gotten easy to maintain solutions going forward.

Downsides??

The only real downside here is ‘adding more tools’. There is always resistance (appropriate more or less) to adding to the enterprise overhead of ‘more tech’. Developers also need to learn more to work in more ‘polygot programmer’ environment. So the ‘more tech is bad’ and ‘devs need to learn more’ arguments are valid arguments. That being said, I think the benefits are outweighing the downsides so far.

The future?

I see more coming in the same spaces. I will be working more on ‘higher level scripting’ of existing .net apps, and build/ integration etc. At some point, I can see doing ‘it all’ with these languages perhaps - but I think the current sweet spot in the stack is adding value at certain points of the abstraction. The right tools for the right job and all that.

How Has Functional Programming Influenced Your Coding Style?

When I have time, I will comment - but needless to say - my exposure to FP really has changed (in a positive way) my coding style.

-Chris

and

These lovely images tell us things like:

• JP Chase seems to have some 84 Trillion in derivs exposure (and 1.3 trillion in actual assets)
• Citi seems to be in 2nd place with some less than half that derivs exposure (and 1.2 trillion in actual assets)
• Lehman and Deutsche are way at the lower end of the spectrum of total exposure
• Most of the big firms have their holdings in rate related contracts
• It is ALL about the OTC market contracts

It is an interesting picture, there are more here, but some folks have mentioned that cannot read this link behind most big bank internet content constraints.

Gabe and Micah are playing nicely together today (it is nice when a 3 and 5 year old brothers can get along so well). I have been fiddling with this code on the sofa and while moving my code, I came to move a c# Equals(object) method and the counterpart GetHashCode(). ReSharper is a great tool - it had given the basis for these overrides. Simple Alt+Insert gives you the ‘generate’ menu and from there you can ask it to generate these methods for you. Essentially - you tell ReSharper the fields that must be ‘unique’ and then it can handle the Equals and GetHashCode methods for you.

Here is what I had in my c# code:

Now this is fine (not exactly) for my C# application as I see it. ReSharper kindly churned out the code I needed also - which is great. When I moved it into ruby however, it looked very un-ruby. Astute readers will also notice that the implementations of Equals and GetHashCode do not match - the Alias property was added later, included in the Equals but NOT in the GetHashCode . Generating code has its issues it would seem.

One of the areas that ruby excels at is helping you write less code. Part of that is identifying the little repeated bits in your code and automating it away with a little ruby.

So what is the pattern above? Well - equals checks for null, checks for direct object reference comparison, then checks a number of fields. It also does a type check. Then it compares a number of fields. If any of these fields fails comparison - it returns false. Ruby does not really need the type check, it just says ‘if you have these props then you are fine by me’ (duck typing).

The folks @ JetBrains recognized the pattern - based on the input of what properties are unique, it generates a code snippet. In ruby, we will not generate code, but handle it like this:

Here include a module called SymbolHashEquals. In this module I define 2 methods symbol_hash and symbol_eql?. These methods take a list of attributes and then apply the same logic that resharper generated. This uses ruby’s symbols that allow you to refer to elements by the ‘name of the code’ or … symbol.

In stead of having our class generate methods based on a heuristic or 2, we simply make a module that holds that abstraction, then we include it, and have our host class use it. Essentially, since ruby lets you ‘do more with less’ we do not have to ‘generate more code’ - we simply can codify more into our solution and have less dumb, generated code. Instead of investing in software to generate more code, we do less!

Here is our module that works on the symbols and handes the standard eql?and hash.

Now - rather than generating code every time I need that same logical heuristic, I simply include that 1 module, and we are good to go. Less code, less repetition, etc.

There you have it - ruby rocks!

=Chris=

So, this weekend, as Gabe is sitting next to me playing his Legend of Zelda; I found this article @ Risk Latte. It says essentially what i thought - and even calls dispersion trading “volatility arbitrage”. I will not re-explain what the article says infinitely better than I could, but I thought it was interesting how simply connecting a concept you know to something you vaguely understand solidifies the understanding.

It is amazing how much I know that I do not know

-Chris

Some ~2 or 3 years ago, MIT’s Journal of Evolutionary Computation had a ’special issue’ on AISs. There were several interesting publications in that particular issue, and it led me on a minor reading splurge regarding AISs. At the time, I had been heavily focused on evolutionary and other bio-inspired algorithms and their applications for automated trading systems. There are a ton of algorithms in this class, many of them have mappings to various sub-problems in automated trading.

As far as I can find, there is no work out there currently that applies AISs to finding mis-priced options, and it seems quite a good fit. Searching over a sea of options to sort out which one seem to be aberrations may be a good match. AISs are relatively efficient algorithms compared to alternatives. Interestingly - it would not even necessarily involve your own option pricing model - you would be basically letting the AIS sort out what seem to be ‘normal’ pricings and what seem to stand out as odd.

This is obviously one component of the puzzle, sorting out if the option should be bought or sold is another component. This particular technique could also be applied to any other derivative market with ample liquidity, not just options.

Just some nonsense that has been meandering around my brain for a week or so, so I figured I would write it up here.

-Chris

There have been several theories on why the 1987 crash happened, but it seems that at least 2 books agree that it was due to ‘everybody using Portfolio Insurance’. The other book was Bookstabber’s Demons of our Own Design (which I quite enjoyed). Essentially, some smart folks came up with portfolio insurance. It worked well for them, they started selling it as a service to other firms. Soon enough, everybody was doing it. When the portfolios started hitting their floors, there was a fantastic circular reaction between markets that were now interconnected via portfolio insurance programs all over the street. Downward spiral ensues. Money disappears, etc.

I just thought it was noteworthy that this is the 2nd book that said outright that the basic issue was primarily portfolio insurance.

-Chris

PS - I am wondering if CPPI (zero coupon bonds usually involved) vs the futures technique referred to in the case of the ‘87 crash are the same? Is CPPI an evolution of that portfolio insurance? Always more to learn.

Martin Fowler + Neal Ford

Steve Yegge’s entertaining rant (his dialect of English)

Essentially - Java (and those like it) are good at making the lowest level bits and pieces of software, but bad at making logical ’sentences’ of software.

All seems interesting….

Back to my non-stop work-athon - build done.

-Chris

There is a ton more to read out there. This is plain nuts, crazy stuff folks!!

=Chris=

Here on cnbc Jim Rogers talks about the current state of the US economy, how idiotic Bernanke has been in his efforts to ’save the us economy’, the commodity markets (agriculture in particular), China, inflation, the debasing of the USD and more.

The strongest point (to the Fed); stop trying to ’save the economy’. Free markets will do a better job. Let people, companies, etc. fail. Stop printing money - it does not work. It seems the current fiscal policy is like the Doritos advertisement campaign of a few years back “crunch all you want, we’ll make more!”. Similarly - Bernanke seems to be saying “Problem? NO problem! You need $? We’ll make more!”. This cannot work. It violates the most basic economic rule; the more of some thing there is - the less it is worth. The very creation of more makes each worth less!!

Anyhow - go listen for a few minutes; well said Mr. Rogers.

=Chris=

Fun;
Chris

PS - I have been doing random .net build things with rake this evening. I gotta say - it is pretty darn nice and thusfar - seems better in many ways than MSBuild. More to come there too, it is still day 1 of playing.