I have worked for some years on genetic, evolutionary, memetic and other optimisation software - specifically in the context of automated/ systematic trading. I have also worked for some years on very multi-threaded software systems. I have been interested in the actor model for writing concurrent software.

Axum

Axum anincubation project at Microsoft implementing the actor model for the CLR. Others have posted on it - Matthew Podwysocki has two posts here and here that are worth a look. The Axum team blog is here. Have a read as you can, they are much more ‘on it’ wrt Axum than I.

Genetic Algorithms

Genetic algorithms are a class of algorithms from the school of ‘evolutionary computation’. I will not go into a study here, just google genetic algos. Needless to say - this example is a very naive example. I tried pretty hard to use any of my ‘real’ evolutionary algorithm libraries, but they just did not work easily enough with Axum. More on that later…

Caveats

Aside from being a really naive GA, the demo is also ver imperative and array/ for loop based. This is not good - but I was focused on getting Axum running. After I was unable to pull in my real evolutionary algo libs I just went for bare bones. We are also doing a ‘pure numeric’ optimization problem, this is a common test alog for testing multi-objective optimizer software. It was easy to convert to axum - so I used it. Perhaps for future examples I will try to apply a GA in a more interesting way (eg; not just a maths problem).

High level steps I followed to getting the demo running

1. Define the data elements flowing between actors. I 1st tried to use a class called ‘individual’ from my standard libs. Axum has a concept called schemas. Here is a post from the Axum team blog on schemas. Schemas are data transfer objects or ’schematized messages‘ that are immutable and used to travel in a type safe way between agents, across channels.
2. Next I defined the channels. Channels are aptly named, they are typed channels that your messages (those schematized object instances from above) travel through from one agent to another.
3. Next I defined the agents that received messages from a channel, and published messages back onto a channel. In the agents are any ‘real code’. the schemas and channels are just construct definitions to establish what can travel around.

Although a simple example, it was fun. I missed having Resharper also ;). I will likely continue on and try to get a more ‘real’ optimizer running using Axum and report back. Love the model, I am skeptical that much will make it into the CLR I hope some does…

Due to the nature of my current project, and more-or-less my career developing trading related software - I have been thinking hard about concurrency lately. This is not a new thing, but I feel for the 1st time in some years that we are really ready to have some substantial changes in the ways that we write highly concurrent software.

Pub/sub messaging

I have had some great experience doing pub/sub messaging work in the past, specifically @ Merrill Lynch, and to a lesser degree at Morgan Stanley years back now. As it goes, many applications in finance tend to be based on pub/sub messaging in one form or another. These types of systems create a few dynamics that are interesting. Incoing messages are coming from elsewhere on the network so they are inherently immutable. As it goes immutability winds up being very important - essentially having as little shared state as possible is key. Messaging systems tend to make this true to some degree at least.

Immutability

We are constantly trying to make more and more immutabe in my current project. I have been looking a whole bunch at BCL Extras (more on that below) for immutable collections. (When we are doing ‘edits’ we are basically copy-on-write since we are still working on mutable systems to one degree or another). In any case, the whole immutability thing simply enables simpler concurrent access; things can’t change underneath you if they can’t change at all…

(Better Concurrent Collections)

This is an aside, but I am also working hard at creating/ finding/ using better concurent collections that help us to use them correctly. There are some good articles on low/ no lock techniques and Jared Par has a good post here.

BCL Extras

BCL Extras is an intestesting library written by Jared Par. The most interesting bits in here are good implementations of immutable collections, including a good immutable map. We are currently trying to find a good way to use immutable collections alongside our better ‘concurrent mutable’ collections (low locking, still mutable, etc). I recommend just checking this library out - it is along many of my favorite trajectories; Concurrency, Immutablity, Functional Programming and LINQ.

Scala, Erlang

Erlang and more recently Scala use the messaging/ actor based concurrency model. People wrote much more intelligently than I on this here, here, here, etc.

Axum

Axum (formely called Maestro) is here. This is an incubation project at Microsoft for implementing natively in the .net framework the actor/ message concurrency model. Their team blog is here. Axum is pretty exciting - if it ever makes it out of incubation.

Retlang

Written mostly by Mike Rettig for use at DRW Trading. Retlang is most interesting to me because #1 it is useable today in the context of my current project (unlike scala/ erlang/ haskell, axum, etc) #2 it is being used today for real world applications. #3 a former colleuge Mike Roberts and I had a few messages on it and he gave it is ‘thumbs up’. Personal vouchers from trustworthy sources are very useful. his slides for his presentation on Jetlang/ Retlang are here. 

Retlang feels like a pub/ sub messaging system, the difference is that you use it in process. The model that has worked successfully for messaging and driven a certain way of working that removes your locking code etc. is just used in process to give you similar dynamics.

Retlang gives you channles and fibers as its basic bits and pieces. You pub at one end of a channel, you sub at the other end of the channel. It gives you execution queues that your components/ subscribers feed from and act as if they are all working on 1 thread. This means you can punt out your locking all over your business logic. 

Again, with this model, within a fiber, you think about everything as single threaded. Dependency injection is for getting the assembly out of the business logic. With retlang (the messaging/ actor model) we get the multi-threaded-ness out of the business logic and it is wired together. Everything in a fiber is just a component/ set of classes that are wired together and are in a single thread together. We now can punt all the locking, etc code OUT of our business logic code. 

Channels are pub/ sub conduits and fibers are the wiring of what is executing on the same thread. You can have 1 fiber that listens to N channels, and all the subscribers on that fiber will be executing in the same thread. This is SO much like dependency injection it astounds me. We get to NOW assume we are single threaded within this component.

This all sounds good to me. I am going to seriously consider Retlang currently. 

Conclusions

I really think we are at the edge of changing the way that we build concurrent applications. We need to. Building concurrent apps the way to do today is too hard. Really it is. I am looking forward to working hard to make it better so I can deliver better software, faster to my desks!

-Chris-

In my current project, I have been using some very interesting (to me at least) tools and techniques that I thought were worth mentioning. There are many vaguely related people doing similar things, but not exactly the same…

Common Probems

Boilerplate code, excessive mapping, and DRY (Don’t Repeat Yourself) violations. It is very easy to have several essentially redundant yet seemingly necessary representations of your domain model manifest as code in your software. Lets take a simplifed example domain - Trade. (pseudo-code)

class Trade { int TradeId, Trader Trader, Security Security, int Quantity } //etc

Now - it is not common to have some mapping to persistence, either your classical ORM, or XML persistence. You might have some code that essentially takes that domain (your *canonical representation*) and maps it (or *projects* the canonical representation) into another domain (xml, database, etc). You get something like:

class TradeDao {void Save (Trade trade) Trade LoadById(int id)}

It is also common to have a messaging/ transportable representation of the type, again, this could be XML, bytes, serializable in some custom form + a message envelope, etc. The point is there is usually some other mapping for ‘messaging’.

class TradeMessageAdapter {Trade FromMessage (Message msg); Message ToMessage(Trade t);}

In addition, it is also common to have a mapping or a set of representations of your types that are specialized for user interface code. UIs do not have the same locking semantics as servers usually, they have databinding ramifications (especially in WPF as our apps are), so you have a mapping or *projection* into that domain. These types of projections are interesting, since in many cases the type system just does not have any real way (in C#/ the ‘raw’ CLR at least) to generalize these types of relationships, more on this another time…

class TradePresentation { UiProperty<Trader> Trader; UiProperty<Counterparty> Counterparty; }

In some cases - you have a pure data projection, our message example is like this. In some cases you have a semantic projection (eg, figure out relevant operations based on the data and do something with it (our above DAO example is like this). 

Data Projection = Mapper

Anytime you are taking a property/ field/ value from 1 place and packing/ unpacking it into an alternate form with essentially the same name, you have some sort of mapper. This is essentially what I will call a data projection, as in our Trade to TradePresentation projection.

Semantic Projection = Boilerplate Code

Anytime you can look at the model and infer some operations, and generate boilerplate code (eg; our DAO example), you have what I will call a semantic projection. In the DAO example, you can look at the relationships in the canonical model and then generate methods/ their bodies in a templated way, since Boilerplate code is really just a form of a template that can’t be automated/ abstracted away by the type system you are using.

Canonical Model = the essence of your domain model

The canonical model is that - the essence of your domain. It is our 1st Trade example above. There is no domain specific information in it, just the most terse, easy to understand version of your domain.

How do we make practical use of these concepts?

C# can make classes, they suffice to represent the canonical domain - you make classes. You can be fancy or simple, but essentially you model your cannonical domain using C# classes. How do we then project the canonical domain into another domain? Using T4 Templates/ Code Generation with your canonical model.

T4 Templates

VS 2008 has a great feature called T4 templates. You can simply add a file with a .tt extension, put some ASP.Net like code into it and it will generate you some new files, usually C# classes. 

Example projection T4 Template

<#@ Output Language=”C#” #>

<# foreach (var type in myModel)  { #>

class <#= type.Name #>Message : BaseMessageTypeOfYourFlavor {

    public void FromInstance(<#= type.Name +” “  type.AsVariableName #>) {

        <# foreach (var property in type.Properties) {#>

              AddValue<<#= property.Type #>( <#=type.AsVariableName #>);

       <#}#>

    }

}

<#~

var myModel = new MyCanonicalModel();

#>

Assuming your canonical model had a few types in it, it would generate something like:

class TradeMessage : BaseMessageTypeOfYourFlavor {

      public void FromInstance(Trade _trade) {

           AddValue<int>(_trade.Id);

           …etc

     }

}

We will have to do some more work to make it handle mapping other types from the canonical model, but that is still the basic idea. We generate any repetitive code, mapping code etc.

If you use partial classes, it turns out you can easily genrate different aspects of the same class into different files, it is really quite nice! More on that next…

Conclusions

So - this works towards a dynamic where you focus on writing ‘framework layers’ and your canonical model, then generating instances of what you need from your domain layer for each of your framework layers… Again, more in the future here.

So - this is some of the stuff on my mind these days. More to come on these topics. The basic ideas are relentless automation, only stating relationships and the general model 1x in your software (DRY), do not write boilerplate code, etc. This gives you a LOT more leverage. It also lets you focus on modeling the actual domain model, but does not restrict you from using it projected into several other areas by making concessions IN your canonical domain model. The last thing we want is ALL of our domains subtley manifesting themselves IN our canonical model…

So - go have a play with T4 and consider ways to do some of what I have mentioned. I think the important ideas are presented herein. I will work on some better practical examples going forward.

-Chris-

The AlloSphere, truly insane !!!!

I have been using the ‘attached behaviors’ pattern more and more in my adoption of WPF. In general, I spurn significant ‘code behind’ in my windows/ controls, etc - unless absolutely necessary. Using attached properties in WPF, you can actually build up a library of behaviors that you can attach to various objects in your application. I have a large # of these behaviors in my current project - especially for Xceed’s WPF DataGridControl.

I will use this simple example to illustrate the concepts.

The goal is to allow a window to resize when a specific tab is focused. I have a window with a number of tabs, and for most tabs it is appropriate to have the same size window (relatively small). There is one tab however that is too squashed to look at in this small-ish size. What I decided to do was to save the users from manually resizing the window each time they went to it. The result was this simple ‘attached behavior’.

Attached properties in WPF are properties that you can attach arbitrarily to any dependency object. In this example, I simply attached my TabBehavior to the TabItem like this in XAML:

<TabItem  Behaviors:TabBehaviors.ResizeOnFocus=”805″ … />

The underlined bit is my attached property. It is simply giving the size I want to resize the parent window to when this tab gets focus.

So - how does this work? Well the effect of adding an attached property is to pass the value in the argument + the object the attached property is decorated on… So you get a call with the TabItem and the 805 value passed into your code.

The 1st step is to declare a public static class called TabBehaviors, next we declare our 1st attached property; ResizeOnFocus - like we see above:

This is very much like a normal Dependecy

Property in WPF. Note the last bit - the callback OnResizeFocusChanged; this is what we get when the property is set on an object.

The OnResizeFocusChanged method is where the magic starts (I have renamed that method after I took my screenshots ). The 1st argument passed in is the object the attached property is attached to - the TabItem in our example. The 2nd argument has the NewValue - the 805 value in our current example. The next bit - the DependencyPropertyDescriptorFromProperty is a way to add a ‘changed’ handler to the IsSelected property of the tab item. This is sort of like using a .net event, but it is just the WPF Dependency Property way to do so. 

I have talked with people for some time about work I had done in the past with copulas… today saw a good ref on Slashdot:

The Formula That Killed Wall Street

I do not like PowerShell. 

I still love Rake for building/ running .net apps

Visual Studio crashes all the time when editing XAML (it hates custom attached properties I think)

Xceed’s latest release of their grid is replete with sucky bugs

I feel overly awake at 11:30, that is what happens when you go to the gym at night

The global economy sucks, everyone is feeling it. I hope the low is not so low we can’t all bear it. I really do not want my wife and kids to be truly effected by it.

I love my iPhone and my MacBook Air

I am pissed that my skateboard went missing during our move

I love living in London

I have been thinking a lot about WPF, what are good practices for larger apps wrt managing XAML/ templates/ styles etc

I want a good lock free (or at least low, low lock) hashtable for C# (not a bad place to start Julian M Bucknall)

Windbg rock

The latest Ants profiler is pretty impressive, well done

OK, that is enough

Well, it has been 55 days since I posted last - that is a record for me! As of my last post, my family and I moved to London. Since then, we have lived in Knightsbridge for a few weeks, gone on holiday to Austria, moved to Chelsea, unpacked our goods, furnished rooms, hooked up bills of all sorts… and so on.

It has been a whirlwind of activity, expense, fun and overall goodness. I have been ‘FaceBooking’ all my small, personal items regularly - it takes very little effort. Now that that business is all through, and life is getting back to ‘normal’, I will be blogging again about my normal technical fare.

Currently I am building some good software using WPF and the other MS stuff. I am also getting back into using Rake for my build and general scripting automation. All in all, things technically have been much the same. I plan to be elaborating more ASAP!

For now, here we are having fun in Austria

-Chris-

It has been a while since I have posted. It has been even longer since I have posted about anything I would normally post about (eg; software/ finance). I WILL post soon on those topics, but today - we have met a milestone - my family has moved to London

It has been a long road - but we are here now. We will be staying for ~2 months in Knightsbridge (~ 1 block from Harrods). Gabe slept the entire flight; before we took off, Gabe fell asleep - and he woke right after we landed. Micah slept the 1st half of the trip and left Shannon awake the whole time :(. I managed to rest for maybe 2-3 hours during he 2nd 1/2 of the flight.

We made it throught the airport easily, some traffic getting into town, and we made it to our appartment. It is cute and comfortable - I will post some pics later today.

All in all, a pretty easy trip. The move itself was also pretty simple - the movers (Continuum I believe) were excellent. Having movers that pack was an excellent experience - we will try to opt for that one again in the future if at all possible - it saves a lot of hassle.

So -  there you have it. We will be getting settled; school interviews with the kids, setting up banks, phones and such in the coming days. Soon enough we will find our own apartment and settle into life.

More to come;

Chris

PS; George Soros “The New Paradigm for Financial Markets” is an interesting read - I am about 1/2 way through, but I would already recommend it.

A few weeks ago, Gabe (and Micah as well - he does what Gabe does) got interested in skateboarding. This led to me buying them 2 crappy skateboards to ‘gauge the depth of their interests’. That led to ME getting interested again and getting myself a new board (I LOVE it!).

I told Gabe that if he kept riding the one we got him - showing us with practice - that I would get him a ‘real skateboard’ not one of the toys r us toys. He has been into it - riding where he has been able so we surfed around online to pick decks he liked etc. Then yesterday the 2 of us went to Blades on Broadway and got him his new board:

We also got him a new set of pads (he already has a good helmet from all his bikes/ scooters, etc.

• 5:00 AM - Wake up, doze
• 5:45 AM - Get up, eat breakfast (Optimum Slim + Soy Milk)
• 6:00 AM - Skateboard over to P.S. 29 to VOTE FOR BARACK OBAMA
• 6:45 AM - Done voting, skateboard home
• 6:50 AM - Send Shannon to vote, sit with Gabe on sofa - watch some cartoons (he was soooo sweet this AM)
• 7:15 AM - Shower, dress
• 7:30 AM - Shannon home, chat with Micah for 2 minutes (he was in a cute sweet mood) off to work - catch the bus, to the subway (4,5,6 green line to 51st & Lexington)
• 8:15 AM - Arrive @ office (side-trip to Starbucks for a coffee)
• … get water, bananas, fruit salad, check email, login to Bloomberg, CNBC on/ off/ on
• … WPF Command Binding to ContextMenus
• … Help co-worker deal with a few small WPF-isms (say ElementSpy/ ToolTip issues)
• … Update RedGate SQL Toolbelt install
• 11:45 AM - Lunch - Salad + Salmon + Steamed Cauliflower
• 12:00 PM - Back to work
• … more WPF stuff, Xceed Data Grid - forcing right click to select the row beneath it (not successful yet!)
• 1:30 PM - Double Tall Soy Latte to prep for some database work
• 1:40 PM - Get to some nitty gritty database work - let’s get rid of some crap in the DB…
• … work work work …
• not a lot of trading today, markets up well positive PnL
• 5:00 PM - Virgin Frequent Flier accounts set up for the whole family
• 5:45 PM - Leave (a little early) Subway @ Lex + 51st, out @ Jay St, B75 Bus to Kane St + Court St, walk home
• 6:35 PM - Home, hugs + kisses from sweet kids + sweet wife, change, hold Micah, watch Gabe do homework
• 7:00 PM - Shannon made a *lovely* orzo soup - *yum* (did I mention she is a ROCKIN’ cook?!), Hang with Micah a bit more
• 7:30 PM - Read books to boys (”Don’t Let the Pigeon Drive the Bus” + “Surprise in the Lunchbox” (something like that))
• 7:45 PM - Skateboard to the gym, boxing!!! cycling, cancel gym membership as of Dec 1 (since that is when we MOVE!)
• 9:00 PM - home with Shannon ! Sofa…
• 9:15 PM - start typing random crap in my blog while I am anxious about the ELECTION
• 9:25 PM - ABC News projects Obama as winner of Ohio! (big deal here folks!!)
• 9:35 PM - Lookin’ grim for MCRAGE!!!!
• 9:40 PM - Stop this nonsense blog entry…

Summary - go Obama, go Skateboard, go Shannon, go Gabe, go Micah, go London

My boys have been getting into skateboarding as of late. After a short time watching them, I got the itch to get a board. Went to CraigsList, found a neighbor with a brand new board they wanted to sell… Got it!

2 minutes later - I was skating through Brooklyn. I felt like a kid - how awesome. Can’t wait to skate some more with the boys!!!