July 2008
M T W T F S S
« Jun    
 123456
78910111213
14151617181920
21222324252627
28293031  

Chris Donnan : Programming - Brooklyn Style

software, trading, family, fun

How I have fit IronPython, Ruby, JRuby and IronRuby into my existing .net projects

Ruby and Python are great languages. I have spent several years now with ruby and I am just getting into python in the past year or so. I will still admit that I prefer ruby in every way, but I have come to like and appreciate python as well.

I have been working on building some very interesting global derivative trading applications - so how do python and ruby factor in?? Well - I could have ‘just used them’ in several places. That of course is not valuable, but I have found a few ways to use these tools to really add value to my projects.

Scripting for your existing .net application

I have been making an effort to make our plugin application suite scriptable. I have been able to get IronPython 2 running along with a syntax hi-lighting code editor. It has proved extremely useful. I have also tried to do the same for IronRuby, but with less useful success. Why you may ask - predominantly for 2 reasons:

IronPython vs IronRuby for scripting your existing applications

Basically, IronRuby does not let you do some things that are a must! You CANNOT unsub from events yet :(. The other big issue is calling existing .net APIs that need generic args. You can see Jon Lam’s response to my queries here. Jon also said:

Our .NET interop is fairly weak today. Because of our current push to get the language running in time for RailsConf, all .NET interop work is being pushed back into June on our schedule.

I understand, but this does make it virtually unusable for using as an existing scripting engine against my current .net applications.

IronPython 2 does a MUCH better job of .net interop. It basically just works as expected in 99% of the cases I have needed it for. This is just a joy I must say. Especially trying to use the early cuts of IronRuby on the DLR, it was painful to get anything interop based between existing .net and IronRuby really working.

Another thing I have done with IronPython is to support ‘calculated formula columns’ in a UI Grid control. We basically implemented this and it is currently ‘not supported’ in production, but in a future iteration we have planned (and requested from users), excel like formula summaries/ aggregations. We will continue this work with IronPython.

So - as you can see - IronPython made itself able to add value.

So how have I fit in Ruby in a meaningful way?

Essentially, we have a plugin framework so that several teams can concurrently develop ‘independent’ applications that can integrate. This means that there are several global teams developing plugins (think eclipse) that can integrate, use and extend one another. This is great, but it means there are all kinds of build, assembly versioning and other application packaging, deployment issues.

In any big bank, or other corporation I expect, there will be different ways of doing things. The continuous builds for differing teams may be different. Some are nant based, some are simple, some are complex, some are msbuild, some do a better job of CruiseControl setup than others, etc, etc. Now if you want to take the work from several teams like this, and build, package, test, deploy in a sane way, there is some orchestration needed. This is where I have used ruby to add real value.

Ruby, Rake/ Python, BuildBot

Rake (and plain old ruby, not j, iron, or another variety) have enabled me to have a single homogenous view of the otherwise complex world. It is basically my tool for integrating multiple teams that are doing things as is appropriate in their own area, but not necessarily what is the ‘best’ for use by everyone else in the world. Ruby has REALLY shined here. In stead of having dozens of XML, Nant, MSBuild scripts, MSBuild extensions, Nant extensions, cruisecontrol configs, bat files, etc - I have ‘owned’ them all with some rake build scripts. I have managed to do this with some ~200 lines of terse, easy to read ruby.

Interestingly, a smart colleague of mine has done something very similar with plain-old-python (and BuildBot). He has managed to take some 50 projects that can be put together in varying ways, built, deployed, etc. He has taken a very difficult problem (managing a large enterprise suite of .net of branched, versioned and VERY WIDELY UTILIZED software) and owned it with some 400-500 lines of python.

What did we get for our efforts with python and ruby?

Number one is just the joy of really getting into daily use of other languages. I have programmed several languages over the years from C/C++, to Java, to SQL, Javascript, Tradestation EL, to Python and Ruby. Working with new languages is a good experience - bottom line. I have had a lot more time with ruby than python. I always love ruby, and I have really enjoyed more real time with python in the past year. I think these languages have each made me a better programmer.

With regards to scripting applications, these languages, in particular IronPython have just made it too easy to add value fast. I have done the work with ANTLR, regex garbage languages, compiling C#, etc. These approaches ALL take much more effort to get similar rewards. I can see myself working in a mode of ‘core plumbing’ in C# and/ java and the higher level stuff in IronRuby JRuby.

That reminds me, a small note. Since we are java server/ .net client apps, I have used JRuby on several occasions to quickly do some send messages around the network, and otherwise mimic stuff coming from the server. I have had more effort on the client/ .net side of things, but I am sure if I was doing more server work, I would be using JRuby all the time.

Regarding build and integration - the big thing that we got IMO was a homogenous solution. Whereas before, there were the xml files, bat files, etc. - we each now have small bits of easy to manage code that does all we need. This is coupled to a small degree of ‘convention over configuration’, but essentially, we have gotten a lot of value for a little code and effort. We have also gotten easy to maintain solutions going forward.

Downsides??

The only real downside here is ‘adding more tools’. There is always resistance (appropriate more or less) to adding to the enterprise overhead of ‘more tech’. Developers also need to learn more to work in more ‘polygot programmer’ environment. So the ‘more tech is bad’ and ‘devs need to learn more’ arguments are valid arguments. That being said, I think the benefits are outweighing the downsides so far.

The future?

I see more coming in the same spaces. I will be working more on ‘higher level scripting’ of existing .net apps, and build/ integration etc. At some point, I can see doing ‘it all’ with these languages perhaps - but I think the current sweet spot in the stack is adding value at certain points of the abstraction. The right tools for the right job and all that.


TradeStation platform interoperation - using GlobalVariables
Many folks in the TradeStation community have requested this vanilla functionality. The GlobalVariable.dll is here:
http://www.tradestationsupport.com/discussions/Topic_Archive.aspx?Topic_ID=19622
The C++ source is there as well. Basically - the C++ code allocates a shared memory block and then the GlobalVariable.dll 
is used to put/ get from it - thus sharing between multiple exe’s. Have fun. 
-Chris
PS - there is also lots of code/ examples of c++ (win32) and c# interop at TradeStationworld.com search c# and you can 
see all my posts and code.
using System;
using System.Runtime.InteropServices;
namespace TradeStation.Interop
{
public class GlobalVariables
{

[DllImport("GlobalVariable.dll")]
public static extern int  GV_SetInteger( int iLocation, int iVal ) ;

[DllImport("GlobalVariable.dll")]
public static extern int  GV_GetInteger( int iLocation ) ;

[DllImport("GlobalVariable.dll")]
public static extern int  GV_SetNamedInt( string sIntVarName, int iVal ) ;

[DllImport("GlobalVariable.dll")]
public static extern int  GV_GetNamedInt( string sIntVarName, int iErrorCode ) ;

[DllImport("GlobalVariable.dll")]
 public static extern int  GV_SetFloat( int iLocation, float fVal ) ;

[DllImport("GlobalVariable.dll")]
 public static extern float  GV_GetFloat( int iLocation ) ;

[DllImport("GlobalVariable.dll")]
public static extern int  GV_SetNamedFloat( string sFloatVarName, float fVal ) ;

[DllImport("GlobalVariable.dll")]
public static extern float  GV_GetNamedFloat( string sFloatVarName, float fErrorCode ) ;

[DllImport("GlobalVariable.dll")]
public static extern int  GV_SetDouble( int iLocation, double dVal ) ;

[DllImport("GlobalVariable.dll")]
public static extern double  GV_GetDouble( int iLocation ) ;

[DllImport("GlobalVariable.dll")]
public static extern int  GV_SetNamedDouble( string sDoubleVarName, double dVal ) ;

[DllImport("GlobalVariable.dll")]
public static extern double  GV_GetNamedDouble( string sDoubleVarName, double dErrorCode ) ;

[DllImport("GlobalVariable.dll")]
public static extern int  GV_SetString( int iLocation, string sVal ) ;

[DllImport("GlobalVariable.dll")]
 public static extern int  GV_SetNamedString( string sStringVarName, string sVal ) ;

[DllImport("GlobalVariable.dll", EntryPoint="GV_GetNamedString")]
static extern IntPtr GetNamedString( string sStringVarName, string sErrorCode ) ;

[DllImport("GlobalVariable.dll", EntryPoint="GV_GetString")]
 static extern IntPtr GetString( int iLocation ) ;

 public static string GV_GetString( int iLocation )
{
IntPtr ptr = GetString(iLocation);
//return Marshal.PtrToStringAuto(ptr);
return Marshal.PtrToStringAnsi(ptr);
}

public static string GV_GetNamedString( string sStringVarName, string sErrorCode )
{
IntPtr ptr = GetNamedString(sStringVarName,sErrorCode);
//return Marshal.PtrToStringAuto(ptr);
return Marshal.PtrToStringAnsi(ptr);
}

}
}

From saturday - new Interactive Brokers C#/.Net interface
Spent the day fiddling with this InteractiveBrokersHost.cs . Good ActiveX Wrapper for a good direct access broker (with great comissions). Took a bit of ‘jimmying’ to get it straight, but it is there. I never was much of an activeX visual component guy, and now in .Net I will reiterate my disaproval. It does however meet an end, next is to make a socket based implementation to the brokerage so I can eliminate the activeX layer and compile on mono.

L8,

Chris