Generating Objective-C code with a ruby DSL

vMAT uses a ruby DSL to do code generation for certain repetitive code patterns. For example, the way coercions between matrices of different types is handled (dynamically, at runtime¹) currently requires implementing an Objective-C++ dispatch method to copy to every numeric type from every numeric type. With 10 distinct numeric types that is 100 Objective-C++ methods that are slight variations of the same method template.

This is the problem that vMATCodeMonkey was designed to solve. Given the following specification in the file copyFromMethods.mk²:

#  copyFromMethods.mk
#  vMAT
#
#  Created by Kaelin Colclasure on 4/23/13.
#  Copyright (c) 2013 Kaelin Colclasure. All rights reserved.

require 'vMATCodeMonkey'

VMATCodeMonkey.new.coercions do |to, fm, to_t, fm_t| <<"EOS"
- (void)_copy_#{to}_from_#{fm}:(vMAT_Array *)matrix;
{
    copyFrom(self, matrix, #{to_t}, #{fm_t});
}
EOS
end

vMATCodeMonkey produces 100 methods are all slight variations on this one:

⋮

- (void)_copy_miDOUBLE_from_miSINGLE:(vMAT_Array *)matrix;
{
    copyFrom(self, matrix, DOUBLE, SINGLE);
}

⋮

“That’s not very Impressive”, I hear you saying. Okay, let’s look at a more complex (but still highly repetitive) coding task: Parsing all those vMAT_API function’s options arrays.

Given this specification from clusterOptions.mk:

#  clusterOptions.mk
#  vMAT
#
#  Created by Kaelin Colclasure on 4/26/13.
#  Copyright (c) 2013 Kaelin Colclasure. All rights reserved.

require 'vMATCodeMonkey'

VMATCodeMonkey.new.options_processor <<EOS, :static
"criterion:"  arg: { choice => { "distance" => set(:useInconsistent, false), "inconsistent" => set(:useInconsistent, true) }}, default: "inconsistent"
"cutoff:"     flag: set(:useCutoff, true), arg: vector(:double)
"depth:"      flag: set(:useInconsistent, true), arg: scalar(:index), default: 2
"maxclust:"   flag: set(:useCutoff, false), arg: vector(:index)
EOS

vMATCodeMonkey produces the following code:

// vMATCodeMonkey's work; do not edit by hand!

struct clusterOptions {
    NSMutableArray * remainingOptions;
    bool useInconsistent;
    bool useCutoff;
    NSString * criterion;
    vMAT_Array * cutoff;
    vMAT_Array * depth;
    vMAT_Array * maxclust;
};

#define WITH_clusterOptions(options, opts) struct clusterOptions opts; clusterOptions(options, &opts)

static void
clusterOptions(NSArray * options, struct clusterOptions * resultsOut)
{
    // "criterion:"  arg: { choice => { "distance" => set(:useInconsistent, false), "inconsistent" => set(:useInconsistent, true) }}, default: "inconsistent"
    // "cutoff:"     flag: set(:useCutoff, true), arg: vector(:double)
    // "depth:"      flag: set(:useInconsistent, true), arg: scalar(:index), default: 2
    // "maxclust:"   flag: set(:useCutoff, false), arg: vector(:index)
    // Initialize resultsOut struct
    resultsOut->remainingOptions = nil;
    resultsOut->criterion = @"inconsistent";
    resultsOut->cutoff = nil;
    resultsOut->depth = vMAT_coerce(@2, @[ @"index" ]);
    resultsOut->maxclust = nil;
    // Flags: [:useInconsistent, :useCutoff]
    bool useInconsistent_wasSet = false;
    resultsOut->useInconsistent = true;
    bool useCutoff_wasSet = false;
    resultsOut->useCutoff = false;
    // Locals
    __block NSMutableArray * remainingOptions = nil;
    __block NSUInteger optidx = NSNotFound;
    id (^ optarg)() = ^ { NSCParameterAssert([remainingOptions count] > optidx + 1); return remainingOptions[optidx + 1]; };
    // Options array normalization
    if ([options count] > 0) {
        remainingOptions = [options mutableCopy];
        resultsOut->remainingOptions = remainingOptions;
        options = nil;
    }
    else return;
    // criterion: {:arg=>{:choice_ie1=>{"distance"=>{:set_ie1=>[:useInconsistent, false]}, "inconsistent"=>{:set_ie2=>[:useInconsistent, true]}}, :key=>:choice_ie1, :slot=>"criterion", :kind=>:choice_with_flags}, :default=>"inconsistent"}
    if ((optidx = [remainingOptions indexOfObject:@"criterion:"]) != NSNotFound) {
        {
            NSMutableArray * remainingOptions = [@[ optarg(), optarg() ] mutableCopy];
            NSUInteger optidx = NSNotFound;
            if ((optidx = [remainingOptions indexOfObject:@"distance"]) != NSNotFound) {
                if (useInconsistent_wasSet) NSCParameterAssert(false == resultsOut->useInconsistent);
                else { resultsOut->useInconsistent = false; useInconsistent_wasSet = true; }
                resultsOut->criterion = [optarg() description];
                [remainingOptions removeObjectsInRange:NSMakeRange(optidx, 2)];
            }
            if ((optidx = [remainingOptions indexOfObject:@"inconsistent"]) != NSNotFound) {
                if (useInconsistent_wasSet) NSCParameterAssert(true == resultsOut->useInconsistent);
                else { resultsOut->useInconsistent = true; useInconsistent_wasSet = true; }
                resultsOut->criterion = [optarg() description];
                [remainingOptions removeObjectsInRange:NSMakeRange(optidx, 2)];
            }
        }
        [remainingOptions removeObjectsInRange:NSMakeRange(optidx, 2)];
        // Default inconsistent
    }
    // cutoff: {:flag=>{:set_ie3=>[:useCutoff, true]}, :arg=>{:vector_ie1=>:double, :type=>:double, :key=>:vector_ie1, :slot=>"cutoff", :kind=>:scalar_or_vector}}
    if ((optidx = [remainingOptions indexOfObject:@"cutoff:"]) != NSNotFound) {
        if (useCutoff_wasSet) NSCParameterAssert(true == resultsOut->useCutoff);
        else { resultsOut->useCutoff = true; useCutoff_wasSet = true; }
        resultsOut->cutoff = vMAT_coerce(optarg(), @[ @"double" ]);
        [remainingOptions removeObjectsInRange:NSMakeRange(optidx, 2)];
    }
    // depth: {:flag=>{:set_ie4=>[:useInconsistent, true]}, :arg=>{:scalar_ie1=>:index, :type=>:index, :key=>:scalar_ie1, :slot=>"depth", :kind=>:scalar_or_vector}, :default=>2}
    if ((optidx = [remainingOptions indexOfObject:@"depth:"]) != NSNotFound) {
        if (useInconsistent_wasSet) NSCParameterAssert(true == resultsOut->useInconsistent);
        else { resultsOut->useInconsistent = true; useInconsistent_wasSet = true; }
        resultsOut->depth = vMAT_coerce(optarg(), @[ @"index" ]);
        [remainingOptions removeObjectsInRange:NSMakeRange(optidx, 2)];
        // Default 2
    }
    // maxclust: {:flag=>{:set_ie5=>[:useCutoff, false]}, :arg=>{:vector_ie2=>:index, :type=>:index, :key=>:vector_ie2, :slot=>"maxclust", :kind=>:scalar_or_vector}}
    if ((optidx = [remainingOptions indexOfObject:@"maxclust:"]) != NSNotFound) {
        if (useCutoff_wasSet) NSCParameterAssert(false == resultsOut->useCutoff);
        else { resultsOut->useCutoff = false; useCutoff_wasSet = true; }
        resultsOut->maxclust = vMAT_coerce(optarg(), @[ @"index" ]);
        [remainingOptions removeObjectsInRange:NSMakeRange(optidx, 2)];
    }
}

As you can see, in this case the DSL specification looks nothing at all like the generated Objective-C code.

Both of these examples essentially entail boilerplate code. And it’s the kind of boilerplate that tends to get programmers in trouble; it’s clearly crying out for abstraction, but what kind of abstraction?

In a germinal version of vMAT (from even before embracing Objective-C++) I used the following idiom to implement the very first of the dispatch methods from the first example:

⋮

- (void)_copy_miSINGLE_from_miDOUBLE:(vMAT_Array *)matrix;
{
#define TypeA float
#define TypeB double
    long lenA = vMAT_Size_prod(_size);
    long lenB = vMAT_Size_prod(matrix.size);
    NSParameterAssert(lenA == lenB);
    TypeA * A = _data.mutableBytes;
    const TypeB * B = matrix.data.bytes;
    for (int i = 0;
         i < lenA;
         i++) {
        A[i] = B[i];
    }
#undef TypeA
#undef TypeB
}

⋮

Clearly, this was disaster waiting to happen. Even with those ugly macros each copy & paste instantiation of this “template” needed to be carefully edited in four places. Unsurprisingly, it was exactly this exercise that pushed me over the barrier and into the realm of Objective-C++. But how much did templates really improve things?

Well, only by half, obviously. C++ allowed me to create the overloaded copyFrom function that is still used in the generated code. But there was still no way to template-ize those Objective-C selectors. And that’s where the first incarnation of vMATCodeMonkey entered the scene.

#  vMATCodeMonkey.rb
#  vMAT
#
#  Created by Kaelin Colclasure on 4/23/13.
#  Copyright (c) 2013 Kaelin Colclasure. All rights reserved.

class VMATCodeMonkey

  MI_NUMERIC_TYPES = %w[
    miINT8
    miUINT8
    miINT16
    miUINT16
    miINT32
    miUINT32
    miSINGLE
    miDOUBLE
    miINT64
    miUINT64
  ]

  MI_NUMERIC_TYPES.each {|to|
      MI_NUMERIC_TYPES.each {|fm|
          to_t = to[2, to.length - 2]
          fm_t = fm[2, fm.length - 2]
          print "- (void)_copy_#{to}_from_#{fm}:(vMAT_Array *)matrix;\n{\n"
          print "    copyFrom(self, matrix, #{to_t}, #{fm_t});\n"
          print "}\n\n"
      }
  }

end

Not a DSL at all; just a quick ruby script to complete what would otherwise be an annoying and error-prone task.³ But once you set foot down this path, it’s hard to stop.

In a (perhaps distant) future post, I’ll walk through a dissection of the current implementation of vMATCodeMonkey. Not that I think it’s a model implementation or anything, mind you. I’ve only been writing ruby code for a short time, and I certainly could not have completed this task without lots of assistance from the ruby community of StackOverflow. But I believe it’s a testament to the versatility of the language that even I, as a relative n00bie @ ruby was able to make this useful-if-highly-specialized tool. [Not that there weren’t a few speed bumps along the way. I’ll talk about those too.]

While C++ templates are a useful tool for handling repetitive code generation at compile-time, they can’t solve this particular vMAT challenge; there’s no notion of method overloading in Objective-C.↩
The extensions of the filenames of gists included in this article differ from the names used in vMAT’s code base. This was done to get more appropriate syntax highlighting.↩
I’d bet that’s how a lot of ruby DSLs got their start.↩

kaelin

Generating Objective-C Code With a Ruby DSL

Comments