Alieniloquent

Movie: Shoot Em Up
Hates a pussy with a gun
After, I need a smoke

I’m an Emacs user, and I run on a Mac. I just use GNU Carbon Emacs out of CVS.

As long as I can remember, my command key mapped to the meta key in Emacs. This was particularly bothersome as it got my fingers in the habit of typing command-w to yank some text. But when I’m editing on a remote server in a Terminal window, that closes the window. Sometimes I manage to do it twice before I hit the option key instead.

I used to have this in my .emacs, although I never really noticed it:

(when stesla-mac-p
  (setq mac-command-key-is-meta nil))

A week ago when I wanted to finally fix this and make my command key do something other than be meta and make my option key be my meta key, I began to baffle as to why it wasn’t that way already. See, Google told me that the code I had in my .emacs should have done what I want.

Well, it turns out that it used to be how to do it. Emacs is cooler now, and lets you specify the behavior of all three special keys. So now what have is this:

(when stesla-mac-p
  (setq mac-command-modifier nil)
  (setq mac-option-modifier 'meta))

This makes Emacs not recognize the command key as a modifier at all and use the option key as meta, which is how I like it.

Quickly on the heels of the initial release of my Base32 library, I have an update. I should have tried to compile it on Linux, as the GCC settings on my Gentoo box caught some silly things I had done.

It’s all better now, and the gem can install on both Mac OS X and Gentoo Linux. I assume other Linuxes are probably fine, as are BSDs and other *NIXes.

To download it go here.

As you may know, I’ve been working with base32 encoding. Well, I decided to share my work with the world in the form of a library.

This first release simply contains the code I needed for my original project, but I’ve packaged it up as a nice Ruby extension.

You can visit the project page here.
You can download the release here.

I’ve scribbled this down so many times, I thought you all might want to benefit a little from my troubles.

|0000 0|000 11|11 111|1 2222| 2222 3|333 33|33 444|4 4444
|0000 0|111 11|22 222|3 3333| 4444 4|555 55|66 666|7 7777

That is a chart to help understand how five octets turn into eight base32-encoded bytes. It should be fairly self explanatory, but when I say that, I’m always wrong.

The top row is the octets. I’ve each group of four represents four bits, and the numbers represent which octet it is. The bottom row represents the quintets, with the same numbering scheme. I spaced the digits the same, so you can see how they match up. The pipes are nice visual borders.

So I’m writing the license-key generation code for the store-front for a shareware program my friend Tyler and I are preparing to release (more about that later). We’ve decided to use cryptography to reduce the likelihood that our licensing schema will be compromised (for relatively little effort on our part). We also decided to base32 encode the actual keys to make them easier to read.

Well, the store-front is going to be a Rails app, of course. Ruby has a module to base64 encode, but it doesn’t have one to base32 encode. So, I wrote one, and I did it test first (of course).

The first four tests were easy. Really short strings, but they worked out most of the kinks. But, I wanted something that would boost my confidence further. So I wrote the following test which ended up being quite patriotic.

def test_constitution_preamble
  plaintext =<<-EOT
    We the people of the United States, in order to form a more perfect union,
    establish justice, insure domestic tranquility, provide for the common
    defense, promote the general welfare, and secure the blessings of liberty
    to ourselves and our posterity, do ordain and establish this Constitution
    for the United States of America.
  EOT
  encoded = %W(
    EAQCAIBAEBLWKIDUNBSSA4DFN5YGYZJAN5TCA5DIMUQFK3TJORSWIICTORQXIZLTFQQGS3RA
    N5ZGIZLSEB2G6IDGN5ZG2IDBEBWW64TFEBYGK4TGMVRXIIDVNZUW63RMBIQCAIBAEAQGK43U
    MFRGY2LTNAQGU5LTORUWGZJMEBUW443VOJSSAZDPNVSXG5DJMMQHI4TBNZYXK2LMNF2HSLBA
    OBZG65TJMRSSAZTPOIQHI2DFEBRW63LNN5XAUIBAEAQCAIDEMVTGK3TTMUWCA4DSN5WW65DF
    EB2GQZJAM5SW4ZLSMFWCA53FNRTGC4TFFQQGC3TEEBZWKY3VOJSSA5DIMUQGE3DFONZWS3TH
    OMQG6ZRANRUWEZLSOR4QUIBAEAQCAIDUN4QG65LSONSWY5TFOMQGC3TEEBXXK4RAOBXXG5DF
    OJUXI6JMEBSG6IDPOJSGC2LOEBQW4ZBAMVZXIYLCNRUXG2BAORUGS4ZAINXW443UNF2HK5DJ
    N5XAUIBAEAQCAIDGN5ZCA5DIMUQFK3TJORSWIICTORQXIZLTEBXWMICBNVSXE2LDMEXAU===).join
  assert_equal(encoded, Base32.encode(plaintext))
end

I recently found myself wanting a Cocoa class that represents a set of 8-bit bytes. Cocoa has NSCharacterSet, but that is for unichar, not uint8_t. So I wrote one. It was easy enough, I gave it an array of UINT8_MAX booleans and said that if a particular element in the array was YES then that byte was in the set, and not if the element was NO.

Initially the class only knew how to answer questions of membership: is a byte in the set or not? But then I found a number of places where I was enumerating all possible values and testing for membership, so I figured adding a method that would return a NSData with just the bytes included in the set would be useful.

So I wrote this:

I had unit tests that proved it worked, and they all passed, so I checked in. All was good in the world.

Five days later, I flip open my laptop and decide to use the program this code is part of. I always try to eat my own dog food, and I prefer the freshest dog food I can get. So, whenever I want to use this application, I delete it, update from our Subversion repository, and build it.

Much to my surprise, when I built it on my laptop, some of those tests did not pass. I was expecting the NSData returned from -dataValue to have certain bytes in it. The NSData I actually got back did have the correct number of bytes, but they were all zeroes.

I banged my head against it for about twenty minutes, until I had a flash of insight. My desktop machine at home is an iMac, and inside it is an Intel Core Duo processor. My laptop is a PowerBook, and inside it is a Motorola G4 processor. The Core Duo, like most other Intel processors, stores numbers in the little-endian format, whereas the G4 stores them in big-endian format.

Endianess is a computer topic that makes a lot of programmers’ heads hurt. Unfortunately, Cocoa programmers do have to think about this now. Since Apple switched from their old, big-endian, Motorola platform to their new, little-endian, Intel platform, applications that are meant to run on both have to be aware of byte-order issues.

Computers store data in bytes, which are eight bits long. However, eight bits is only enough to store a number up to 255. In order to store larger numbers, computers just concatenate bytes together. A 16-bit number is comprised of two bytes, and a 32-bit number is comprised of four. The endianess of a system determines what order those bytes are stored in.

When you read a decimal number like 4242, you read it from left to right. The most significant digit is the left-most digit. Similarly, when you read a binary number like 1000010010010, the most significant digit is the left-most digit. If we divide that number into bytes, 00010000 10010010, the left-most byte is called the most significant byte, or the high-order byte. The right-most byte is called the least significant byte, or the low-order byte.

A big-endian processor, like the G4, stores numbers exactly like you’d read them. So if you read a 16-bit integer in big-endian order, the first byte you read is the high-order byte. Now, if the number is less than 255, for example 42, you’ll get this: 00000000 00101010.

A little-endian processor, like the Core Duo, stores numbers just the opposite of how you’d expect. The first byte you read is the least significant byte, followed by the next most significant byte, and then so on. So when we read our binary number in we’ll get 10010010 00010000 instead of what we expected. Now, if we look at that small number again, you’d get this: 00101010 00000000.

So, to bring this back to my bug. The unsigned type is actually an unsigned 32-bit integer. Since my code was manipulating a set of 8-bit numbers, every single number would fit into the low-order byte of that unsigned, thus leaving the other three bytes all zero.

The line of code where I do this:

Is a clever little trick I’ve used to avoid having to actually declare a one-byte array when I want to append just one byte. It works great if i is actually an uint8_t. It also works great if i is an unsigned and stored in little-endian format, since the first byte happens to be the byte I’m interested in. However, on a big-endian processor, that will reference the most significant byte of the number instead, and since i never gets any bigger than UINT8_MAX (which is 11111111 in binary), that byte will always be zero.

So now the code looks like this:

The compiler knows to do the correct conversion between the 32-bit and 8-bit types when assigning from one to another, so the new code now works on both of my machines.

Update: The title is a joke that Erica made up when I told her about this bug. All blame for its terribleness should go to her, I just recognized how apropos it was for the post.

I’m reading Agile Web Development With Rails and I just have to share this most amusing quote:

AJAX (which once stood for Asynchronous JavaScript and XML but
now just means Making Browsers Suck Less)

Thank you Dave and Dave for calling things like they really are.

Two years ago when my employer adopted Extreme Programming, we began to write automated tests for our code. My memory is fuzzy at this point, but as I recall we wanted to be able to write tests and keep the source files anywhere in our source tree, and then have our test runner automagically find them, or something like that. So when we created it we called it MasterTestXMLReportApp because it was cool enough to deserve such a long name.

As time passed those tests got slower and slower, so we split them into two suites. One that was meant to be fast and one that was meant to be slow. The slow ones were run once a night. We called that suite MasterTestNightlyXMLReportApp because it ran every night.

Fast forward to yesterday. The “nightly” tests haven’t run nightly in a long time. They just run continuously, albeit very slowly. I can never remember the name of the projects, and neither can any of my teammates. So finally we got fed up. We renamed them to FastTests and SlowTests.

We held a little memorial service, and gave them a little plot on our 0.04 acres of whiteboard, complete with a headstone.

So we know how to create a DLL in C++ that exposes .NET code to the Win32 world. We also know how to consume that DLL from Delphi. We know how to instantiate an object, call methods on it, and destroy it. So now, let’s do something interesting. Let’s make a progress bar.

Since this is just an example, we’re going to do something really simple. We’ll make an object that runs for a number of cycles and calls our event handler each cycle after sleeping for a little bit. Here’s the code for Clock:

Now the first thing to notice there is the ref keyword. This is how you let the compiler know this is a managed class. Next, you’re all probably wondering what ProgressCallback is. That is the class that takes care of all the magic behind simulating method pointers from Delphi.

A brief aside to talk about just what method pointers are. In C and C++ you can declare a pointer type like this:

Then you can use that type like this:

You can do the exact same thing in Delphi like this:

But Delphi also offers another kind of function pointer called a method pointer. You declare it like this:

Those two words of object make all the difference. What this does is it lets you use a pointer to a method on a specific instance of an object. When you call that method the Self pointer is set to the correct value so that you can access the state on the object. It is really powerful. This is typically how progress bars are driven in VCL applications. You just do something like this:

And then that method can do something such as adjust a progress bar or log to a file. It’s really slick.

Well, we want to display a progress bar in our Delphi GUI that moves as our Clock ticks across. But we can only pass plain old procedure pointers (the kind without of object) to the DLL functions because the code in the DLL doesn’t know how to do the magic that makes method pointers so nice. So we’ll just have to make that magic happen ourselves by passing the object pointer in explicitly along with a procedure pointer that takes the object pointer in its parameter list. We can cast the pointer and then call a method on the object with the rest of the parameters.

So now that we have the basic strategy in mind. Let me show you the code that encapsulates this method pointer idea:

Note that I store the pointers as IntPtr references. This is the type that all of the methods on System::Runtime::InteropServices::Marshal return pointers as. So, it’s useful to make your fields that way. You can always call ToPointer() on it.

Now, the last bit that we need is to export stuff in the DLL. But you’ll notice that all of the classes I’ve made so far have been managed classes. We can’t send a pointer to a managed object out of the DLL, but we can send a pointer to an unmanaged object that has a reference to our managed object. So we make this wrapper:

So all that’s left is to export the DLL functions like before. Just to keep them separate I’ll make another delete method, even though it’s identical in every way except the name.

Then on the Delphi side:

Some things to note. First off, the Progress method on TForm1 is private, and yet I call it from ProgressCallback. This is because of how scoping works in Delphi. Any code in the same unit as a private or protected method can call that method. In Delphi 2005, the keywords strict private and strict protected were introduced to prevent this ability, but in this case we actually want the behavior because we don’t want to expose that event to anybody else.

Next, notice that the procedure pointer also has cdecl on it. This has to be this way because it’s a pointer that will be passed into the DLL. So, it needs to be declared with the same calliing convention that’ll be used on the other side.

That’s how you hook up a Delphi progress bar to a .NET object. It isn’t much more work to use the ProcedureOfObject pattern and call that from a delegate on the .NET side, which is useful when you are hooking events on sub-objects.

One big gotcha with this pattern that isn’t demonstrated in this code is hooking the callback in a constructor. The Self pointer does not point to what you think it does. So if you send that along to the other side and call back to it later, you’re referencing offsets off of something else entirely, and you will get access violations.

So now we know how to make a .NET DLL that we can call into from the Win32 world. We know how to consume that in Delphi, and we know how to simulate Delphi’s method pointers. With these building blocks, you can map just about anything in the .NET world into your Delphi applications.