Juri Pakaste - The Blog

ZeroMQ, iOS and Python

2012-05-01T11:35:49Z

I wrote some example code for you.

Background: last week a coworker asked me what's the flavor du jour in two-way communication between a network server and an iOS app, should he just go with BSD sockets or is there something better? I suggested he should take a look at ZeroMQ. Not that I actually knew very much at all about it, but I had heard the name and seen that some people were pretty enthustiastic about it.

Turns out he had deadlines and wasn't too keen on building the library with an outdated zsh script that didn't seem to work.

I started fiddling with it myself on the weekend to have some idea about what I'm talking about. True enough that the script provided on the web page wasn't up to date, and once I fixed it I got only an ARM binary which isn't very useful in development when you want to test on the iPhone Simulator. After some more poking I finally managed to massage a working fat library out of it. And objc-zmq provided a nice simple Objective-C wrapper around the C API.

Not satisfied with playing with just one unknown technology, I thought I'd do the server in Clojure. After shaving the JNI yak for a couple of hours, trying to build a working version if the Java binding, I decided it wasn't worth it. Python was an easy fallback, but to keep things at least moderately exciting I used gevent which was also something I hadn't tried before.

It worked out pretty nice. The Python server is extremely simple and the iOS app isn't complex either. This hardly counts as a strenuous test of ZeroMQ, but at least from the code perspective it was really pleasant to work with. No hassle with buffers, just complete messages from an extremely simple API that enables the most common communication patterns with a couple of keywords. They wisely keep out of the marshalling business, telling people to use Protocol Buffers or something else for defining message formats.

The example project is available on Bitbucket. It works on my computer at the moment; see the Bitbucket page for details about the bits it requires.

Enjoy.

Unit testing Cocoa code with MacRuby

2010-11-30T18:45:10Z

Announcing RCRunner, a GUI test runner for MacRuby and Cocoa.

Cocoa unit testing can be a pain. In addition to the usual difficulties of writing tests for user interface heavy code, the Apple sanctioned solution, SenTestingKit, can isn't the greatest testing framework around and the default way of using it rules out debugger.

GHUnit helps somewhat. It's a GUI test runner with additional testing methods. However, with it you are still writing your tests in Objective-C. On the plus side it's the same language you're probably writing your app in. On the minus side Objective-C can be verbose and sometimes, especially when writing test code, brevity would be welcome.

Enter MacRuby. You get the conciseness of Ruby with full access to your Objective-C classes. And Ruby probably has the greatest density of testing frameworks per active programmer among all the languages in popular use today.

There's a nice article about TDD, Objective-C and MacRuby on the MacRuby site. However, the approach taken in it still uses a Xcode build phase script to accomplish testing. That makes debugging hard and you have to hunt through the build logs for your errors.

RCRunner is a separate GUI program you run. You tell it names of Ruby modules and it uses any test cases it finds[1]. You can breakpoint your code and thanks to Ruby, reload the test code. You can inspect errors and log output test by test.

Enjoy.

[1] At the moment it supports only MiniTest but adding support for other frameworks isn't difficult.

Using Firefox as Flash playing Safari fallback

2010-11-10T16:11:28Z

If you want to go Flashless on Mac and Safari, it's possible to use Firefox as a fallback, too, not just Chrome. While Firefox does load Plugins from /Library/Internet Plug-Ins and ~/Library/Internet Plug-Ins, it looks in other places too. I just tested and it seems to work fine from ~/Library/Application Support/Firefox/Profiles/<profile name>/plugins and I suspect /Applications/Firefox.app/Contents/MacOS/plugins would work too.

So you can copy Flash Player.plugin, NP-PPC-Dir-Shockwave and flashplayer.xpt to one of the Firefox specific folders and launch Firefox from Safari's Developer menu. Chrome starts up faster, though.

xibgraph: Interface Builder overviews

2010-06-19T16:17:13Z

When putting together user interfaces with Interface Builder, you connect things together with bindings, actions and outlets and it's good. Understanding the result later on is a completely different matter. It can be time consuming and difficult to browse the objects inside one by one, trying to comprehend the whole. Even more so if you're trying to read someone else's work.

Out of that frustration came xibgraph. It takes a XIB file and outputs the connections contained inside:

At the moment it supports bindings and actions. Outlets are next.

xibgraph supports a couple of different output formats. JSON is supported out of the box and if you install pydot, you get DOT, the format understood by Graphviz and OmniGraffle too.

It hasn't been tested on a particularly wide variety of XIBs, so it's very plausible it will produce wonky results or just outright refuse to work with your files. If so, patches and bug reports are welcome.

xibgraph is MIT licensed and written in Python. It requires PyObjC (it seemed like the easiest way to get XPath support on OS X) and probably Python 2.6. Everything but the DOT support should work without additional requirements on OS X 10.6.

hg-status-sections

2010-06-04T17:14:13Z

I usually use Murky, dvc or some other shell for Mercurial. Not always though, for various reasons, and when running hg status I'm always frustrated when copy and pasting file names. bzr provides neat, non-cluttered lines that can be copied whole to get a file name without a hassle, but hg takes the traditional one-character prefix approach to status display and as a result makes you manually select a part of a line instead of just grabbing a whole line.

So I wrote a small extension to help. Meet hg-status-sections.

A better @synthesize

2010-05-22T11:03:49Z

The biggest problem with Objective-C's @synthesize directive for properties is how difficult it's to augment the synthesized code. You often need to add logic to a property setter, but while you're adding it, you're losing the probably correct implementation Apple's code creates for property flags like atomic and retain.

At the moment, when you synthesize a readwrite property called foo you get a setter method called setFoo. If you need to add logic around it, you can either store the value in a private property and add a public property with a different name or use a subclass. Both are a bit of a hassle. Usually I just end up writing my own method, including the logic for implementing the modifiers correctly.

In an ideal world the language would support something like around/before/after methods in CLOS, but those features are rare. There's a simple way @synthesize could make things easier without requiring massive changes to the runtime. It could give you for both the getter and setter two methods. There would be the public methods they create in the current implementation, but there'd also be methods with names like __synthesized_property and __synthesized_setProperty. The public methods would rely on the semi-private methods to actually implement all their logic. Then if you needed to add logic around the accessors, you could override the public methods and call the semi-private ones to get access to the synthesized accessor logic without jumping through hoops or risking getting the implementation wrong.

Block indentation in Emacs

2010-03-08T17:28:30Z

There are several small things Emacs could be doing to make it nicer to write code. One I was missing was making it possible to go with one press of the return key between braces in a C derived language from this:

if (test) { }

to this:

if (test) {
   <-- insertion point here
}

That is, pressing return before the closing parentheses, brace or bracket should move the closing character two lines down, indent it properly, and move the insertion point to the new empty line in the middle and indent it property. The way TextMate does it.

Here are a few of elisp functions to accomplish this:

(defun char-isws (c)
  "Is character c a whitespace character?"
  (or (char-equal c ?\ )
      (char-equal c ?\t)
      (char-equal c ?\n)))

(defun line-next-non-ws ()
  "Return the next non-whitespace character on the current line or nil."
  (let ((cc (char-after)))
    (if (and cc (char-isws cc))
        (save-excursion
          (if (re-search-forward "[^[:space:]]" (save-excursion     (end-of-line) (point)) t)
              (char-before)
            nil))
      cc)))

(defun newline-and-indent-extra-for-closing-paren ()
  "Insert a newline and indent. If the next non-whitespace character is a closing paren, insert two newlines and indent the two  new lines correctly, placing the point on the first of the two new lines."
  (interactive)
  (let ((nc (line-next-non-ws)))
    (if (or (null nc)
            (not (= (char-syntax nc) ?))))
        (newline-and-indent)
      (progn
        (just-one-space)
        (newline-and-indent)
        (newline-and-indent)
        (previous-line)
        (indent-for-tab-command)))))

Now you can bind return to newline-and-indent-extra-for-closing-paren in a suitable language keymap. I've been using this with scala-mode and it works well there.

Exporting geolocation data from iPhoto with AppleScript

2009-07-29T19:04:21Z

I recently transferred all my photos to iPhoto. I share them on Flickr, but I've been unhappy with iPhoto's built-in Flickr support — it has an arbitrary 500 photo limit on web album size, it's crashy, it does weird synchronizations that take ages when combined with lots of large photos and a slow internet connection, it has multiple times failed to send all the full-resolution images — so I've been exploring alternatives. There's at least FlickrExport and Flickr's own Uploadr.

Although the tools work, there's a downside compared to iPhoto's built-in Flickr support. iPhoto has wonderful geotagging support, as does Flickr, but iPhoto doesn't write the data to EXIF tags and that poses a problem for the tools. Uploadr reads just the files and so never sees the data, and apparently iPhoto doesn't provide the data to FlickrExport, either. The result is that Flickr won't know the locations of the photos.

There's a way to work around this problem. The solution is AppleScript. iPhoto exports photo objects that can tell you their location as set inside iPhoto. The downside to this approach is that it's AppleScript, but apparently the alternatives like Python or JSTalk aren't quite up to tasks like these without application support.

This script will write the locations of the selected photos relies on ExifTool. It will litter your photo directory with files ending with _original that should contain the unmodified images. You should make sure the modified files are ok before deleting the originals. The usual caveats apply: I'm no AppleScript expert and this has not been tested particularly rigorously. I'd be careful especially if you don't live in the NE hemisphere. And you might want to reduce the number of dialogs. Do what you want with it.

-- This applescript will geotag the selected photos with the
-- location information set in iPhoto.
--  
--   You must have exiftool installed; by default it's loaded from
-- /opt/local/bin, where MacPorts installs it from the package
-- p5-image-exiftool.
--
-- Author: Juri Pakaste (http://www.juripakaste.fi/)
--
-- Based on the Set Geo Data.scpt script by
-- Andrew Turner (http://highearthorbit.com)
-- 
property exifToolOriginal : "_original"
property exifToolPath : "/opt/local/bin/exiftool"

on extract_decimal(realnum)
   set res to realnum - (round realnum rounding down)
   res
end extract_decimal

on roundFloat(n, precision)
    set x to 10 ^ precision
    (((n * x) + 0.5) div 1) / x
end roundFloat

on d2s(degs)
    log "enter d2s"
    if the degs < 0 then
        set the degs to degs * -1
    end if

    set the degrees to round degs rounding down
    set the minssecs to extract_decimal(degs)

    log "minssecs: " & minssecs

    set the minssecs to minssecs * 60
    set the mins to round minssecs rounding down

    set the minssecs to extract_decimal(minssecs)
    log "minssecs 2: " & minssecs
    set the secs to minssecs * 60

    "" & degrees & "," & mins & "," & roundFloat(secs, 2)
end d2s

on exportCoords(image_file, lat, lng, alt)
    set the northSouth to "N"
    set the eastWest to "E"

    if the lat is less than 0 then
        set the northSouth to "S"
        set the lat to the lat * -1
    end if

    if the lng is less than 0 then
        set the eastWest to "W"
        set the lng to the lng * -1
    end if

    log "calling d2s on " & lat
    set the latstr to my d2s(lat)
    set the lngstr to my d2s(lng)

    set exifCommand to exifToolPath & " -GPSMapDatum=WGS-84     -gps:GPSLatitude='" & latstr & "' -gps:GPSLatitudeRef='" & northSouth     ¬
        & "' -gps:GPSLongitude='" & lngstr & "'     -gps:GPSLongitudeRef='" & eastWest ¬
        & "' -xmp:GPSLatitude='" & latstr & northSouth & "'     -xmp:GPSLongitude='" & lngstr & eastWest & "'     -xmp:GPSMapDatum='WGS-84'" & " -xmp:GPSVersionID='2.2.0.0'" & " " & quoted form of image_file
    display dialog of ("running: " & exifCommand)
    set output to do shell script exifCommand
    display dialog of output
    --do shell script "rm '" & image_file & "'" &     exifToolOriginal    
end exportCoords

tell application "iPhoto"
    activate
    try
        copy (my selected_images()) to these_images
        if these_images is false or (the count of     these_images) is 0 then ¬
            error "Please select one or more images."

        repeat with i from 1 to the count of these_images
            set this_photo to item i of these_images
            tell this_photo
                set the image_file to the image path
                set lat to the latitude
                set lng to the longitude
                set alt to the altitude
            end tell

            if lat < 90.1 and lng < 180.1 then
                my exportCoords(image_file, lat, lng,     alt)
            else
                display alert ("No location set for "     & name of this_photo)
            end if

            log "read image, lat: " & lat & ", lng: " &     lng

        end repeat
        display dialog "Geo Exif write complete."
    on error error_message number error_number
        if the error_number is not -128 then
            display dialog of ("failed on: " & image_file)
            display dialog error_message buttons     {"Cancel"} default button 1
        end if
    end try
end tell


on selected_images()
    tell application "iPhoto"
        try
            -- get selection
            set these_items to the selection
            -- check for single album selected
            if the class of item 1 of these_items is album     then error
            -- return the list of selected photos
            return these_items
        on error
            return false
        end try
    end tell
end selected_images

Update (2010-02-28): Thank you to @simonmark on Twitter, who pointed out the script had some issues. My version broke if file names had single quotes in them and Simon's version broke with double quotes (admittedly probably rarer in file names.) I was going to leave it as it was, but found the quoted form method of text objects in AppleScript Language Guide which, assuming it works correctly, should make the script always work properly (at least in terms of file name handling.) I also copied Simon's better error display code and replaced the extract_decimal implementation with something that isn't quite as silly as my previous version was.

Update (2010-06-02): I set up a Bitbucket repository for this and other scripts I've written for iPhoto.

Announcing Chipmunk Backup

2009-02-27T16:31:01Z

I put up on Launchpad a backup utility I wrote called Chipmunk Backup. It's not extremely configurable nor does it have a huge set of features. It's a simple tool for maintaining a number of GnuPG encrypted full backups of a directory in a remote, rsync-accessible location.

There's no ready to download archive, but checking out lp:chipmunk-backup with bzr should give you a working version.

It's written in PLT Scheme and is known to work with version 4.1.4.

Emacs tips: Navigate CamelCase words

2009-01-06T14:26:02Z

Emacs tip #0: Always search EmacsWiki when you think you might need something.

Emacs tip #1: To navigate studlyCapped words, M-x c-subword-mode, as found on the CamelCase page. I had to add the following lines to my .emacs to get it work with C-left/C-right, M-b/M-f worked right out of the box:

(define-key global-map [(control right)] 'forward-word)
(define-key global-map [(control left)] 'backward-word)

Before that, they were bound to the -nomark variants.

Inline admin forms with admin site links in Django

2008-12-23T14:52:49Z

I have a somewhat difficult relationship with Django's admin site. It's a very useful feature, but I haven't really done enough with it to know when I'm going to hit a wall, if that wall's in the code or in my understanding, and how hard it's going to be to climb over the wall.

This time I wanted to have inline admin forms, except that I didn't actually want to have the forms there, I just wanted to have links to the objects — and not their views on the actual site, but on the admin site. As far as I can tell, there's no built-in support for this.

According to the admin docs, there are two subclasses of InlineModelAdmin: TabularInline and StackedInline. Looking at django/contrib/admin/options.py confirms this. And as the docs say, the only difference is the template they use. The stacked version comes pretty close when we add all the fields to an InlineModelAdmin subclass's exclude array, but it doesn't have the link.

To solve this we first create a new subclass:

class LinkedInline(admin.options.InlineModelAdmin):
    template = "admin/edit_inline/linked.html"

When you want to create inline links to a model, you subclass this new LinkedInline class. So to use a slightly contrived example, if we have a Flight with Passengers:

class PassengerInline(LinkedInline):
    model = models.Passenger
    extra = 0
    exclude = [ "name", "sex" ] # etc

class FlightAdmin(admin.ModelAdmin):
    inlines = [ PassengerInline ]

And yes, we have to exclude all the fields explicitly: an empty fields tuple or list is ignored.

The new template is easiest to create by cutting down aggressively the stacked template. Like this:

{% load i18n %}
<div class="inline-group">
  <h2>{{ inline_admin_formset.opts.verbose_name_plural|title}}</h2>
{{ inline_admin_formset.formset.management_form }}
{{ inline_admin_formset.formset.non_form_errors }}

{% for inline_admin_form in inline_admin_formset %}
<div class="inline-related {% if forloop.last %}last-related{% endif %}">
  <h3><b>{{ inline_admin_formset.opts.verbose_name|title }}:</b>&nbsp;{% if inline_admin_form.original %}{{ inline_admin_form.original }}{% else %} #{{ forloop.counter }}{% endif %}
    {% if inline_admin_formset.formset.can_delete and inline_admin_form.original %}<span class="delete">{{ inline_admin_form.deletion_field.field }} {{ inline_admin_form.deletion_field.label_tag }}</span>{% endif %}
  </h3>
  {{ inline_admin_form.pk_field.field }}
  {{ inline_admin_form.fk_field.field }}
</div>
{% endfor %}
</div>

The primary/foreign key fields are necessary to keep Django happy.

The result looks about right, it just lacks the links. It seems that Django doesn't give the template all the information we need to make them work: there's root_path that gives us /admin/, app_label contains the application's name and inline_admin_form.original.id contains the id of the inline object. What is lacking is the path component that names the model. I don't think it's available by default (is there a clean way to ask Django what's available in a template's context?), so we need to add it. Amend LinkedInline to look like this:

class LinkedInline(admin.options.InlineModelAdmin):
    template = "admin/edit_inline/linked.html"
    admin_model_path = None

    def __init__(self, *args):
        super(LinkedInline, self).__init__(*args)
        if self.admin_model_path is None:
            self.admin_model_path = self.model.__name__.lower()

Now inline_admin_formset.opts.admin_model_path will be bound to the lowercase name of the inline object's model, which is what the admin site uses in its paths.

With this, we can now replace the inline-related div in the template with this:

<div class="inline-related {% if forloop.last %}last-related{%  endif %}">
  <h3><b>{{ inline_admin_formset.opts.verbose_name|title  }}:</b>&nbsp;<a href="{{ root_path }}{{ app_label }}/{{ inline_admin_formset.opts.admin_model_path }}/{{ inline_admin_form.original.id }}/">{% if inline_admin_form.original %}{{ inline_admin_form.original }}{% else %} #{{ forloop.counter }}{% endif %}</a>
    {% if inline_admin_formset.formset.can_delete and inline_admin_form.original %}<span class="delete">{{ inline_admin_form.deletion_field.field }} {{ inline_admin_form.deletion_field.label_tag }}</span>{% endif %}
  </h3>
  {{ inline_admin_form.pk_field.field }}
  {{ inline_admin_form.fk_field.field }}
</div>

That's it. Now Flights get links to Passengers without big forms cluttering up the page.

iPhone shuffle

2008-11-21T18:31:33Z

These days I use an iPhone as my mobile music device. I have a bit over 1800 songs on it. I usually use shuffle and had it stuck in a weird state a couple of weeks ago — it was constantly playing me just a few tracks. I usually listen for just half an hour to an hour at a time, so I don't know if it would have started looping or what, but those were basically always there for a week's worth of commutes. I finally restarted the phone and that seemed to help, but what do you know, a couple of weeks, several restarts and one operating system upgrade later, it's again playing me exactly the same tracks.

As much as I love the The Roots, honestly, at this points Adrenaline!'s "Once a-again, once a-gain..." start makes me mostly think "once again indeed."

Crashes with NSURLConnection

2008-10-04T14:30:53Z

Speaking of Cocoa (and iPhone) programming, for a change.

Having trouble with spurious EXC_BAD_ACCESS crashes when using NSURLConnection? NSZombie giving you not very clear messages about [Not A Type retain], pointing to an address that malloc_history says has been allocated somewhere with only framework code in the call stack? See Amro Mousa's blog entry about the subject.

In a nutshell, don't send the start message to a NSURLConnection object you've initialized with a +connectionWithRequest:delegate: or -initWithRequest:delegate:. It'll break stuff.

Apple, how about a warning about this in the docs? The docs for -start say "Causes the receiver to begin loading data, if it has not already.", not "Will break your program and make you waste uncounted hours debugging if receiver has already started." Or how about preventing this in the code? Is there some scenario where you'd want to call start after the object has already started?

Various Python related things

2008-09-18T15:14:52Z

Python Magazine published my article "Using Dependency Injection in Python" in their August issue. Doug Hellman saw my blog entry about DI when I was switching web hosting and managed to repost old stuff to Planet Python, contacted me to ask if I wanted to expand on it a bit, I said yes, and now I've been published. Which is nice. I basically argue in the article that dependency injection is a good idea and you don't need to buy into a framework to get many of the benefits and provide some examples on how to do it. Which seems kind of apropos, seeing as there's yet another new contestant in that area.
I was hacking on something I wrote in Python recently and was doing it in a pretty heavily dependency injected manner, writing tests as I was going along. It came to me that one really basic, really nice thing about Python that isn't discussed all that much is that callables are duck typed and there are multiple choices on how to implement them. When you write a function that takes as a parameter something it calls to acquire values it doesn't have to care what the thing it's been given actually is. It can be a function, a callable, a bound method, a constructor, a lambda expression or even a generator. The caller doesn't have to care. It's not a language feature that's easy to fit in a bullet point or to appreciate without using it, but it is useful.
I was excited to see class decorators are coming in Python 2.6. Then I realized I can't remember what I wanted to use them for when I was really frustrated by their absence, but still, it's good. I always found their absence rather puzzling.

Flow 08: Saturday and Sunday

2008-08-18T20:04:36Z

On Saturday, it was raining cats and dogs. We were appropriately equipped and sniggered at the people in trainers etc trying to dodge the puddles. However, didn't see too many acts — saw a bit of Sébastien Tellier, but decided the crowds were too much and went to find some food (which was excellent and at 25 € for a three course vegetarian menu pretty good value.)

Next up on our schedule was CSS, which was pretty good. For some reason, it has never quite clicked for me, but still, they were busy as hell and obviously having fun.

And finally, The Roots. What a great show. No bling, no diva manners, just excellent hip hop with a surprising amount of jazz thrown in, just like on their early albums. And an incredibly diverse band, with Captain Kirk shredding his guitar and Tuba Gooding Jr on sousaphone.

Saturday was the only day it really felt like there were too many people stuffed into too small a space. Maybe it was the rain, maybe it was the fact that it was sold out, even though I don't think the other two days were that far behind.

On Sunday we were definitely starting to feel old and tired. It's surprising how tiring three days of festival gets, even without excessive drinking. Maybe I'm just too old. While eating on Saturday, I had the idea that they really should offer a show and dinner version of the festival; they already have an excellent restaurant on board, now just stretch out the dinner experience a bit and place the restaurant in a suitable location, and hey, the middle aged among us could be nice and comfy while checking out the gigs. And I really think they should have put the restaurant on the roof of the newer (if it is newer, the black one) gasometer.

We caught a glimpse of Astro Can Caravan, who had the weirdo jazz thing pretty well covered. Next up was The Five Corners Quintet, whose retro jazz was excellent as always. After that, we first checked out Plutonium 74, and had enough after one and a half songs. We listened to the first two or three songs from Señor Coconut and while their version of Daft Punk's Around the World wasn't horrible it wasn't nowhere near as good as Christian Prommer's on Friday, and by the time they hit Eurythmics' Sweet Dreams, we decided we had had enough of that too.

The next act we saw was José James who was one of the high points of the festival. He did a surprisingly jazzy gig and the crowd appreciated. After his gig, we listened for a couple of songs by Cut Copy, who really revealed themselves to be very summery party pop. Loud summery party pop. Their album In Ghost Colors was decent but I didn't get into it all that much, but they were better live. However, we were just too tired at that point and after a while headed home.