Juri Pakaste: BlogPersonal blog of Juri PakasteZola2023-08-12T08:30:00+03:00https://juripakaste.fi/atom.xmlSplitting a Xcode project with SPM2023-08-12T08:30:00+03:002023-08-12T08:30:00+03:00Unknownhttps://juripakaste.fi/split-swift-project-with-spm/<p>I was <a href="https://mastodon.social/@juri/110871248068997878">talking on Mastodon</a> about splitting an Xcode project into smaller pieces. Here's an elaboration.</p>
<p>Background: With SwiftUI previews you want small focused Xcode schemes. The larger your scheme, the less likely the preview is to succeed. You also get faster compilation if you don't always build the whole app. You can get smaller schemes with framework build targets or with Swift Package Manager. Frameworks are a pain in the ass, SPM mostly isn't.</p>
<p>A note about the confusing terminology, in case you're not familiar with both Xcode and SPM: Xcode has projects, targets and schemes. SPM has products and targets. SPM products show up in the scheme selector of Xcode.</p>
<p>Here's how you set up my ideal Xcode app project:</p>
<p>Create a root directory for your project. This is where you have your <code>.git</code> folder or equivalent. In the root directory, create a new Xcode project, so you have <code>root/MyApp/MyApp.xcodeproj</code> and next to it <code>root/MyApp/MyApp/</code> for the app files.</p>
<p>Then create a SPM package directory next to <code>MyApp</code>, something like <code>root/MyAppCore/</code>. Run <code>swift package init --type library</code> in it. Now open <code>root</code> in Finder and drag <code>MyAppCore</code> into <code>MyApp</code> in the Xcode Project Navigator sidebar (you don't need a workspace if you drop it onto the project.) After this you should have two entries under the MyApp project in Xcode: MyAppCore and MyApp. You should also see MyAppCore in the Xcode scheme selector.</p>
<p>After this you only have to add MyAppCore to the app build target in the Frameworks, Libraries and Embedded Content section of the project editor's General tab, and you're good to go.</p>
<p>With a setup like this, you can keep very little code on the app side, and put almost everything in the SPM package. You'll probably need your SwiftUI App, app delegate or scene delegate, depending on what and how you're building, on the app side, but other than that, just shove everything into the package. Every file you add to the package instead of the app is one change to the project file you avoided.</p>
<p>Ideally you want a wide but shallow SPM project structure with as many targets as is necessary, but structured so that they don't build up into deep dependency trees. You don't need to create a SPM product for each target; it's enough to create a projects as you need them from previews or for when you want to have Xcode compile just one part of the package.</p>
<p>To make imports easier in the app, you can use the <code>MyAppCore</code> product as an umbrella module. That means the <code>MyAppCore</code> target depends on every other target in the package, and then in <code>MyAppCore/MyAppCore.swift</code> calls <a href="https://github.com/apple/swift/blob/main/docs/ReferenceGuides/UnderscoredAttributes.md#_exported"><code>@_exported import</code></a> on all of them. <code>@_exported</code> is an underscored attribute and thus may break at some point, but it doesn't seem like a huge risk. So in <code>Package.swift</code> you can have this:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span>.target(
</span><span> name: </span><span style="color:#a3be8c;">"MyAppCore"</span><span>,
</span><span> dependencies: [
</span><span> </span><span style="color:#a3be8c;">"DB"</span><span>,
</span><span> </span><span style="color:#a3be8c;">"Networking"</span><span>,
</span><span> </span><span style="color:#a3be8c;">"PreferencesUI"</span><span>,
</span><span> </span><span style="color:#65737e;">/* all the other targets */
</span><span> ]
</span><span>),
</span></code></pre>
<p>and then in <code>MyAppCore.swift</code>:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">@_exported import </span><span>DB
</span><span style="color:#b48ead;">@_exported import </span><span>Networking
</span><span style="color:#b48ead;">@_exported import </span><span>PreferencesUI
</span><span style="color:#65737e;">/* all the other targets */
</span></code></pre>
<p>Now when you do <code>import MyAppCore</code> on the app side, you get everything.</p>
<p>With a setup like you rarely need to touch the Xcode project file. It reduces the need for a tool like <a href="https://github.com/yonaskolb/XcodeGen">XcodeGen</a>, which is priceless on a more monolithic project. If you want to avoid the project editor completely, I recommend looking into it.</p>
A fast timestamp parser in Swift2023-07-10T09:40:00+03:002023-07-10T09:40:00+03:00Unknownhttps://juripakaste.fi/parse3339/<p>I wrote a timestamp parser in Swift. It's called <a href="https://github.com/juri/Parse3339">Parse3339</a>.</p>
<p>It's well known that <code>DateFormatter</code>, the main timestamp formatter and parser Apple ships in Foundation, is not particularly fast. It's flexible and it's correct, but it takes its time. The newer <code>ISO8601DateFormatter</code> has similar performance.</p>
<p>I haven't much worried about that in the recent years. A while ago I had a problem with slow parsing but that time a caching layer solved it, as the data had a ton of identical dates. Last week I saw date parsers rear their heads in an Instruments trace again, and this time the data wasn't amenable to caching. It was happening on a background queue and it was only a few hundred milliseconds. But once you start optimizing, it's hard to let go.</p>
<p>In the past a common solution was using <code>strptime(3)</code>. This time I wanted to see how far I could get with Swift. I've usually used parser combinators for parsing, but these time stamps are so regular a simpler approach felt sufficient.</p>
<p>Time formats have two relevant standards: ISO 8601 and RFC 3339. RFC 3339 is <a href="https://ijmacd.github.io/rfc3339-iso8601/">mostly</a> a subset of ISO 8601, but even it allows for all kinds of irrelevant silliness. Realistically the only format I care about is a full timestamp with a <code>T</code> between date and time and possibly a <code>Z</code> for time zone. When I don't support any other variations, the resulting parser is really small, straightforward and easy to modify if your needs are different.</p>
<p>It's also very fast. The parser itself easily breezes through hundreds of thousands timestamps per second on my Mac. However, in most cases, you need something other than a struct with a bunch of numbers. Things slowed down once I started creating <code>DateComponents</code> and converting those to <code>Date</code>s with a <code>Calendar</code>. The code was still a few times faster than the platform parsers, but it was clearly leaving performance on the table. It was also using a lot of memory even though the parser itself was completely running on stack, allocating nothing. Benchmarking revealed that my code was using a similar amount of memory as <code>DateFormatter</code>, whereas <code>ISO8601DateFormatter</code> had a lighter footprint.</p>
<p>So back to the old Unix functions. Swift's C bridging is first-class; <code>struct tm</code> and <code>timegm(3)</code> are right there. After changing the code to use those, the whole <code>String</code> to <code>Date</code> conversion runs completely without heap allocations and it's around 6–7 times faster than when doing the detour via <code>DateComponents</code>.</p>
<p>The final result, according to <a href="https://github.com/ordo-one/package-benchmark">Benchmark</a>, is maybe around 15x the speed of the Foundation parsers. I'm pretty happy with it. It's available as a <a href="https://swiftpackageindex.com/juri/Parse3339">Swift package</a>, there's <a href="https://swiftpackageindex.com/juri/Parse3339/main/documentation/parse3339">documentation</a>, and if you need something different or don't feel like using a package, all the parsing code is in just one file you can copy over to your project.</p>
Git worktrees helper2022-11-01T17:00:00+00:002022-11-01T17:00:00+00:00Unknownhttps://juripakaste.fi/worktrees/<p>I recently became an avid user of Git worktrees. However, the command line interface to them is about as great as Git command line interfaces always are. Just <code>git worktree add</code> is a confusing maze of options.</p>
<p>I wrote a shell script to help myself and then I wanted to check parameters and then I decided that was a bridge too far with shell today and went for Swift instead.</p>
<p>My <a href="https://github.com/juri/worktree">Worktrees</a> tool has just one command for now, because that's the pain point I ran into today. <code>worktrees add-new-branch username/ mybranch main</code> creates a new branch called <code>username/mybranch</code> and a worktree called <code>mybranch</code> for working on it, branching it off <code>main</code>. You could very well argue that it's no easier than <code>git worktree add</code>, but the idea is that I can define a repo-local alias for it. Almost all the branches I open are from the mainline branch of the repo I'm working in, and I want to prefix my branches with <code>juri/</code>. So I can add this alias, if the repo is using <code>main</code>:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[alias]
</span><span> wta = !worktree add-new-branch juri/ main
</span></code></pre>
<p>And now running <code>git wta mybranch</code> does opens me a new worktree the way I like it.</p>
Better diff hunk headers with Swift2022-10-18T19:00:00+00:002022-10-18T19:00:00+00:00Unknownhttps://juripakaste.fi/swift-git-attributes/<p>When you run <code>git diff</code> — or look at diffs in at least <a href="https://git-fork.com">Fork</a> — on a modified Swift project you see things like this:</p>
<pre data-lang="diff" style="background-color:#2b303b;color:#c0c5ce;" class="language-diff "><code class="language-diff" data-lang="diff"><span>@@ -251,7 +251,7 @@ </span><span style="color:#8fa1b3;">extension AppUITests {
</span><span> let container = app.scrollViews["scroll"]
</span><span> XCTAssertTrue(container.waitForExistence(timeout: 2))
</span><span style="color:#bf616a;">- XCTAssertTrue(container.buttons["Restore"].exists)
</span><span style="color:#a3be8c;">+ XCTAssertTrue(container.buttons["Restore Purchases"].exists)
</span><span> XCTAssertTrue(container.buttons["View Subscriptions"].exists)
</span><span> }
</span></code></pre>
<p>While staring at a bunch of these today I realized just how unhelpful that line with <code>@@</code>s and <code>extension</code> is. The name of the extension where these lines live is not the most relevant context; the name of the function is.</p>
<p>This article on <a href="https://tekin.co.uk/2020/10/better-git-diff-output-for-ruby-python-elixir-and-more">improving diff output for a bunch of other languages</a> was helpful. I'll adapt its guidance for Swift here.</p>
<h2 id="step-1-diff-driver">Step 1: Diff driver</h2>
<p>Define a <a href="https://www.git-scm.com/docs/gitattributes#_defining_an_external_diff_driver">diff driver</a> in your <code>$HOME/.gitconfig</code>. The <code>xfuncname</code> configuration specifies a regular expression that is used to match a line you want to see in the hunk header after the <code>@@</code> bit. Covering all possible options with a regexp probably isn't possible, but this should cover most of the cases:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[diff "swift"]
</span><span> xfuncname = ^[ \t]*(((private |public |internal |final |open )*class|(private |public |internal )*struct|(private |public |internal )*actor|(private |public |internal )*func|(private |public |internal )*extension|(private |public |internal )*enum)[ \t].*)$
</span></code></pre>
<h2 id="step-2-global-git-attributes">Step 2: Global git attributes</h2>
<p>If you don't have a global git attributes file configured, set one up:</p>
<pre data-lang="sh" style="background-color:#2b303b;color:#c0c5ce;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="color:#bf616a;">git</span><span> config</span><span style="color:#bf616a;"> --global</span><span> core.attributesfile </span><span style="color:#bf616a;">~</span><span>/.gitattributes
</span></code></pre>
<h2 id="step-3-configure-the-swift-driver-for-swift-files">Step 3: Configure the <code>swift</code> driver for Swift files</h2>
<p>Edit the <code>~/.gitattributes</code> file to make Git use your newly defined diff driver for Swift files. Add the following line:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>*.swift diff=swift
</span></code></pre>
<p>After these changes, the above diff will look like this:</p>
<pre data-lang="diff" style="background-color:#2b303b;color:#c0c5ce;" class="language-diff "><code class="language-diff" data-lang="diff"><span>@@ -251,7 +251,7 @@ </span><span style="color:#8fa1b3;">func testButtons() {
</span><span> let container = app.scrollViews["scroll"]
</span><span> XCTAssertTrue(container.waitForExistence(timeout: 2))
</span><span style="color:#bf616a;">- XCTAssertTrue(container.buttons["Restore"].exists)
</span><span style="color:#a3be8c;">+ XCTAssertTrue(container.buttons["Restore Purchases"].exists)
</span><span> XCTAssertTrue(container.buttons["View Subscriptions"].exists)
</span><span> }
</span></code></pre>
<p>That's a lot more helpful.</p>
Creating icons in Xcode playgrounds2022-05-15T15:00:00+00:002022-05-15T15:00:00+00:00Unknownhttps://juripakaste.fi/xcode-playground-drawing/<p>I'm no good at drawing. I have Affinity Designer and I like it well enough, but it requires more expertise than I have, really. Usually when I want to draw things, I prefer to retreat back to code.</p>
<p>Xcode playgrounds are pretty OK for writing your graphics code. Select your drawing technology of choice to create an image, create a view that displays it, make it the live view with <code>PlaygroundPage.current.setLiveView</code> and you're done. Well, almost. How do you get the image out of there?</p>
<p>Say you're creating icons for an iOS project. You want a bunch of variously sized versions of the same icon (I'm assuming here you aren't finessing the different versions too much, or otherwise you wouldn't be reading a tutorial on how to generate images in code), and you want to get them into an asset catalog in Xcode. Xcode's asset catalog editor can accept dragged files, so that seems like a something we could try enable.</p>
<p>SwiftUI makes it really easy.</p>
<p>Start with a function that draws the icon into a CGImage. This one just draws a purplish rectangle. It won't win any ADAs, but it'll serve for this tutorial:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">func </span><span>makeImage(size: CGSize) -> CGImage {
</span><span> </span><span style="color:#b48ead;">let</span><span> ctx = CGContext(
</span><span> data: </span><span style="color:#d08770;">nil</span><span>,
</span><span> width: Int(size.width),
</span><span> height: Int(size.height),
</span><span> bitsPerComponent: </span><span style="color:#d08770;">8</span><span>,
</span><span> bytesPerRow: </span><span style="color:#d08770;">0</span><span>,
</span><span> space: CGColorSpaceCreateDeviceRGB(),
</span><span> bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue
</span><span> )!
</span><span> </span><span style="color:#b48ead;">let</span><span> rect = CGRect(origin: .zero, size: size)
</span><span> ctx.setFillColor(red: </span><span style="color:#d08770;">0</span><span>.</span><span style="color:#d08770;">9</span><span>, green: </span><span style="color:#d08770;">0</span><span>.</span><span style="color:#d08770;">4</span><span>, blue: </span><span style="color:#d08770;">0</span><span>.</span><span style="color:#d08770;">6</span><span>, alpha: </span><span style="color:#d08770;">1</span><span>.</span><span style="color:#d08770;">0</span><span>)
</span><span> ctx.fill(rect)
</span><span> </span><span style="color:#b48ead;">let</span><span> image = ctx.makeImage()!
</span><span> </span><span style="color:#b48ead;">return</span><span> image
</span><span>}
</span></code></pre>
<p>Next define a bunch of values for the icon sizes that Xcode likes. As of Xcode 13 and iOS 15, something like this is a good representation of what you need:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">enum</span><span> IconSize: CGFloat {
</span><span> </span><span style="color:#b48ead;">case</span><span> phoneNotification = </span><span style="color:#d08770;">20</span><span>.</span><span style="color:#d08770;">0
</span><span> </span><span style="color:#b48ead;">case</span><span> phoneSettings = </span><span style="color:#d08770;">29</span><span>.</span><span style="color:#d08770;">0
</span><span> </span><span style="color:#b48ead;">case</span><span> phoneSpotlight = </span><span style="color:#d08770;">40</span><span>.</span><span style="color:#d08770;">0
</span><span> </span><span style="color:#b48ead;">case</span><span> phoneApp = </span><span style="color:#d08770;">60</span><span>.</span><span style="color:#d08770;">0
</span><span> </span><span style="color:#b48ead;">case</span><span> padApp = </span><span style="color:#d08770;">76</span><span>.</span><span style="color:#d08770;">0
</span><span> </span><span style="color:#b48ead;">case</span><span> padProApp = </span><span style="color:#d08770;">83</span><span>.</span><span style="color:#d08770;">5
</span><span>}
</span><span>
</span><span style="color:#b48ead;">extension</span><span> IconSize: CustomStringConvertible {
</span><span> </span><span style="color:#b48ead;">var</span><span> description: String {
</span><span> </span><span style="color:#b48ead;">switch self</span><span> {
</span><span> </span><span style="color:#b48ead;">case </span><span>.phoneNotification: </span><span style="color:#b48ead;">return </span><span style="color:#a3be8c;">"iPhone/iPad Notification (</span><span>\(</span><span style="color:#b48ead;">self</span><span>.rawValue)</span><span style="color:#a3be8c;">)"
</span><span> </span><span style="color:#b48ead;">case </span><span>.phoneSettings: </span><span style="color:#b48ead;">return </span><span style="color:#a3be8c;">"iPhone/iPad Settings (</span><span>\(</span><span style="color:#b48ead;">self</span><span>.rawValue)</span><span style="color:#a3be8c;">)"
</span><span> </span><span style="color:#b48ead;">case </span><span>.phoneSpotlight: </span><span style="color:#b48ead;">return </span><span style="color:#a3be8c;">"iPhone/iPad Spotlight (</span><span>\(</span><span style="color:#b48ead;">self</span><span>.rawValue)</span><span style="color:#a3be8c;">)"
</span><span> </span><span style="color:#b48ead;">case </span><span>.phoneApp: </span><span style="color:#b48ead;">return </span><span style="color:#a3be8c;">"iPhone App (</span><span>\(</span><span style="color:#b48ead;">self</span><span>.rawValue)</span><span style="color:#a3be8c;">)"
</span><span> </span><span style="color:#b48ead;">case </span><span>.padApp: </span><span style="color:#b48ead;">return </span><span style="color:#a3be8c;">"iPad App (</span><span>\(</span><span style="color:#b48ead;">self</span><span>.rawValue)</span><span style="color:#a3be8c;">)"
</span><span> </span><span style="color:#b48ead;">case </span><span>.padProApp: </span><span style="color:#b48ead;">return </span><span style="color:#a3be8c;">"iPad Pro App (</span><span>\(</span><span style="color:#b48ead;">self</span><span>.rawValue)</span><span style="color:#a3be8c;">)"
</span><span> }
</span><span> }
</span><span>}
</span></code></pre>
<p>Then define a struct that holds one extra bit of information: the scale we're working at.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">struct</span><span> IconVariant {
</span><span> </span><span style="color:#b48ead;">let</span><span> size: IconSize
</span><span> </span><span style="color:#b48ead;">let</span><span> scale: CGFloat
</span><span>
</span><span> </span><span style="color:#b48ead;">var</span><span> scaledSize: CGSize {
</span><span> </span><span style="color:#b48ead;">let</span><span> scaled = </span><span style="color:#b48ead;">self</span><span>.scale * </span><span style="color:#b48ead;">self</span><span>.size.rawValue
</span><span> </span><span style="color:#b48ead;">return</span><span> CGSize(width: scaled, height: scaled)
</span><span> }
</span><span>}
</span><span>
</span><span style="color:#b48ead;">extension</span><span> IconVariant: CustomStringConvertible {
</span><span> </span><span style="color:#b48ead;">var</span><span> description: String { </span><span style="color:#a3be8c;">"</span><span>\(</span><span style="color:#b48ead;">self</span><span>.size) </span><span style="color:#a3be8c;">@ </span><span>\(</span><span style="color:#b48ead;">self</span><span>.scale)</span><span style="color:#a3be8c;">x"</span><span> }
</span><span>}
</span><span>
</span><span style="color:#b48ead;">extension</span><span> IconVariant: Identifiable {
</span><span> </span><span style="color:#b48ead;">var</span><span> id: String { </span><span style="color:#b48ead;">self</span><span>.description }
</span><span>}
</span></code></pre>
<p>The descriptions are useful for you, the human; the <code>Identifiable</code> conformance will be helpful when you set up a SwiftUI view showing the variants.</p>
<p>Next define all the variants you want:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">let</span><span> icons: [IconVariant] = [
</span><span> IconVariant(size: .phoneNotification, scale: </span><span style="color:#d08770;">2</span><span>),
</span><span> IconVariant(size: .phoneNotification, scale: </span><span style="color:#d08770;">3</span><span>),
</span><span> IconVariant(size: .phoneSettings, scale: </span><span style="color:#d08770;">2</span><span>),
</span><span> IconVariant(size: .phoneSettings, scale: </span><span style="color:#d08770;">3</span><span>),
</span><span> IconVariant(size: .phoneSpotlight, scale: </span><span style="color:#d08770;">2</span><span>),
</span><span> IconVariant(size: .phoneSpotlight, scale: </span><span style="color:#d08770;">3</span><span>),
</span><span> IconVariant(size: .phoneApp, scale: </span><span style="color:#d08770;">2</span><span>),
</span><span> IconVariant(size: .phoneApp, scale: </span><span style="color:#d08770;">3</span><span>),
</span><span> IconVariant(size: .phoneNotification, scale: </span><span style="color:#d08770;">1</span><span>),
</span><span> IconVariant(size: .phoneSettings, scale: </span><span style="color:#d08770;">1</span><span>),
</span><span> IconVariant(size: .phoneSpotlight, scale: </span><span style="color:#d08770;">1</span><span>),
</span><span> IconVariant(size: .padApp, scale: </span><span style="color:#d08770;">1</span><span>),
</span><span> IconVariant(size: .padApp, scale: </span><span style="color:#d08770;">2</span><span>),
</span><span> IconVariant(size: .padProApp, scale: </span><span style="color:#d08770;">2</span><span>),
</span><span>]
</span></code></pre>
<p>Then let's start work on getting those variants on screen. We'll use a simple SwiftUI view with stacks for it; it won't be pretty, but it'll do what's needed.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">struct</span><span> IconView: View {
</span><span> </span><span style="color:#b48ead;">var</span><span> body: some View {
</span><span> VStack {
</span><span> ForEach(icons) { icon </span><span style="color:#b48ead;">in
</span><span> HStack {
</span><span> </span><span style="color:#b48ead;">let</span><span> cgImage = makeImage(size: icon.scaledSize)
</span><span> Text(String(describing: icon))
</span><span> Image(cgImage, scale: </span><span style="color:#d08770;">1</span><span>.</span><span style="color:#d08770;">0</span><span>, label: Text(String(describing: icon)))
</span><span> }
</span><span> }
</span><span> }
</span><span> }
</span><span>}
</span><span>
</span><span>PlaygroundPage.current.setLiveView(IconView())
</span></code></pre>
<p>As promised, functionality over form:</p>
<p><img src="/xcode-playground-drawing/screenshot.png" alt="Screenshot of labeled squares" /></p>
<p>Now we need just the glue to enable dragging. Add a CGImage extension that makes it easier to export the image as PNG data:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">extension</span><span> CGImage {
</span><span> </span><span style="color:#b48ead;">var</span><span> png: Data? {
</span><span> </span><span style="color:#b48ead;">guard let</span><span> mutableData = CFDataCreateMutable(</span><span style="color:#d08770;">nil</span><span>, </span><span style="color:#d08770;">0</span><span>),
</span><span> </span><span style="color:#b48ead;">let</span><span> destination = CGImageDestinationCreateWithData(mutableData, </span><span style="color:#a3be8c;">"public.png" </span><span>as CFString, </span><span style="color:#d08770;">1</span><span>, </span><span style="color:#d08770;">nil</span><span>)
</span><span> </span><span style="color:#b48ead;">else</span><span> { </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">nil</span><span> }
</span><span> CGImageDestinationAddImage(destination, </span><span style="color:#b48ead;">self</span><span>, </span><span style="color:#d08770;">nil</span><span>)
</span><span> </span><span style="color:#b48ead;">guard</span><span> CGImageDestinationFinalize(destination) </span><span style="color:#b48ead;">else</span><span> { </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">nil</span><span> }
</span><span> </span><span style="color:#b48ead;">return</span><span> mutableData as Data
</span><span> }
</span><span>}
</span></code></pre>
<p>To make the images in the view draggable, you'll need to use the <code>onDrag</code> view modifier. It requires a function that returns a <code>NSItemProvider</code>. The nicest way to create one is probably with a custom class that conforms to <code>NSItemProviderWriting</code>. Something like this:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">final class</span><span> IconProvider: NSObject, NSItemProviderWriting {
</span><span> </span><span style="color:#b48ead;">struct</span><span> UnrecognizedTypeIdentifierError: Error {
</span><span> </span><span style="color:#b48ead;">let</span><span> identifier: String
</span><span> }
</span><span>
</span><span> </span><span style="color:#b48ead;">let</span><span> image: CGImage
</span><span>
</span><span> </span><span style="color:#b48ead;">init</span><span>(image: CGImage) {
</span><span> </span><span style="color:#b48ead;">self</span><span>.image = image
</span><span> }
</span><span>
</span><span> </span><span style="color:#b48ead;">func </span><span>loadData(
</span><span> withTypeIdentifier typeIdentifier: String,
</span><span> forItemProviderCompletionHandler completionHandler: </span><span style="color:#b48ead;">@escaping</span><span> (Data?, Error?) -> Void
</span><span> ) -> Progress? {
</span><span> </span><span style="color:#b48ead;">guard</span><span> typeIdentifier == </span><span style="color:#a3be8c;">"public.png" </span><span style="color:#b48ead;">else</span><span> {
</span><span> completionHandler(</span><span style="color:#d08770;">nil</span><span>, UnrecognizedTypeIdentifierError(identifier: typeIdentifier))
</span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">nil
</span><span> }
</span><span> completionHandler(</span><span style="color:#b48ead;">self</span><span>.image.png, </span><span style="color:#d08770;">nil</span><span>)
</span><span> </span><span style="color:#65737e;">// Progress: all done in one step.
</span><span> </span><span style="color:#b48ead;">let</span><span> progress = Progress(parent: </span><span style="color:#d08770;">nil</span><span>)
</span><span> progress.totalUnitCount = </span><span style="color:#d08770;">1
</span><span> progress.completedUnitCount = </span><span style="color:#d08770;">1
</span><span> </span><span style="color:#b48ead;">return</span><span> progress
</span><span> }
</span><span>
</span><span> </span><span style="color:#b48ead;">static var</span><span> writableTypeIdentifiersForItemProvider: [String] {
</span><span> [</span><span style="color:#a3be8c;">"public.png"</span><span>]
</span><span> }
</span><span>}
</span></code></pre>
<p>And then the last thing needed is the <code>onDrag</code> handler. Add it to the <code>Image</code> line in the <code>IconView</code> you created earlier.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span>Image(cgImage, scale: </span><span style="color:#d08770;">1</span><span>.</span><span style="color:#d08770;">0</span><span>, label: Text(String(describing: icon)))
</span><span> .onDrag {
</span><span> NSItemProvider(object: IconProvider(image: cgImage))
</span><span> }
</span></code></pre>
<p>Refresh the playground preview and the images are waiting for you to drag them into the asset catalog.</p>
Date component ranges in Swift2022-01-16T17:12:00+00:002022-01-16T17:12:00+00:00Unknownhttps://juripakaste.fi/swift-date-ranges/<p>Ever needed to iterate over a list of days or months in Swift? Ever needed to have a random-access collection of those?</p>
<p>The first thing they teach you in the How to not Operate on Dates Horribly Wrong class (aka <a href="https://yourcalendricalfallacyis.com">Calendrical Fallacies</a>) is to forget about using seconds for calendar correct calculations. On the Apple platforms you should be using Foundation's Calendar type instead. You will still write bugs, but hopefully they'll be more interesting than calendar drift caused by leap seconds and days.</p>
<p>To get a list of months you can start with an integer range and do fancy operations with it. Define a few example values to get started:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">let</span><span> now = Date()
</span><span style="color:#b48ead;">let</span><span> valueRange = </span><span style="color:#d08770;">1 </span><span>... </span><span style="color:#d08770;">120
</span><span style="color:#b48ead;">let</span><span> calendar = Calendar(identifier: .gregorian)
</span></code></pre>
<p>Making a list of dates corresponding to that <code>valueRange</code> is easy enough:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">let</span><span> months = valueRange.map {
</span><span> calendar.date(byAdding: .month, value: $</span><span style="color:#d08770;">0</span><span>, to: now)!
</span><span>}
</span></code></pre>
<p>The force unwrap operator, <code>!</code>, is uncomfortable there, but it's caused by Calendar's somewhat dodgy API design: because <code>Calendar.Component</code> contains cases such as <code>calendar</code> and <code>timeZone</code>, the calculation function has to return an optional. If you're hard coding <code>.month</code>, I think using a force unwrap there is quite OK.</p>
<p>Anyway, after that, <code>months</code> is a nice array of dates and we're ready to go home, right?</p>
<p>Well, sure. If the size of the list really is 120, that's pretty much it. But it wouldn't be much a blog post then. If you start dealing with larger ranges, the list will start to get a little cumbersome. The <code>valueRange</code> we started with it is an example of where we'd like to end up at: it's just two integers wrapped in a <code>ClosedRange</code>. If you change it to <code>1 ... 1200</code>, it doesn't get any larger in memory. But moments in time are represented with <code>Date</code>s, which are really just floating point values representing the number of seconds since a point in history, and if you created a range like <code>now ... now.addingTimeInterval(100000)</code> you'll just get timestamps separated by seconds, it won't conform to <code>RandomAccessCollection</code> because <code>Date</code> doesn't conform to <code>Strideable</code> and you'd run into problems with calendar math if you tried to build something fancier based on that.</p>
<p>So we need something else.</p>
<p>Swift has some helpers for creating sequences, such as, well, the <code>sequence</code> function, but nothing that builds on ranges, as far as I can tell. But building our own type that wraps a range and adds calendar logic on top of it isn't a big task, once you figure out the necessary protocols. That's what we really want, in the end: logically a lazy map over an integer range, one that doesn't create a list but just maps one integer to a date, while getting the calendar math right. The <code>map</code> function is out because it builds an array.</p>
<p>We can get started by defining a type:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">struct</span><span> CalendarRange {
</span><span> </span><span style="color:#b48ead;">let</span><span> calendar: Calendar
</span><span> </span><span style="color:#b48ead;">let</span><span> component: Calendar.Component
</span><span> </span><span style="color:#b48ead;">let</span><span> epoch: Date
</span><span> </span><span style="color:#b48ead;">let</span><span> values: ClosedRange<Int>
</span><span>}
</span></code></pre>
<p>We use <code>epoch</code> as the zero point. It's extremely arguable if it is necessary as a property here; you could just hard code something like <code>Date(timeIntervalSinceReferenceDate: 0)</code> and that would most probably be just fine. Some people would leave <code>Calendar</code> out too, just relying on <code>.autoupdatingCurrent</code>, but that's a an easy way to write untestable code. Adding convenience initializers is permitted.</p>
<p>Now we just need to add the necessary conformances to build up to <code>RandomAccessCollection</code>. The conformance tower goes like this, starting from the top: <code>RandomAccessCollection</code> → <code>BidirectionalCollection</code> → <code>Collection</code> → <code>Sequence</code>. We can skip over implementing <code>Sequence</code> because <code>Collection</code> gives it us for free:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">extension</span><span> CalendarRange: Collection {
</span><span> </span><span style="color:#b48ead;">typealias</span><span> Index = ClosedRange<Int>.Index
</span><span>
</span><span> </span><span style="color:#b48ead;">var</span><span> startIndex: Index { </span><span style="color:#b48ead;">self</span><span>.values.startIndex }
</span><span> </span><span style="color:#b48ead;">var</span><span> endIndex: Index { </span><span style="color:#b48ead;">self</span><span>.values.endIndex }
</span><span>
</span><span> </span><span style="color:#b48ead;">func </span><span>index(after i: Index) -> Index { </span><span style="color:#b48ead;">self</span><span>.values.index(after: i) }
</span><span>
</span><span> </span><span style="color:#b48ead;">subscript</span><span>(index: Index) -> Date {
</span><span> </span><span style="color:#b48ead;">self</span><span>.calendar.date(
</span><span> byAdding: </span><span style="color:#b48ead;">self</span><span>.component,
</span><span> value: </span><span style="color:#b48ead;">self</span><span>.values[index],
</span><span> to: </span><span style="color:#b48ead;">self</span><span>.epoch
</span><span> )!
</span><span> }
</span><span>}
</span></code></pre>
<p>The idea is simple: delegate everything else to the range but execute the calculation in <code>subscript</code>. There's again the awkward bang, but there's going to be some variation of it regardless of how you package this up, as long as you're using <code>Calendar</code> which is the right thing to do. You can dress it up in a <code>preconditionFailure</code> if you prefer.</p>
<p>Next: add <code>BidirectionalCollection</code>.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">extension</span><span> CalendarRange: BidirectionalCollection {
</span><span> </span><span style="color:#b48ead;">func </span><span>index(before i: Index) -> Index { </span><span style="color:#b48ead;">self</span><span>.values.index(before: i) }
</span><span>}
</span></code></pre>
<p>That wasn't extremely complicated. Let's see about <code>RandomAccessCollection</code> next.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">extension</span><span> CalendarRange: RandomAccessCollection {}
</span></code></pre>
<p>Nice. <code>RandomAccessCollection</code> doesn't require any extra members, it just places restrictions on what kind of index you can use. The one we inherited from <code>ClosedRange</code> suits it fine.</p>
<p>Now we can test it. Make a debug function and call it with some values:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">func </span><span>show<T>(_ values: T) where T: RandomAccessCollection {
</span><span> print(</span><span style="color:#a3be8c;">"----- values count: </span><span>\(values.count)</span><span style="color:#a3be8c;">"</span><span>)
</span><span> print(values.startIndex, String(describing: values.first))
</span><span> print(values.endIndex, String(describing: values.last))
</span><span>}
</span><span>
</span><span>show(CalendarRange(calendar: gregorian, component: .month, epoch: Date(), values: </span><span style="color:#d08770;">-12_000 </span><span>... </span><span style="color:#d08770;">12_000</span><span>))
</span><span>show(CalendarRange(calendar: gregorian, component: .day, epoch: Date(), values: </span><span style="color:#d08770;">-12_000 </span><span>... </span><span style="color:#d08770;">12_000</span><span>))
</span></code></pre>
<p>And hey, it works! It's not exactly fast, though. Running that takes noticeably long. To figure out what's happening with code this simple you don't even need to break out Instruments. Just throw a couple of debug prints in there and you'll discover that <code>count</code> will call <code>index(after:)</code> for each index. It'll also probably put Xcode in a state where you have force quit it, but such is the price of science.</p>
<p>What we're observing here is the magic of Swift's default protocol requirement implementations: you get <code>count</code> for free, but it's the lower common denominator. Sometimes that's the best you can do, and sometimes you can help make it suck less. In this case, improving <code>count</code> is trivial. <code>ClosedRange</code> has a perfectly good implementation, and we just need to, once again, delegate to it:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">extension</span><span> CalendarRange {
</span><span> </span><span style="color:#b48ead;">var</span><span> count: Int { </span><span style="color:#b48ead;">self</span><span>.values.count }
</span><span>}
</span></code></pre>
<p>Now it's blazing fast. It's possible there are other methods it would be beneficial to specialize; I haven't looked that deep into it.</p>
<p>One more thing we could do, if we find ourselves using this kind of thing a lot, is return to that statement earlier about what we need: a range with a lazy mapping function. If we think about it a bit more, in the general case, it wouldn't even have to be a range. In some cases it might be useful to do this with a non-closed <code>Range</code> or an <code>Array</code> or some other collection. And as I said, what we're doing in this code is just delegating everything but the <code>subscript</code> implementation to the wrapped range.</p>
<p>A generalized version of this is not much work:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">struct</span><span> MappedRandomAccess<Wrapped, Output>: RandomAccessCollection
</span><span style="color:#b48ead;">where</span><span> Wrapped: RandomAccessCollection {
</span><span> </span><span style="color:#b48ead;">typealias</span><span> Index = Wrapped.Index
</span><span> </span><span style="color:#b48ead;">let</span><span> values: Wrapped
</span><span> </span><span style="color:#b48ead;">let</span><span> f: (Wrapped.Element) -> Output
</span><span>
</span><span> </span><span style="color:#b48ead;">var</span><span> startIndex: Index { </span><span style="color:#b48ead;">self</span><span>.values.startIndex }
</span><span> </span><span style="color:#b48ead;">var</span><span> endIndex: Index { </span><span style="color:#b48ead;">self</span><span>.values.endIndex }
</span><span> </span><span style="color:#b48ead;">func </span><span>index(after i: Index) -> Index { </span><span style="color:#b48ead;">self</span><span>.values.index(after: i) }
</span><span> </span><span style="color:#b48ead;">func </span><span>index(before i: Index) -> Index { </span><span style="color:#b48ead;">self</span><span>.values.index(before: i) }
</span><span> </span><span style="color:#b48ead;">var</span><span> count: Int { </span><span style="color:#b48ead;">self</span><span>.values.count }
</span><span> </span><span style="color:#b48ead;">subscript</span><span>(index: Index) -> Output { </span><span style="color:#b48ead;">self</span><span>.f(</span><span style="color:#b48ead;">self</span><span>.values[index]) }
</span><span>}
</span></code></pre>
<p>This version requires the conversion to be supplied from outside, but that's easy enough too:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">let</span><span> epoch = Date(timeIntervalSinceReferenceDate: </span><span style="color:#d08770;">0</span><span>)
</span><span>
</span><span>show(MappedRandomAccess(values: </span><span style="color:#d08770;">-12_000 </span><span>... </span><span style="color:#d08770;">12_000</span><span>) {
</span><span> calendar.date(
</span><span> byAdding: .month,
</span><span> value: $</span><span style="color:#d08770;">0</span><span>,
</span><span> to: epoch
</span><span> )!
</span><span>})
</span></code></pre>
<p>The type of that value is <code>MappedRandomAccess<ClosedRange<Int>, Date></code>, but you could make that a bit easier to deal with using a typealias.</p>
<p>It's very possible this last generalization step isn't worth it in your code base, even if you need the date stepping functionality; using a closure does have a cost and the type is decidedly pointier than the non-general version, even if you manage to hide some of the time with a typealias. But there is a neatness to it.</p>
<p>Whichever version you decide to go with, you now have a memory efficient method of storing a large number of dates separated by a constant calendar unit. It can come useful sometimes.</p>
Async Swift and ArgumentParser2022-01-09T10:00:00+00:002022-01-09T10:00:00+00:00Unknownhttps://juripakaste.fi/async-argumentparser/<p>Swift 5.5 brought us async functions. <a href="https://github.com/apple/swift-argument-parser/">ArgumentParser</a> is the most popular way
to write command line interfaces with Swift. Swift 5.5 supports an asynchronous main
function, but ArgumentParser does not, as of version 1.0.2.</p>
<p>To bridge this gap, you can call ArgumentParser manually from your asynchronous main
function, like this:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">import </span><span>ArgumentParser
</span><span>
</span><span style="color:#b48ead;">struct</span><span> MyCommand: ParsableCommand {
</span><span> </span><span style="color:#b48ead;">@Argument var</span><span> arg: String
</span><span>
</span><span> </span><span style="color:#65737e;">// Usually you'd write a `run` function, but it has to be synchronous.
</span><span> </span><span style="color:#b48ead;">func </span><span>runAsync() async throws {
</span><span> </span><span style="color:#65737e;">/* … */
</span><span> }
</span><span>}
</span><span>
</span><span style="color:#b48ead;">@main
</span><span style="color:#b48ead;">enum</span><span> Main {
</span><span> </span><span style="color:#b48ead;">static func </span><span>main() async throws {
</span><span> </span><span style="color:#b48ead;">let</span><span> args = Array(CommandLine.arguments.dropFirst())
</span><span> do {
</span><span> </span><span style="color:#b48ead;">let</span><span> command = </span><span style="color:#b48ead;">try</span><span> MyCommand.parse(args)
</span><span> </span><span style="color:#b48ead;">try</span><span> await command.runAsync()
</span><span> } catch {
</span><span> MyCommand.exit(withError: error)
</span><span> }
</span><span> }
</span><span>}
</span></code></pre>
Converting between NSBezierPath and CGPath2021-07-14T12:00:00+00:002021-07-14T12:00:00+00:00Unknownhttps://juripakaste.fi/nzbezierpath-cgpath/<p>The macOS SDK ships with at least two graphics path types: <a href="https://developer.apple.com/documentation/appkit/nsbezierpath">NSBezierPath</a> and <a href="https://developer.apple.com/documentation/coregraphics/cgpath">CGPath</a>. They are mostly used in different contexts but sometimes it would be useful to convert between them. On iOS <a href="https://developer.apple.com/documentation/uikit/uibezierpath">UIBezierPath</a> has tools for the conversion, but on macOS we have to do it manually.</p>
<p>Here are the two conversions, based on Stack Overflow answers (<a href="https://stackoverflow.com/a/39385101">NSBezierPath to CGPath</a>, <a href="https://stackoverflow.com/a/49011112">CGPath to NSBezierPath</a>), converted to Swift 5.5 and tested on macOS 11.4.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">extension</span><span> NSBezierPath {
</span><span> </span><span style="color:#65737e;">// https://stackoverflow.com/a/39385101
</span><span> </span><span style="color:#b48ead;">var</span><span> cgPath: CGPath {
</span><span> </span><span style="color:#b48ead;">let</span><span> path = CGMutablePath()
</span><span> </span><span style="color:#b48ead;">var</span><span> points = [CGPoint](repeating: .zero, count: </span><span style="color:#d08770;">3</span><span>)
</span><span> </span><span style="color:#b48ead;">for</span><span> i </span><span style="color:#b48ead;">in </span><span style="color:#d08770;">0 </span><span>..< </span><span style="color:#b48ead;">self</span><span>.elementCount {
</span><span> </span><span style="color:#b48ead;">let</span><span> type = </span><span style="color:#b48ead;">self</span><span>.element(at: i, associatedPoints: &points)
</span><span> </span><span style="color:#b48ead;">switch</span><span> type {
</span><span> </span><span style="color:#b48ead;">case </span><span>.moveTo: path.move(to: points[</span><span style="color:#d08770;">0</span><span>])
</span><span> </span><span style="color:#b48ead;">case </span><span>.lineTo: path.addLine(to: points[</span><span style="color:#d08770;">0</span><span>])
</span><span> </span><span style="color:#b48ead;">case </span><span>.curveTo: path.addCurve(to: points[</span><span style="color:#d08770;">2</span><span>], control1: points[</span><span style="color:#d08770;">0</span><span>], control2: points[</span><span style="color:#d08770;">1</span><span>])
</span><span> </span><span style="color:#b48ead;">case </span><span>.closePath: path.closeSubpath()
</span><span> </span><span style="color:#b48ead;">@unknown default</span><span>: fatalError(</span><span style="color:#a3be8c;">"Unknown element </span><span>\(type)</span><span style="color:#a3be8c;">"</span><span>)
</span><span> }
</span><span> }
</span><span> </span><span style="color:#b48ead;">return</span><span> path
</span><span> }
</span><span>
</span><span> </span><span style="color:#65737e;">// https://stackoverflow.com/a/49011112
</span><span> </span><span style="color:#b48ead;">convenience init</span><span>(cgPath: CGPath) {
</span><span> </span><span style="color:#b48ead;">self</span><span>.</span><span style="color:#b48ead;">init</span><span>()
</span><span> cgPath.applyWithBlock { (elementPointer: UnsafePointer<CGPathElement>) </span><span style="color:#b48ead;">in
</span><span> </span><span style="color:#b48ead;">let</span><span> element = elementPointer.pointee
</span><span> </span><span style="color:#b48ead;">let</span><span> points = element.points
</span><span> </span><span style="color:#b48ead;">switch</span><span> element.type {
</span><span> </span><span style="color:#b48ead;">case </span><span>.moveToPoint:
</span><span> </span><span style="color:#b48ead;">self</span><span>.move(to: points.pointee)
</span><span> </span><span style="color:#b48ead;">case </span><span>.addLineToPoint:
</span><span> </span><span style="color:#b48ead;">self</span><span>.line(to: points.pointee)
</span><span> </span><span style="color:#b48ead;">case </span><span>.addQuadCurveToPoint:
</span><span> </span><span style="color:#b48ead;">let</span><span> qp0 = </span><span style="color:#b48ead;">self</span><span>.currentPoint
</span><span> </span><span style="color:#b48ead;">let</span><span> qp1 = points.pointee
</span><span> </span><span style="color:#b48ead;">let</span><span> qp2 = points.successor().pointee
</span><span> </span><span style="color:#b48ead;">let</span><span> m = </span><span style="color:#d08770;">2</span><span>.</span><span style="color:#d08770;">0</span><span>/</span><span style="color:#d08770;">3</span><span>.</span><span style="color:#d08770;">0
</span><span> </span><span style="color:#b48ead;">let</span><span> cp1 = NSPoint(
</span><span> x: qp0.x + ((qp1.x - qp0.x) * m),
</span><span> y: qp0.y + ((qp1.y - qp0.y) * m)
</span><span> )
</span><span> </span><span style="color:#b48ead;">let</span><span> cp2 = NSPoint(
</span><span> x: qp2.x + ((qp1.x - qp2.x) * m),
</span><span> y: qp2.y + ((qp1.y - qp2.y) * m)
</span><span> )
</span><span> </span><span style="color:#b48ead;">self</span><span>.curve(to: qp2, controlPoint1: cp1, controlPoint2: cp2)
</span><span> </span><span style="color:#b48ead;">case </span><span>.addCurveToPoint:
</span><span> </span><span style="color:#b48ead;">let</span><span> cp1 = points.pointee
</span><span> </span><span style="color:#b48ead;">let</span><span> cp2 = points.advanced(by: </span><span style="color:#d08770;">1</span><span>).pointee
</span><span> </span><span style="color:#b48ead;">let</span><span> target = points.advanced(by: </span><span style="color:#d08770;">2</span><span>).pointee
</span><span> </span><span style="color:#b48ead;">self</span><span>.curve(to: target, controlPoint1: cp1, controlPoint2: cp2)
</span><span> </span><span style="color:#b48ead;">case </span><span>.closeSubpath:
</span><span> </span><span style="color:#b48ead;">self</span><span>.close()
</span><span> </span><span style="color:#b48ead;">@unknown default</span><span>:
</span><span> fatalError(</span><span style="color:#a3be8c;">"Unknown type </span><span>\(element.type)</span><span style="color:#a3be8c;">"</span><span>)
</span><span> }
</span><span> }
</span><span> }
</span><span>}
</span></code></pre>
Swift networking with AsyncHTTPClient2021-03-30T18:12:00+00:002021-03-30T18:12:00+00:00Unknownhttps://juripakaste.fi/swift-asynchttpclient/<p>When you need to access resources over HTTP in Swift, in most cases the answer is <code>URLSession</code> from Foundation. On server side that's most probably not the right choice; there you are most likely running on <a href="https://github.com/apple/swift-nio">SwiftNIO</a> and you'll want something that integrates with it. On command line it's a toss up; on macOS <code>URLSession</code> is great, on other platforms… well, hope you don't run into any unimplemented corners.</p>
<p>I needed a command line tool that fetches a JSON file, parses it, downloads the files listed in the JSON file, and saves the JSON file too. I'm mostly on macOS so <code>URLSession</code> would have been fine, but I wanted to explore my options. SwiftNIO ships with <a href="https://apple.github.io/swift-nio/docs/current/NIOHTTP1/index.html">a low-level HTTP client implementation</a>, but that's not the right choice for a quick utility. The good news is there's also a higher-level implementation: <a href="https://github.com/swift-server/async-http-client">AsyncHTTPClient</a>. It's a lovely future based (while we wait for async/await in Swift) asynchronous HTTP client that makes this task a breeze.</p>
<p>The format of the JSON manifest file looks like this:</p>
<pre data-lang="json" style="background-color:#2b303b;color:#c0c5ce;" class="language-json "><code class="language-json" data-lang="json"><span>{
</span><span> "</span><span style="color:#a3be8c;">files</span><span>": [
</span><span> {
</span><span> "</span><span style="color:#a3be8c;">file</span><span>": "</span><span style="color:#a3be8c;">filename1</span><span>",
</span><span> "</span><span style="color:#a3be8c;">bytes</span><span>": </span><span style="color:#d08770;">42</span><span>,
</span><span> "</span><span style="color:#a3be8c;">sha256</span><span>": "</span><span style="color:#a3be8c;">8e079926d7340822e6b4c501811af5d1edc47d796b97f56c1cbe3177b47d588b</span><span>"
</span><span> },
</span><span>
</span><span> {
</span><span> "</span><span style="color:#a3be8c;">file</span><span>": "</span><span style="color:#a3be8c;">filename2</span><span>",
</span><span> "</span><span style="color:#a3be8c;">bytes</span><span>": </span><span style="color:#d08770;">10</span><span>,
</span><span> "</span><span style="color:#a3be8c;">sha256</span><span>": "</span><span style="color:#a3be8c;">4998ab8c155e03ebf843d4adf51d702b9560dc0fbafe35405c826d9a76460289</span><span>"
</span><span> }
</span><span> ]
</span><span>}
</span></code></pre>
<p>And so on. That's just a couple of structs:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">struct</span><span> ManifestFile: Codable {
</span><span> </span><span style="color:#b48ead;">let</span><span> file: String
</span><span>}
</span><span>
</span><span style="color:#b48ead;">struct</span><span> Manifest: Codable {
</span><span> </span><span style="color:#b48ead;">let</span><span> files: [ManifestFile]
</span><span>}
</span></code></pre>
<p>I'll just ignore <code>bytes</code> and <code>sha256</code>. They are relevant in other contexts, but here they don't matter.</p>
<h2 id="command-line">Command line</h2>
<p>Let's start by defining a command line interface. We'll use <a href="https://github.com/apple/swift-argument-parser">Swift Argument Parser</a>:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">import </span><span>ArgumentParser
</span><span style="color:#b48ead;">import </span><span>Foundation
</span><span>
</span><span style="color:#b48ead;">struct</span><span> DownloadManifestFiles: ParsableCommand {
</span><span> </span><span style="color:#b48ead;">static var</span><span> configuration = CommandConfiguration(
</span><span> commandName: </span><span style="color:#a3be8c;">"manifestdl"</span><span>,
</span><span> abstract: </span><span style="color:#a3be8c;">"Download manifest files"
</span><span> )
</span><span>
</span><span> </span><span style="color:#b48ead;">@Argument</span><span>(help: </span><span style="color:#a3be8c;">"URL to the manifest"</span><span>)
</span><span> </span><span style="color:#b48ead;">var</span><span> url: String
</span><span>
</span><span> </span><span style="color:#b48ead;">@Option</span><span>(help: </span><span style="color:#a3be8c;">"Target directory, default to working directory"</span><span>)
</span><span> </span><span style="color:#b48ead;">var</span><span> directory: String = </span><span style="color:#a3be8c;">"."
</span><span>
</span><span> </span><span style="color:#b48ead;">mutating func </span><span>validate() throws {
</span><span> </span><span style="color:#b48ead;">guard</span><span> URL(string: </span><span style="color:#b48ead;">self</span><span>.url) != </span><span style="color:#d08770;">nil </span><span style="color:#b48ead;">else</span><span> {
</span><span> </span><span style="color:#b48ead;">throw</span><span> ValidationError(</span><span style="color:#a3be8c;">"url </span><span>\(</span><span style="color:#b48ead;">self</span><span>.url) </span><span style="color:#a3be8c;">was not valid"</span><span>)
</span><span> }
</span><span> }
</span><span>
</span><span> </span><span style="color:#b48ead;">mutating func </span><span>run() throws {
</span><span> </span><span style="color:#b48ead;">guard let</span><span> reallyURL = URL(string: </span><span style="color:#b48ead;">self</span><span>.url) </span><span style="color:#b48ead;">else</span><span> {
</span><span> </span><span style="color:#b48ead;">throw</span><span> ValidationError(</span><span style="color:#a3be8c;">"URL </span><span>\(</span><span style="color:#b48ead;">self</span><span>.url) </span><span style="color:#a3be8c;">was not valid"</span><span>)
</span><span> }
</span><span> </span><span style="color:#b48ead;">let</span><span> cwd = URL(fileURLWithPath: FileManager.</span><span style="color:#b48ead;">default</span><span>.currentDirectoryPath, isDirectory: </span><span style="color:#b48ead;">true</span><span>)
</span><span> </span><span style="color:#b48ead;">let</span><span> directory = URL(fileURLWithPath: </span><span style="color:#b48ead;">self</span><span>.directory, isDirectory: </span><span style="color:#b48ead;">true</span><span>, relativeTo: cwd)
</span><span>
</span><span> fatalError(</span><span style="color:#a3be8c;">"TODO"</span><span>)
</span><span> }
</span><span>}
</span><span>
</span><span>DownloadManifestFiles.main()
</span></code></pre>
<p>Nice and easy. An interesting wrinkle is that ArgumentParser doesn't support URLs, probably because they're in Foundation and ArgumentParser uses only stdlib. And <code>NSURL</code>, which backs <code>URL</code>, is a class with far more responsibilities than is comfortable in a simple data wrapper.</p>
<h2 id="downloading">Downloading…</h2>
<p>Next we need the actual networking. Let's wrap it in a helper class:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">import </span><span>AsyncHTTPClient
</span><span style="color:#b48ead;">import </span><span>Foundation
</span><span style="color:#b48ead;">import </span><span>NIO
</span><span style="color:#b48ead;">import </span><span>NIOHTTP1
</span><span>
</span><span style="color:#b48ead;">class</span><span> Downloader {
</span><span> </span><span style="color:#b48ead;">let</span><span> httpClient = HTTPClient(eventLoopGroupProvider: .createNew)
</span><span>
</span><span> </span><span style="color:#b48ead;">func </span><span>syncShutdown() throws {
</span><span> </span><span style="color:#b48ead;">try self</span><span>.httpClient.syncShutdown()
</span><span> }
</span><span>
</span><span> </span><span style="color:#b48ead;">func </span><span>downloadListedFiles(url: URL, directory: URL) -> EventLoopFuture<Void> {
</span><span> fatalError(</span><span style="color:#a3be8c;">"TODO"</span><span>)
</span><span> }
</span><span>}
</span></code></pre>
<p>We don't really care about any details about the download, we just want to know if it succeeded or not, hence <code>EventLoopFuture<Void></code>. <code>EventLoopFuture</code> uses untyped errors, so we don't need to include an error in the type signature. It makes a bit odd next to <code>Result</code> and Combine's <code>Publisher</code>, but it does help when integrating with Swift's exceptions.</p>
<p>Next let's implement the <code>downloadListedFiles</code> method.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">class</span><span> Downloader {
</span><span> </span><span style="color:#65737e;">/* … */
</span><span> </span><span style="color:#b48ead;">func </span><span>downloadListedFiles(url: URL, directory: URL) -> EventLoopFuture<Void> {
</span><span> </span><span style="color:#b48ead;">let</span><span> future = </span><span style="color:#b48ead;">self</span><span>.downloadManifest(url: manifestURL)
</span><span> .flatMap { manifest, data </span><span style="color:#b48ead;">in
</span><span> </span><span style="color:#b48ead;">self</span><span>.downloadManifestContent(
</span><span> manifest: manifest,
</span><span> manifestURL: manifestURL,
</span><span> directory: directory
</span><span> ).map { data }
</span><span> }
</span><span> .flatMap { data </span><span style="color:#b48ead;">in
</span><span> </span><span style="color:#b48ead;">self</span><span>.saveManifest(directory: directory, data: data)
</span><span> }
</span><span> </span><span style="color:#b48ead;">return</span><span> future
</span><span> }
</span><span>
</span><span> </span><span style="color:#b48ead;">func </span><span>downloadManifest(url: URL) -> EventLoopFuture<(manifest: Manifest, data: Data)> {
</span><span> fatalError(</span><span style="color:#a3be8c;">"TODO"</span><span>)
</span><span> }
</span><span>
</span><span> </span><span style="color:#b48ead;">func </span><span>downloadManifestContent(
</span><span> manifest: Manifest,
</span><span> manifestURL: URL,
</span><span> directory: URL
</span><span> ) -> EventLoopFuture<Void> {
</span><span> fatalError(</span><span style="color:#a3be8c;">"TODO"</span><span>)
</span><span> }
</span><span>
</span><span> </span><span style="color:#b48ead;">func </span><span>saveManifest(directory: URL, data: Data) -> EventLoopFuture<Void> {
</span><span> fatalError(</span><span style="color:#a3be8c;">"TODO"</span><span>)
</span><span> }
</span><span>}
</span></code></pre>
<p>That looks like an acceptable outline for it. Give <code>downloadListedFiles</code> an URL to a listing file and a directory to download to and it'll download the file manifest, parse it, and then download the listed files, and finally save the manifest too. I'll fill in the blanks one by one.</p>
<p>Next let's look how <code>downloadManifest</code> should look.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">func </span><span>downloadManifest(url: URL) -> EventLoopFuture<(manifest: Manifest, data: Data)> {
</span><span> </span><span style="color:#b48ead;">self</span><span>.httpClient.</span><span style="color:#b48ead;">get</span><span>(url: url.absoluteString)
</span><span> .flatMapThrowing { response </span><span style="color:#b48ead;">in
</span><span> </span><span style="color:#b48ead;">guard var</span><span> body = response.body,
</span><span> </span><span style="color:#b48ead;">let</span><span> data = body.readData(length: body.readableBytes)
</span><span> </span><span style="color:#b48ead;">else</span><span> {
</span><span> </span><span style="color:#b48ead;">throw</span><span> MissingBodyError()
</span><span> }
</span><span> </span><span style="color:#b48ead;">return</span><span> (manifest: </span><span style="color:#b48ead;">try</span><span> JSONDecoder().decode(Manifest.</span><span style="color:#b48ead;">self</span><span>, from: data), data: data)
</span><span> }
</span><span>}
</span></code></pre>
<p>As you can see, just like <code>ArgumentParser</code>, <code>HTTPClient</code> eschews <code>URL</code> as a Foundation type. Other than that, <code>HTTPClient</code> gives us a really easy interface. Just <code>.get</code> a String containing an URL, and then do whatever you need to do with a chaining method like <code>.map</code>, <code>.flatMap</code>, or as in this case, <code>.flatMapThrowing</code>.</p>
<p>Next we can tackle <code>downloadManifestContent</code>. It's the function that's responsible for downloading all the listed files.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">func </span><span>downloadManifestContent(
</span><span> manifest: Manifest,
</span><span> manifestURL: URL,
</span><span> directory: URL
</span><span>) -> EventLoopFuture<Void> {
</span><span> </span><span style="color:#b48ead;">let</span><span> baseURL = manifestURL.deletingLastPathComponent()
</span><span> </span><span style="color:#b48ead;">let</span><span> requestFutures: [EventLoopFuture<Void>]
</span><span> do {
</span><span> requestFutures = </span><span style="color:#b48ead;">try</span><span> manifest.files.map { manifestFile </span><span style="color:#b48ead;">in
</span><span> </span><span style="color:#b48ead;">let</span><span> localURL = directory.appendingPathComponent(manifestFile.file)
</span><span> </span><span style="color:#b48ead;">try</span><span> FileManager.</span><span style="color:#b48ead;">default</span><span>.createDirectory(
</span><span> at: localURL.deletingLastPathComponent(),
</span><span> withIntermediateDirectories: </span><span style="color:#b48ead;">true</span><span>,
</span><span> attributes: </span><span style="color:#d08770;">nil
</span><span> )
</span><span> </span><span style="color:#b48ead;">let</span><span> localPath = localURL.path
</span><span> </span><span style="color:#b48ead;">let</span><span> delegate = </span><span style="color:#b48ead;">try</span><span> FileDownloadDelegate(path: localPath, pool: </span><span style="color:#b48ead;">self</span><span>.threadPool)
</span><span> </span><span style="color:#b48ead;">let</span><span> request = </span><span style="color:#b48ead;">try</span><span> HTTPClient.Request(
</span><span> url: baseURL.appendingPathComponent(manifestFile.file).absoluteString
</span><span> )
</span><span> </span><span style="color:#b48ead;">return self</span><span>.httpClient.execute(request: request, delegate: delegate)
</span><span> .futureResult
</span><span> .map { _ </span><span style="color:#b48ead;">in</span><span> () }
</span><span> }
</span><span> } catch {
</span><span> </span><span style="color:#b48ead;">return self</span><span>.httpClient.eventLoopGroup.next().makeFailedFuture(error)
</span><span> }
</span><span> </span><span style="color:#b48ead;">return</span><span> EventLoopFuture.andAllSucceed(requestFutures, on: </span><span style="color:#b48ead;">self</span><span>.eventLoopGroup.next())
</span><span>}
</span><span>
</span><span style="color:#b48ead;">var</span><span> eventLoopGroup: EventLoopGroup {
</span><span> </span><span style="color:#b48ead;">self</span><span>.httpClient.eventLoopGroup
</span><span>}
</span></code></pre>
<p>This one's not quite as simple, but it's not too bad. For each listed file we create a future for downloading it. The download to disk, as opposed to memory, happens with the help of <a href="https://swift-server.github.io/async-http-client/docs/current/AsyncHTTPClient/Classes/FileDownloadDelegate.html">FileDownloadDelegate</a>, a delegate included in AsyncHTTPClient that can write downloads to disk and report progress. Then once we have a list of futures, we smash them all together with <code>andAllSucceeded</code>. Again we don't care about anything else other than success, so <code>Void</code> is a perfectly fine value type.</p>
<p>One detail I need to point out here is the <code>eventLoopGroup</code> property. SwiftNIO works with EventLoops, and EventLoops are apparently usually threads. While we're working only with with networking code it's probably not a problem to ask <code>HTTPClient</code> for its <code>EventLoopGroup</code> instance.</p>
<h2 id="file-i-o">File I/O</h2>
<p>We have read the manifest and written the files listed in it to disk. One thing left to do: saving the manifest. Writing the file with SwiftNIO isn't quite as friendly as AsyncHTTPClient is, and if you were doing more of this you'd want to put a nicer façade on it, but here we just need it this once.</p>
<p>To prepare for this, lets first set up a bit of scaffolding. It feels cleaner to move the management of the <code>EventLoopGroup</code> to our own code now that we're using it for not just the HTTP client, and we'll also need a thread pool for the file I/O.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">class</span><span> Downloader {
</span><span> </span><span style="color:#b48ead;">let</span><span> eventLoopGroup: EventLoopGroup </span><span style="color:#65737e;">// replaces the computed property
</span><span> </span><span style="color:#b48ead;">let</span><span> httpClient: HTTPClient
</span><span> </span><span style="color:#b48ead;">let</span><span> threadPool: NIOThreadPool
</span><span>
</span><span> </span><span style="color:#b48ead;">init</span><span>() {
</span><span> </span><span style="color:#b48ead;">self</span><span>.eventLoopGroup = NIOTSEventLoopGroup()
</span><span> </span><span style="color:#b48ead;">self</span><span>.httpClient = HTTPClient(eventLoopGroupProvider: .shared(</span><span style="color:#b48ead;">self</span><span>.eventLoopGroup))
</span><span> </span><span style="color:#b48ead;">self</span><span>.threadPool = NIOThreadPool(numberOfThreads: </span><span style="color:#d08770;">1</span><span>)
</span><span>
</span><span> </span><span style="color:#b48ead;">self</span><span>.threadPool.start()
</span><span> }
</span><span>
</span><span> </span><span style="color:#b48ead;">func </span><span>syncShutdown() throws {
</span><span> </span><span style="color:#b48ead;">try self</span><span>.httpClient.syncShutdown()
</span><span> </span><span style="color:#b48ead;">try self</span><span>.threadPool.syncShutdownGracefully()
</span><span> </span><span style="color:#b48ead;">try self</span><span>.eventLoopGroup.syncShutdownGracefully()
</span><span> }
</span><span>
</span><span> </span><span style="color:#65737e;">/* … */
</span><span>}
</span></code></pre>
<p>Forcing <code>NIOTSEventLoopGroup</code> here probably ties this code to macOS. For portability, there are other implementations. Here's what AsyncHTTPClient does when you ask it to create the event loop group itself:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span>#</span><span style="color:#b48ead;">if</span><span> canImport(Network)
</span><span> </span><span style="color:#b48ead;">if</span><span> #available(OSX </span><span style="color:#d08770;">10</span><span>.</span><span style="color:#d08770;">14</span><span>, iOS </span><span style="color:#d08770;">12</span><span>.</span><span style="color:#d08770;">0</span><span>, tvOS </span><span style="color:#d08770;">12</span><span>.</span><span style="color:#d08770;">0</span><span>, watchOS </span><span style="color:#d08770;">6</span><span>.</span><span style="color:#d08770;">0</span><span>, *) {
</span><span> </span><span style="color:#b48ead;">self</span><span>.eventLoopGroup = NIOTSEventLoopGroup()
</span><span> } </span><span style="color:#b48ead;">else</span><span> {
</span><span> </span><span style="color:#b48ead;">self</span><span>.eventLoopGroup = MultiThreadedEventLoopGroup(numberOfThreads: </span><span style="color:#d08770;">1</span><span>)
</span><span> }
</span><span>#</span><span style="color:#b48ead;">else
</span><span> </span><span style="color:#b48ead;">self</span><span>.eventLoopGroup = MultiThreadedEventLoopGroup(numberOfThreads: </span><span style="color:#d08770;">1</span><span>)
</span><span>#endif
</span></code></pre>
<p>Doing something similar in your own code should help make this more cross platform.</p>
<p>With that setup done, we can dive into the file writing itself.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">func </span><span>saveManifest(directory: URL, data: Data) -> EventLoopFuture<Void> {
</span><span> </span><span style="color:#b48ead;">let</span><span> io = NonBlockingFileIO(threadPool: </span><span style="color:#b48ead;">self</span><span>.threadPool)
</span><span> </span><span style="color:#b48ead;">let</span><span> eventLoop = </span><span style="color:#b48ead;">self</span><span>.eventLoopGroup.next()
</span><span> </span><span style="color:#b48ead;">let</span><span> buffer = ByteBuffer(data: data)
</span><span> </span><span style="color:#b48ead;">return</span><span> io
</span><span> .openFile(
</span><span> path: directory.appendingPathComponent(</span><span style="color:#a3be8c;">"manifest.json"</span><span>).path,
</span><span> mode: .write,
</span><span> flags: .allowFileCreation(),
</span><span> eventLoop: eventLoop
</span><span> ).flatMap { handle </span><span style="color:#b48ead;">in
</span><span> io.write(fileHandle: handle, buffer: buffer, eventLoop: eventLoop)
</span><span> .map { handle }
</span><span> }.flatMapThrowing { handle </span><span style="color:#b48ead;">in
</span><span> </span><span style="color:#b48ead;">try</span><span> handle.close()
</span><span> }
</span><span>}
</span></code></pre>
<p>It's two async operations and one synchronous one in a pipeline. Open file asynchronously, write file asynchronously, close it synchronously. The flatMaps can feel a little daunting if you're not used to them, as always with future libraries. But once you get used to them, it's pretty OK. Async/await will hopefully help.</p>
<p>After all that work we're ready to loop back to our <code>run</code> method. We left it calling fatalError after processing the arguments. Now we can finish it up:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">mutating func </span><span>run() throws {
</span><span> </span><span style="color:#b48ead;">guard let</span><span> reallyURL = URL(string: </span><span style="color:#b48ead;">self</span><span>.url) </span><span style="color:#b48ead;">else</span><span> {
</span><span> </span><span style="color:#b48ead;">throw</span><span> ValidationError(</span><span style="color:#a3be8c;">"URL </span><span>\(</span><span style="color:#b48ead;">self</span><span>.url) </span><span style="color:#a3be8c;">was not valid"</span><span>)
</span><span> }
</span><span> </span><span style="color:#b48ead;">let</span><span> cwd = URL(fileURLWithPath: FileManager.</span><span style="color:#b48ead;">default</span><span>.currentDirectoryPath, isDirectory: </span><span style="color:#b48ead;">true</span><span>)
</span><span> </span><span style="color:#b48ead;">let</span><span> directory = URL(fileURLWithPath: </span><span style="color:#b48ead;">self</span><span>.directory, isDirectory: </span><span style="color:#b48ead;">true</span><span>, relativeTo: cwd)
</span><span>
</span><span> </span><span style="color:#b48ead;">let</span><span> downloader = Downloader()
</span><span> </span><span style="color:#b48ead;">let</span><span> dlFuture = downloader.downloadListedFiles(url: reallyURL, directory: directory)
</span><span>
</span><span> </span><span style="color:#b48ead;">defer</span><span> {
</span><span> do {
</span><span> </span><span style="color:#b48ead;">try</span><span> downloader.syncShutdown()
</span><span> } catch {
</span><span> print(</span><span style="color:#a3be8c;">"Error shutting down: </span><span>\(error)</span><span style="color:#a3be8c;">"</span><span>)
</span><span> }
</span><span> }
</span><span> </span><span style="color:#b48ead;">try</span><span> dlFuture.wait()
</span><span>}
</span></code></pre>
<p>And that's it! Create a <code>Downloader</code>, call it to get a future, set up cleanup, then wait until the future is done. </p>
<h2 id="conclusion">Conclusion</h2>
<p>SwiftNIO is a fantastic library that powers the Swift on the server ecosystem. It's also great with command line tooling. It can occasionally be a bit more involved than Foundation, but especially with HTTP requests the difference is negligible. You'd have had to bring in Combine too to make URLSessions composable.</p>
<p>The Foundation/standard library split is a bit awkward here, as it often is when working with Swift command line tools. It's not that Foundation doesn't work, but it's clear that often there's the Swift way and then there's the Foundation way. And Foundation's cross platform story has been a bit rocky.</p>
<p>As Swift's async story progresses a lot of this code can be simplified, I hope. In the ideal case the structure would stay pretty much as is, but those nested <code>map</code>s and <code>flatMap</code>s could be replaced with more straightforward code. However, I don't think you need to wait for async/await and all the related enhancements to arrive. This is already pretty great.</p>
Database connections in Vapor 42021-02-07T12:00:00+00:002021-02-07T12:00:00+00:00Unknownhttps://juripakaste.fi/db-in-vapor4/<p>Version 4 of the Swift web framework <a href="https://vapor.codes">Vapor</a> was released a while ago. Vapor emphasizes their ORM, Fluent, and it seems that version 4 has changed how a database connection can be acquired if you prefer to write the SQL yourself. They've also skipped documenting it, so getting things working requires some digging. In this post I'll explain how to do it. I'm using PostgreSQL.</p>
<p>You need a connection pool. The right place to set it up is your app's <code>configure(_:Application)</code> method. Use an environment variable to feed a database URL to your app:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">import </span><span>Vapor
</span><span>
</span><span style="color:#b48ead;">public func </span><span>configure(_ app: Application) throws {
</span><span> </span><span style="color:#b48ead;">guard let</span><span> dbUrlString = Environment.</span><span style="color:#b48ead;">get</span><span>(</span><span style="color:#a3be8c;">"DBURL"</span><span>) </span><span style="color:#b48ead;">else</span><span> {
</span><span> preconditionFailure(</span><span style="color:#a3be8c;">"Missing DBURL"</span><span>)
</span><span> }
</span><span> </span><span style="color:#65737e;">/* … */
</span><span>
</span><span> </span><span style="color:#65737e;">// register routes
</span><span> </span><span style="color:#b48ead;">try</span><span> routes(app)
</span><span>}
</span></code></pre>
<p>Now that you have a URL, import <code>PostgresKit</code> and set up the pool:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">import </span><span>PostgresKit
</span><span style="color:#b48ead;">import </span><span>Vapor
</span><span>
</span><span style="color:#b48ead;">public func </span><span>configure(_ app: Application) throws {
</span><span> </span><span style="color:#b48ead;">guard let</span><span> dbUrlString = Environment.</span><span style="color:#b48ead;">get</span><span>(</span><span style="color:#a3be8c;">"DBURL"</span><span>) </span><span style="color:#b48ead;">else</span><span> {
</span><span> preconditionFailure(</span><span style="color:#a3be8c;">"Missing DBURL"</span><span>)
</span><span> }
</span><span>
</span><span> </span><span style="color:#b48ead;">let</span><span> postgresConfiguration = PostgresConfiguration(url: dbUrlString)!
</span><span> </span><span style="color:#b48ead;">let</span><span> pool = EventLoopGroupConnectionPool(
</span><span> source: PostgresConnectionSource(configuration: postgresConfiguration),
</span><span> on: app.eventLoopGroup
</span><span> )
</span><span> </span><span style="color:#65737e;">/* … */
</span><span>
</span><span> </span><span style="color:#65737e;">// register routes
</span><span> </span><span style="color:#b48ead;">try</span><span> routes(app)
</span><span>}
</span></code></pre>
<p>Next we need to make the pool available to our request handlers and make sure it's shut down correctly. Making it available to request handlers happens by inserting it into the <code>Application</code> object's <code>storage</code>. Shutdown requires implementing Vapor's <code>LifecycleHandler</code> and registering it with the <code>Application</code>.</p>
<p>First define a struct that wraps the pool:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">struct</span><span> DatabaseService {
</span><span> </span><span style="color:#b48ead;">let</span><span> pool: EventLoopGroupConnectionPool<PostgresConnectionSource>
</span><span>}
</span></code></pre>
<p>To keep the service in <code>Application.storage</code>, we need a key type:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">struct</span><span> DatabaseServiceKey: StorageKey {
</span><span> </span><span style="color:#b48ead;">typealias</span><span> Value = DatabaseService
</span><span>}
</span></code></pre>
<p>Add an <code>Application</code> extension property to make it easier to access the service in the storage:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">extension</span><span> Application {
</span><span> </span><span style="color:#b48ead;">var</span><span> databaseService: DatabaseService? {
</span><span> </span><span style="color:#b48ead;">get</span><span> { </span><span style="color:#b48ead;">self</span><span>.storage[DatabaseServiceKey.</span><span style="color:#b48ead;">self</span><span>] }
</span><span> </span><span style="color:#b48ead;">set</span><span> { </span><span style="color:#b48ead;">self</span><span>.storage[DatabaseServiceKey.</span><span style="color:#b48ead;">self</span><span>] = newValue }
</span><span> }
</span><span>}
</span></code></pre>
<p>The lifecycle implementation looks like this:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">extension</span><span> DatabaseService: LifecycleHandler {
</span><span> </span><span style="color:#b48ead;">func </span><span>shutdown(_ application: Application) {
</span><span> </span><span style="color:#b48ead;">self</span><span>.pool.shutdown()
</span><span> }
</span><span>}
</span></code></pre>
<p>Now you just have slot these pieces in place in <code>configure</code>:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">import </span><span>PostgresKit
</span><span style="color:#b48ead;">import </span><span>Vapor
</span><span>
</span><span style="color:#b48ead;">public func </span><span>configure(_ app: Application) throws {
</span><span> </span><span style="color:#b48ead;">guard let</span><span> dbUrlString = Environment.</span><span style="color:#b48ead;">get</span><span>(</span><span style="color:#a3be8c;">"DBURL"</span><span>) </span><span style="color:#b48ead;">else</span><span> {
</span><span> preconditionFailure(</span><span style="color:#a3be8c;">"Missing DBURL"</span><span>)
</span><span> }
</span><span>
</span><span> </span><span style="color:#b48ead;">let</span><span> postgresConfiguration = PostgresConfiguration(url: dbUrlString)!
</span><span> </span><span style="color:#b48ead;">let</span><span> pool = EventLoopGroupConnectionPool(
</span><span> source: PostgresConnectionSource(configuration: postgresConfiguration),
</span><span> on: app.eventLoopGroup
</span><span> )
</span><span>
</span><span> </span><span style="color:#b48ead;">let</span><span> dbService = DatabaseService(pool: pool)
</span><span> app.databaseService = dbService
</span><span> app.lifecycle.use(dbService)
</span><span>
</span><span> </span><span style="color:#65737e;">// register routes
</span><span> </span><span style="color:#b48ead;">try</span><span> routes(app)
</span><span>}
</span></code></pre>
<p>Now you have a working database setup. If you want to run migrations, the <code>configure</code> method is probably a good place to do it, before you set up the routes. I keep the DB code in a domain specific <code>DBClient</code> type; you can use any division of responsibilities you like.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span> app.lifecycle.use(dbService)
</span><span>
</span><span> </span><span style="color:#b48ead;">let</span><span> db = dbService.pool.database(logger: app.logger)
</span><span> </span><span style="color:#b48ead;">let</span><span> dbClient = DBClient(database: db)
</span><span> app.logger.info(</span><span style="color:#a3be8c;">"Will run migrate on DB"</span><span>)
</span><span> _ = </span><span style="color:#b48ead;">try</span><span> dbClient.migrate().wait()
</span><span> app.logger.info(</span><span style="color:#a3be8c;">"DB migration done"</span><span>)
</span><span>
</span><span> </span><span style="color:#65737e;">// register routes
</span></code></pre>
<p>When handling requests, you'll just have to get the database service from <code>Request.application</code>. I like to create a struct called <code>RequestEnvironment</code> that encapsulates acquisition of the service and creation of domain logic services. Something like this:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">struct</span><span> RequestEnvironment {
</span><span> </span><span style="color:#b48ead;">var</span><span> makeFooService: () -> FooService
</span><span>
</span><span> </span><span style="color:#b48ead;">static func </span><span>makeDefault(req: Request) -> RequestEnvironment {
</span><span> </span><span style="color:#b48ead;">guard let</span><span> dbService = req.application.databaseService </span><span style="color:#b48ead;">else</span><span> {
</span><span> fatalError(</span><span style="color:#a3be8c;">"Missing DatabaseService"</span><span>)
</span><span> }
</span><span> </span><span style="color:#b48ead;">let</span><span> db = dbService.pool.database(logger: req.logger)
</span><span> </span><span style="color:#b48ead;">let</span><span> dbClient = DBClient(database: db)
</span><span> </span><span style="color:#b48ead;">return</span><span> RequestEnvironment(
</span><span> makeFooService: { FooService(dbClient: dbClient, request: req) }
</span><span> )
</span><span> }
</span><span>}
</span></code></pre>
<p>Now when your controller handles a request, create the <code>RequestEnvironment</code> object and use it to call your services with the database client:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">struct</span><span> FooController {
</span><span> </span><span style="color:#b48ead;">func </span><span>create(_ req: Request) -> EventLoopFuture<FooExternal> {
</span><span> </span><span style="color:#b48ead;">let</span><span> newFoo = </span><span style="color:#b48ead;">try</span><span> req.content.decode(NewFooIncoming.</span><span style="color:#b48ead;">self</span><span>)
</span><span> </span><span style="color:#b48ead;">let</span><span> env = RequestEnvironment.makeDefault(req: req)
</span><span> </span><span style="color:#b48ead;">return</span><span> env.makeFooService().makeFoo(newFoo)
</span><span> }
</span><span>}
</span></code></pre>
<p>That's it! Go forth and SQL, swiftly.</p>
Alfred Script Filter with find and jq2021-01-30T12:00:00+00:002021-01-30T12:00:00+00:00Unknownhttps://juripakaste.fi/jq-alfred-script-filter/<p><a href="https://juripakaste.fi/diff_two_modified_jsons_in_fish.html">Looks</a> <a href="https://juripakaste.fi/jq_copy_value.html">like</a> this is a <a href="https://github.com/stedolan/jq">jq</a> blog now, so here's another one.</p>
<p>I work on an iOS repository that's used to create a large number of apps and a few frameworks. Each app has a directory with configuration and a script that regenerates the associated Xcode project with <a href="https://github.com/yonaskolb/XcodeGen">XcodeGen</a>.</p>
<p>You can run the script from the shell, or from Finder. Both of these require that you navigate to the appropriate directory or find a window that's already there. Both approaches work, and both are unsatisfactory.</p>
<p>I use <a href="https://alfredapp.com">Alfred</a> for launching apps and all sorts of other things on macOS. One of the things it allows is workflows, a sort of Automator-like thing where after typing in a keyword Alfred will prompt you for input and execute things and so on. I built a workflow for helping with launching those regenerate scripts. Alfred's workflow sharing thing isn't great, as it creates hard to inspect zip files, and besides my specific circumstances probably aren't relevant to many people. I'll explain here in prose how it works. Adapt it to your needs as necessary.</p>
<p>The repository contains publisher folders. Inside the publisher folders are app folders. In each app folder is a script called <code>regenerate-project.command</code>. The hierarchy looks something like this:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>├── publisher1
</span><span>│ ├── app1
</span><span>│ │ └── regenerate-project.command
</span><span>│ └── app2
</span><span>│ └── regenerate-project.command
</span><span>└── publisher2
</span><span> └── app1
</span><span> └── regenerate-project.command
</span></code></pre>
<p>We want Alfred to ask us which one of the scripts to run after we've typed a keyword.</p>
<p>Let's see how we can make it happen. First, to get a list of those files we can run <code>find</code> in the terminal:</p>
<pre data-lang="sh" style="background-color:#2b303b;color:#c0c5ce;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="color:#bf616a;">find</span><span> .</span><span style="color:#bf616a;"> -maxdepth</span><span> 3</span><span style="color:#bf616a;"> -mindepth</span><span> 3</span><span style="color:#bf616a;"> -name</span><span> regenerate-project.command</span><span style="color:#bf616a;"> -print
</span></code></pre>
<p>This gives us a list of files, one per line, like:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>./publisher1/app1/regenerate-project.command
</span><span>./publisher1/app2/regenerate-project.command
</span><span>./publisher2/app1/regenerate-project.command
</span></code></pre>
<p>etc<sup class="footnote-reference"><a href="#1">1</a></sup>.</p>
<p>Now, looking at Alfred's documentation, looks like we need to create a document in the <a href="https://www.alfredapp.com/help/workflows/inputs/script-filter/json/">Script Filter JSON Format</a>. It should look like this:</p>
<pre data-lang="json" style="background-color:#2b303b;color:#c0c5ce;" class="language-json "><code class="language-json" data-lang="json"><span>{
</span><span> "</span><span style="color:#a3be8c;">items</span><span>": [
</span><span> {
</span><span> "</span><span style="color:#a3be8c;">uid</span><span>": "</span><span style="color:#a3be8c;">publisher1/app1</span><span>",
</span><span> "</span><span style="color:#a3be8c;">type</span><span>": "</span><span style="color:#a3be8c;">file</span><span>",
</span><span> "</span><span style="color:#a3be8c;">title</span><span>": "</span><span style="color:#a3be8c;">publisher1/app1</span><span>",
</span><span> "</span><span style="color:#a3be8c;">arg</span><span>": "</span><span style="color:#a3be8c;">publisher1/app1</span><span>",
</span><span> "</span><span style="color:#a3be8c;">match</span><span>": "</span><span style="color:#a3be8c;">publisher1: app1</span><span>"
</span><span> }
</span><span> ]
</span><span>}
</span></code></pre>
<p>And so on. The one thing that breaks the monotony of identical keys is the <code>match</code>
value. Its purpose there is to make Alfred give better completions. Alfred
has a "word boundary" matching logic, but apparently <code>/</code> doesn't count as a word
boundary.</p>
<p>What do we do when we need to handle JSON on the command line? We reach for jq.</p>
<p>Jq has a number of parameters that modify how it handles input. To get it to ingest the list of strings produced by <code>find</code>, what seemed to work was using a combination of the <code>--raw-input/-R</code> and <code>--null-input/-n</code> flags, and the <code>inputs</code> builtin function. So the first thing to do is to build the wrapping object.</p>
<pre data-lang="sh" style="background-color:#2b303b;color:#c0c5ce;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="color:#bf616a;">find</span><span> .</span><span style="color:#bf616a;"> -maxdepth</span><span> 3</span><span style="color:#bf616a;"> -mindepth</span><span> 3</span><span style="color:#bf616a;"> -name</span><span> regenerate-project.command</span><span style="color:#bf616a;"> -print </span><span>| </span><span style="color:#bf616a;">jq -nR </span><span>'</span><span style="color:#a3be8c;">{ "items": [inputs] }</span><span>'
</span></code></pre>
<p>Running that produces output like this:</p>
<pre data-lang="json" style="background-color:#2b303b;color:#c0c5ce;" class="language-json "><code class="language-json" data-lang="json"><span>{
</span><span> "</span><span style="color:#a3be8c;">items</span><span>": [
</span><span> "</span><span style="color:#a3be8c;">./publisher1/app2/regenerate-project.command</span><span>",
</span><span> "</span><span style="color:#a3be8c;">./publisher1/app1/regenerate-project.command</span><span>",
</span><span> "</span><span style="color:#a3be8c;">./publisher2/app1/regenerate-project.command</span><span>"
</span><span> ]
</span><span>}
</span></code></pre>
<p>You could pipe <code>find</code> through <code>sort</code> or you could use jq's <code>sort</code> function, but the order doesn't matter as Alfred will reorder the choices by usage anyway, which is nice.</p>
<p>Next, just because we're careful developers, let's filter out empty entries, just in case we're ever using this with some other source of data:</p>
<pre data-lang="sh" style="background-color:#2b303b;color:#c0c5ce;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="color:#bf616a;">find</span><span> … | </span><span style="color:#bf616a;">jq -nR </span><span>'</span><span style="color:#a3be8c;">{ "items": [inputs | select(length>0)] }</span><span>'
</span></code></pre>
<p>When you're running this with find, it shouldn't affect the output, but if you ever end up feeding it a text file it might be a different story.</p>
<p>Next drop the extra bits from the lines. We don't care about the leading <code>./</code> or the script name. They're all the same on all the lines. To lose them split the line into path components, take the two central elements and recombine them:</p>
<pre data-lang="sh" style="background-color:#2b303b;color:#c0c5ce;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="color:#bf616a;">find</span><span> … | </span><span style="color:#bf616a;">jq -nR </span><span>'</span><span style="color:#a3be8c;">{
</span><span style="color:#a3be8c;"> "items": [
</span><span style="color:#a3be8c;"> inputs |
</span><span style="color:#a3be8c;"> select(length>0) |
</span><span style="color:#a3be8c;"> split("/")[1:3] |
</span><span style="color:#a3be8c;"> join("/")
</span><span style="color:#a3be8c;"> ]
</span><span style="color:#a3be8c;">}</span><span>'
</span></code></pre>
<pre data-lang="json" style="background-color:#2b303b;color:#c0c5ce;" class="language-json "><code class="language-json" data-lang="json"><span>{
</span><span> "</span><span style="color:#a3be8c;">items</span><span>": [
</span><span> "</span><span style="color:#a3be8c;">publisher1/app2</span><span>",
</span><span> "</span><span style="color:#a3be8c;">publisher1/app1</span><span>",
</span><span> "</span><span style="color:#a3be8c;">publisher2/app1</span><span>"
</span><span> ]
</span><span>}
</span></code></pre>
<p>One thing we have to do to before we can build the object literals is capture the values — both the parts array and the combined string — in variables. This is a slightly longer version of the above jq snippet. It produces exactly the same output, but it defines the variables we need in the next step:</p>
<pre data-lang="sh" style="background-color:#2b303b;color:#c0c5ce;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="color:#bf616a;">find</span><span> … | </span><span style="color:#bf616a;">jq -nR </span><span>'</span><span style="color:#a3be8c;">{
</span><span style="color:#a3be8c;"> "items": [
</span><span style="color:#a3be8c;"> inputs |
</span><span style="color:#a3be8c;"> select(length>0) |
</span><span style="color:#a3be8c;"> split("/")[1:3] as $parts |
</span><span style="color:#a3be8c;"> $parts |
</span><span style="color:#a3be8c;"> join("/") as $file |
</span><span style="color:#a3be8c;"> $file
</span><span style="color:#a3be8c;"> ]
</span><span style="color:#a3be8c;">}</span><span>'
</span></code></pre>
<p>OK, good. Now we have a the two folders as an array in <code>$parts</code> and as a string in <code>$file</code>. Then just replace that last bit that produces the array elements with an object literal.</p>
<pre data-lang="sh" style="background-color:#2b303b;color:#c0c5ce;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="color:#bf616a;">find</span><span> … | </span><span style="color:#bf616a;">jq -nR </span><span>'</span><span style="color:#a3be8c;">{
</span><span style="color:#a3be8c;"> "items": [
</span><span style="color:#a3be8c;"> inputs |
</span><span style="color:#a3be8c;"> select(length>0) |
</span><span style="color:#a3be8c;"> split("/")[1:3] as $parts |
</span><span style="color:#a3be8c;"> $parts |
</span><span style="color:#a3be8c;"> join("/") as $file |
</span><span style="color:#a3be8c;"> {
</span><span style="color:#a3be8c;"> "uid": $file,
</span><span style="color:#a3be8c;"> "type": "file",
</span><span style="color:#a3be8c;"> "title": $file,
</span><span style="color:#a3be8c;"> "arg": $file,
</span><span style="color:#a3be8c;"> match: $parts | join(": ")
</span><span style="color:#a3be8c;"> }
</span><span style="color:#a3be8c;"> ]
</span><span style="color:#a3be8c;">}</span><span>'
</span></code></pre>
<p>That's a whole lot of <code>$file</code> and one special element that produces the value for the <code>match</code> field. Now the output looks like this:</p>
<pre data-lang="json" style="background-color:#2b303b;color:#c0c5ce;" class="language-json "><code class="language-json" data-lang="json"><span>{
</span><span> "</span><span style="color:#a3be8c;">items</span><span>": [
</span><span> {
</span><span> "</span><span style="color:#a3be8c;">uid</span><span>": "</span><span style="color:#a3be8c;">publisher1/app2</span><span>",
</span><span> "</span><span style="color:#a3be8c;">type</span><span>": "</span><span style="color:#a3be8c;">file</span><span>",
</span><span> "</span><span style="color:#a3be8c;">title</span><span>": "</span><span style="color:#a3be8c;">publisher1/app2</span><span>",
</span><span> "</span><span style="color:#a3be8c;">arg</span><span>": "</span><span style="color:#a3be8c;">publisher1/app2</span><span>",
</span><span> "</span><span style="color:#a3be8c;">match</span><span>": "</span><span style="color:#a3be8c;">publisher1: app2</span><span>"
</span><span> },
</span><span> {
</span><span> "</span><span style="color:#a3be8c;">uid</span><span>": "</span><span style="color:#a3be8c;">publisher1/app1</span><span>",
</span><span> "</span><span style="color:#a3be8c;">type</span><span>": "</span><span style="color:#a3be8c;">file</span><span>",
</span><span> "</span><span style="color:#a3be8c;">title</span><span>": "</span><span style="color:#a3be8c;">publisher1/app1</span><span>",
</span><span> "</span><span style="color:#a3be8c;">arg</span><span>": "</span><span style="color:#a3be8c;">publisher1/app1</span><span>",
</span><span> "</span><span style="color:#a3be8c;">match</span><span>": "</span><span style="color:#a3be8c;">publisher1: app1</span><span>"
</span><span> },
</span><span> {
</span><span> "</span><span style="color:#a3be8c;">uid</span><span>": "</span><span style="color:#a3be8c;">publisher2/app1</span><span>",
</span><span> "</span><span style="color:#a3be8c;">type</span><span>": "</span><span style="color:#a3be8c;">file</span><span>",
</span><span> "</span><span style="color:#a3be8c;">title</span><span>": "</span><span style="color:#a3be8c;">publisher2/app1</span><span>",
</span><span> "</span><span style="color:#a3be8c;">arg</span><span>": "</span><span style="color:#a3be8c;">publisher2/app1</span><span>",
</span><span> "</span><span style="color:#a3be8c;">match</span><span>": "</span><span style="color:#a3be8c;">publisher2: app1</span><span>"
</span><span> }
</span><span> ]
</span><span>}
</span></code></pre>
<p>All right, that's what we were after! Now we need to glue things together. In Alfred's Preferences, go to Workflows and create a new blank workflow. First tap on the "[𝑥]" button to set up variables. You'll need at least one, to specify where your project lives. Call it <code>root</code>, specify your folder as the value, and uncheck "Don't Export" as you want it as an environment variable in your script.</p>
<p>Next ctrl-click in the workflow background to get the context menu and select Inputs > Script Filter. In the filter configuration panel, give your workflow a keyword — I call mine <code>regenios</code>, this is how I invoke it in Alfred — uncheck "with space", and select "Argument Required". Select <code>/bin/bash</code> as the script language, and as text add this:</p>
<pre data-lang="sh" style="background-color:#2b303b;color:#c0c5ce;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="color:#96b5b4;">cd </span><span>$</span><span style="color:#bf616a;">root
</span><span style="color:#bf616a;">find</span><span> .</span><span style="color:#bf616a;"> -maxdepth</span><span> 3</span><span style="color:#bf616a;"> -mindepth</span><span> 3</span><span style="color:#bf616a;"> -name</span><span> regenerate-project.command</span><span style="color:#bf616a;"> -print </span><span>| </span><span style="color:#bf616a;">jq -nR </span><span>'</span><span style="color:#a3be8c;">{
</span><span style="color:#a3be8c;"> "items": [
</span><span style="color:#a3be8c;"> inputs |
</span><span style="color:#a3be8c;"> select(length>0) |
</span><span style="color:#a3be8c;"> split("/")[1:3] as $parts |
</span><span style="color:#a3be8c;"> $parts |
</span><span style="color:#a3be8c;"> join("/") as $file |
</span><span style="color:#a3be8c;"> {
</span><span style="color:#a3be8c;"> "uid": $file,
</span><span style="color:#a3be8c;"> "type": "file",
</span><span style="color:#a3be8c;"> "title": $file,
</span><span style="color:#a3be8c;"> "arg": $file,
</span><span style="color:#a3be8c;"> match: $parts | join(": ")
</span><span style="color:#a3be8c;"> }
</span><span style="color:#a3be8c;"> ]
</span><span style="color:#a3be8c;">}</span><span>'
</span></code></pre>
<p>Now click Save to save your Script Filter. Then ctrl-click in the workflow background again and this time select Actions > Terminal Command. Insert the following as the terminal command:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>{var:root}/{query}/regenerate-project.command && exit
</span></code></pre>
<p>Again click save. Finally in the workflow editor drag a connection from the Script Filter shape to the Terminal Command box and you're done.</p>
<p>Now when you open the Alfred command window and type <code>regenios</code> and two spaces, you should get a full list of all the items your script produced. If you start typing after the first space, Alfred will match the beginning of each word of the <code>match</code> field of the JSON objects we produced and give a list of the matching items.</p>
<p>As I said at the start of this article, this probably isn't of much use to you as is. But it might be useful as inspiration.</p>
<div class="footnote-definition" id="1"><sup class="footnote-definition-label">1</sup>
<p>Yes, I'm aware of <code>-print0</code>, but it seems <code>jq</code> isn't.</p>
</div>
Diff two modified JSON files in fish2020-08-24T09:00:00+00:002020-08-24T09:00:00+00:00Unknownhttps://juripakaste.fi/diff-two-modified-jsons-in-fish/<p>Another interesting command line JSON exercise: you have two JSON files, you want to diff a modified version of one to the other, and your shell is <a href="https://fishshell.com">fish</a>.</p>
<p>For making JSON diffable, <a href="https://github.com/TomNomNom/gron">gron</a> is a great choice: it transforms a JSON file into a list of assignment lines. Like this:</p>
<pre data-lang="sh" style="background-color:#2b303b;color:#c0c5ce;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="color:#bf616a;">$</span><span> echo '</span><span style="color:#a3be8c;">{"a": ["b", "c"]}</span><span>' | </span><span style="color:#bf616a;">gron
</span><span style="color:#bf616a;">json</span><span> = {};
</span><span style="color:#bf616a;">json.a</span><span> = </span><span style="color:#b48ead;">[]</span><span>;
</span><span style="color:#bf616a;">json.a[0]</span><span> = "</span><span style="color:#a3be8c;">b</span><span>";
</span><span style="color:#bf616a;">json.a[1]</span><span> = "</span><span style="color:#a3be8c;">c</span><span>";
</span></code></pre>
<p>gron doesn't help us with modifications, so <a href="https://github.com/stedolan/jq">jq</a> is <a href="https://juripakaste.fi/jq_copy_value.html">again</a> a good choice there. In my case, I had one file that contained an array of objects, and wanted to compare the first element of that array to the content of a second file. Additionally I wanted to set the value of one key of the object from the first file to <code>null</code>. With jq that's quick work:</p>
<pre data-lang="sh" style="background-color:#2b303b;color:#c0c5ce;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="color:#bf616a;">jq </span><span>'</span><span style="color:#a3be8c;">.[0] | .large_field = null</span><span>' file1.json | </span><span style="color:#bf616a;">gron
</span></code></pre>
<p>The second file I wanted as-is, and that's just</p>
<pre data-lang="sh" style="background-color:#2b303b;color:#c0c5ce;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="color:#bf616a;">gron</span><span> file2.json
</span></code></pre>
<p>The last problem I wanted to solve was how to actually get the output from those two commands to diff (or <a href="https://www.colordiff.org">colordiff</a>). I could just pipe them to files and compare those, but that's untidy. As usual it's much easier to find answers to this for <a href="https://www.gnu.org/software/bash/">bash</a> than for fish. In bash (and zsh, and ksh) it looks like this:</p>
<pre data-lang="sh" style="background-color:#2b303b;color:#c0c5ce;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="color:#bf616a;">colordiff -u </span><span><(</span><span style="color:#bf616a;">jq </span><span>'</span><span style="color:#a3be8c;">.[0] | .large_field = null</span><span>' file1.json | </span><span style="color:#bf616a;">gron</span><span>) <(</span><span style="color:#bf616a;">gron</span><span> file2.json)
</span></code></pre>
<p>A quick peek at the bash manual page reveals that <code><(</code> is used for "process substitution". Search the web and you'll find that in fish that's accomplished with the magic command <a href="http://fishshell.com/docs/current/cmds/psub.html">psub</a>:</p>
<pre data-lang="sh" style="background-color:#2b303b;color:#c0c5ce;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="color:#bf616a;">colordiff -u</span><span> (jq '</span><span style="color:#a3be8c;">.[0] | .large_field = null</span><span>' file1.json | </span><span style="color:#bf616a;">gron </span><span>| </span><span style="color:#bf616a;">psub</span><span>) (</span><span style="color:#bf616a;">gron</span><span> file2.json | </span><span style="color:#bf616a;">psub</span><span>)
</span></code></pre>
<p>And that's it. Pretty colored diff of the parts you're interested in, and no temporary files to remove afterwards.</p>
Copy value with jq2020-08-23T12:00:00+00:002020-08-23T12:00:00+00:00Unknownhttps://juripakaste.fi/jq-copy-value/<p>I use <a href="https://github.com/stedolan/jq">jq</a> heavily in my day to day work. It's a powerful tool but not always easy, so I have piles of notes about how to do things with it.</p>
<p>I had to do a few iterations of copying a value from one JSON file to another the other day, and the files are large, and copying and pasting and launching editors was getting old, so I reached for jq. And after digging for a while and reading Stack Overflow answers interpreting the manual page for me, here's how to do it, in <a href="https://fishshell.com">fish</a>:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>jq --argjson js (jq .path.field a.json) '.path.field = $js' b.json | sponge b.json
</span></code></pre>
<p>The subshell invoked by <code>()</code> reads the value from <code>a.json</code>, <code>--argjson</code> sets the value the variable <code>js</code> without extra quotes or escapes, as it's already a valid JSON value, and then the assignment just sets the value to the document read from <code>b.json</code>.</p>
<p>It also requires <a href="https://joeyh.name/code/moreutils/">sponge</a> because the jq developers don't want it to have an <code>-i</code> option. Oh well.</p>
Parsing and evaluating mathematical expressions in Swift / Part 3: Interpreter2020-04-06T15:20:00+00:002020-04-06T15:20:00+00:00Unknownhttps://juripakaste.fi/expressions-3/<p>Ever needed to interpret mathematical expressions with variables, like <code>a.field1 + (a.field2 - b.field1) * 2</code>, in Swift? I did. This series of blog posts will walk you through my solution. This is part 3 of the series:</p>
<ol>
<li><a href="https://juripakaste.fi/expressions-3/expressions-1.html">Tokenization</a></li>
<li><a href="https://juripakaste.fi/expressions-3/expressions-2.html">Building a syntax tree</a></li>
<li>Evaluating the syntax tree</li>
</ol>
<h2 id="evaluating-the-syntax-tree">Evaluating the syntax tree</h2>
<p>To recap, we started with the string <code>a.field1 + (a.field2 - b.field1) * 2</code> and ended up with this Swift structure:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span>ExpressionTreeNode.operation(
</span><span> .addition,
</span><span> .atom(.name([</span><span style="color:#a3be8c;">"a"</span><span>, </span><span style="color:#a3be8c;">"field1"</span><span>])),
</span><span> .operation(
</span><span> .multiplication,
</span><span> .operation(
</span><span> .subtraction,
</span><span> .atom(.name([</span><span style="color:#a3be8c;">"a"</span><span>, </span><span style="color:#a3be8c;">"field2"</span><span>])),
</span><span> .atom(.name([</span><span style="color:#a3be8c;">"b"</span><span>, </span><span style="color:#a3be8c;">"field1"</span><span>]))
</span><span> ),
</span><span> .atom(.number(</span><span style="color:#d08770;">2</span><span>))
</span><span> )
</span><span>)
</span></code></pre>
<p>Now we want to compute the value of that tree. That requires two things: a depth-first walk of the tree to compute the nested values first, and a context where we can find values for the names. Let's look at the name context first.</p>
<p>Binding values to names is something we're very well equipped for. It's just a simple dictionary. One thing we do have to consider is the types of values. My implementation deals in floating point values, but we have those names as arrays, allowing nested contexts. You might be able to go with enum values, but in this case I ended up with plain dynamic typing: values typed as <code>Any</code>, types checked at runtime. So our dictionary is typed as <code>[String: Any]</code>.</p>
<p>Do we want to build up dictionaries of all our values? I suppose we could. But as you might have guessed from the names in our expressions, the values are actually derived from some actual live objects. It might be nicer to feed those objects directly to the evaluator, right? We can use a protocol to make it happen. Swift's dictionaries use Swift's <a href="https://docs.swift.org/swift-book/LanguageGuide/Subscripts.html">subscript</a> functionality to allow access to values, and you can adopt that in your own types, too. So let's define a protocol:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">protocol</span><span> ExpressionEvaluatorNameContext {
</span><span> </span><span style="color:#b48ead;">subscript</span><span>(name: String) -> Any? { </span><span style="color:#b48ead;">get</span><span> }
</span><span>}
</span></code></pre>
<p>So what do we want, the dictionary I discussed earlier or types implementing that protocol? Swift allows us to have our cake and eat it too:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">extension </span><span>Dictionary: ExpressionEvaluatorNameContext </span><span style="color:#b48ead;">where</span><span> Key == String, Value == Any {}
</span></code></pre>
<p>And if we want to wrap our objects in that protocol, we can define simple wrapper types:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">struct</span><span> EvaluatorContextUIViewWrapper {
</span><span> </span><span style="color:#b48ead;">private let</span><span> view: UIView
</span><span>
</span><span> </span><span style="color:#b48ead;">init</span><span>(view: UIView) {
</span><span> </span><span style="color:#b48ead;">self</span><span>.view = view
</span><span> }
</span><span>}
</span><span>
</span><span style="color:#b48ead;">extension</span><span> EvaluatorContextUIViewWrapper: ExpressionEvaluatorNameContext {
</span><span> </span><span style="color:#b48ead;">subscript</span><span>(name: String) -> Any? {
</span><span> </span><span style="color:#b48ead;">switch</span><span> name {
</span><span> </span><span style="color:#b48ead;">case </span><span style="color:#a3be8c;">"height"</span><span>: </span><span style="color:#b48ead;">return </span><span>Double(</span><span style="color:#b48ead;">self</span><span>.view.bounds.height)
</span><span> </span><span style="color:#b48ead;">default</span><span>: </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">nil
</span><span> }
</span><span> }
</span><span>}
</span></code></pre>
<p>Now that we have our context types defined, we can start digging for the values. First let's define a couple of error types that we'll use in our evaluator. <code>EvaluationError</code> is what we'll use to signal problems outside, and <code>NameLookupInContextFailed</code> is used to accumulate errors in recursive name resolution.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">enum</span><span> EvaluationError: Error {
</span><span> </span><span style="color:#b48ead;">case</span><span> emptyName
</span><span> </span><span style="color:#b48ead;">case</span><span> unknownName([String], Error)
</span><span> </span><span style="color:#b48ead;">case</span><span> invalidOperation(ExpressionOperator, Any, Any)
</span><span>}
</span><span>
</span><span style="color:#b48ead;">struct</span><span> NameLookupInContextFailed: Error {
</span><span> </span><span style="color:#b48ead;">let</span><span> name: String
</span><span> </span><span style="color:#b48ead;">let</span><span> cause: Error?
</span><span>}
</span></code></pre>
<p>Then define a couple of functions for finding values in those nested contexts. One provides the outside interface, the other recursively calls itself with names from the list until reaches the end or runs into a problem:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">func </span><span>contextValue(</span><span style="color:#b48ead;">for</span><span> name: [String], context: ExpressionEvaluatorNameContext) throws -> Any {
</span><span> </span><span style="color:#b48ead;">guard let</span><span> nameHead = name.first </span><span style="color:#b48ead;">else</span><span> { </span><span style="color:#b48ead;">throw</span><span> EvaluationError.emptyName }
</span><span> do {
</span><span> </span><span style="color:#b48ead;">return try</span><span> contextValue(</span><span style="color:#b48ead;">for</span><span>: nameHead, tail: name.dropFirst(), </span><span style="color:#b48ead;">in</span><span>: context)
</span><span> } catch {
</span><span> </span><span style="color:#b48ead;">throw</span><span> EvaluationError.unknownName(name, error)
</span><span> }
</span><span>}
</span><span>
</span><span style="color:#b48ead;">func </span><span>contextValue(
</span><span> </span><span style="color:#b48ead;">for</span><span> nameHead: String,
</span><span> tail: ArraySlice<String>,
</span><span> </span><span style="color:#b48ead;">in</span><span> context: ExpressionEvaluatorNameContext
</span><span>) throws -> Any {
</span><span> </span><span style="color:#b48ead;">let</span><span> maybeValue = context[nameHead]
</span><span>
</span><span> </span><span style="color:#b48ead;">guard let</span><span> value = maybeValue </span><span style="color:#b48ead;">else</span><span> {
</span><span> </span><span style="color:#b48ead;">throw</span><span> NameLookupInContextFailed(name: nameHead, cause: </span><span style="color:#d08770;">nil</span><span>)
</span><span> }
</span><span>
</span><span> </span><span style="color:#b48ead;">if let</span><span> tailHead = tail.first {
</span><span> </span><span style="color:#b48ead;">guard let</span><span> subContext = value as? ExpressionEvaluatorNameContext </span><span style="color:#b48ead;">else</span><span> {
</span><span> </span><span style="color:#b48ead;">throw</span><span> NameLookupInContextFailed(name: nameHead, cause: </span><span style="color:#d08770;">nil</span><span>)
</span><span> }
</span><span>
</span><span> do {
</span><span> </span><span style="color:#b48ead;">return try</span><span> contextValue(</span><span style="color:#b48ead;">for</span><span>: tailHead, tail: tail.dropFirst(), </span><span style="color:#b48ead;">in</span><span>: subContext)
</span><span> } catch {
</span><span> </span><span style="color:#b48ead;">throw</span><span> NameLookupInContextFailed(name: nameHead, cause: error)
</span><span> }
</span><span> } </span><span style="color:#b48ead;">else</span><span> {
</span><span> </span><span style="color:#b48ead;">return</span><span> value
</span><span> }
</span><span>}
</span></code></pre>
<p>That's pretty straightforward — look for a value, check if we have still more parts of the name to resolve, if so recurse, otherwise return the value, check types and throw errors as necessary. With those functions in place, we can resolve names in our expressions. </p>
<p>We can now move on to expression node evaluation. We have two kinds of expression nodes: ones with just a simple value, and ones with an operation on other nodes. </p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">func </span><span>evaluate(expression: ExpressionTreeNode, context: ExpressionEvaluatorNameContext) throws -> Any {
</span><span> </span><span style="color:#b48ead;">switch</span><span> expression {
</span><span> </span><span style="color:#b48ead;">case let </span><span>.atom(atom):
</span><span> </span><span style="color:#b48ead;">return try</span><span> evaluate(atom: atom, context: context)
</span><span> </span><span style="color:#b48ead;">case let </span><span>.operation(op, n1, n2):
</span><span> </span><span style="color:#b48ead;">return try</span><span> evaluate(operation: op, node1: n1, node2: n2, context: context)
</span><span> }
</span><span>}
</span></code></pre>
<p>For atoms, it can be a number which we can return directly, or a name, in which case we'll use the <code>contextValue</code> functions we defined earlier.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">func </span><span>evaluate(atom: ExpressionTreeAtom, context: ExpressionEvaluatorNameContext) throws -> Any {
</span><span> </span><span style="color:#b48ead;">switch</span><span> atom {
</span><span> </span><span style="color:#b48ead;">case let </span><span>.name(name): </span><span style="color:#b48ead;">return try</span><span> contextValue(</span><span style="color:#b48ead;">for</span><span>: name, context: context)
</span><span> </span><span style="color:#b48ead;">case let </span><span>.number(num): </span><span style="color:#b48ead;">return</span><span> num
</span><span> }
</span><span>}
</span></code></pre>
<p>And for operators we need to find a function that implements the operator and then evaluate both sides so we get the final operand values for the operation.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">func </span><span>evaluate(
</span><span> operation: ExpressionOperator,
</span><span> node1: ExpressionTreeNode,
</span><span> node2: ExpressionTreeNode,
</span><span> context: ExpressionEvaluatorNameContext
</span><span>) throws -> Any {
</span><span> </span><span style="color:#b48ead;">let</span><span> opf = opFunc(operation)
</span><span> </span><span style="color:#b48ead;">let</span><span> val1 = </span><span style="color:#b48ead;">try</span><span> evaluate(expression: node1, context: context)
</span><span> </span><span style="color:#b48ead;">let</span><span> val2 = </span><span style="color:#b48ead;">try</span><span> evaluate(expression: node2, context: context)
</span><span> </span><span style="color:#b48ead;">return try</span><span> opf(val1, val2)
</span><span>}
</span></code></pre>
<p>All right, almost done! Except for the implementation of <code>opFunc</code> which should give us actual operator implementations. We need functions that are ready to deal with two <code>Any</code> values and do math of them. I'm going to present simple versions here that deal only with Doubles for brevity. In reality you may want to support other numeric types too, as well as deal with Optionals. The <code>op</code> function is generic and will happily deal with any types you throw at it, but optionals would require a bit more work to unwrap them.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">func </span><span>opFunc(_ op: ExpressionOperator) -> (Any, Any) throws -> Any {
</span><span> </span><span style="color:#b48ead;">switch</span><span> op {
</span><span> </span><span style="color:#b48ead;">case </span><span>.addition: </span><span style="color:#b48ead;">return</span><span> opAddition(_:_:)
</span><span> </span><span style="color:#b48ead;">case </span><span>.subtraction: </span><span style="color:#b48ead;">return</span><span> opSubtraction(_:_:)
</span><span> </span><span style="color:#b48ead;">case </span><span>.multiplication: </span><span style="color:#b48ead;">return</span><span> opMultiplication(_:_:)
</span><span> </span><span style="color:#b48ead;">case </span><span>.division: </span><span style="color:#b48ead;">return</span><span> opDivision(_:_:)
</span><span> }
</span><span>}
</span><span>
</span><span style="color:#b48ead;">func </span><span>opAddition(_ lhs: Any, _ rhs: Any) throws -> Any {
</span><span> </span><span style="color:#b48ead;">if let</span><span> result = op(Double.</span><span style="color:#b48ead;">self</span><span>, +, lhs, rhs) { </span><span style="color:#b48ead;">return</span><span> result }
</span><span> </span><span style="color:#b48ead;">throw</span><span> EvaluationError.invalidOperation(.addition, lhs, rhs)
</span><span>}
</span><span>
</span><span style="color:#b48ead;">func </span><span>opSubtraction(_ lhs: Any, _ rhs: Any) throws -> Any {
</span><span> </span><span style="color:#b48ead;">if let</span><span> result = op(Double.</span><span style="color:#b48ead;">self</span><span>, -, lhs, rhs) { </span><span style="color:#b48ead;">return</span><span> result }
</span><span> </span><span style="color:#b48ead;">throw</span><span> EvaluationError.invalidOperation(.subtraction, lhs, rhs)
</span><span>}
</span><span>
</span><span style="color:#b48ead;">func </span><span>opMultiplication(_ lhs: Any, _ rhs: Any) throws -> Any {
</span><span> </span><span style="color:#b48ead;">if let</span><span> result = op(Double.</span><span style="color:#b48ead;">self</span><span>, *, lhs, rhs) { </span><span style="color:#b48ead;">return</span><span> result }
</span><span> </span><span style="color:#b48ead;">throw</span><span> EvaluationError.invalidOperation(.multiplication, lhs, rhs)
</span><span>}
</span><span>
</span><span style="color:#b48ead;">func </span><span>opDivision(_ lhs: Any, _ rhs: Any) throws -> Any {
</span><span> </span><span style="color:#b48ead;">if let</span><span> result = op(Double.</span><span style="color:#b48ead;">self</span><span>, /, lhs, rhs) { </span><span style="color:#b48ead;">return</span><span> result }
</span><span> </span><span style="color:#b48ead;">throw</span><span> EvaluationError.invalidOperation(.division, lhs, rhs)
</span><span>}
</span><span>
</span><span style="color:#b48ead;">func </span><span>op<T>(_ type: T.</span><span style="color:#b48ead;">Type</span><span>, _ operation: (T, T) -> T, _ lhs: Any, _ rhs: Any) -> T? {
</span><span> </span><span style="color:#b48ead;">if let</span><span> lhsValue = lhs as? T, </span><span style="color:#b48ead;">let</span><span> rhsValue = rhs as? T {
</span><span> </span><span style="color:#b48ead;">return</span><span> operation(lhsValue, rhsValue)
</span><span> }
</span><span>
</span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">nil
</span><span>}
</span></code></pre>
<h2 id="conclusion">Conclusion</h2>
<p>That's it! That'll take you from the expression as a string to a final double value, as long as you have suitable values in the context. If we give it a spin, the following results in 42:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">let</span><span> context: [String: Any] = [
</span><span> </span><span style="color:#a3be8c;">"b"</span><span>: [</span><span style="color:#a3be8c;">"field1"</span><span>: </span><span style="color:#d08770;">4</span><span>.</span><span style="color:#d08770;">0</span><span>] as [String: Any],
</span><span> </span><span style="color:#a3be8c;">"a"</span><span>: [</span><span style="color:#a3be8c;">"field1"</span><span>: </span><span style="color:#d08770;">10</span><span>.</span><span style="color:#d08770;">0</span><span>, </span><span style="color:#a3be8c;">"field2"</span><span>: </span><span style="color:#d08770;">20</span><span>.</span><span style="color:#d08770;">0</span><span>] as [String: Any]
</span><span>]
</span><span>
</span><span style="color:#b48ead;">let</span><span> tokenized = </span><span style="color:#b48ead;">try</span><span> tokenizeExpression(input: </span><span style="color:#a3be8c;">"a.field1 + (a.field2 - b.field1) * 2"</span><span>)
</span><span style="color:#b48ead;">let</span><span> node = </span><span style="color:#b48ead;">try</span><span> parseTokenizedExpression(expr: tokenized)
</span><span style="color:#b48ead;">try</span><span> evaluate(expression: node, context: context)
</span></code></pre>
<p>This was a fun problem to work on. I got to use a very functional approach to a problem I hadn't had to tackle before. I'm sure a lot of this is the kind of thing the first chapters of a compilers course would cover, but it was unfamiliar territory for me.</p>
<p>There's lots of room to expand. In the part about parser combinators I left lots of details uncovered. The evaluator only supports non-optional Doubles. And the expression language is very limited with only four operators and no function calls (spoiler: they're easy to add.) But if you ever need to work on something like this and don't have a modern compiler text book handy, this might get you off to a decent start.</p>
Parsing and evaluating mathematical expressions in Swift / Part 2: Building a syntax tree2020-04-06T15:19:00+00:002020-04-06T15:19:00+00:00Unknownhttps://juripakaste.fi/expressions-2/<p>Ever needed to interpret mathematical expressions with variables, like <code>a.field1 + (a.field2 - b.field1) * 2</code>, in Swift? I did. This series of blog posts will walk you through my solution. This is part 2 of the series:</p>
<ol>
<li><a href="https://juripakaste.fi/expressions-2/expressions-1.html">Tokenization</a></li>
<li>Building a syntax tree</li>
<li><a href="https://juripakaste.fi/expressions-2/expressions-3.html">Evaluating the syntax tree</a></li>
</ol>
<h2 id="parsing-a-list-into-a-syntax-tree">Parsing a list into a syntax tree</h2>
<p>A refresher: we're looking at this expression: <code>a.field1 + (a.field2 - b.field1) * 2</code>. We've now chopped it up into a list of tokens.</p>
<p>What you want next: something that lets you interpret it easily. One option would be a stack, like in <a href="https://en.wikipedia.org/wiki/Forth_(programming_language)">Forth</a> or <a href="https://en.wikipedia.org/wiki/Reverse_Polish_notation">Reverse Polish notation</a>. That would mean transforming the expression to into a Swift array like this: </p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span>[.name([</span><span style="color:#a3be8c;">"a"</span><span>, </span><span style="color:#a3be8c;">"field1"</span><span>]), .name([</span><span style="color:#a3be8c;">"a"</span><span>, </span><span style="color:#a3be8c;">"field2"</span><span>]), .name([</span><span style="color:#a3be8c;">"b"</span><span>, </span><span style="color:#a3be8c;">"field1"</span><span>]), .subtraction, .number(</span><span style="color:#d08770;">2</span><span>), .multiplication, .addition]
</span></code></pre>
<p>That would allow you interpret it with a stack machine, popping operands and operators off the stack one by one. </p>
<p>Another option, and the option I'm going for, is a syntax tree. It's a tree you can walk to evaluate your expression. Make a tree where each operator is a node and its operands subnodes, then when evaluating first evaluate the subnodes before evaluating the node itself. A diagram of the tree would look like this:</p>
<pre data-lang="text" style="background-color:#2b303b;color:#c0c5ce;" class="language-text "><code class="language-text" data-lang="text"><span> ┌───┐
</span><span> ┌───│ + │───┐
</span><span> ▼ └───┘ ▼
</span><span>┌──────────┐ ┌───┐
</span><span>│ a.field1 │ ┌──│ * │──┐
</span><span>└──────────┘ │ └───┘ │
</span><span> ▼ ▼
</span><span> ┌───┐ ┌───┐
</span><span> ┌────│ - │────┐│ 2 │
</span><span> │ └───┘ │└───┘
</span><span> ▼ ▼
</span><span>┌──────────┐ ┌──────────┐
</span><span>│ a.field2 │ │ b.field1 │
</span><span>└──────────┘ └──────────┘
</span></code></pre>
<p>To model that tree in Swift, you'll want a few types. Indirect enums are great for tree nodes:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">enum</span><span> ExpressionOperator {
</span><span> </span><span style="color:#b48ead;">case</span><span> addition
</span><span> </span><span style="color:#b48ead;">case</span><span> subtraction
</span><span> </span><span style="color:#b48ead;">case</span><span> multiplication
</span><span> </span><span style="color:#b48ead;">case</span><span> division
</span><span>}
</span><span>
</span><span style="color:#b48ead;">enum</span><span> ExpressionTreeAtom {
</span><span> </span><span style="color:#b48ead;">case</span><span> number(Double)
</span><span> </span><span style="color:#b48ead;">case</span><span> name([String])
</span><span>}
</span><span>
</span><span>indirect </span><span style="color:#b48ead;">enum</span><span> ExpressionTreeNode {
</span><span> </span><span style="color:#b48ead;">case</span><span> atom(ExpressionTreeAtom)
</span><span> </span><span style="color:#b48ead;">case</span><span> operation(ExpressionOperator, ExpressionTreeNode, ExpressionTreeNode)
</span><span>}
</span></code></pre>
<p>To get from the token list to a data structure built with those types, the thing we have to figure out is operator precedence. There are a bunch of algorithms for solving that. I picked Eli Bendersky's article on <a href="https://eli.thegreenplace.net/2012/08/02/parsing-expressions-by-precedence-climbing">precedence climbing</a> as my guide and was happy with the result. The idea is to define precedence levels (<code>*</code> is solved before <code>+</code>) and associativity (in what order is a list of equal-precedence operators solved) for the operators we support, then recursively handle the tokens. </p>
<p>Let's start by defining precedences and associativities:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">enum</span><span> Associativity: Equatable {
</span><span> </span><span style="color:#b48ead;">case left
</span><span> </span><span style="color:#b48ead;">case right
</span><span>}
</span><span>
</span><span style="color:#b48ead;">struct</span><span> OperatorInfo {
</span><span> </span><span style="color:#b48ead;">let precedence</span><span>: Int
</span><span> </span><span style="color:#b48ead;">let associativity</span><span>: Associativity
</span><span>}
</span><span>
</span><span style="color:#b48ead;">extension</span><span> ExpressionOperator {
</span><span> </span><span style="color:#b48ead;">var</span><span> operatorInfo: OperatorInfo {
</span><span> </span><span style="color:#b48ead;">switch self</span><span> {
</span><span> </span><span style="color:#b48ead;">case </span><span>.addition: </span><span style="color:#b48ead;">return</span><span> OperatorInfo(</span><span style="color:#b48ead;">precedence</span><span>: </span><span style="color:#d08770;">1</span><span>, </span><span style="color:#b48ead;">associativity</span><span>: .</span><span style="color:#b48ead;">left</span><span>)
</span><span> </span><span style="color:#b48ead;">case </span><span>.subtraction: </span><span style="color:#b48ead;">return</span><span> OperatorInfo(</span><span style="color:#b48ead;">precedence</span><span>: </span><span style="color:#d08770;">1</span><span>, </span><span style="color:#b48ead;">associativity</span><span>: .</span><span style="color:#b48ead;">left</span><span>)
</span><span> </span><span style="color:#b48ead;">case </span><span>.multiplication: </span><span style="color:#b48ead;">return</span><span> OperatorInfo(</span><span style="color:#b48ead;">precedence</span><span>: </span><span style="color:#d08770;">2</span><span>, </span><span style="color:#b48ead;">associativity</span><span>: .</span><span style="color:#b48ead;">left</span><span>)
</span><span> </span><span style="color:#b48ead;">case </span><span>.division: </span><span style="color:#b48ead;">return</span><span> OperatorInfo(</span><span style="color:#b48ead;">precedence</span><span>: </span><span style="color:#d08770;">2</span><span>, </span><span style="color:#b48ead;">associativity</span><span>: .</span><span style="color:#b48ead;">left</span><span>)
</span><span> }
</span><span> }
</span><span>}
</span></code></pre>
<p>With those definitions ready, we're ready for the algorithm itself. It's two mutually recursive functions, walking an ArraySlice of the tokens with the space tokens filtered out. <code>computeAtom</code> handles individual atoms, <code>computeExpr</code> compound expressions. When <code>computeAtom</code> runs into an parentheses it calls <code>computeExpr</code>, and <code>computeExpr</code> calls <code>computeAtom</code> to get a value for the first operand and itself to solve the second one. It's a pretty direct translation of Bendersky's example to Swift, with the difference that he computed values while walking the token list, whereas my code creates the syntax tree for evaluation later.</p>
<p>The <code>precedence</code> values define how expressions are divided into subexpressions, and so control how the recursive <code>computeExpr</code> calls consume parts of the expression.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">func </span><span>computeAtom(tokens: </span><span style="color:#b48ead;">inout</span><span> ArraySlice<ExpressionToken>) throws -> ExpressionTreeNode {
</span><span> </span><span style="color:#b48ead;">guard let</span><span> tok = tokens.first </span><span style="color:#b48ead;">else</span><span> {
</span><span> </span><span style="color:#b48ead;">throw</span><span> ExpressionParseError.unexpectedEndOfExpression
</span><span> }
</span><span> </span><span style="color:#b48ead;">switch</span><span> tok {
</span><span> </span><span style="color:#b48ead;">case </span><span>.openParen:
</span><span> tokens = tokens.dropFirst()
</span><span> </span><span style="color:#b48ead;">let</span><span> subNode = </span><span style="color:#b48ead;">try</span><span> computeExpr(tokens: &tokens, minimumPrecedence: </span><span style="color:#d08770;">1</span><span>)
</span><span> </span><span style="color:#b48ead;">guard case </span><span>.closeParen = tokens.first </span><span style="color:#b48ead;">else</span><span> {
</span><span> </span><span style="color:#b48ead;">throw</span><span> ExpressionParseError.unbalancedParentheses
</span><span> }
</span><span> tokens = tokens.dropFirst()
</span><span> </span><span style="color:#b48ead;">return</span><span> subNode
</span><span> </span><span style="color:#b48ead;">case </span><span>.addition,
</span><span> .subtraction,
</span><span> .multiplication,
</span><span> .division:
</span><span> </span><span style="color:#b48ead;">guard let</span><span> op = ExpressionOperator(token: tok) </span><span style="color:#b48ead;">else</span><span> {
</span><span> preconditionFailure(</span><span style="color:#a3be8c;">"Failed to convert token </span><span>\(tok) </span><span style="color:#a3be8c;">to Operator"</span><span>)
</span><span> }
</span><span> </span><span style="color:#b48ead;">throw</span><span> ExpressionParseError.unexpectedOperator(op)
</span><span> </span><span style="color:#b48ead;">case </span><span>.closeParen:
</span><span> </span><span style="color:#b48ead;">throw</span><span> ExpressionParseError.unbalancedParentheses
</span><span> </span><span style="color:#b48ead;">case let </span><span>.number(num):
</span><span> tokens = tokens.dropFirst()
</span><span> </span><span style="color:#b48ead;">return </span><span>.atom(.number(num))
</span><span> </span><span style="color:#b48ead;">case let </span><span>.name(name):
</span><span> tokens = tokens.dropFirst()
</span><span> </span><span style="color:#b48ead;">return </span><span>.atom(.name(name))
</span><span> </span><span style="color:#b48ead;">case </span><span>.space:
</span><span> preconditionFailure(</span><span style="color:#a3be8c;">"Unexpected space token"</span><span>)
</span><span> }
</span><span>}
</span><span>
</span><span style="color:#b48ead;">private func </span><span>computeExpr(
</span><span> tokens: </span><span style="color:#b48ead;">inout</span><span> ArraySlice<ExpressionToken>,
</span><span> minimumPrecedence: Int
</span><span>) throws -> ExpressionTreeNode {
</span><span> </span><span style="color:#b48ead;">var</span><span> node1 = </span><span style="color:#b48ead;">try</span><span> computeAtom(tokens: &tokens)
</span><span> </span><span style="color:#b48ead;">while let</span><span> tok = tokens.first {
</span><span> </span><span style="color:#b48ead;">guard let</span><span> op = ExpressionOperator(token: tok) </span><span style="color:#b48ead;">else</span><span> { </span><span style="color:#b48ead;">break </span><span>}
</span><span> </span><span style="color:#b48ead;">let</span><span> opInfo = op.operatorInfo
</span><span> </span><span style="color:#b48ead;">guard</span><span> opInfo.</span><span style="color:#b48ead;">precedence </span><span>>= minimumPrecedence </span><span style="color:#b48ead;">else</span><span> { </span><span style="color:#b48ead;">break</span><span> }
</span><span> </span><span style="color:#b48ead;">let</span><span> nextMinimumPrecedence = opInfo.</span><span style="color:#b48ead;">precedence </span><span>+ (opInfo.</span><span style="color:#b48ead;">associativity </span><span>== .</span><span style="color:#b48ead;">left </span><span>? </span><span style="color:#d08770;">1 </span><span>: </span><span style="color:#d08770;">0</span><span>)
</span><span> tokens = tokens.dropFirst()
</span><span> </span><span style="color:#b48ead;">let</span><span> node2 = </span><span style="color:#b48ead;">try</span><span> computeExpr(tokens: &tokens, minimumPrecedence: nextMinimumPrecedence)
</span><span> node1 = .operation(op, node1, node2)
</span><span> }
</span><span> </span><span style="color:#b48ead;">return</span><span> node1
</span><span>}
</span></code></pre>
<p>We can kick off the computation by filtering out spaces and converting our tokens array into an ArraySlice.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">func </span><span>parseTokenizedExpression(expr: TokenizedExpression) throws -> ExpressionTreeNode {
</span><span> </span><span style="color:#b48ead;">var</span><span> tokens = expr.tokens.filter { $</span><span style="color:#d08770;">0 </span><span>!= .space }[...]
</span><span> </span><span style="color:#b48ead;">return try</span><span> computeExpr(tokens: &tokens, minimumPrecedence: </span><span style="color:#d08770;">1</span><span>)
</span><span>}
</span></code></pre>
<p>That'll give us this tree:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span>ExpressionTreeNode.operation(
</span><span> .addition,
</span><span> .atom(.name([</span><span style="color:#a3be8c;">"a"</span><span>, </span><span style="color:#a3be8c;">"field1"</span><span>])),
</span><span> .operation(
</span><span> .multiplication,
</span><span> .operation(
</span><span> .subtraction,
</span><span> .atom(.name([</span><span style="color:#a3be8c;">"a"</span><span>, </span><span style="color:#a3be8c;">"field1"</span><span>])),
</span><span> .atom(.name([</span><span style="color:#a3be8c;">"b"</span><span>, </span><span style="color:#a3be8c;">"field2"</span><span>]))
</span><span> ),
</span><span> .atom(.number(</span><span style="color:#d08770;">2</span><span>))
</span><span> )
</span><span>)
</span></code></pre>
<p>Whew. We have now tokenized our string and generated a syntax tree. Only one thing left to do in <a href="https://juripakaste.fi/expressions-2/expressions-3.html">part 3</a>: evaluating the expression so we can get a value out of it.</p>
Parsing and evaluating mathematical expressions in Swift / Part 1: Tokenization2020-04-06T15:18:00+00:002020-04-06T15:18:00+00:00Unknownhttps://juripakaste.fi/expressions-1/<p>Ever needed to interpret mathematical expressions with variables, like <code>a.field1 + (a.field2 - b.field1) * 2</code>, in Swift? I did. This series of blog posts will walk you through my solution. This is part 1 of the series:</p>
<ol>
<li>Tokenization</li>
<li><a href="https://juripakaste.fi/expressions-1/expressions-2.html">Building a syntax tree</a></li>
<li><a href="https://juripakaste.fi/expressions-1/expressions-3.html">Evaluating the syntax tree</a></li>
</ol>
<h2 id="background">Background</h2>
<p>I needed to handle math expressions with a solution that would allow me to parse them in one place and later evaluate the parsed expressions in another place. The second place had to be Swift and the first place could be Swift, so all-Swift it is.</p>
<p>I must lead with a disclaimer: I have no background in interpreters or compilers, so it's possible I'm doing something very unoptimal here. But it does seem to work.</p>
<h2 id="tokenization-with-parser-combinators">Tokenization with parser combinators</h2>
<p>First thing you have to do is break the input expression into tokens. Tokens in the case of the example expression above are <code>a.field1</code>, <code>+</code>, <code>(</code>, <code>a.field2</code>, <code>-</code>, <code>b.field1</code>, <code>)</code>, <code>*</code>, and <code>2</code>.</p>
<p>A nice tool for tokenization is <a href="https://en.wikipedia.org/wiki/Parser_combinator">parser combinators</a>. They're a parsing technique from the functional programming world, focusing on small, composable parser functions. For an in-depth introduction, I heartily recommend <a href="https://pointfree.co">Point-Free</a>'s multi-episode series on them. Some of my code is cribbed straight from their examples, and I'm leaving out some building blocks they had.</p>
<p>Parser combinators consist of two things: simple parser functions that extract data, and other functions that combine those parsers into larger parsers.</p>
<p>A parser function takes in an input and consumes anything at the start of the input it's interested in. It may return the prefix it consumed or just swallow it, and it'll always return rest of the input it didn't care about. A parser that parsed integers would, given the input of <code>"123abc"</code> return <code>123</code> and <code>"abc"</code>. If you fed it only the <code>"abc"</code>, you'd get back <code>nil</code> and still the same <code>"abc"</code>.</p>
<p>In fully generic Swift we'd write the type as <code>(Input) -> (Output?, Input)</code>. We can be a bit more specific here: first of all, we're dealing with strings and can optimize things by using the <code>Substring</code> type. And instead of accepting the <code>Substring</code> and returning it, we can use an <code>inout</code> parameter. Our type will look like this: <code>(inout Substring) -> Output?</code>.</p>
<p>To make the functions easier to deal with, it's useful to wrap them in a struct. Like this:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">public struct</span><span> Parser<A> {
</span><span> </span><span style="color:#b48ead;">public let</span><span> run: (</span><span style="color:#b48ead;">inout</span><span> Substring) -> A?
</span><span>}
</span></code></pre>
<p>A parser that returns the first character of input looks like this:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">let</span><span> char = Parser<Character> { str </span><span style="color:#b48ead;">in
</span><span> </span><span style="color:#b48ead;">guard </span><span>!str.isEmpty </span><span style="color:#b48ead;">else</span><span> { </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">nil</span><span> }
</span><span> </span><span style="color:#b48ead;">return</span><span> str.removeFirst()
</span><span>}
</span></code></pre>
<p>You can imagine plenty of other, similar parsers: ones that parse a fixed string, ones that parse integers, ones that parse decimal numbers, ones that parse dates, etc. The other half of the parser combinator story is the combinators, the thing you use to glue parsers together. The most fundamental ones are things like <code>map</code> (convert the return value to something else), <code>flatMap</code> (convert the return value to another parser) and <code>zip</code> (run multiple parsers in sequence):</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">extension</span><span> Parser {
</span><span> </span><span style="color:#b48ead;">func </span><span>map<B>(_ f: </span><span style="color:#b48ead;">@escaping</span><span> (A) -> B) -> Parser<B> {
</span><span> Parser<B> { str -> B? </span><span style="color:#b48ead;">in
</span><span> </span><span style="color:#b48ead;">self</span><span>.run(&str).map(f)
</span><span> }
</span><span> }
</span><span>
</span><span> </span><span style="color:#b48ead;">func </span><span>flatMap<B>(_ f: </span><span style="color:#b48ead;">@escaping</span><span> (A) -> Parser<B>) -> Parser<B> {
</span><span> Parser<B> { str -> B? </span><span style="color:#b48ead;">in
</span><span> </span><span style="color:#b48ead;">let</span><span> original = str
</span><span> </span><span style="color:#b48ead;">let</span><span> parserB = </span><span style="color:#b48ead;">self</span><span>.run(&str).map(f)
</span><span> </span><span style="color:#b48ead;">guard let</span><span> matchB = parserB?.run(&str) </span><span style="color:#b48ead;">else</span><span> {
</span><span> str = original
</span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">nil
</span><span> }
</span><span> </span><span style="color:#b48ead;">return</span><span> matchB
</span><span> }
</span><span> }
</span><span>}
</span><span>
</span><span style="color:#b48ead;">func </span><span>zip<A, B>(_ a: Parser<A>, _ b: Parser<B>) -> Parser<(A, B)> {
</span><span> Parser<(A, B)> { str -> (A, B)? </span><span style="color:#b48ead;">in
</span><span> </span><span style="color:#b48ead;">let</span><span> original = str
</span><span> </span><span style="color:#b48ead;">guard let</span><span> matchA = a.run(&str) </span><span style="color:#b48ead;">else</span><span> { </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">nil </span><span>}
</span><span> </span><span style="color:#b48ead;">guard let</span><span> matchB = b.run(&str) </span><span style="color:#b48ead;">else</span><span> {
</span><span> str = original
</span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">nil
</span><span> }
</span><span> </span><span style="color:#b48ead;">return</span><span> (matchA, matchB)
</span><span> }
</span><span>}
</span></code></pre>
<p>There's a lot you can build on those. I won't reproduce my complete hierarchy of parsers here, but let's take a closer look at parsing the dotted names. I don't need to get too ambitious with the language; just ASCII and some pretty basic rules. </p>
<p>One additional thing I want to do is break apart those dotted names: they represent hierarchical name spaces so it makes sense to deal with an array of name components instead of one long string.</p>
<p>Let's start with a couple of rules, one for the first character of the name and another for the rest of it:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">func </span><span>isValidNameStart(_ c: Character) -> Bool { c.isASCII && (c.isLetter || c == </span><span style="color:#a3be8c;">"_"</span><span>) }
</span><span style="color:#b48ead;">func </span><span>isValidNameContinuation(_ c: Character) -> Bool { isValidNameStart(c) || (c.isASCII && c.isNumber) }
</span></code></pre>
<p>Then let's define a parser for the first character. Use the <code>char</code> parser defined earlier and <code>flatMap</code> the result with a function that checks if the character from the <code>char</code> parser is valid and if so, return a <code>Parser</code> that returns the character, otherwise return a <code>Parser</code> that returns <code>nil</code>:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">let</span><span> nameStart: Parser<Character> = Parsers.char.flatMap {
</span><span> isValidNameStart($</span><span style="color:#d08770;">0</span><span>) ? Parsers.always($</span><span style="color:#d08770;">0</span><span>) : Parsers.failing()
</span><span>}
</span></code></pre>
<p>Next define a parser for the rest of the name. Use a parser called <code>prefix</code> that eats up 0–<em>n</em> first characters that satisfy a test:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">let</span><span> nameContinuation: Parser<Substring> = Parsers.</span><span style="color:#b48ead;">prefix</span><span>(</span><span style="color:#b48ead;">while</span><span>: isValidNameContinuation)
</span></code></pre>
<p>Then declare combinations: a <code>namePart</code> is <code>nameStart</code> followed by <code>nameContinuation</code>, a <code>dottedNamePart</code> is a period followed by a <code>namePart</code>, a <code>nameTail</code> is zero or more <code>dottedNamePart</code>s, and a complete name is a <code>namePart</code> followed by a <code>nameTail</code>.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">let</span><span> namePart: Parser<String> = zip(nameStart, nameContinuation).map { </span><span style="color:#a3be8c;">"</span><span>\($</span><span style="color:#d08770;">0</span><span>)\($</span><span style="color:#d08770;">1</span><span>)</span><span style="color:#a3be8c;">"</span><span> }
</span><span style="color:#b48ead;">let</span><span> dottedNamePart: Parser<String> = zip(Parsers.literal(</span><span style="color:#a3be8c;">"."</span><span>), namePart).map { $</span><span style="color:#d08770;">1</span><span> }
</span><span style="color:#b48ead;">let</span><span> nameTail: Parser<[String]> = Parsers.zeroOrMore(dottedNamePart, separatedBy: Parsers.always(()))
</span><span>
</span><span style="color:#b48ead;">let</span><span> namePartsParser: Parser<[String]> = zip(namePart, nameTail)
</span><span> .map { [$</span><span style="color:#d08770;">0</span><span>] + $</span><span style="color:#d08770;">1</span><span> }
</span></code></pre>
<p>Then just declare all the things we support and join them up with a couple of other combinators left as an excercise to the reader, <code>oneOf</code> and <code>zeroOrMore</code>.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">let</span><span> numberParser: Parser<ExpressionToken> = Parsers.double.map(ExpressionToken.number)
</span><span style="color:#b48ead;">let</span><span> additionParser: Parser<ExpressionToken> = Parsers.literal(</span><span style="color:#a3be8c;">"+"</span><span>).map { ExpressionToken.addition }
</span><span style="color:#b48ead;">let</span><span> subtractionParser: Parser<ExpressionToken> = Parsers.literal(</span><span style="color:#a3be8c;">"-"</span><span>).map { ExpressionToken.subtraction }
</span><span style="color:#b48ead;">let</span><span> multiplicationParser: Parser<ExpressionToken> = Parsers.literal(</span><span style="color:#a3be8c;">"*"</span><span>).map { ExpressionToken.multiplication }
</span><span style="color:#b48ead;">let</span><span> divisionParser: Parser<ExpressionToken> = Parsers.literal(</span><span style="color:#a3be8c;">"/"</span><span>).map { ExpressionToken.division }
</span><span style="color:#b48ead;">let</span><span> openParenParser: Parser<ExpressionToken> = Parsers.literal(</span><span style="color:#a3be8c;">"("</span><span>).map { ExpressionToken.openParen }
</span><span style="color:#b48ead;">let</span><span> closeParenParser: Parser<ExpressionToken> = Parsers.literal(</span><span style="color:#a3be8c;">")"</span><span>).map { ExpressionToken.closeParen }
</span><span style="color:#b48ead;">let</span><span> spaceParser: Parser<ExpressionToken> = Parsers.oneOrMoreSpaces.map { ExpressionToken.space }
</span><span>
</span><span style="color:#b48ead;">let</span><span> nameParser = namePartsParser.map(ExpressionToken.name)
</span><span>
</span><span style="color:#b48ead;">let</span><span> tokenParser = Parsers.oneOf([
</span><span> numberParser,
</span><span> additionParser,
</span><span> subtractionParser,
</span><span> multiplicationParser,
</span><span> divisionParser,
</span><span> openParenParser,
</span><span> closeParenParser,
</span><span> nameParser,
</span><span> spaceParser,
</span><span>])
</span><span>
</span><span style="color:#b48ead;">let</span><span> tokensParser = Parsers.zeroOrMore(tokenParser, separatedBy: Parsers.always(()))
</span></code></pre>
<p><code>tokensParser</code> is the final product. Run it on an expression and you'll get the a list of tokens out, like this:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span>[
</span><span> name([</span><span style="color:#a3be8c;">"a"</span><span>, </span><span style="color:#a3be8c;">"field1"</span><span>]),
</span><span> space,
</span><span> addition,
</span><span> space,
</span><span> openParen,
</span><span> name([</span><span style="color:#a3be8c;">"a"</span><span>, </span><span style="color:#a3be8c;">"field2"</span><span>]),
</span><span> space,
</span><span> subtraction,
</span><span> space,
</span><span> name([</span><span style="color:#a3be8c;">"b"</span><span>, </span><span style="color:#a3be8c;">"field1"</span><span>]),
</span><span> closeParen,
</span><span> space,
</span><span> multiplication,
</span><span> space,
</span><span> number(</span><span style="color:#d08770;">2</span><span>.</span><span style="color:#d08770;">0</span><span>)
</span><span>]
</span></code></pre>
<p>All right, we're getting somewhere with this. Next up: how to mangle that list into a shape that when evaluated gives us expected results? We'll look into that it <a href="https://juripakaste.fi/expressions-1/expressions-2.html">part 2</a>.</p>
Looser dependencies with Swift2019-07-24T09:51:00+00:002019-07-24T09:51:00+00:00Unknownhttps://juripakaste.fi/dependency-protocols-structs/<p>What kind of types do you use to manage dependencies between objects in Swift? How do you keep your objects that depend on others testable? How do you prevent an addition of a method from causing changes to test code? Couplings should be loose, but how to achieve that?</p>
<h2 id="assumptions">Assumptions</h2>
<p>I’m making a few assumptions in this article:</p>
<ol>
<li>Your software has semi-permanent pieces — let’s call them services — that other parts of your software depend on. Model, networking, database layers, etc.</li>
<li>Your services aren’t laser-focused, perfectly factored little gems that each provide just the right set of methods that everyone using them requires.</li>
<li>You want to test the code that uses those services.</li>
</ol>
<h2 id="everyone-gets-an-interface">Everyone gets an interface</h2>
<p>If you spent time in the enterprisey Java mines of the 90s and 00s like I did, you probably learned that your classes should have separate interfaces. And on the projects I saw, more often than not, there was nothing abstract about the interfaces: it was one interface per class, because most of them didn’t describe behavior, they just were there so it was possible to substitute them with dummies/mocks/fakes/whatevers¹ when writing tests. </p>
<p>In Objective-C land, it’s possible to substitute a class with another because it’s not particularly strongly typed, but that same paradigm has been pretty popular there too, with the difference that it’s called a protocol, not an interface. And the same thing in Swift.</p>
<p>So you have a <code>ProfileService</code>, and it implements <code>ProfileServiceInterface</code>, with the exact same set of public methods. You can agonize over whether it should be called <code>ProfileServiceInterface</code> or just <code>ProfileService</code> and have the implementation carry a more awkward name, if you want to pretend there’s abstraction happening here. Whatever makes you feel good.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">struct</span><span> Profile {
</span><span> </span><span style="color:#b48ead;">let</span><span> name: String
</span><span>}
</span><span>
</span><span style="color:#b48ead;">protocol</span><span> ProfileServiceProtocol {
</span><span> </span><span style="color:#b48ead;">func </span><span>readLoggedInProfile() -> Profile
</span><span>}
</span><span>
</span><span>class ProfileService: ProfileServiceProtocol {
</span><span> </span><span style="color:#b48ead;">func </span><span>readLoggedInProfile() -> Profile {
</span><span> fatalError()
</span><span> }
</span><span>}
</span><span>
</span><span>
</span><span style="color:#b48ead;">class</span><span> ProfileViewModel {
</span><span> </span><span style="color:#b48ead;">private let</span><span> profileService: ProfileServiceProtocol
</span><span>
</span><span> </span><span style="color:#b48ead;">init</span><span>(profileService: ProfileServiceProtocol) {
</span><span> </span><span style="color:#b48ead;">self</span><span>.profileService = profileService
</span><span> }
</span><span>}
</span></code></pre>
<h2 id="square-pegs">Square pegs</h2>
<p>There’s a bunch of problems with the approach I described above, including:</p>
<ol>
<li>It’s arguably a misuse of protocols. Protocols are well suited to things where you have different behaviors hiding behind similar interfaces — collection type hierarchies being a favorite example — and in Swift’s case about code reuse. You aren’t describing behavior here, you’re just copying the implementation declarations.</li>
<li>It cements a supply side view of the interface. Services evolve and rarely manage to stay tightly focused over their lifetimes. Once your interface has more than one method and more than one user, it’s almost certain some users are only going to care about a subset.</li>
<li>Adding things to the protocol causes all implementations of that protocol — of which there can be several in tests — to change.</li>
</ol>
<p>While we ignore that first point, Swift gives us a way to address the other issues: you can declare conformance to protocols outside the implementation. For dependencies, this means that the units that need profiles can each define their own protocol and declare that <code>ProfileService</code> implements it, moving from a supply side definition to a demand side one. Like this:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">class</span><span> ProfileViewModel {
</span><span> </span><span style="color:#b48ead;">private let</span><span> profileService: ProfileViewModelProfileServiceAPI
</span><span>
</span><span> </span><span style="color:#b48ead;">init</span><span>(profileService: ProfileViewModelProfileServiceAPI) {
</span><span> </span><span style="color:#b48ead;">self</span><span>.profileService = profileService
</span><span> }
</span><span>}
</span><span>
</span><span style="color:#b48ead;">protocol</span><span> ProfileViewModelProfileServiceAPI {
</span><span> </span><span style="color:#b48ead;">func </span><span>readLoggedInProfile() -> Profile
</span><span>}
</span><span>
</span><span>extension ProfileService: ProfileViewModelProfileServiceAPI {}
</span></code></pre>
<p>And voilà: as long as the service implements the method described in the protocol, you’re done. You don’t need to worry about implementing the rest of it in tests and you can even use that protocol to evolve <code>ProfileViewModel</code> at a different pace from <code>ProfileService</code>.</p>
<h2 id="less-protocol-more-action">Less protocol, more action</h2>
<p>Really the above, with the depender defining the protocol of its dependency, is not too bad. </p>
<p>There’s a few small downsides:</p>
<ol>
<li>Protocols are top level objects in Swift, at least for now.</li>
<li>Having a protocol means you have to have a type that implements it. That’s fine with the “just protocolify a concrete service” case, but can be a hassle in tests.</li>
<li>Not usually an issue in this case, but protocols can be treacherous in that their behavior radically changes if you decide an associated type would be appropriate.</li>
</ol>
<p>None of these are deal breakers, but there’s a way I find to be nicer: instead of defining a protocol for your dependency, just define a struct that carries closures. Swift's structs are both syntactically and performance-wise extremely lightweight and perfectly suited to the task of providing an insulation layer.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">class</span><span> ProfileViewModel {
</span><span> </span><span style="color:#b48ead;">struct</span><span> ProfileServiceAPI {
</span><span> </span><span style="color:#b48ead;">let</span><span> readLoggedInProfile: () -> Profile
</span><span>
</span><span> </span><span style="color:#b48ead;">init</span><span>(readLoggedInProfile: </span><span style="color:#b48ead;">@escaping</span><span> () -> Profile) {
</span><span> </span><span style="color:#b48ead;">self</span><span>.readLoggedInProfile = readLoggedInProfile
</span><span> }
</span><span> }
</span><span>
</span><span> </span><span style="color:#b48ead;">private let</span><span> profileService: ProfileServiceAPI
</span><span>
</span><span> </span><span style="color:#b48ead;">init</span><span>(profileService: ProfileServiceAPI) {
</span><span> </span><span style="color:#b48ead;">self</span><span>.profileService = profileService
</span><span> }
</span><span>}
</span><span>
</span><span style="color:#b48ead;">extension</span><span> ProfileViewModel.ProfileServiceAPI {
</span><span> </span><span style="color:#b48ead;">init</span><span>(profileService: ProfileService) {
</span><span> </span><span style="color:#b48ead;">self</span><span>.</span><span style="color:#b48ead;">init</span><span>(
</span><span> readLoggedInProfile: profileService.readLoggedInProfile)
</span><span> }
</span><span>}
</span></code></pre>
<p>Now you don’t have an extra top-level protocol and you don’t need a type to implement the protocol with, which can make the code a lot nicer to test: with protocols, more often than not, I find myself writing the equivalent of <code>ProfileViewModel.ProfileServiceAPI</code> in tests anyway, and have it implement the protocol. This cuts out that one extra step. And the struct retains basically all the benefits of the one protocol per depender approach, keeping your dependencies loose and adaptable.</p>
<p>One downside can be that closures don’t have named parameters. If you’re wrapping an API with lots of those, it’s simple enough to add methods to your struct that restore the names:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">class</span><span> ProfileService {
</span><span> </span><span style="color:#b48ead;">func </span><span>readLoggedInProfile() -> Profile {
</span><span> fatalError()
</span><span> }
</span><span>
</span><span> </span><span style="color:#b48ead;">func </span><span>readProfile(ofUser name: String) -> Profile? {
</span><span> fatalError()
</span><span> }
</span><span>}
</span><span>
</span><span style="color:#b48ead;">class</span><span> OtherUserViewModel {
</span><span> </span><span style="color:#b48ead;">struct</span><span> ProfileServiceAPI {
</span><span> </span><span style="color:#b48ead;">private let</span><span> readUserProfile: (String) -> Profile?
</span><span>
</span><span> </span><span style="color:#b48ead;">init</span><span>(readUserProfile: </span><span style="color:#b48ead;">@escaping</span><span> (String) -> Profile?) {
</span><span> </span><span style="color:#b48ead;">self</span><span>.readUserProfile = readUserProfile
</span><span> }
</span><span>
</span><span> </span><span style="color:#b48ead;">func </span><span>readProfile(ofUser name: String) -> Profile? {
</span><span> </span><span style="color:#b48ead;">return self</span><span>.readUserProfile(name)
</span><span> }
</span><span> }
</span><span>
</span><span> </span><span style="color:#b48ead;">private let</span><span> profileService: ProfileServiceAPI
</span><span>
</span><span> </span><span style="color:#b48ead;">init</span><span>(profileService: ProfileServiceAPI) {
</span><span> </span><span style="color:#b48ead;">self</span><span>.profileService = profileService
</span><span> }
</span><span>}
</span><span>
</span><span style="color:#b48ead;">extension</span><span> OtherUserViewModel.ProfileServiceAPI {
</span><span> </span><span style="color:#b48ead;">init</span><span>(profileService: ProfileService) {
</span><span> </span><span style="color:#b48ead;">self</span><span>.</span><span style="color:#b48ead;">init</span><span>(
</span><span> readUserProfile: profileService.readProfile(ofUser:))
</span><span> }
</span><span>}
</span></code></pre>
<p>You could continue one step further: in some cases it may make sense to forego that struct completely and instead just inject the closures directly into the class using them. But having the struct there may make it easier to see afterwards where they're coming from.</p>
<p>Over the course of these changes we've made the view model depend only on the signatures of the functions it requires, regardless of who implements them and what else they implement. It's now easier to test and to evolve. None of this will prevent you from tying your codebase and yourself into knots with a badly designed dependency graph, but if should help at least locally.</p>
<h3 id="footnotes">Footnotes</h3>
<ol>
<li>If you remember the difference between a dummy, a mock and a fake you’re better, or more thoroughly brainwashed, than me. Take your pick.</li>
</ol>
GraphQL with Swift2019-06-02T18:44:00+00:002019-06-02T18:44:00+00:00Unknownhttps://juripakaste.fi/graphqler/<p>Know that thing where you start writing a tool for something, discover it needs (for some values of “need”) something slightly complicated, you decide to write a library for said complicated thing, then discover you don’t need the tool you were working on in the first place but decide to write the library anyway?</p>
<p>So I wrote <a href="https://github.com/juri/graphqler">GraphQLer</a> (pronounced “graph quiller”), a library for generating GraphQL from Swift. It’s not the only Swift GraphQL library around, but I believe it’s far simpler — mostly by being far less ambitious — than the other solutions. Throwing in the kitchen sink seems to be popular in this space, forcing you to shape your code around the libraries. That may work out great in some situations, but it wasn’t what I wanted.</p>
<p>GraphQLer just defines a bunch of data types that model the things found in <a href="https://graphql.github.io/graphql-spec/June2018/">the spec</a>, and functions for turning those data types into strings. Its only purpose is to make it easier to write syntactically valid GraphQL documents. It has no dependencies — it doesn’t even import Foundation — and you handle all networking and responses as you see fit.</p>
<p>It works with the Swift Package Manager. There’s also a Xcode workspace and a playground, but you need to run SPM to get an Xcode project. I'll add an Xcode project if I have to to make it work with Carthage, but the WWDC 2019 keynote is tomorrow and I'm hoping there will be surprises related to SPM.</p>
<p>PS. A shout out to <a href="https://github.com/realm/jazzy">jazzy</a> and <a href="https://pages.github.com">GitHub Pages</a>: it's ridiculous how easy it is to publish <a href="https://juri.github.io/graphqler/">documentation</a> for a Swift project these days.</p>
annotate-git-commit2018-11-16T18:29:00+00:002018-11-16T18:29:00+00:00Unknownhttps://juripakaste.fi/annotate-git-commit/<p>I got into the feature branch workflow back when I was using Mercurial. I’ve since discovered that Mercurial’s branches aren’t really suitable for short-lived branches and switched to Git for a variety of reasons, but I still wish my commits carried with them metadata about what they were related to.</p>
<p>I also still wish my VCS had a sane UI, but that’s an unrelated topic.</p>
<p>Anyway, my Git branches, when I’m working with a ticketing system, usually carry in their name a ticket identifier in an easily parseable format. I wrote a small too for extracting that ticket identifier and pasting it to the bottom of my commit messages: <a href="https://github.com/juri/annotate-git-commit">annotate-git-commit</a>. You can use it with Git’s <code>prepare-commit-msg</code> hook.</p>
TODO.swift2018-09-01T10:35:00+00:002018-09-01T10:35:00+00:00Unknownhttps://juripakaste.fi/todo-swift/<p>I saw this Swift trick a few years ago on Twitter. I can’t recall who it was, sorry. It sounded way too clever at first, but after a while I tried and decided I love it. Now it’s one of those things I add to every project.</p>
<p>Sometimes you just want to get on with declaring your functions without worrying about the actual implementations. A small enough function with good enough name and types is pretty much done. Filling out the body is just something that will distract you and make you lose the big picture by bogging you down in the details.</p>
<p>The problem with incomplete functions is that they make the type checker unhappy. Have a return type but no return statement? That’s a compilation error. And the more interesting your types, the harder it is to make a temporary hack to placate the compiler.</p>
<p>Swift offers an escape hatch: the <code>Never</code> type. Call a function that returns <code>Never</code> and you don’t need to worry about writing a return. So this will compile:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">func </span><span>foo(zap: Bar) -> Fnord {
</span><span> fatalError(</span><span style="color:#a3be8c;">"Unimplemented"</span><span>)
</span><span>}
</span></code></pre>
<p>The code will compile, it will just crash at run time. Exactly what we wanted. But we can do better.</p>
<p>You can write your own function that returns <code>Never</code> as long as you ensure it never returns. Like, for example, by calling <code>fatalError</code> as we did in the previous example:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">func </span><span>TODO() -> Never { fatalError(</span><span style="color:#a3be8c;">"Unimplemented"</span><span>) }
</span><span>
</span><span style="color:#b48ead;">func </span><span>foo(zap: Bar) -> Fnord {
</span><span> TODO()
</span><span>}
</span></code></pre>
<p>That’s an clear improvement: less need for repeating yourself, and easier to find in the code. The next small step will be to ditch the parentheses by switching to a computed property:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">var</span><span> TODO: Never { fatalError(</span><span style="color:#a3be8c;">"Unimplemented"</span><span>) }
</span><span>
</span><span style="color:#b48ead;">func </span><span>foo(zap: Bar) -> Fnord {
</span><span> TODO
</span><span>}
</span></code></pre>
<p>Not a huge improvement, but better, anyway.</p>
<p>There’s one more thing we can do: we can make it harder to forget these in the code. <code>TODO</code> doesn’t cause errors, but we can make it cause a warning by abusing Swift’s deprecation annotation. And because you care for the quality of your code base you fix all warnings and so can’t help but notice the new ones when you compile.</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span style="color:#b48ead;">@available</span><span>(*, deprecated, message: </span><span style="color:#a3be8c;">"Unimplemented"</span><span>)
</span><span style="color:#b48ead;">var</span><span> TODO: Never { fatalError(</span><span style="color:#a3be8c;">"Unimplemented"</span><span>) }
</span><span>
</span><span style="color:#b48ead;">func </span><span>foo(zap: Bar) -> Fnord {
</span><span> TODO
</span><span>}
</span></code></pre>
<p>Now the <code>TODO</code> line inside <code>Foo</code> will cause a warning:</p>
<blockquote>
<p>'TODO' is deprecated: Not implemented</p>
</blockquote>
<p>You’ll get a nice warning reminding you to implement the function before merging your code in, but you can do it when you’re ready.</p>
Converting UNIX dates to a readable format on Mac2018-02-15T19:03:00+00:002018-02-15T19:03:00+00:00Unknownhttps://juripakaste.fi/convert-unix-date/<p>I tend to run into UNIX dates — large numbers representing points in time as seconds since the first second of 1970, something like 1500000000 for 2017-07-14 05:40:00 — all the time in my work, dealing with servers, web APIs, etc.</p>
<p>Here's how to make them readable on a Mac. All text below assumes macOS 10.13.</p>
<h2 id="terminal">Terminal</h2>
<p>The most straightforward solution is probably to open Terminal.app and run <code>date</code>:</p>
<pre data-lang="sh" style="background-color:#2b303b;color:#c0c5ce;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="color:#65737e;"># Outputs Fri Jul 14 05:40:00 EEST 2017
</span><span style="color:#bf616a;">date -r</span><span> 1500000000
</span><span>
</span><span style="color:#65737e;"># Outputs 2017-07-14 05:40:00
</span><span style="color:#bf616a;">date -r</span><span> 1500000000 +'</span><span style="color:#a3be8c;">%F %T</span><span>'
</span></code></pre>
<h2 id="services-shell-applescript">Services: shell + AppleScript</h2>
<p>Command line isn't your only option. macOS has a Services menu where you can add all sorts of useful things that operate on your selection:</p>
<p><img src="https://juripakaste.fi/convert-unix-date/services-menu.png" alt="Services menu screenshot" /></p>
<p>You can create a Service menu item pretty easily from the <code>date</code> invocation above.</p>
<ol>
<li>
<p>Launch Automator</p>
</li>
<li>
<p>Create a new document. Select Service when Automator prompts for a type.</p>
</li>
<li>
<p>Add "Run Shell Script" action. Select <code>/bin/sh</code> as the shell. Set the following as the content:</p>
<pre data-lang="sh" style="background-color:#2b303b;color:#c0c5ce;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="color:#bf616a;">xargs -I</span><span> INPUT date</span><span style="color:#bf616a;"> -r</span><span> INPUT +'</span><span style="color:#a3be8c;">%F %T</span><span>'
</span></code></pre>
</li>
<li>
<p>Add a "Run AppleScript" action. Set the following as the content:</p>
<pre data-lang="applescript" style="background-color:#2b303b;color:#c0c5ce;" class="language-applescript "><code class="language-applescript" data-lang="applescript"></code></pre>
</li>
</ol>
<p>on run {input, parameters}
tell me to display dialog input with title "Date" buttons "OK"
return input
end run
```
5. Save the document, select text anywhere, select the <Program Name> → Services → <Your Automator document name> menu item, and you'll get a dialog box with the converted date.</p>
<h2 id="services-javascript">Services: Javascript</h2>
<p>Starting a shell, invoking (with no input validation) a couple of command line tools, then piping that to AppleScript is not very satisfactory.</p>
<p>You might be able to eliminate the first step and do all of it in AppleScript. But that would mean writing more AppleScript. I prefer to avoid that.</p>
<p>macOS offers another option: JavaScript for Automation, also known as JXA. Not exactly the greatest of languages either, but it's not as read only as AppleScript. And while its standard library is not great, it can handle dates. So we can reduce our Automator script to one action and improve our input validation at the same time.</p>
<ol>
<li>
<p>Start by again creating a new service in Automator.</p>
</li>
<li>
<p>Add a "Run JavaScript" action. Set the following as content:</p>
<pre data-lang="javascript" style="background-color:#2b303b;color:#c0c5ce;" class="language-javascript "><code class="language-javascript" data-lang="javascript"><span style="color:#b48ead;">function </span><span style="color:#8fa1b3;">run</span><span>(</span><span style="color:#bf616a;">input</span><span>, </span><span style="color:#bf616a;">parameters</span><span>) {
</span><span> </span><span style="color:#b48ead;">const </span><span style="color:#bf616a;">app </span><span>= </span><span style="color:#bf616a;">Application</span><span>.</span><span style="color:#8fa1b3;">currentApplication</span><span>();
</span><span> </span><span style="color:#bf616a;">app</span><span>.</span><span style="color:#bf616a;">includeStandardAdditions </span><span>= </span><span style="color:#d08770;">true</span><span>;
</span><span>
</span><span> </span><span style="color:#b48ead;">const </span><span style="color:#bf616a;">inputInt </span><span>= </span><span style="color:#96b5b4;">parseInt</span><span>(</span><span style="color:#bf616a;">input</span><span>);
</span><span> </span><span style="color:#b48ead;">if </span><span>(</span><span style="color:#96b5b4;">isNaN</span><span>(</span><span style="color:#bf616a;">inputInt</span><span>)) {
</span><span> </span><span style="color:#bf616a;">app</span><span>.</span><span style="color:#8fa1b3;">displayAlert</span><span>("</span><span style="color:#a3be8c;">Not an integer: </span><span>" + </span><span style="color:#bf616a;">input</span><span>);
</span><span> </span><span style="color:#b48ead;">return </span><span style="color:#bf616a;">input</span><span>;
</span><span> }
</span><span>
</span><span> </span><span style="color:#b48ead;">const </span><span style="color:#bf616a;">d </span><span>= new Date(</span><span style="color:#bf616a;">inputInt </span><span>* </span><span style="color:#d08770;">1000</span><span>);
</span><span>
</span><span> </span><span style="color:#bf616a;">app</span><span>.</span><span style="color:#8fa1b3;">displayDialog</span><span>(</span><span style="color:#bf616a;">d</span><span>.</span><span style="color:#8fa1b3;">toISOString</span><span>(), {
</span><span> withTitle: "</span><span style="color:#a3be8c;">Date</span><span>",
</span><span> buttons: ["</span><span style="color:#a3be8c;">Close</span><span>"],
</span><span> defaultButton: "</span><span style="color:#a3be8c;">Close</span><span>",
</span><span> });
</span><span>
</span><span> </span><span style="color:#b48ead;">return </span><span style="color:#bf616a;">input</span><span>;
</span><span>}
</span></code></pre>
</li>
<li>
<p>Now again save the document. The new service is available in the Services menu.</p>
</li>
</ol>
<p>Here's the window you get:</p>
<p><img src="https://juripakaste.fi/convert-unix-date/date-dialog.png" alt="Date dialog screenshot" /></p>
<p>The output from <code>toISOString</code> isn't quite as nice as <code>%F %T</code> from <code>date</code>, but if you're used to ISO dates, it's not bad. Making it prettier is left as an exercise to the reader. Start with Mozilla's documentation for <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date">Date</a>.</p>
<p>There you have it! Maybe some day we can use Swift and the Foundation libraries for automation, but meanwhile I can live with the JS solution.</p>
Netstrings for Swift2017-05-25T08:30:00+00:002017-05-25T08:30:00+00:00Unknownhttps://juripakaste.fi/swift-netstring/<p>I published another small Swift library: <a href="https://github.com/juri/swift-netstring">swift-netstring</a> implements reader and writer for D. J. Bernstein's <a href="https://cr.yp.to/proto/netstrings.txt">Netstrings</a> format in Swift.</p>
<p>Netstrings is a specification for length-prefixed and delimited byte strings, useful mostly as a low-level building block for network protocols. With swift-netsring you can parse incoming data from a socket or some other stream by wrapping the stream in a simple closure that reads bytes synchronously. Or you can feed it an array of bytes.</p>
<p>You can find examples in the repo in the README, in a Swift playground and in tests. There's also <a href="https://juri.github.io/swift-netstring/">API documentation</a> (a big thank you to Jesse Squires for <a href="http://www.jessesquires.com/swift-documentation/">an excellent guide</a> for setting up docs on GitHub pages for a Swift project.)</p>
Flue: Fluent API for value extraction and conversion for Swift2016-10-02T17:35:00+00:002016-10-02T17:35:00+00:00Unknownhttps://juripakaste.fi/flue/<p><a href="http://github.com/juri/flue">Flue</a> is a Swift (3.0, as of this writing) library for extracting, validating and converting values from user input. It tries to do this with a fluent interface:</p>
<pre data-lang="swift" style="background-color:#2b303b;color:#c0c5ce;" class="language-swift "><code class="language-swift" data-lang="swift"><span>vp.extract(</span><span style="color:#a3be8c;">"100"</span><span>).maxLength(</span><span style="color:#d08770;">6</span><span>).regexp(</span><span style="color:#a3be8c;">"1.*"</span><span>)!.asInt().</span><span style="color:#b48ead;">required</span><span>()
</span></code></pre>
<p>It can output readable help and error messages from your conversions. See the <a href="https://github.com/juri/flue/blob/master/README.md">README</a> for details.</p>
<p>Enjoy.</p>
Talking to servers: WebSockets2013-06-06T18:37:00+00:002013-06-06T18:37:00+00:00Unknownhttps://juripakaste.fi/ios-websockets-clojure/<p>This is a sort of follow-up post to <a href="https://juripakaste.fi/zeromq-ios-python/">ZeroMQ, iOS and
Python</a> from a year
ago. I again wrote a test app and server. Of the earlier components, iOS
stayed, but ZeroMQ I replaced with WebSockets and the server is this time
written in Clojure.</p>
<p>Idea of the exercise is the same as last time: two-way communication between
an iOS client and server over a persistent connection. WebSockets is a more
mainstream technology than ZeroMQ with a wide variety of servers available,
even if it is a young spec and not quite everything has stabilized yet.</p>
<p>On the iOS side there's two options, short of writing the protocol
implementation yourself: some kind of horrible Javascript-UIWebView bridge and
Square's <a href="https://github.com/square/SocketRocket">SocketRocket</a>. I went for
SocketRocket. It's available from <a href="http://cocoapods.org">CocoaPods</a> and at
least in this brief test worked fine.</p>
<p>Last time I was planning on doing the server in Clojure but ran out of
patience. This time the technology stack was easier. I pretty much followed
<a href="http://blog.jayfields.com/2011/02/clojure-web-socket-introduction.html">Jay Fields' example</a> with just a few changes here and there and had a server
running in no time.</p>
<p>The system doesn't do anything fancy, or do it particularly beautifully: the
app waits for a button tap, sends a number wrapped in JSON to the server, the
server increments it and sends it back.</p>
<p>The <a href="https://bitbucket.org/juri/socketrockettest">code</a> is on BitBucket as
always. Enjoy.</p>
NSStringinfying enums from AppCode2013-04-27T19:03:00+00:002013-04-27T19:03:00+00:00Unknownhttps://juripakaste.fi/nsstringfromenum-appcode/<p>I added a <code>--text</code> option to <a href="https://bitbucket.org/juri/nsstringgen/">nsstringfromenumgen</a> so it can be used with JetBrains' excellent Objective-C IDE, <a href="http://www.jetbrains.com/objc/">AppCode</a>. AppCode doesn't integrate with OS X services, but you can configure nsstringfromenumgen as an "External Tool" and provide the selected text as a parameter. There's still an extra copy & paste step not needed with Xcode and Services, but it's not too bad.</p>
NSStringinfying enums2012-12-29T10:18:00+00:002012-12-29T10:18:00+00:00Unknownhttps://juripakaste.fi/nsstringify/<p>Making sense of enum values in C is too difficult. Usually sooner rather than later while debugging you need to see what's inside an enum value, but C provides no introspection tools. You just get an integer value, no mapping to the original symbolic name. That's why every enum should be accompanied by a stringifying function, but that requires manual labor. Few programmers are happy to write those functions and fewer are going to update them. Automation to the rescue.</p>
<p>I'm working in the context of Objective-C, and so is Wolf Rentzsch, who <a href="http://rentzsch.tumblr.com/post/37512716957/enum-nsstring">recently blogged</a> his script for generating NSStringFromEnumName functions. I have used a very similar solution for a long time, and I expect so have many others.</p>
<p>But you don't have to be <a href="http://regex.info/blog/2006-09-15/247">jwz</a> to doubt the wisdom of parsing C with a bunch of ad hoc regexps. I was always unhappy with it and knew there were various not-so-corner cases I wasn't handling properly. Using a real C parser seemed like the right thing to do, but it hasn't really been feasible until pretty recently, after the advent of <a href="http://clang.llvm.org/doxygen/group__CINDEX.html">libclang</a>.</p>
<p>So, here it is, using Python and clang, <a href="https://bitbucket.org/juri/nsstringgen/">my most recent take on NSStringFromEnum</a>. It might not be any <em>better</em> for being based on libclang, but it's <em>different</em>.</p>
ZeroMQ, iOS and Python2012-05-01T14:35:00+00:002012-05-01T14:35:00+00:00Unknownhttps://juripakaste.fi/zeromq-ios-python/<p>I wrote some example code for you.</p>
<p>Background: last week a coworker asked me what's the flavor <em>du jour</em> in two-way communication between a network server and an iOS app, should he just go with BSD sockets or is there something better? I suggested he should take a look at <a href="http://www.zeromq.org/">ZeroMQ</a>. Not that I actually knew very much at all about it, but I had heard the name and seen that some people were pretty enthustiastic about it.</p>
<p>Turns out he had deadlines and wasn't too keen on building the library with an outdated zsh script that didn't seem to work.</p>
<p>I started fiddling with it myself on the weekend to have some idea about what I'm talking about. True enough that the script provided on the web page wasn't up to date, and once I fixed it I got only an ARM binary which isn't very useful in development when you want to test on the iPhone Simulator. After some more poking I finally managed to massage a working fat library out of it. And <a href="https://github.com/jeremy-w/objc-zmq">objc-zmq</a> provided a nice simple Objective-C wrapper around the C API.</p>
<p>Not satisfied with playing with just one unknown technology, I thought I'd do the server in Clojure. After shaving the JNI yak for a couple of hours, trying to build a working version if the Java binding, I decided it wasn't worth it. Python was an easy fallback, but to keep things at least moderately exciting I used <a href="http://www.gevent.org/">gevent</a> which was also something I hadn't tried before.</p>
<p>It worked out pretty nice. The Python server is extremely simple and the iOS app isn't complex either. This hardly counts as a strenuous test of ZeroMQ, but at least from the code perspective it was really pleasant to work with. No hassle with buffers, just complete messages from an extremely simple API that enables the most common communication patterns with a couple of keywords. They wisely keep out of the marshalling business, telling people to use Protocol Buffers or something else for defining message formats.</p>
<p>The <a href="https://bitbucket.org/juri/zeromq-ios-py">example project</a> is available on Bitbucket. It works on my computer at the moment; see the Bitbucket page for details about the bits it requires.</p>
<p>Enjoy.</p>
Unit testing Cocoa code with MacRuby2010-11-30T20:45:00+00:002010-11-30T20:45:00+00:00Unknownhttps://juripakaste.fi/rcrunner/<p>Announcing <a href="https://bitbucket.org/juri/rcrunner/wiki/Home">RCRunner</a>, a GUI test runner for <a href="http://www.macruby.org/">MacRuby</a> and Cocoa.</p>
<p>Cocoa unit testing can be a pain. In addition to the usual difficulties of writing tests for user interface heavy code, the Apple sanctioned solution, SenTestingKit, can isn't the greatest testing framework around and the default way of using it rules out debugger.</p>
<p><a href="https://github.com/gabriel/gh-unit">GHUnit</a> helps somewhat. It's a GUI test runner with additional testing methods. However, with it you are still writing your tests in Objective-C. On the plus side it's the same language you're probably writing your app in. On the minus side Objective-C can be verbose and sometimes, especially when writing test code, brevity would be welcome.</p>
<p>Enter MacRuby. You get the conciseness of Ruby with full access to your Objective-C classes. And Ruby probably has the greatest density of testing frameworks per active programmer among all the languages in popular use today.</p>
<p>There's a nice article about <a href="http://www.macruby.org/recipes/tdd-in-objective-c-with-macruby.html">TDD, Objective-C and MacRuby</a> on the MacRuby site. However, the approach taken in it still uses a Xcode build phase script to accomplish testing. That makes debugging hard and you have to hunt through the build logs for your errors.</p>
<p>RCRunner is a separate GUI program you run. You tell it names of Ruby modules and it uses any test cases it finds[1]. You can breakpoint your code and thanks to Ruby, reload the test code. You can inspect errors and log output test by test.</p>
<p>Enjoy.</p>
<p>[1] At the moment it supports only MiniTest but adding support for other frameworks isn't difficult.</p>
Using Firefox as Flash playing Safari fallback2010-11-10T18:11:00+00:002010-11-10T18:11:00+00:00Unknownhttps://juripakaste.fi/safari-flash-firefox-fallback/<p>If you want to go <a href="http://daringfireball.net/2010/11/flash_free_and_cheating_with_google_chrome">Flashless</a> on Mac and Safari, it's possible to use Firefox as a fallback, too, not just Chrome. While Firefox does load Plugins from <code>/Library/Internet Plug-Ins</code> and <code>~/Library/Internet Plug-Ins</code>, it looks in other places too. I just tested and it seems to work fine from <code>~/Library/Application Support/Firefox/Profiles/<profile name>/plugins</code> and I suspect <code>/Applications/Firefox.app/Contents/MacOS/plugins</code> would work too.</p>
<p>So you can copy <code>Flash Player.plugin</code>, <code>NP-PPC-Dir-Shockwave</code> and <code>flashplayer.xpt</code> to one of the Firefox specific folders and launch Firefox from Safari's Developer menu. Chrome starts up faster, though.</p>
xibgraph: Interface Builder overviews2010-06-19T19:17:00+00:002010-06-19T19:17:00+00:00Unknownhttps://juripakaste.fi/xibgraph/<p>When putting together user interfaces with Interface Builder, you connect things together with bindings, actions and outlets and it's good. Understanding the result later on is a completely different matter. It can be time consuming and difficult to browse the objects inside one by one, trying to comprehend the whole. Even more so if you're trying to read someone else's work.</p>
<p>Out of that frustration came <a href="http://bitbucket.org/juri/xibgraph/">xibgraph</a>. It takes a XIB file and outputs the connections contained inside:</p>
<p><img src="https://juripakaste.fi/xibgraph/xibgraph1.png" alt="Example xibgraph output" /></p>
<p>At the moment it supports bindings and actions. Outlets are next.</p>
<p>xibgraph supports a couple of different output formats. JSON is supported out of the box and if you install <a href="http://code.google.com/p/pydot/">pydot</a>, you get DOT, the format understood by <a href="http://www.graphviz.org/">Graphviz</a> and <a href="http://www.omnigroup.com/products/omnigraffle/">OmniGraffle</a> too.</p>
<p>It hasn't been tested on a particularly wide variety of XIBs, so it's very plausible it will produce wonky results or just outright refuse to work with your files. If so, patches and bug reports are welcome.</p>
<p>xibgraph is MIT licensed and written in Python. It requires PyObjC (it seemed like the easiest way to get XPath support on OS X) and probably Python 2.6. Everything but the DOT support should work without additional requirements on OS X 10.6.</p>
hg-status-sections2010-06-04T20:14:00+00:002010-06-04T20:14:00+00:00Unknownhttps://juripakaste.fi/hg-status-sections/<p>I usually use <a href="http://bitbucket.org/snej/murky/wiki/Home">Murky</a>, <a href="http://www.xsteve.at/prg/emacs_dvc/dvc.html">dvc</a> or some other shell for Mercurial. Not always though, for various reasons, and when running <code>hg status</code> I'm always frustrated when copy and pasting file names. <code>bzr</code> provides neat, non-cluttered lines that can be copied whole to get a file name without a hassle, but <code>hg</code> takes the traditional one-character prefix approach to status display and as a result makes you manually select a part of a line instead of just grabbing a whole line.</p>
<p>So I wrote a small extension to help. Meet <a href="http://bitbucket.org/juri/hg-status-sections/src">hg-status-sections</a>.</p>
A better @synthesize2010-05-22T14:03:00+00:002010-05-22T14:03:00+00:00Unknownhttps://juripakaste.fi/better-synthesize/<p>The biggest problem with Objective-C's <code>@synthesize</code> directive for <a href="http://developer.apple.com/mac/library/documentation/cocoa/conceptual/objectivec/articles/ocProperties.html">properties</a> is how difficult it's to augment the synthesized code. You often need to add logic to a property setter, but while you're adding it, you're losing the probably correct implementation Apple's code creates for property flags like <code>atomic</code> and <code>retain</code>.</p>
<p>At the moment, when you synthesize a <code>readwrite</code> property called <code>foo</code> you get a setter method called <code>setFoo</code>. If you need to add logic around it, you can either store the value in a private property and add a public property with a different name or use a subclass. Both are a bit of a hassle. Usually I just end up writing my own method, including the logic for implementing the modifiers correctly. </p>
<p>In an ideal world the language would support something like around/before/after methods in <a href="http://en.wikipedia.org/wiki/Common_Lisp_Object_System">CLOS</a>, but those features are rare. There's a simple way <code>@synthesize</code> could make things easier without requiring massive changes to the runtime. It could give you for both the getter and setter two methods. There would be the public methods they create in the current implementation, but there'd also be methods with names like <code>__synthesized_property</code> and <code>__synthesized_setProperty</code>. The public methods would rely on the semi-private methods to actually implement all their logic. Then if you needed to add logic around the accessors, you could override the public methods and call the semi-private ones to get access to the synthesized accessor logic without jumping through hoops or risking getting the implementation wrong.</p>
Block indentation in Emacs2010-03-08T19:28:00+00:002010-03-08T19:28:00+00:00Unknownhttps://juripakaste.fi/emacs-block-indent/<p>There are several small things Emacs could be doing to make it nicer to write code. One I was missing was making it possible to go with one press of the return key between braces in a C derived language from this:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>if (test) { }
</span></code></pre>
<p>to this:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>if (test) {
</span><span> <-- insertion point here
</span><span>}
</span></code></pre>
<p>That is, pressing return before the closing parentheses, brace or bracket should move the closing character two lines down, indent it properly, and move the insertion point to the new empty line in the middle and indent it property. The way TextMate does it.</p>
<p>Here are a few of elisp functions to accomplish this:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(defun char-isws (c)
</span><span> "Is character c a whitespace character?"
</span><span> (or (char-equal c ?\ )
</span><span> (char-equal c ?\t)
</span><span> (char-equal c ?\n)))
</span><span>
</span><span>(defun line-next-non-ws ()
</span><span> "Return the next non-whitespace character on the current line or nil."
</span><span> (let ((cc (char-after)))
</span><span> (if (and cc (char-isws cc))
</span><span> (save-excursion
</span><span> (if (re-search-forward "[^[:space:]]" (save-excursion (end-of-line) (point)) t)
</span><span> (char-before)
</span><span> nil))
</span><span> cc)))
</span><span>
</span><span>(defun newline-and-indent-extra-for-closing-paren ()
</span><span> "Insert a newline and indent. If the next non-whitespace character is a closing paren, insert two newlines and indent the two new lines correctly, placing the point on the first of the two new lines."
</span><span> (interactive)
</span><span> (let ((nc (line-next-non-ws)))
</span><span> (if (or (null nc)
</span><span> (not (= (char-syntax nc) ?\))))
</span><span> (newline-and-indent)
</span><span> (progn
</span><span> (just-one-space)
</span><span> (newline-and-indent)
</span><span> (newline-and-indent)
</span><span> (previous-line)
</span><span> (indent-for-tab-command)))))
</span></code></pre>
<p>Now you can bind return to <code>newline-and-indent-extra-for-closing-paren</code> in a suitable language keymap. I've been using this with <code>scala-mode</code> and it works well there.</p>
Exporting geolocation data from iPhoto with AppleScript2009-07-29T22:04:00+00:002009-07-29T22:04:00+00:00Unknownhttps://juripakaste.fi/iphoto-geodata-applescript/<p>I recently transferred all my photos to iPhoto. I <a href="http://www.flickr.com/photos/juripakaste/">share</a> them on Flickr, but I've been unhappy with iPhoto's built-in Flickr support — it has an arbitrary 500 photo limit on web album size, it's crashy, it does weird synchronizations that take ages when combined with lots of large photos and a slow internet connection, it has multiple times failed to send all the full-resolution images — so I've been exploring alternatives. There's at least <a href="http://connectedflow.com/flickrexport/">FlickrExport</a> and Flickr's own <a href="http://www.flickr.com/tools/uploadr/">Uploadr</a>.</p>
<p>Although the tools work, there's a downside compared to iPhoto's built-in Flickr support. iPhoto has wonderful geotagging support, as does Flickr, but iPhoto doesn't write the data to EXIF tags and that poses a problem for the tools. Uploadr reads just the files and so never sees the data, and apparently iPhoto doesn't provide the data to FlickrExport, either. The result is that Flickr won't know the locations of the photos.</p>
<p>There's a way to work around this problem. The solution is AppleScript. iPhoto exports <code>photo</code> objects that can tell you their location as set inside iPhoto. The downside to this approach is that it's AppleScript, but apparently the alternatives like Python or JSTalk aren't quite up to tasks like these without application support.</p>
<p>This script will write the locations of the selected photos relies on <a href="http://www.sno.phy.queensu.ca/~phil/exiftool/">ExifTool</a>. It will litter your photo directory with files ending with <code>_original</code> that should contain the unmodified images. You should make sure the modified files are ok before deleting the originals. The usual caveats apply: I'm no AppleScript expert and this has not been tested particularly rigorously. I'd be careful especially if you don't live in the NE hemisphere. And you might want to reduce the number of dialogs. Do what you want with it.</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>-- This applescript will geotag the selected photos with the
</span><span>-- location information set in iPhoto.
</span><span>--
</span><span>-- You must have exiftool installed; by default it's loaded from
</span><span>-- /opt/local/bin, where MacPorts installs it from the package
</span><span>-- p5-image-exiftool.
</span><span>--
</span><span>-- Author: Juri Pakaste (http://www.juripakaste.fi/)
</span><span>--
</span><span>-- Based on the Set Geo Data.scpt script by
</span><span>-- Andrew Turner (http://highearthorbit.com)
</span><span>--
</span><span>property exifToolOriginal : "_original"
</span><span>property exifToolPath : "/opt/local/bin/exiftool"
</span><span>
</span><span>on extract_decimal(realnum)
</span><span> set res to realnum - (round realnum rounding down)
</span><span> res
</span><span>end extract_decimal
</span><span>
</span><span>on roundFloat(n, precision)
</span><span> set x to 10 ^ precision
</span><span> (((n * x) + 0.5) div 1) / x
</span><span>end roundFloat
</span><span>
</span><span>on d2s(degs)
</span><span> log "enter d2s"
</span><span> if the degs < 0 then
</span><span> set the degs to degs * -1
</span><span> end if
</span><span>
</span><span> set the degrees to round degs rounding down
</span><span> set the minssecs to extract_decimal(degs)
</span><span>
</span><span> log "minssecs: " & minssecs
</span><span>
</span><span> set the minssecs to minssecs * 60
</span><span> set the mins to round minssecs rounding down
</span><span>
</span><span> set the minssecs to extract_decimal(minssecs)
</span><span> log "minssecs 2: " & minssecs
</span><span> set the secs to minssecs * 60
</span><span>
</span><span> "" & degrees & "," & mins & "," & roundFloat(secs, 2)
</span><span>end d2s
</span><span>
</span><span>on exportCoords(image_file, lat, lng, alt)
</span><span> set the northSouth to "N"
</span><span> set the eastWest to "E"
</span><span>
</span><span> if the lat is less than 0 then
</span><span> set the northSouth to "S"
</span><span> set the lat to the lat * -1
</span><span> end if
</span><span>
</span><span> if the lng is less than 0 then
</span><span> set the eastWest to "W"
</span><span> set the lng to the lng * -1
</span><span> end if
</span><span>
</span><span> log "calling d2s on " & lat
</span><span> set the latstr to my d2s(lat)
</span><span> set the lngstr to my d2s(lng)
</span><span>
</span><span> set exifCommand to exifToolPath & " -GPSMapDatum=WGS-84 -gps:GPSLatitude='" & latstr & "' -gps:GPSLatitudeRef='" & northSouth ¬
</span><span> & "' -gps:GPSLongitude='" & lngstr & "' -gps:GPSLongitudeRef='" & eastWest ¬
</span><span> & "' -xmp:GPSLatitude='" & latstr & northSouth & "' -xmp:GPSLongitude='" & lngstr & eastWest & "' -xmp:GPSMapDatum='WGS-84'" & " -xmp:GPSVersionID='2.2.0.0'" & " " & quoted form of image_file
</span><span> display dialog of ("running: " & exifCommand)
</span><span> set output to do shell script exifCommand
</span><span> display dialog of output
</span><span> --do shell script "rm '" & image_file & "'" & exifToolOriginal
</span><span>end exportCoords
</span><span>
</span><span>tell application "iPhoto"
</span><span> activate
</span><span> try
</span><span> copy (my selected_images()) to these_images
</span><span> if these_images is false or (the count of these_images) is 0 then ¬
</span><span> error "Please select one or more images."
</span><span>
</span><span> repeat with i from 1 to the count of these_images
</span><span> set this_photo to item i of these_images
</span><span> tell this_photo
</span><span> set the image_file to the image path
</span><span> set lat to the latitude
</span><span> set lng to the longitude
</span><span> set alt to the altitude
</span><span> end tell
</span><span>
</span><span> if lat < 90.1 and lng < 180.1 then
</span><span> my exportCoords(image_file, lat, lng, alt)
</span><span> else
</span><span> display alert ("No location set for " & name of this_photo)
</span><span> end if
</span><span>
</span><span> log "read image, lat: " & lat & ", lng: " & lng
</span><span>
</span><span> end repeat
</span><span> display dialog "Geo Exif write complete."
</span><span> on error error_message number error_number
</span><span> if the error_number is not -128 then
</span><span> display dialog of ("failed on: " & image_file)
</span><span> display dialog error_message buttons {"Cancel"} default button 1
</span><span> end if
</span><span> end try
</span><span>end tell
</span><span>
</span><span>
</span><span>on selected_images()
</span><span> tell application "iPhoto"
</span><span> try
</span><span> -- get selection
</span><span> set these_items to the selection
</span><span> -- check for single album selected
</span><span> if the class of item 1 of these_items is album then error
</span><span> -- return the list of selected photos
</span><span> return these_items
</span><span> on error
</span><span> return false
</span><span> end try
</span><span> end tell
</span><span>end selected_images
</span></code></pre>
<p><strong>Update (2010-02-28)</strong>: Thank you to <a href="http://twitter.com/simonmark/">@simonmark</a> on Twitter, who <a href="http://twitter.com/simonmark/status/9758717258">pointed out</a> the script had some issues. My version broke if file names had single quotes in them and Simon's version broke with double quotes (admittedly probably rarer in file names.) I was going to leave it as it was, but found the <code>quoted form</code> method of <code>text</code> objects in <a href="http://developer.apple.com/mac/library/documentation/AppleScript/Conceptual/AppleScriptLangGuide/introduction/ASLR_intro.html">AppleScript Language Guide</a> which, assuming it works correctly, should make the script always work properly (at least in terms of file name handling.) I also copied Simon's better error display code and replaced the <code>extract_decimal</code> implementation with something that isn't quite as silly as my previous version was.</p>
<p><strong>Update (2010-06-02)</strong>: I set up a Bitbucket
<a href="http://bitbucket.org/juri/iphoto-scripts/src">repository</a> for this
and other scripts I've written for iPhoto.</p>
Announcing Chipmunk Backup2009-02-27T18:31:00+00:002009-02-27T18:31:00+00:00Unknownhttps://juripakaste.fi/chipmunk-backup/<p>I put up on Launchpad a backup utility I wrote called <a href="https://launchpad.net/chipmunk-backup">Chipmunk Backup</a>. It's not extremely configurable nor does it have a huge set of features. It's a simple tool for maintaining a number of GnuPG encrypted full backups of a directory in a remote, rsync-accessible location.</p>
<p>There's no ready to download archive, but checking out <code>lp:chipmunk-backup</code> with <a href="http://bazaar-vcs.org/">bzr</a> should give you a working version.</p>
<p>It's written in <a href="http://www.plt-scheme.org/">PLT Scheme</a> and is known to work with version 4.1.4.</p>
Emacs tips: Navigate CamelCase words2009-01-06T16:26:00+00:002009-01-06T16:26:00+00:00Unknownhttps://juripakaste.fi/emacs-tip/<p>Emacs tip #0: Always search <a href="http://www.emacswiki.org/">EmacsWiki</a> when you think you might need something.</p>
<p>Emacs tip #1: To navigate studlyCapped words, <code>M-x c-subword-mode</code>, as found on the <a href="http://www.emacswiki.org/emacs/CamelCase">CamelCase page</a>. I had to add the following lines to my <code>.emacs</code> to get it work with <code>C-left</code>/<code>C-right</code>, <code>M-b</code>/<code>M-f</code> worked right out of the box:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(define-key global-map [(control right)] 'forward-word)
</span><span>(define-key global-map [(control left)] 'backward-word)
</span></code></pre>
<p>Before that, they were bound to the <code>-nomark</code> variants.</p>
Inline admin forms with admin site links in Django2008-12-23T16:52:00+00:002008-12-23T16:52:00+00:00Unknownhttps://juripakaste.fi/django-admin-inline-link/<p>I have a somewhat difficult relationship with <a href="http://www.djangoproject.com/">Django</a>'s admin site. It's a very useful feature, but I haven't really done enough with it to know when I'm going to hit a wall, if that wall's in the code or in my understanding, and how hard it's going to be to climb over the wall.</p>
<p>This time I wanted to have <a href="http://docs.djangoproject.com/en/dev/ref/contrib/admin/#inlinemodeladmin-objects">inline admin forms</a>, except that I didn't actually want to have the forms there, I just wanted to have links to the objects — and not their views on the actual site, but on the admin site. As far as I can tell, there's no built-in support for this. </p>
<p>According to the admin docs, there are two subclasses of <code>InlineModelAdmin</code>: <code>TabularInline</code> and <code>StackedInline</code>. Looking at <a href="http://code.djangoproject.com/browser/django/trunk/django/contrib/admin/options.py?rev=9383#L804">django/contrib/admin/options.py</a> confirms this. And as the docs say, the only difference is the template they use. The stacked version comes pretty close when we add all the fields to an <code>InlineModelAdmin</code> subclass's exclude array, but it doesn't have the link.</p>
<p>To solve this we first create a new subclass:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>class LinkedInline(admin.options.InlineModelAdmin):
</span><span> template = "admin/edit_inline/linked.html"
</span></code></pre>
<p>When you want to create inline links to a model, you subclass this new <code>LinkedInline</code> class. So to use a slightly contrived example, if we have a Flight with Passengers:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>class PassengerInline(LinkedInline):
</span><span> model = models.Passenger
</span><span> extra = 0
</span><span> exclude = [ "name", "sex" ] # etc
</span><span>
</span><span>class FlightAdmin(admin.ModelAdmin):
</span><span> inlines = [ PassengerInline ]
</span></code></pre>
<p>And yes, we have to exclude all the fields explicitly: an empty <code>fields</code> tuple or list is ignored.</p>
<p>The new template is easiest to create by cutting down aggressively the stacked template. Like this:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>{% load i18n %}
</span><span><div class="inline-group">
</span><span> <h2>{{ inline_admin_formset.opts.verbose_name_plural|title}}</h2>
</span><span>{{ inline_admin_formset.formset.management_form }}
</span><span>{{ inline_admin_formset.formset.non_form_errors }}
</span><span>
</span><span>{% for inline_admin_form in inline_admin_formset %}
</span><span><div class="inline-related {% if forloop.last %}last-related{% endif %}">
</span><span> <h3><b>{{ inline_admin_formset.opts.verbose_name|title }}:</b>&nbsp;{% if inline_admin_form.original %}{{ inline_admin_form.original }}{% else %} #{{ forloop.counter }}{% endif %}
</span><span> {% if inline_admin_formset.formset.can_delete and inline_admin_form.original %}<span class="delete">{{ inline_admin_form.deletion_field.field }} {{ inline_admin_form.deletion_field.label_tag }}</span>{% endif %}
</span><span> </h3>
</span><span> {{ inline_admin_form.pk_field.field }}
</span><span> {{ inline_admin_form.fk_field.field }}
</span><span></div>
</span><span>{% endfor %}
</span><span></div>
</span></code></pre>
<p>The primary/foreign key fields are necessary to keep Django happy.</p>
<p>The result looks about right, it just lacks the links. It seems that Django doesn't give the template all the information we need to make them work: there's <code>root_path</code> that gives us <code>/admin/</code>, <code>app_label</code> contains the application's name and <code>inline_admin_form.original.id</code> contains the id of the inline object. What is lacking is the path component that names the model. I don't think it's available by default (is there a clean way to ask Django what's available in a template's context?), so we need to add it. Amend <code>LinkedInline</code> to look like this:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>class LinkedInline(admin.options.InlineModelAdmin):
</span><span> template = "admin/edit_inline/linked.html"
</span><span> admin_model_path = None
</span><span>
</span><span> def __init__(self, *args):
</span><span> super(LinkedInline, self).__init__(*args)
</span><span> if self.admin_model_path is None:
</span><span> self.admin_model_path = self.model.__name__.lower()
</span></code></pre>
<p>Now <code>inline_admin_formset.opts.admin_model_path</code> will be bound to the lowercase name of the inline object's model, which is what the admin site uses in its paths.</p>
<p>With this, we can now replace the <code>inline-related</code> <code>div</code> in the template with this:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span><div class="inline-related {% if forloop.last %}last-related{% endif %}">
</span><span> <h3><b>{{ inline_admin_formset.opts.verbose_name|title }}:</b>&nbsp;<a href="{{ root_path }}{{ app_label }}/{{ inline_admin_formset.opts.admin_model_path }}/{{ inline_admin_form.original.id }}/">{% if inline_admin_form.original %}{{ inline_admin_form.original }}{% else %} #{{ forloop.counter }}{% endif %}</a>
</span><span> {% if inline_admin_formset.formset.can_delete and inline_admin_form.original %}<span class="delete">{{ inline_admin_form.deletion_field.field }} {{ inline_admin_form.deletion_field.label_tag }}</span>{% endif %}
</span><span> </h3>
</span><span> {{ inline_admin_form.pk_field.field }}
</span><span> {{ inline_admin_form.fk_field.field }}
</span><span></div>
</span></code></pre>
<p>That's it. Now Flights get links to Passengers without big forms cluttering up the page.</p>
iPhone shuffle2008-11-21T20:31:00+00:002008-11-21T20:31:00+00:00Unknownhttps://juripakaste.fi/iphone-shuffle/<p>These days I use an iPhone as my mobile music device. I have a bit over 1800 songs on it. I usually use shuffle and had it stuck in a weird state a couple of weeks ago — it was <a href="http://www.last.fm/music/The+Roots/_/Act+Too+(The+Love+of+My+Life)+(feat.+Common)">constantly</a> <a href="http://www.last.fm/music/Kanye+West/_/Addiction">playing</a> <a href="http://www.last.fm/music/Kanye+West/_/Addiction">me</a> <a href="http://www.last.fm/music/Marvin+Gaye/_/After+the+Dance+%28instrumental%29">just</a> <a href="http://www.last.fm/music/The+Roots/_/Adrenaline%21">a</a> <a href="http://www.last.fm/music/Jos%C3%A9+Padilla/_/Adi%C3%B3s+Ayer">few</a> <a href="http://www.last.fm/music/4hero/_/The+Action+%28Shawn+J+Period+remix%29">tracks</a>. I usually listen for just half an hour to an hour at a time, so I don't know if it would have started looping or what, but those were basically always there for a week's worth of commutes. I finally restarted the phone and that seemed to help, but what do you know, a couple of weeks, several restarts and one operating system upgrade later, it's again playing me exactly the same tracks.</p>
<p>As much as I love the The Roots, honestly, at this points Adrenaline!'s "Once a-again, once a-gain..." start makes me mostly think "once again indeed."</p>
Crashes with NSURLConnection2008-10-04T17:30:00+00:002008-10-04T17:30:00+00:00Unknownhttps://juripakaste.fi/nsurlconnection-start-crash/<p>Speaking of Cocoa (and iPhone) programming, for a change.</p>
<p>Having trouble with spurious EXC_BAD_ACCESS crashes when using NSURLConnection? NSZombie giving you not very clear messages about [Not A Type retain], pointing to an address that malloc_history says has been allocated somewhere with only framework code in the call stack? See <a href="http://amromousa.com/2008/08/27/nsurlconnection-timeoutconnection-amdev-woes/">Amro Mousa's blog entry</a> about the subject.</p>
<p>In a nutshell, don't send the start message to a NSURLConnection object you've initialized with a +connectionWithRequest:delegate: or -initWithRequest:delegate:. It'll break stuff.</p>
<p>Apple, how about a warning about this in the docs? The docs for -start say "Causes the receiver to begin loading data, if it has not already.", not "Will break your program and make you waste uncounted hours debugging if receiver has already started." Or how about preventing this in the code? Is there some scenario where you'd want to call start after the object has already started?</p>
Various Python related things2008-09-18T18:14:00+00:002008-09-18T18:14:00+00:00Unknownhttps://juripakaste.fi/di-python-stuff/<ul>
<li><a href="http://pymag.phparch.com/">Python Magazine</a> published my article "Using Dependency Injection in Python" in their <a href="http://pymag.phparch.com/c/issue/view/79">August issue</a>. <a href="http://blog.doughellmann.com/">Doug Hellman</a> saw <a href="https://juripakaste.fi/dependency-injection/">my blog entry about DI</a> when I was switching web hosting and managed to repost old stuff to Planet Python, contacted me to ask if I wanted to expand on it a bit, I said yes, and now I've been published. Which is nice. I basically argue in the article that dependency injection is a good idea and you don't need to buy into a framework to get many of the benefits and provide some examples on how to do it. Which seems kind of apropos, seeing as there's yet another <a href="http://www.traceback.org/2008/09/08/ann-snake-guice-preview-a-dependency-injection-framework/">new contestant</a> in that area.</li>
<li>I was hacking on something I wrote in Python recently and was doing it in a pretty heavily dependency injected manner, writing tests as I was going along. It came to me that one really basic, really nice thing about Python that isn't discussed all that much is that callables are duck typed and there are multiple choices on how to implement them. When you write a function that takes as a parameter something it calls to acquire values it doesn't have to care what the thing it's been given actually is. It can be a function, a callable, a bound method, a constructor, a lambda expression or even a generator. The caller doesn't have to care. It's not a language feature that's easy to fit in a bullet point or to appreciate without using it, but it is useful.</li>
<li>I was excited to see class decorators are coming in Python 2.6. Then I realized I can't remember what I wanted to use them for when I was really frustrated by their absence, but still, it's good. I always found their absence rather puzzling.</li>
</ul>
Flow 08: Saturday and Sunday2008-08-18T23:04:00+00:002008-08-18T23:04:00+00:00Unknownhttps://juripakaste.fi/flow08-saturday-sunday/<p>On Saturday, it was raining cats and dogs. We were appropriately equipped and sniggered at the people in trainers etc trying to dodge the puddles. However, didn't see too many acts — saw a bit of <a href="http://www.myspace.com/sebastientellier">Sébastien Tellier</a>, but decided the crowds were too much and went to find some food (which was excellent and at 25 € for a three course vegetarian menu pretty good value.)</p>
<p>Next up on our schedule was <a href="http://www.csshurts.com/">CSS</a>, which was pretty good. For some reason, it has never quite clicked for me, but still, they were busy as hell and obviously having fun.</p>
<p>And finally, <a href="http://www.theroots.com/">The Roots</a>. What a great show. No bling, no diva manners, just excellent hip hop with a surprising amount of jazz thrown in, just like on their early albums. And an incredibly diverse band, with <a href="http://en.wikipedia.org/wiki/Captain_Kirk_Douglas">Captain Kirk</a> shredding his guitar and Tuba Gooding Jr on <a href="http://en.wikipedia.org/wiki/Sousaphone">sousaphone</a>.</p>
<p>Saturday was the only day it really felt like there were too many people stuffed into too small a space. Maybe it was the rain, maybe it was the fact that it was sold out, even though I don't think the other two days were that far behind.</p>
<p>On Sunday we were definitely starting to feel old and tired. It's surprising how tiring three days of festival gets, even without excessive drinking. Maybe I'm just too old. While eating on Saturday, I had the idea that they really should offer a show and dinner version of the festival; they already have an excellent restaurant on board, now just stretch out the dinner experience a bit and place the restaurant in a suitable location, and hey, the middle aged among us could be nice and comfy while checking out the gigs. And I really think they should have put the restaurant on the <a href="http://www.flickr.com/photos/juripakaste/2767663750/in/set-72157606759737007/">roof</a> of the newer (if it is newer, the black one) gasometer.</p>
<p>We caught a glimpse of <a href="http://www.myspace.com/astrocancaravan">Astro Can Caravan</a>, who had the weirdo jazz thing pretty well covered. Next up was <a href="http://www.thefivecornersquintet.com/">The Five Corners Quintet</a>, whose retro jazz was excellent as always. After that, we first checked out <a href="http://www.plutonium74.net/">Plutonium 74</a>, and had enough after one and a half songs. We listened to the first two or three songs from Señor Coconut and while their version of Daft Punk's Around the World wasn't horrible it wasn't nowhere near as good as Christian Prommer's on Friday, and by the time they hit Eurythmics' Sweet Dreams, we decided we had had enough of that too.</p>
<p>The next act we saw was <a href="http://www.myspace.com/josejamesquartet">José James</a> who was one of the high points of the festival. He did a surprisingly jazzy gig and the crowd appreciated. After his gig, we listened for a couple of songs by <a href="http://www.myspace.com/cutcopy">Cut Copy</a>, who really revealed themselves to be very summery party pop. Loud summery party pop. Their album In Ghost Colors was decent but I didn't get into it all that much, but they were better live. However, we were just too tired at that point and after a while headed home.</p>
Flow08, Day 12008-08-16T11:45:00+00:002008-08-16T11:45:00+00:00Unknownhttps://juripakaste.fi/flow08-friday/<p><a href="http://www.flowfestival.com/">Flow08</a> started off a whole lot better than <a href="https://juripakaste.fi/flow07-friday/">last year</a>. Everything worked smoothly despite the fact that there were twice as many people and twice as large an area as last year. In fact, the enlarged space felt better than the more constrained area of last year, maybe because we got to see more of the Suvilahti grounds.</p>
<p><a href="http://www.flickr.com/photos/juripakaste/2767662120/" title="Jamie Lidell on stage"><img src="http://farm4.static.flickr.com/3144/2767662120_5ab73f50ef_m.jpg" width="240" height="180" alt="Jamie Lidell" /></a></p>
<p>Artists we saw: </p>
<ul>
<li><a href="http://www.jamielidell.com/">Jamie Lidell</a> was a positive surprise. I found a video I saw earlier a bit meh, but his show was energetic and fun, moving from style to style effortlessly.</li>
<li><a href="http://www.kingsofconvenience.com/">Kings of Convenience</a> and <a href="http://www.mum.is/">Múm</a> were both nice enough and lots of people seemed to enjoy them a lot, but... eh, not really Friday night music? Múm did occasionally manage to get a bit of a rhythm going, but they were more in the artsy fartsy moody camp I'd rather listen on my headphones. Both would have been better on Saturday or Sunday afternoon.</li>
<li><a href="http://www.22-pistepirkko.net/">22-Pistepirkko</a> was excellent. It's not like they are a new band, but it was the first time I saw them live. They were loud, some of the songs got completely different treatments than on their albums, and the feeling was great.</li>
<li><a href="http://www.myspace.com/christianprommer">Christian Prommer's Drumlesson</a> was fabulous. The guys got an incredible groove going from the start, jazzing straight into the hedonistic heart of dance music.</li>
</ul>
<p>Some mostly lousy pics <a href="http://www.flickr.com/photos/juripakaste/">on Flickr</a>. Maybe <a href="http://www.flickr.com/photos/markonen/">Marko</a> will post some better ones.</p>
Using custom widgets with Django's newforms-admin2008-07-27T17:22:00+00:002008-07-27T17:22:00+00:00Unknownhttps://juripakaste.fi/custom-django-newforms-admin-widgets/<p>The following isn't magic but it was unclear to me and required reading both documentation and source code and some additional Googling to get right. Maybe that's because I'm a Django newbie, but hey, I'm probably not the only one. By the way, the following applies to Django SVN revision 8068. That's roughly Django 1.0 alpha.</p>
<p>Anyway, I have in my model a field that's basically a time-stamped boolean: a field called deleted of type DateTimeField. If it's NULL, the thing, let's call it Foo, is not deleted; if it has a value, it tells us the Foo in question was marked as deleted back then. It's not the only way I could have implemented it but it meets my requirements and I didn't want to change it.</p>
<p>How to display that in admin? I could have left it as just a split datetime field, but that doesn't really communicate the intent. A checkbox with additional text telling the date is a much better representation.</p>
<p>I found some help in in Stuart Langridge's <a href="http://www.kryogenix.org/days/2008/03/28/overriding-a-single-field-in-the-django-admin-using-newforms-admin">Overriding a single field in the Django admin, using newforms-admin</a>, but it seems it's slightly outdated and didn't contain all the details.</p>
<p>Turns out for this to work well, I needed three bits: a widget class, a field class and a formfield_for_dbfield method in my ModelAdmin class.</p>
<p>Here's my widget class:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>class BooleanDateWidget(forms.CheckboxInput):
</span><span> def __init__(self, attrs=None, check_test=lambda v: v is not None):
</span><span> super(BooleanDateWidget, self).__init__(attrs, check_test)
</span><span>
</span><span> def render(self, name, value, attrs=None):
</span><span> final_attrs = self.build_attrs(attrs, type='checkbox', name=name)
</span><span> try:
</span><span> result = self.check_test(value)
</span><span> except: # Silently catch exceptions
</span><span> result = False
</span><span> if result:
</span><span> final_attrs['checked'] = 'checked'
</span><span> dt = ' <label for="%s" class="vCheckboxLabel">(%s)</label>' % (final_attrs["id"], value)
</span><span> else:
</span><span> dt = ""
</span><span> if value not in ('', True, False, None):
</span><span> # Only add the 'value' attribute if a value is non-empty.
</span><span> final_attrs['value'] = force_unicode(value)
</span><span> return mark_safe(u'<input%s />%s' % (flatatt(final_attrs), dt))
</span></code></pre>
<p>The render method has mostly been copied and pasted from django.forms.CheckboxInput.render, with a few modifications to create an additional label. When database value is rendered to the admin form, that method gets called. If deleted is NULL, it creates an empty checkbox; if there's a date there, it creates a checked checkbox with an additional label that contains the timestamp (not very prettily formatted, though.)</p>
<p>Next, the field class:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>class BooleanDateField(fields.BooleanField):
</span><span> widget = BooleanDateWidget
</span><span>
</span><span> def clean(self, value):
</span><span> v = super(BooleanDateField, self).clean(value)
</span><span> if v:
</span><span> return datetime.datetime.now()
</span><span> return None
</span></code></pre>
<p>That's pretty simple. I just specify the widget to use and make the clean method which is called when the value sent by the browser is validated for storage in the model return either None if the checkbox was checked or None otherwise.</p>
<p>And finally the last piece is formfield_for_dbfield method in the ModelAdmin class.</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>class FooAdmin(admin.ModelAdmin):
</span><span> def formfield_for_dbfield(self, db_field, **kwargs):
</span><span> if db_field.name == 'deleted':
</span><span> field = db_field.formfield(form_class=booleandate.BooleanDateField)
</span><span> else:
</span><span> field = super(FooAdmin,self).formfield_for_dbfield(db_field,**kwargs)
</span><span> return field
</span></code></pre>
<p>That method is invoked when the newforms-admin interface needs the field object for displaying and handling the deleted value. It checks the field name to specialize the interface. For the deleted class, we call the db_field object's formfield method, otherwise we delegate to ModelAdmin. ModelAdmin's formfield_for_dbfield is pretty simple itself, it mostly just sets the widget type for a few field types and calls the formfield method of the db_field object. That's what we could have done here, too; instead of specifying the widget type in the Field class, we could have done it here.</p>
<p>That's all it takes! With those modifications, the split datetime field disappears and instead you get a datetime-labeled checkbox.</p>
A moment of C++ hate2008-07-25T17:17:00+00:002008-07-25T17:17:00+00:00Unknownhttps://juripakaste.fi/a-moment-of-cplusplus-hate/<p>Apologies, I'm going to indulge myself for a moment. If you aren't interested in C++ ranting, skip this.</p>
<p>I'm in the process of converting some C++ code to heap allocate objects instead of putting them in the stack, because I need to use them in Objective-C++ and stack-allocated objects aren't the best idea there.</p>
<p>Who in their right mind wants to spend programming time worrying about where to allocate objects? Why do I have to care? There's an actual problem domain here with business objects, and I'm twiddling object allocation. I feel like bashing my head against the keyboard.</p>
JUGC 0.2 with Python support2008-07-18T18:34:00+00:002008-07-18T18:34:00+00:00Unknownhttps://juripakaste.fi/jugc-0-2/<p>I just released version 0.2 (and 0.2.1, now with a NEWS file!) of <a href="http://code.google.com/p/jugc/">JUGC</a>, the unit conversion library generator. It allows you to specify a set of measuring units and translations in XML files and generate a conversion library from them, thus avoiding parsing a definition file at startup. The generator portion is still written in Java, but now in addition to Java, it can also generate Python code. There are small examples on the <a href="http://code.google.com/p/jugc/wiki/Examples">Examples wiki page</a>. There isn't much documentation, but the example data files and tests should be clear enough.</p>
pgrok2008-05-04T14:40:00+00:002008-05-04T14:40:00+00:00Unknownhttps://juripakaste.fi/pgrok/<p>I put up <a href="http://launchpad.net/pgrok/">pgrok</a> ("Project Grokking for Emacs"), a simple Elisp package for project settings and tools, on Launchpad. It's a tidied up version of a couple of things I've been using for ages, basically for loading project (a source tree) settings, functions or whatever you need from files that are looked up in your directory hierarchy. There are also two (at the moment) functions for locating stuff in your project with find and grep.</p>
<p>There are no releases yet, but it's just one file in the bzr branch hosted on Launchpad. I use GNU Emacs 23.0.60, no guarantees about any other Emacsen.</p>
Software Engineering Radio2008-04-17T09:33:00+00:002008-04-17T09:33:00+00:00Unknownhttps://juripakaste.fi/seradio/<p>They keep on asking their listeners to plug them, so here goes. I've been listening to podcasts quite a bit lately, as I got a phone that can do them and my commute is too short to really concentrate on reading articles, and the best of the ones I regularly follow is <a href="http://www.se-radio.net/">Software Engineering Radio</a>. There's a lot of backlog to go through and at least the interviews they've done with <a href="http://www.se-radio.net/podcast/2008-03/episode-89-joe-armstrong-erlang">Joe Armstrong</a>, <a href="http://www.se-radio.net/podcast/2008-03/episode-88-singularity-research-os-galen-hunt">Galen Hunt</a> and <a href="http://www.se-radio.net/podcast/2008-02/episode-86-interview-dave-thomas">Dave "not PragDave" Thomas</a> have been really nice.</p>
AudioFormat 0.62008-01-13T14:36:00+00:002008-01-13T14:36:00+00:00Unknownhttps://juripakaste.fi/audioformat-0-6/<p>Waking the project up, at least momentarily, from a nine-month slumber, I released version 0.6 of <a href="http://www.juripakaste.fi/audioformat/">AudioFormat</a>, the friendly audio file format converter for the GNOME desktop.</p>
<p>I've again found more use for the software and discovered that it was in several ways still embarassingly broken, from refusing to install on Python versions with micro version numbers (like 2.5.1) to still continuing conversions after clicking cancel, even without the window, when using as a Nautilus extension. </p>
<p>It should be at least a bit better now.</p>
Dependency injection: I'm not going back2007-12-11T19:38:00+00:002007-12-11T19:38:00+00:00Unknownhttps://juripakaste.fi/dependency-injection/<p>InfoQ carried <a href="http://www.infoq.com/news/2007/12/does-di-pay-off">an article about dependency injection</a>, with the title "Does Dependency Injection pay off?".</p>
<p>Answer: it does.</p>
<p>You learn something useful from most languages, frameworks and libraries you get to know. One of the more surprising revelations the Java world had for me, after working in it for the best part of a decade, came in the form of <a href="http://springframework.org/">Spring</a>. It's a huge pile of stuff, but in the center sits a pretty good idea. It's not an original idea, but Spring is where I got to know it. It's <a href="http://martinfowler.com/articles/injection.html">dependency injection</a>. It's about classes not instantiating their dependencies or looking them up from some sort of global scope, but exposing them as constructor arguments or properties. </p>
<p>In Spring, you define the dependencies in an XML file and let the framework set things up. It works well enough, but I've been merrily using the same design in non-Java environments ever since getting to know it without any of that angle bracket noise and haven't really missed it; you get by just as well with a piece of code that creates all the objects, hooks them up and presses play.</p>
<p>Everything is so much cleaner and simpler than it used to be and as an extra bonus, it's suddenly a lot easier to test, too. The cleanliness DI brings to your design has the quality of being self-supporting: once you start building the system with it, the path of least resistance is to keep on using it instead of letting the dependencies spread. It also makes using singleton a lot less tempting (it's easier to just instantiate an ordinary class and give that instance everywhere it's needed), which is another big plus.</p>
<p>Someone in the InfoQ article invokes <a href="http://c2.com/xp/YouArentGonnaNeedIt.html">YAGNI</a>. It's a false argument. You have to set up the dependencies anyway somewhere. You are going to need <em>that</em>. So the question is, should you do it in one place or scattered throughout the system? Which is cleaner? Another question you can ask is the old OO design principle, whose responsibility is it anyway? With the benefits DI brings, it's an easy decision to make.</p>
Using SSH from the Gnome menus2007-12-08T09:55:00+00:002007-12-08T09:55:00+00:00Unknownhttps://juripakaste.fi/ssh-gnome-menu/<p>Some time ago I wrote in a <a href="http://sshmenu.sf.net/">SSHMenu</a> related blog posting's comment thread that I found the Gnome panel's menu combined with gnome-terminal's profiles feature sufficient for my SSH needs. As a followup, I got a email a couple of days ago by someone asking if I knew how to do the same thing except opening Nautilus windows for the remote hosts. I replied, but apparently the email could never be delivered, so here's a quick recap.</p>
<p>Below, I'll use <em>foo</em> as the remote host name. Replace it with the name of whatever host you want to use.</p>
<p>Steps:</p>
<ul>
<li>Open the menu editor, either by right-clicking on the menu or from System > Preferences > Main Menu.</li>
<li>Click on New menu to create a new submenu, call it "Remote hosts" or something.</li>
<li>Select the new menu, click on New Item to create a new menu item.</li>
</ul>
<p>For terminals:</p>
<ul>
<li>In the Launcher Properties window that opened after clicking New Item, set the launcher type to Application, give it a descriptive name like "ssh <em>foo</em>" and the following command: gnome-terminal --window-with-profile="Remote <em>foo</em>". You can also give the launcher an icon. I usually like to give an icon that I use also for the gnome-terminal profile. See below.</li>
<li>The launcher won't do much useful yet, beyond opening up a new terminal, so now you have to create the gnome-terminal profile "Remote <em>foo</em>".</li>
<li>Open up a gnome-terminal window. </li>
<li>If you value your screen real estate, you probably have hidden the terminal menu bar. You have to make it visible to access the new profile creation functionality, so bring up the context menu with the right mouse button and select Show Menubar.</li>
<li>In the gnome-terminal menu, select Edit > Profiles. Click on New to create a new profile and give it the name "Remote <em>foo</em>". Click Create.</li>
<li>In the General tab, you can give the profile an icon. As I said, I like to use an icon here and in the menu, but that's up to you. You can also specify host specific colors in the Colors tab.</li>
<li>In the Title and Command tab, check "Run a custom command instead of my shell", and specify ssh <em>foo</em> as the custom command.</li>
</ul>
<p>Now you are set, and selecting ssh <em>foo</em> in your new Remote hosts should open up a new terminal window that prompts you for authentication information for <em>foo</em>.</p>
<p>Turns out using this for Nautilus is even less work. Assuming things work out.</p>
<ul>
<li>In the Launcher Properties window, instead of specifying gnome-terminal as the command, type in nautilus ssh://<em>login</em>@<em>foo</em> .</li>
</ul>
<p>Replace <em>login</em> with your login on <em>foo</em>.</p>
<p>That's it. No extra steps needed.</p>
<p>Now, if gnome-vfs does its magic, you'll get a new Nautilus window when you select your new menu entry, showing the contents of your home folder on <em>foo</em>. The other option is that you get a Nautilus window showing nothing, which seems to happen to me for some hosts. No idea what that's all about.</p>
Shuffling the stack: first faltering steps on Factor2007-12-03T18:33:00+00:002007-12-03T18:33:00+00:00Unknownhttps://juripakaste.fi/first-factor/<p>In my neverending quest of avoiding actually getting any of my projects ready, I started playing with <a href="http://factorcode.org/">Factor</a>. In four years, Slava Pestov and the other folks hacking on Factor have created an extremely impressive language, beating many other dynamic languages like Python and Ruby in speed and providing an environment that doesn't make the <a href="http://factorcode.org/development.fhtml">Factor 2.0 goal</a> "Best interactive development environment of any programming language" sound like an impossible idea.</p>
<p>But it does have a steep learning curve. The concepts of the language aren't difficult — in many ways it resembles other dynamic languages — but it feels like I'm clumsily trying to perform mental gymnastics instead of actually being productive. See, it's a Forth-style stack shuffling <a href="http://en.wikipedia.org/wiki/Concatenative_programming">concatenative programming language</a>. There are named variables (dynamically scoped, which feels weird, but that does have its uses when used sparingly) and these days there are even <a href="http://factor-language.blogspot.com/2007/08/named-local-variables-and-lexical.html">lexically scoped named locals</a>, but depending on those instead of getting used to stack manipulation would be cheating, no? So now I'm slowly <a href="http://factorcode.org/responder/help/show-word?word=dup&vocab=kernel">dup</a>ping, <a href="http://factorcode.org/responder/help/show-word?word=over&vocab=kernel">over</a>ing, <a href="http://factorcode.org/responder/help/show-word?word=tuck&vocab=kernel">tuck</a>ing and <a href="http://factorcode.org/responder/help/show-word?word=swap&vocab=kernel">swap</a>ping my way through simple algorithms, trying to switch my head into the right gear.</p>
This is how all software should be2007-11-08T18:09:00+00:002007-11-08T18:09:00+00:00Unknownhttps://juripakaste.fi/couchdb-relax/<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> $sudo -u couchdb /opt/couchdb/bin/couchdb
</span><span> couch 0.7.0a572 (LogLevel=info)
</span><span> CouchDB is starting.
</span><span> CouchDB has started. Time to relax.
</span></code></pre>
<p>It's a bit like having a dictionary with the words "Don't panic" printed on the cover in big, friendly letters.</p>
Resin: Javascript in JSPs?2007-10-07T13:05:00+00:002007-10-07T13:05:00+00:00Unknownhttps://juripakaste.fi/javascript-jsp/<p>Am I missing something here? <a href="http://www.caucho.com/articles/jsp_templates.xtp">JSP Templates</a> vs.</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/test.jsp:1: 'javascript' is not supported as a JSP scripting language.
</span><span>
</span><span>1: <%@ page language="javascript" %>
</span></code></pre>
<p>That's what Resin 3.1.2 tells me.</p>
<p><strong>Update</strong>: Right, now I noticed this in Resin 3.0.0 release notes: </p>
<blockquote>
<p>Since the JavaScript for JSP was used by too few users, it no longer makes sense to continue support.</p>
</blockquote>
<p>Arf.</p>
<p>Apparently Tomcat, Jetty and Glassfish all support only Java in JSPs, not any javax.script implementations. Now I'm trying to figure out how to use EJS from <a href="https://scripting.dev.java.net/">scripting.dev.java.net</a>. Any other Javascript templating options for servlets? I know <a href="http://helma.org/">Helma</a>, but I'm interested in the front-end code (templates, value objects, possibly controllers) only.</p>
Cirque du Soleil's Delirium: meh2007-09-23T10:12:00+00:002007-09-23T10:12:00+00:00Unknownhttps://juripakaste.fi/cds-delirium-meh/<p>We went to see Cirque du Soleil's show Delirium, now in Helsinki on their tour of Europe, yesterday. It was the first time I've seen any of their shows live. It wasn't exactly bad, but not what I was hoping for. The most important part seemed to be the music and the video stuff, and the acrobatics were a bit of an extra. Trouble is, the acrobatics were what we went there to see.</p>
<p>The music was an A-Z of genres with no carrying theme, going from rock with guitar heroics to Africa flavoured world fusion to soul to pop to salsa to tango. And few of the songs were all that great, either, basically sounding like filler tracks on a pretty ok album. A different album in each case, of course.</p>
<p>The video show was amazing, mostly computer generated and in all cases heavily processed, with two big screens on each side of the stage, one behind it and a translucent one part of the time in front of it.</p>
<p>And occasionally somewhere there were a few acrobats. They were mostly extremely impressive, too. At least when you could see and concentrate on them. Too often, though, even if they were there, there was the huge A/V show going on all around them, drowning them with its volume, both in terms of sound and screen area. And as if that wasn't enough, sometimes they had the translucent screen in front of the stage showing water or something, forcing you to squint to see the people who you thought were supposed to be the most important part of the whole experience.</p>
<p>According to the official web site, "For the first time Cirque du Soleil created a show where melodies, musicians and singers are the driving force". I guess I'll just have to keep that it mind when deciding which show to go see next time.</p>
Glade-3 toolbars2007-08-27T19:21:00+00:002007-08-27T19:21:00+00:00Unknownhttps://juripakaste.fi/glade3-toolbars/<p>This note is here so I can grep or Google for it the next time I spend ten minutes clicking around the Glade-3 window, trying to remember what the trick was: Glade-3 has put the menu and toolbar editors behind an edit button <strong>below</strong> the the properties notebook.</p>
Python-style enumerate for Nemerle2007-08-25T12:50:00+00:002007-08-25T12:50:00+00:00Unknownhttps://juripakaste.fi/nemerle-enumerate/<p>So instead of just whining about Nemerle, here's a something potentially useful. I found I missed Python's built-in <code>enumerate</code> function. It wasn't too hard to implement in Nemerle:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>public class Enumerator['a] : IEnumerable[(int * 'a)] {
</span><span> private e: IEnumerable['a];
</span><span>
</span><span> public this(e: IEnumerable['a]) {
</span><span> this.e = e;
</span><span> }
</span><span>
</span><span> public GetEnumerator(): IEnumerator[(int * 'a)] {
</span><span> mutable i = 0;
</span><span> foreach (elem in this.e) {
</span><span> yield (i, elem);
</span><span> i++;
</span><span> }
</span><span> }
</span><span>}
</span></code></pre>
<p>Usage:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>foreach ((index, item) in Enumerator(list)) {
</span><span> System.Console.WriteLine($"index for $item: $index");
</span><span>}
</span></code></pre>
<p>The naming might not be optimal, with potential confusion with the standard libraries, but I don't care at the moment. Use and enjoy.</p>
Flow 07: Sunday and Regina & Husky Rescue at the Huvila Festival Tent2007-08-21T18:45:00+00:002007-08-21T18:45:00+00:00Unknownhttps://juripakaste.fi/flow07-sunday-regina-husky/<p>Sunday was a whole lot better at <a href="http://www.flowfestival.com/">Flow</a> than <a href="https://juripakaste.fi/flow07-friday/">Friday</a>. Everything worked better and the music was lovely. Kudos to the organizers for seeing the problems and acting to correct them.</p>
<p>Highlights:</p>
<ul>
<li><a href="http://www.fonal.com/risto/">Risto</a> is the king of the world.</li>
<li><a href="http://www.myspace.com/organkane">Tuomo</a> played just the kind of music you'd love to hear on a sunny Sunday afternoon.</li>
<li><a href="http://www.myspace.com/korpiensemble">Korpi Ensemble</a> was easygoing and fun.</li>
</ul>
<p>On the not quite as good front, it was the first time I had heard of Jenny Wilson and wasn't really very impressed. Nice enough, but didn't really generate any strong feelings. Bebel Gilberto was better, but she arrived late on the stage and I don't think she really captured the audience. We left in the middle of her set.</p>
<p>Monday evening we went to see <a href="http://www.reginaregina.com/">Regina</a> and <a href="http://www.husky-rescue.com/">Husky Rescue</a> at the Helsinki Festival Huvila Tent. Both were excellent, even though Regina would have clearly worked even better in an envinronment where people wouldn't have been sitting down — the tent doesn't really encourage you to move your ass.</p>
<p>All in all, you get the feeling that Finnish music is doing pretty well.</p>
Flow 07: Friday done2007-08-18T12:28:00+00:002007-08-18T12:28:00+00:00Unknownhttps://juripakaste.fi/flow07-friday/<p>First day of <a href="http://www.flowfestival.com/">Flow</a> is behind. This year, we bought tickets for Friday and Sunday, we'll skip Saturday. I have <a href="http://www.flickr.com/photos/juripakaste/sets/72157601516779802/">a few not-so-great photos</a> online.</p>
<p>The good:</p>
<ul>
<li>The artists. Pepe Deluxé was good, sometimes great. Terry Callier was brilliant. No complaints about the other stuff, but those were the only two complete sets we heard. Well, CocoRosie was, er, slightly too weird for my tastes.</li>
<li>The location, the old Suvilahti power station. The outdoor space isn't quite as excellent as <a href="http://flickr.com/photos/juripakaste/sets/72157594248567259/">last year</a>, but it is pretty nice anyway. And even closer to home. And having the club space in the same place justifies the move pretty well.</li>
</ul>
<p>The bad:</p>
<ul>
<li>Queues. Queues everywhere. One of the main attractions of Friday for us was Nicole Willis and The Soul Investigators. We spent the time they were on stage in the line outside the festival area, waiting to get in. Partially a timing problem (starting the first set at six o'clock on a Friday evening is just too early), partially a organizational problem (why does it have to be so slow to let people in?) Time to get a beer from the bars next to the main stage: 20-30 minutes. And of course the bar areas are fenced and you aren't allowed to go buy your drinks from another bar and take them to the main stage. Queues to the toilets, too.</li>
<li>The main reason causing the queues: too little staff. They would have probably doubled their sales if there was actually enough people staffing the bars. When they opened the club space, there was apparently one person staffing the three bars, serving a couple of hundred potential customers.</li>
<li>No schedules anywhere. If you didn't print out the schedule yourself, too bad.</li>
<li>Too small venues. The main stage was ok and we didn't really go to the club hall except right when it opened, so no idea how that was later on. But the tent where The Five Corners Quintet, Jukka Eskola, The Stance Brothers and Timo Lassy were the last acts was woefully small, with horrible crowding. We couldn't stand it so had to skip them, which meant that with Nicole Willis missed earlier, we didn't see half the acts we were trying to see.</li>
<li>The hot dog stand ran out of buns before hotdogs and gave out all the hot dogs to people when there were still others in the line. Thanks. We would have paid for those. After that, the only food left was a small piece of dry lasagna costing ten euros. This looks like a pretty good business.</li>
<li>The beer was Fosters. The cider was Upcider. The wines were JP Chenet.</li>
</ul>
<p>If we hadn't already bought the tickets for Sunday, we might seriously reconsider our plans: good music, but they made it way too difficult to enjoy it.</p>
Slightly disappointed in Nemerle2007-08-12T16:16:00+00:002007-08-12T16:16:00+00:00Unknownhttps://juripakaste.fi/disappointed-in-nemerle/<p>I've been coding with Nemerle a bit during the last couple of months. I've discovered that I had set my expectations too high. It doesn't quite live up to them. Might be as much my fault as Nemerle's, but that doesn't really change anything.</p>
<p>I was hoping for a experience closer to OCaml (only with better cross platform library support) than I got. With OCaml, once your code compiles, you know you are doing pretty good. With Nemerle, not so much. First of all, it feels like you get to fight a bit more with the type inferer than in OCaml. This is probably partially due to the maturity of the implementation, partially due to the fact that with OCaml, I didn't use as serious class hierarchies as I do with Nemerle, thanks to the .NET standard library and heavier Gtk+ usage.</p>
<p>The second problem I've encountered is that when you interface a lot with C# code, you don't really get much mileage out of the functional stuff in Nemerle. When you have a List generated in C#, you don't get to fold over it. While Nemerle does have the ICollection interface extending System.Collections.Generic.ICollection with all the functional iteration tools, there are no wrapper classes and no standalone functions. If you want a foldable List, you'll have to copy stuff. Not nice.</p>
<p>And third, while the variables are by default non-mutable, which is nice, and when you define your types in Nemerle you can do matching and get the safety that brings, you end up interacting so much with .NET code that you have to do the normal null checks all over the place anyway.</p>
<p>In short, the benefits of Nemerle over the current features of C# are feel pretty limited. Maybe it would be better if you were working with pure Nemerle projects, but at least interacting with C# eliminates most of them.</p>
Making Emacs understand ncc errors2007-05-17T14:27:00+00:002007-05-17T14:27:00+00:00Unknownhttps://juripakaste.fi/ncc-emacs-errors/<p>Add this to your .emacs or equivalent to make Emacs' compilation-mode make proper links out of the <a href="http://nemerle.org/">Nemerle</a> compiler's error messages:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(setq compilation-error-regexp-alist
</span><span> (append compilation-error-regexp-alist
</span><span> '(("^\\(.+\\):\\([0-9]+\\):\\([0-9]*\\):\\(?:[0-9]*\\):\\(?:[0-9]*\\):"
</span><span> 1 2 3))))
</span></code></pre>
<p><strong>Update</strong>:</p>
<p>Or even better, using rx (thanks to <a href="http://www.nobugs.org/blog/archives/2007/05/16/emacs-regexps/">this hint</a> for reminding me about it):</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(setq compilation-error-regexp-alist
</span><span> (append compilation-error-regexp-alist
</span><span> `((,(rx line-start
</span><span> (group (1+ (not (any ":"))))
</span><span> ":"
</span><span> (group (1+ digit))
</span><span> ":"
</span><span> (group (1+ digit))
</span><span> ":"
</span><span> (*? digit)
</span><span> ":"
</span><span> (*? digit)
</span><span> ": ") 1 2 3))))
</span></code></pre>
C# export for Gaphor2007-04-28T10:47:00+00:002007-04-28T10:47:00+00:00Unknownhttps://juripakaste.fi/gaphor-csharp/<p><a href="http://gaphor.devjavu.com/">Gaphor</a> is the best free UML tool for X I've seen yet. It's far from finished but at least it produces pretty graphs. I started writing a C# export plugin for it. The first somewhat working version is <a href="http://www.juripakaste.fi/dl/csharpexport.tar.bz2">available</a>. This has been developed with 0.10.4, no guarantees of it working with any other version. No guarantees of it working with that version either, to be exact, but at least it works for me.</p>
<p>Extract it inside Gaphor's source directory, in gaphor/data/plugins.</p>
Python, Queue, Pygame and the GLib mainloop2007-04-27T19:26:00+00:002007-04-27T19:26:00+00:00Unknownhttps://juripakaste.fi/python-queue-mainloop/<p>The GLib mainloop is what you're probably running in your Python program when you are using PyGTK or GStreamer.</p>
<p>Sometimes you want to integrate other event sources to it. The documentation for this is a bit thin; all I've seen is <a href="http://developer.gnome.org/doc/API/glib/glib-the-main-event-loop.html#GSOURCEFUNCS">GLib's page on the main event loop, GSourceFuncs section</a> and <a href="http://svn.gnome.org/viewcvs/pygobject/trunk/tests/test_source.py?view=markup">test-source.py in pygobject</a>. Here I'm presenting two snippets displaying how to do it with two systems I've needed to work with. No guarantees on their correctness, but they do seem to work. Both are parts of larger modules, so they aren't usable by themselves and aren't really tidied up to look like proper examples. However, they should give you the general idea.</p>
<p>Here's how to do it with Python's <a href="http://docs.python.org/lib/module-Queue.html">Queue</a>:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>class QueueSource(gobject.Source):
</span><span> def __init__(self, queue):
</span><span> gobject.Source.__init__(self)
</span><span> self._queue = queue
</span><span> self.logger = loghelper.get_logger(self)
</span><span>
</span><span> def prepare(self):
</span><span> return False
</span><span>
</span><span> def check(self):
</span><span> return not self._queue.empty()
</span><span>
</span><span> def dispatch(self, callback, args):
</span><span> self.logger.debug("dispatch called, callback: " + str(callback))
</span><span> if callback is not None:
</span><span> self.logger.debug("dispatch calling callback: " + str(callback))
</span><span> v = callback(self._queue.get(), *args)
</span><span> self.logger.debug("dispatch returned " + str(v))
</span><span> return True
</span><span>
</span><span> def finalize(self):
</span><span> self.logger.debug("finalize called")
</span><span>
</span><span>
</span><span>m = QueueSource(self._queue)
</span><span>def my_callback(command, *args):
</span><span> self.logger.debug("my_callback: args: " + str(command) + ", " + str(args))
</span><span> self.logger.debug("my_callback: thread: " + str(threading.currentThread()))
</span><span> if command == self.PLAY:
</span><span> self._player.play()
</span><span> elif command == self.STOP:
</span><span> self._player.stop()
</span><span> elif command == self.SKIP:
</span><span> self._player.skip()
</span><span>
</span><span>m.set_callback(my_callback, mainloop)
</span><span>m.attach()
</span></code></pre>
<p>And here's <a href="http://www.pygame.org/">Pygame</a>:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>class PygameEventSource(gobject.Source):
</span><span> def __init__(self):
</span><span> gobject.Source.__init__(self)
</span><span> self.logger = loghelper.get_logger(self)
</span><span>
</span><span> def prepare(self):
</span><span> return False
</span><span>
</span><span> def check(self):
</span><span> try:
</span><span> pygame.event.pump()
</span><span> p = pygame.event.peek(
</span><span> [KEYUP, KEYDOWN, VIDEOEXPOSE, VIDEORESIZE, NUMEVENTS, QUIT,
</span><span> ACTIVEEVENT, USEREVENT])
</span><span> return p
</span><span> except Exception, ex:
</span><span> self.logger.error("Caught an exception while checking for events",
</span><span> exc_info=True)
</span><span>
</span><span> def dispatch(self, callback, args):
</span><span> if callback is not None:
</span><span> try:
</span><span> e = pygame.event.poll()
</span><span> v = callback(e, *args)
</span><span> return v
</span><span> except (KeyboardInterrupt, SystemExit):
</span><span> self.logger.warn(
</span><span> "Caught a terminating exception while dispatching to the "
</span><span> "callback, re-raising", exc_info=True)
</span><span> pygame.display.quit()
</span><span> raise
</span><span> except Exception, ex:
</span><span> self.logger.error(
</span><span> "Caught an exception while dispatching to the callback",
</span><span> exc_info=True)
</span><span> return True
</span><span>
</span><span> def finalize(self):
</span><span> pass
</span><span>
</span><span>def my_callback(self, event, *args):
</span><span> if event.type == QUIT:
</span><span> sys.exit()
</span><span> if event.type == KEYDOWN:
</span><span> self.logger.debug("my_callback: keydown: event.key: " + str(event.key))
</span><span> if event.key in (K_q,):
</span><span> sys.exit()
</span><span> elif event.key in (K_n, K_SPACE):
</span><span> self._player.skip()
</span><span> elif event.key in (K_p,):
</span><span> if self._player.playing:
</span><span> self._player.stop()
</span><span> else:
</span><span> self._player.play()
</span><span>
</span><span> return True
</span><span>
</span><span>
</span><span>def start_mainloop(self, mainloop):
</span><span> self.logger.debug("Starting mainloop")
</span><span> if not self._started:
</span><span> s = PygameEventSource()
</span><span> s.set_callback(self.my_callback, mainloop)
</span><span> s.attach()
</span><span> return
</span></code></pre>
One of Those Things You Don't Know If You Should Post in Your Blog Or on TheDailyWTF2007-04-14T13:43:00+00:002007-04-14T13:43:00+00:00Unknownhttps://juripakaste.fi/rails-json-wtf/<p>From Rails, or more specifically, ActiveSupport's json.rb:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># When +true+, Hash#to_json will omit quoting string or symbol keys
</span><span># if the keys are valid JavaScript identifiers. Note that this is
</span><span># technically improper JSON (all object keys must be quoted), so if
</span><span># you need strict JSON compliance, set this option to +false+.
</span><span>mattr_accessor :unquote_hash_key_identifiers
</span><span>@@unquote_hash_key_identifiers = true
</span></code></pre>
AudioFormat 0.4 released2007-03-09T23:44:00+00:002007-03-09T23:44:00+00:00Unknownhttps://juripakaste.fi/audioformat-0-4/<p>After a longish break, released a new <a
href="http://www.juripakaste.fi/audioformat/">AudioFormat</a> when I
noticed how aggravating the current version was :-)</p>
<p>The new version, 0.4, allows you to add files to the list of files to
convert via a file selector or by dragging. It also should handle and
report errors a bit better. And maybe it'll even recognize Vorbis
files? Although I suspect it'll get very confused if it encounters
something like Ogg Tarkin files. Will have to work on that a bit more.</p>
<p>Enjoy.</p>
Unit conversion in Java2006-12-26T12:49:00+00:002006-12-26T12:49:00+00:00Unknownhttps://juripakaste.fi/jugc/<p><a href="http://code.google.com/p/jugc/">JUGC</a> is a Java library for unit conversions. It allows you to define a set of units and their relationships (a kilogram is a thousand grams, a tonne is a thousand kilograms, etc), translations for those units (gramma is fi_FI for gram), generate a Java library from those units (well, most of the library won't be generated, but the unit definitions will be) and then deploy that library in a project.</p>
<p>So basically you'll go from unit definitions like this:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> <units>
</span><span> <unit>
</span><span> <id>fahrenheit</id>
</span><span> <alias>F</alias>
</span><span> <derive>celsius</derive>
</span><span> <from>9 5 / * 32 +</from>
</span><span> <to>32 - 9 5 / /</to>
</span><span> </unit>
</span><span>
</span><span> <unit>
</span><span> <id>rankine</id>
</span><span> <alias>R</alias>
</span><span> <derive>fahrenheit</derive>
</span><span> <from>459.67 +</from>
</span><span> </unit>
</span><span> </units>
</span></code></pre>
<p>to converting units like this:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> ConverterFactory.getConverter("celsius", "rankine").convert(42);
</span></code></pre>
<p>And the translation support is pretty simple, too:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> <units>
</span><span> <unit>
</span><span> <id>gram</id>
</span><span> <alias>g</alias>
</span><span> </unit>
</span><span>
</span><span> <unit>
</span><span> <id>kilogram</id>
</span><span> <alias>kg</alias>
</span><span> <derive>gram</derive>
</span><span> <from>1000 /</from>
</span><span> </unit>
</span><span> </units>
</span><span>
</span><span> <unittrans>
</span><span> <locale>fi_FI</locale>
</span><span>
</span><span> <translation>
</span><span> <unit>gram</unit>
</span><span> <trans>gramma</trans>
</span><span> </translation>
</span><span>
</span><span> <translation>
</span><span> <unit>kilogram</unit>
</span><span> <trans>kilogramma</trans>
</span><span> </translation>
</span><span> </unittrans>
</span></code></pre>
<p>And then in your code:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> ConverterFactory.getConverter("gramma", "kilogramma", "fi_FI").convert(42);
</span></code></pre>
<p>The conversion expressions are in RPN.</p>
<p>I have yet to make a release and it's not totally finished — it should be possible to ask for the conversions to be done with arbitrary precision math, for instance — but I believe it's pretty usable already.</p>
<p>License: BSD sans advertising clause.</p>
Stamp out the ants2006-12-23T14:22:00+00:002006-12-23T14:22:00+00:00Unknownhttps://juripakaste.fi/stamp-out-the-ants/<p>If learning a simple, conceptually mainstream language such as Ruby is too high a price to pay to get away from the madness of Ant and Maven, as it seems to be for some people commenting on <a href="http://raven.rubyforge.org/">Raven</a>, I'm inclined to think you are in the wrong line of work. And Ant and Maven are hardly zero learning curve tools just because the syntax uses XML.</p>
<p>It's possible that Raven (or <a href="http://blog.foemmel.com/jrake">JRake</a>) isn't the right solution, but the irrational fear evident, for example, in <a href="http://www.theserverside.com/news/thread.tss?thread_id=42923">this TheServerSide thread</a>, would be hard to believe if it wasn't just business as usual in the Java world.</p>
Bad excuses overheard while waiting for a burger in the night while it was raining2006-12-02T10:18:00+00:002006-12-02T10:18:00+00:00Unknownhttps://juripakaste.fi/bad-excuses/<p>Loosely translated, he was talking on the phone:</p>
<blockquote>
<p>Look, Johanna, I'm sorry, but I've got a dentist on Monday, and Hannu has a time for an ornithologist [...] I can't help it, I've got a time for an ornithologist and Hannu has an appointment with a dentist.</p>
</blockquote>
Writing to envy2006-11-17T20:12:00+00:002006-11-17T20:12:00+00:00Unknownhttps://juripakaste.fi/writing-to-envy/<p>Two remarkable pieces of writing in the blogosphere today, both ending up in the instant classics section of their respective categories.</p>
<p>The first one is a tutorial style piece exposing the inner life of a search engine. <a href="http://eigenclass.org/hiki.rb?simple+full+text+search+engine">A simple full-text search engine in 200 lines of Ruby</a>. It's concise, it's clear and after reading it anyone can write a simple search engine. If only all things in the world of code were explained that well.</p>
<p>The second one has been linked from all over the place, including some A-listers, so this probably doesn't come as news. But it's so good it deserves all the attention it can get. Socrates takes on WS-Mumbojumbo: <a href="http://wanderingbarque.com/nonintersecting/2006/11/15/the-s-stands-for-simple/">The S stands for Simple</a>.</p>
Die, Ur-Quan scum2006-11-05T02:01:00+00:002006-11-05T02:01:00+00:00Unknownhttps://juripakaste.fi/star-control-2/<p>Oh man. Oh man oh man oh man. <a href="http://sc2.sourceforge.net/">Star Control II: The Ur-Quan Masters</a>. WHY WASN'T I INFORMED EARLIER. <code>aptitude install uqm</code>!</p>
<p>It's been 14 years, and it still rocks. The good thing is I don't actually remember any of the plot.</p>
Caboodle 0.52006-11-04T23:00:00+00:002006-11-04T23:00:00+00:00Unknownhttps://juripakaste.fi/caboodle-05/<p>There's now a new version, 0.5, of <a href="http://www.iki.fi/juri/caboodle/">Caboodle</a> available. Caboodle is a Planarity knock-off for GNOME. No changes beyond making it work again in Ubuntu Edgy (seems the Cairo — or at least cairo-ocaml — API has changed a bit during the last year.)</p>
Edgy's install a bit too edgy2006-10-28T18:23:00+00:002006-10-28T18:23:00+00:00Unknownhttps://juripakaste.fi/edgy-install/<p>I think this was my first complete (well, not counting the cruft in the home directory) reinstall since Debian 0.93, but now I'm running a fresh Edgy installation. The experience was less than pleasant; the installer tried constantly to use a display mode my monitor couldn't handle (ANALOG OUT OF RANGE 81.3 kHz / 65 Hz) with the result that I had to burn the alternate install CD to get the text-mode installer. Which worked well enough, except for the fact that it configured my X again with the mode that didn't work. Oh well. Good thing I had my old X configuration around and knew how to use virtual consoles.</p>
<p>Also, the installer could have shown me the time when it asked if the hardware clock was in UTC or the local time. I guessed wrong.</p>
Noise2006-10-13T19:48:00+00:002006-10-13T19:48:00+00:00Unknownhttps://juripakaste.fi/noise/<p>A bit over a week to us moving (back closer to the center of Helsinki, yay) and my computer's power supply decided it had had enough. Every time I switched it on, it blew a fuse (a big thank you to whoever invented the kind of fuse where you don't have to put in a new ceramic thingy everytime it goes.) I was seriously considering just going out and buying a new computer, especially as I wasn't quite sure if it was the PSU or some other part, but decided that we're going to have quite enough other things to spend money on in the coming few weeks. So I settled on getting myself a new PSU, and oh boy, the beast lives again. </p>
<p>Because I wasn't quite sure what was the source of the problem and I was sort of planning on getting myself a new computer anyway in a few months, I got the cheapest power supply in the store. And you can tell. I've never been too impressed with the Antec Sonata's much-hyped quietness, but this feels like sitting next to a blow dryer.</p>
Audio format conversion2006-09-27T22:18:00+00:002006-09-27T22:18:00+00:00Unknownhttps://juripakaste.fi/audioformat/<p>Music management applications don't always do what you want when you convert audio, even if they do support it some way, and nautilus-audio-convert can be a bit of a PITA with all those windows popping up. <a href="http://www.iki.fi/juri/audioformat/">AudioFormat</a>'s supposed to be a simple, definitely not in your face, way to convert audio files from Nautilus. It uses GStreamer and the audio profiles you have defined. There's still rough bits, mainly due to me starting from zero with gst, but it'll get better. You'll need PyGTK, GStreamer 0.10 and python-nautilus. Probably newish versions of the Python stuff. And there's no automatic installation yet.</p>
<p>Screenshots behind the AudioFormat link too.</p>
When everything else fails, try Emacs2006-08-18T18:21:00+00:002006-08-18T18:21:00+00:00Unknownhttps://juripakaste.fi/when-everything-else-fails-try-emacs/<p>Even though I've defected to Eclipse for Java coding[1], and even
occasionally use Vim[2], there are times when I flee back to the tender
loving embrace of Emacs.</p>
<p>We've been doing a bit of software archaeology at work, mapping old
requirements specifications to a newer code base. The chosen method was to
append specially formatted comments to the end of each line participating in the implementation of a requirement, in the form of
<code>// req <requirement identifier></code>. How to do this in Eclipse? It doesn't have
a macro facility; there are the snippets or whatever they are called, but
using it for this purpose would be pretty painful. And the lines are
formatted to be long enough (padded with spaces) to make it easier
to read the code with the comments inserted. Which is nice and all,
but with my screen I can't even see what lines have the comments which
don't. Not good.</p>
<p>But, hey, sometimes everything isn't a nail even though you have a
hammer. There's always Emacs. 50 lines of elisp later and I have it
prompting me for the requirement id, inserting and removing the comments
with one key press and highlighting the lines with the current requirement
id. Make it truncate long lines and the only thing different from the
normal display is the pink background on the lines I'm interested
in. Because the Emacs manual (and function doc strings) is as good as it
is, all that was stupidly easy to accomplish even though I had never played
with faces and overlays before. And of course no restarts were required
while I was hacking the code, trying out every change as soon as I made
it. I'm still waiting for another environment that is as malleable as
Emacs.</p>
<p>[1] So I can't live without all the searches and other magic. Its editor is
still pretty weak.</p>
<p>[2] Vi style indentation is better than broken Emacs style indentation.
Some Emacs language modes are less than perfect. Also, AA fonts[3].</p>
<p>[3] The good news is that the <a href="http://www.emacswiki.org/cgi-bin/wiki/XftGnuEmacs">Xft-enabled emacs-unicode-2 branch</a> has been working for me now happily for a day. Smooth Bitstream Vera is love. If only it were packaged in Ubuntu, the way the main development branch is in the emacs-snapshot package.</p>
New job2006-05-26T17:53:00+00:002006-05-26T17:53:00+00:00Unknownhttps://juripakaste.fi/new-job-profium/<p>I changed employers last week. My previous employer, Entra e-Solutions, was recently merged with TietoEnator Telecom & Media. The merger was slightly unpleasant and pushed me finally into looking for a new job. I saw an interesting sounding opening for a semantic web r&d position, applied, did a six hour programming assignment, was hired. I now work for <a href="http://www.profium.com/">Profium</a>. I'm all about RDF now, baby.</p>
<p>Also, apologies for not responding to email, if you've sent me any recently: I've suffered from a network outage and I'm also having trouble connecting to my IMAP mailbox. Evolution only says "Loading..." where it should show my folders.</p>
Outcry: streaming again2006-04-24T22:35:00+00:002006-04-24T22:35:00+00:00Unknownhttps://juripakaste.fi/outcry/<p>Some former colleagues might think I'm reinventing my own wheel here. They would be right. I didn't feel like resurrecting a Python 1.5/omniORB zombie (or did it use SOAP later on? Or maybe both? I forget the details...) that didn't do everything I needed it to do and probably was mostly horrible. Anyway.</p>
<p>No guarantees for it working for anyone else, or for it to be interesting for anyone else, but <a href="http://www.juripakaste.fi/dl/outcry">Outcry</a> is a remotely controllable (via plain old XML over HTTP messages) <a href="http://www.icecast.org/">Icecast</a> streamer. It's written in <a href="http://www.ruby-lang.org/">Ruby</a> (which I'm enjoying, a lot) and requires <a href="http://www.icecast.org/download.php">libshout</a> and <a href="http://ruby-shout.rubyforge.org/">ruby-shout</a>. It streams MP3, converting FLACs and Vorbis files on the fly.</p>
<p>It's the latest step in my ongoing quest to play music in our living room without CDs. My <a href="http://www.kiss-technology.com/?p=dp1500&v=users">KiSS DP-1500</a> isn't really very good at playing OGGs (it crashes a lot) and even worse at FLACs (it doesn't do them). Also, it doesn't play WAVs. My computer can't convert from FLAC to MP3 quickly enough for things to work out OK when I let the KiSS box control what it's playing, so next try is moving the control to this end of the pipe. Which kind of sucks. Oh well.</p>
<p>I should probably release my own kissd replacement written in OCaml, too, but it's even less ready for the masses. Email if you're interested.</p>
Tag, Milou, Tag2006-03-15T17:23:00+00:002006-03-15T17:23:00+00:00Unknownhttps://juripakaste.fi/tomboy-leaftag/<p><a href="http://www.beatniksoftware.com/tomboy/">Tomboy</a> meets <a href="http://www.chipx86.com/wiki/Leaftag">Leaftag</a>.</p>
<img src="http://www.iki.fi/juri/tomboy-leaftag/tomboy-leaftag-1.png" alt="Tag note" />
<img src="http://www.iki.fi/juri/tomboy-leaftag/tomboy-leaftag-2.png" alt="Select tags" />
<img src="http://www.iki.fi/juri/tomboy-leaftag/tomboy-leaftag-3.png" alt="Find the tagged note with Deskbar" />
<p>And finally, after <a href="http://www.juripakaste.fi/dl/tomboy-leaftag/darcs/README">convincing</a> gnome-open to use Tomboy to open note:// URIs, selecting the entry in the Deskbar menu will open up the freshly-tagged note.</p>
<p>Yeah, it's bit of a hack, but it's a start.</p>
<p>You need Leaftag (the latest version from SVN plus, at least at the moment, a <a href="http://www.juripakaste.fi/dl/libleaftag-tag-dialog-response-new-sources.patch">patch</a> for libleaftag-gtk), .NET bindings for leaftag (no tarballs yet but a <a href="http://www.juripakaste.fi/dl/leaftag-sharp/darcs">mirror</a> of my darcs repo is available) and the Tomboy plugin (again, there's a darcs <a href="http://www.juripakaste.fi/dl/tomboy-leaftag/darcs">mirror</a>.)</p>
Thank you2005-10-06T21:15:00+00:002005-10-06T21:15:00+00:00Unknownhttps://juripakaste.fi/new-copyright-law-passed/<p>Right, so we got the utter turd of a new copyright law. Here are the <a href="http://www.eduskunta.fi/triphome/bin/thw/trip/?${BASE}=aanestys&${THWIDS}=3.57/434472&${html}=aan5000&${THWURLSAVE}=57/434472&${maxhits}=1000">results</a> of the final vote in the parliament. "Jaa" is for the new law, "ei" against the law, "poissa" is away. </p>
<p>A friendly message to anyone there who might be listening: I won't vote in any election for anyone who wasn't against the law.</p>
Caboodle 0.3 and 0.42005-08-12T13:08:00+00:002005-08-12T13:08:00+00:00Unknownhttps://juripakaste.fi/caboodle-04/<p>Two new versions of <a href="http://www.iki.fi/juri/caboodle/">Caboodle</a>: now it has better level generation logic, it caches level images so the whole thing doesn't get redrawn all the time, it includes a randomize level menu command and an option to display line intersections interactively as you drag vertices around. The last thing will probably grind the whole thing to a halt on larger levels.</p>
Caboodle 0.22005-08-08T13:45:00+00:002005-08-08T13:45:00+00:00Unknownhttps://juripakaste.fi/caboodle-02/<p>So I <a href="http://www.iki.fi/juri/caboodle/">released</a> 0.1 and 0.2. Now Caboodle has a menu bar (whee), an icon and a .desktop file.</p>
<p>To clarify something: you do not need OCaml or the Gtk+ and Cairo OCaml bindings to run Caboodle, just to compile it. I do not provide binaries at the moment, but if there's demand I can put up a pre-compiled Linux/x86 version.</p>
Caboodle2005-08-06T19:30:00+00:002005-08-06T19:30:00+00:00Unknownhttps://juripakaste.fi/caboodle/<p><a href="http://www.iki.fi/juri/caboodle/">Caboodle</a> is a puzzle game. See:</p>
<img src="http://www.iki.fi/juri/caboodle/Screenshot-Caboodle.png" />
<p>As you are probably able to tell, it's a clone of <a href="http://planarity.net/">Planarity</a>. My excuse for the blatant clone is that I wanted to learn <a href="http://ocaml.org/">OCaml</a> and the game was a nice, simple project to tackle. And at the same time, I got to learn a bit of <a href="http://cairographics.org/">Cairo</a>, too. Of course, I was — and mostly still am — totally clueless about graphs and the related math.</p>
<p>Still, the end result, even its current state, isn't too bad. It's definitely faster than the original in most aspects, which shouldn't come as a surprise given the ass-kickingness of the OCaml compiler especially compared to Flash. The most significant slowness is due to the currently very naïve redraw logic: every time something moves, everything is redrawn. As you can probably guess, it starts to slow down with a large graph and fast mouse movement. I'll have to see about fixing that. And I'm not totally satisfied with the level generator, even though that's the part I've spent the most time with (no, the math didn't come back easy after too many years of not using it at all, you can all point at the source code and laugh); it does generate too many vertices with just two edges. But I think that should be pretty easy to fix.</p>
<p>About OCaml: a nice language with an impressive compiler, and quick to learn, at least up to a point. I haven't used any objects in Caboodle, only the stuff that comes with LablGTK2. And I didn't write any functors. Or a bunch of other stuff. And I still don't find code written by others very quick to read and the compiler error messages can be pretty puzzling, at times. Part of the learning curve is probably caused by the fact that I've never before used a language with a modern (you know, from the 70s) type system, just the basic stuff provided by C, Java, C# and friends. Still, I'm not quite sure I like the ML syntax, but I suppose it's not too bad. And I wonder if there are some limits to the language that I would have hit on a larger project. It is mostly very, very strict, and doesn't offer any of the runtime type frobbing that Java does with its happy casting back and forth, reflection and stuff.</p>
<p>Cairo turned out to be surprisingly easy, too, at least for this simple a project. I have to thank Olivier Andrieu's cairo-ocaml examples, though, without them the going would have been a lot rougher. But I can't really compare it to other systems, because I've never done much graphics programming.</p>
<p>Also, turns out I'm not the first one with the same idea: back before I had anything showing up on screen, xiphmont was on #gtk+ posting screenshots of <a href="http://www.livejournal.com/users/xiphmont/3266.html">his version</a>. Oh well. Seemed to be a silly reason to stop, given that my objective wasn't to write the game as much as to learn the tools, so I ploughed on.</p>
<p>I haven't yet done a release, but it should be ready for 0.1. I'll post it at <a href="http://gnomefiles.org/">gnomefiles</a> once I get around to it. In the mean time, it's available in the darcs repo from the Caboodle home page.</p>
<p>If you wonder what the name is about, it's just a random word I got out of /usr/dict/words. That's my preferred way of naming software.</p>
My First Gtk+ Patch2005-07-07T22:12:00+00:002005-07-07T22:12:00+00:00Unknownhttps://juripakaste.fi/my-first-gtk-patch/<p>Whee, my first Gtk+ patch — or at least something not completely unlike it — was <a href="http://bugzilla.gnome.org/show_bug.cgi?id=145121">accepted</a>.</p>
River of Gods, The Family Trade2005-06-18T11:41:00+00:002005-06-18T11:41:00+00:00Unknownhttps://juripakaste.fi/river-of-gods-and-family-trade/<p><a href="http://www.livejournal.com/users/ianmcdonald/">Ian McDonald</a>'s <em>River of Gods</em> is absolutely fantastic, it rocked my world. The picture it paints of a near future India is incredibly vivid and compelling. Made me want to visit India. I haven't read any of the other <a href="http://dpsinfo.com/awardweb/hugos/2005.html">2005 Hugo nominees</a>, but at least this book is totally deserving.</p>
<p>Also, I shouldn't have read (well, haven't yet finished, I've been reading it on the commute) <a href="http://www.antipope.org/charlie/blosxom.cgi/2005/02/21/#writing-107"><em>The Family Trade</em></a> by <a href="http://www.antipope.org/charlie/index.html">Charles Stross</a>: now I have to wait for the sequels. It's smart, fun and totally not your run of the mill fantasy. And just the right size: as the author commented on <a href="http://nielsenhayden.com/makinglight">Making Light</a>, <a href="http://nielsenhayden.com/makinglight/archives/006389.html#83554">short books are the new long</a>.</p>
Notes from a honeymoon to Florence and Milan.2005-06-04T12:33:00+00:002005-06-04T12:33:00+00:00Unknownhttps://juripakaste.fi/notes-from-the-honeymoon/<ul>
<li>Florence is beautiful. Consistent renaissance style with a few dabs
of the medieval and modern makes for a wonderful city. Milan less so,
with mostly buildings that are ugly, pompous or <a
href="http://en.wikipedia.org/wiki/Milan_Central_Station">both</a>.
However, Milan has more greenery, which is nice: Florence could really
use a few parks in the center of the city, especially of the
non-fenced kind.</li>
<li>Milan felt like a real city, whereas Florence had this slight
feeling of being an open-air museum and a tourist trap. But with that
many tourists in that small a city, it's inevitable, I guess.</li>
<li>An awful lot of those tourists were Americans.</li>
<li>We got mostly indifferent or unfriendly service in Milan. However,
we were there only for one day, so it's plausible we only had bad
luck. Florence was better.</li>
<li>The weather was totally scorchio. However, a nominally cloudless sky
in Milan with 31 degrees celcius was actually a greyish blue. We
couldn't figure out if this was because of pollution.</li>
<li>Avoid the restaurant of Una Hotel Century in Milan like the plague.
When "a glass of white wine" (no wine list was forthcoming) means the
bottom of a chardonnay bottle with some sauvignon blanc on top and the
arrival of the dessert is marked by the unmistakable sounds of the
microwave, you know you aren't in for a great dining experience.
Otherwise the hotel was pretty decent.</li>
<li>Booking a room at Hotel Medici in Florence was a mistake. Maybe we
expected too much from a two star hotel, but we went roaming the city
streets for a nicer place to stay at as soon as we arrived. Sofitel
was pricey, but I guess every hotel in that city is.</li>
<li>The food (and wine) was mostly fantastic, as long as you avoided the most obvious
touristy places with menus in six languages and people in front trying to lure more customers in.</li>
</ul>
This language thing2005-05-12T19:20:00+00:002005-05-12T19:20:00+00:00Unknownhttps://juripakaste.fi/gnome-languages/<p>Not that I particularly expect this GNOME language issue to be resolved anytime soon, but still: assuming that the choices are Java and C#, do people actually want to standardise/bless/whatever the language, the runtime standard (that is, "JVM" or "CLR"), one particular runtime ("Kaffe" or "Mono"), or some combination thereof?</p>
<p>On a totally unrelated note, I always wonder why on earth Microsoft chose a totally ungoogleable name for their language.</p>
Introducing Lukutoukka2005-05-07T17:18:00+00:002005-05-07T17:18:00+00:00Unknownhttps://juripakaste.fi/introducing-lukutoukka/<p><a href="http://www.iki.fi/juri/lukutoukka/">Lukutoukka</a> is a speed reader for the GNOME desktop. Inspired by <a href="http://trevor.typepad.com/blog/2004/07/speed_read_your.html">this post</a> (or really, originally, <a href="http://trevor.smith.name/EST/">the Eastern Standard Tribe speed reader</a>.)</p>
<p>It doesn't look like much, especially not without animation, but here you go, anyway:</p>
<img src="http://www.helsinki.fi/~pakaste/lukutoukka/Screenshot-Lukutoukka.png" />
<p>The idea is to push words from a text file to the screen one at a time at a quick pace. After a while it feels like your brain is melting, but you do end up reading the text far faster than you would scanning a page because your eyes and mind don't get a chance to wander. Depends on your viewpoint and the text you're reading whether that is good or bad :-)</p>
<p>It's also the first significant piece of code I've written in Scheme. I used Guile. It's not the fastest or the most feature-packed implementation out there (compare to <a href="http://www.plt-scheme.org/software/mzscheme/">MzScheme</a>, <a href="http://www-sop.inria.fr/mimosa/fp/Bigloo/">Bigloo</a> and <a href="http://www.call-with-current-continuation.org/">Chicken</a>) but it does have <a href="http://www.gnu.org/software/guile-gnome/">a rocking Gtk+/GNOME binding</a>.</p>
<p>Oh, and to keep life exciting, I'm using <a href="http://www.darcs.net/">Darcs</a> for version control. And loving it.</p>
Two literary links2005-04-30T14:48:00+00:002005-04-30T14:48:00+00:00Unknownhttps://juripakaste.fi/two-literary-links/<p>From Charles Stross, on why <a href="http://www.antipope.org/charlie/blosxom.cgi/2005/04/18#state-of-sf" title="On swings, roundabouts, and long-term trends in science fiction">British SF is these days <em>teh rawk</em></a>.</p>
<p>And a great <a href="http://www.believermag.com/issues/200504/interview_mieville.php" title="The Believer - Interview With China Miéville">interview with China Miéville</a>.</p>
<p>Books by both are highly recommended, by the way.</p>
Ctrl-Alt-Backspace aargh2005-04-29T22:28:00+00:002005-04-29T22:28:00+00:00Unknownhttps://juripakaste.fi/ctrl-alt-backspace/<p>I've lately acquired a habit of using ctrl-backspace when editing text. I also tend to use alt quite a bit when moving around text, especially if said text consists of sexps. And suddenly I see why the DontZap option in the X server is a pretty good idea.</p>
Married and stuff2005-03-05T11:28:00+00:002005-03-05T11:28:00+00:00Unknownhttps://juripakaste.fi/married/<p>So we got married last Saturday. Yay! The night after, we started suffering from what appeared to be food poisoning. My mother and one of the bridesmaids had the same problem. That was not so cool. I ended up staying home sick until Wednesday, Minttu the whole week.</p>
<p>The honeymoon will have to wait a bit, though. We're going to Florence, and we'd rather go there when it's not freezing in northern Italy.</p>
Python is not Java2005-01-30T12:50:00+00:002005-01-30T12:50:00+00:00Unknownhttps://juripakaste.fi/python-is-not-java/<p>I recently read some old stuff on <a href="http://dirtsimple.org/">Phillip J. Eby's blog</a>, and the <a href="http://dirtsimple.org/2004/12/python-is-not-java.html">Python Is Not Java</a> entry really hit home. I'm not a new Python programmer, I've been using it from around 1996 or so, about as long as Java. But every now and then I see Java-isms creep into my code. It's usually after I try some fancy approach and later on come to wonder why my code stinks. I've been especially guilty about writing completely unnecessary getters and setters and doing (or even converting module level functions to) class methods that end up being just painful to use.</p>
<p>The parts that are more Lisp-inspired tend to be a whole lot better. I've never yet regretted returning a function from a function. Too bad about Python's lame lexicals.</p>
Sweet screenshots and various other things2005-01-11T22:38:00+00:002005-01-11T22:38:00+00:00Unknownhttps://juripakaste.fi/launch-screenshots-and-stuff/<p><a href="http://micke.hallendal.net/archives/2005/01/imendio_hackath.html">GNOME Launch Box</a> is looking sweet. Well, at least from the screenshots, not that I've tried it yet. I was hacking in the autumn on something a bit similar; it's called Pepper but it sits abandoned for the moment. (For anyone interested, it's written in Python and available with tla at juri@iki.fi--2004a/pepper--mainline--0.1, http://www.helsinki.fi/~pakaste/arch/) If the hack by the Imendio boys is any good, it can stay abandoned.</p>
<p>In other news, feeling that PyGTK is becoming mainstream, I seeked out a new ghetto and started hacking with <a href="http://common-lisp.net/project/lambda-gtk/">λgtk</a> and SBCL. λgtk is a bit painful to install at the moment, requiring a new core with the sbcl-af stuff for SBCL, but it looks like Brian Mastenbrook is <a href="http://article.gmane.org/gmane.lisp.steel-bank.devel/4083">working on it</a>. I was thinking of using OCaml, but then decided that the type inference stuff of CMUCL/SBCL is good enough a compromise between decent typing and flexibility. If anything comes of this project, I'll probably have to take a look at the libglade bindings I pointed at previously and also bind gconf and stuff.</p>
<p>Also, in the category of nifty languages building on other people's VMs, yet another one I wasn't previously aware of: <a href="http://nice.sourceforge.net/index.html">Nice</a> doesn't look too bad. Also see <a href="http://scala.epfl.ch/">Scala</a>, <a href="http://boo.codehaus.org/">Boo</a> and <a href="http://nemerle.org/">Nemerle</a>. For the record, I think Groovy looks like what happens when a dozen languages collide with each other and someone is picking up the pieces and trying to match them together. With little success.</p>
Moving, part two2004-10-30T12:06:00+00:002004-10-30T12:06:00+00:00Unknownhttps://juripakaste.fi/moving2/<p>We've now lived here for one week, and it's slowly starting to look like home. We've unpacked half of the boxes and assembled most of our furniture. We even have some curtains, although not quite enough. And we've painted one wall; the rest are still all white, we are going to paint three more walls at some point.</p>
<p>The new place is farther from city centre than the previous apartment, but I've discovered it's not quite as bad as I thought it would be. With luck and good timing, you can make downtown in less than half an hour. The commute time varies a lot: I've made it in something like 35 minutes, but it's also taken an hour once.</p>
Geeks and cameras2004-09-18T11:52:00+00:002004-09-18T11:52:00+00:00Unknownhttps://juripakaste.fi/geeks-and-cameras/<p>Reading Planet GNOME recently, I'm reminded of something a co-worker said while we were in Prague: It didn't take more than the invention of the digital camera to turn this bunch of engineers into Japanese tourists.</p>
<p>I was one of the people who was snapping pictures all the time.</p>
Prague2004-09-06T21:00:00+00:002004-09-06T21:00:00+00:00Unknownhttps://juripakaste.fi/prague/<p>Went to Prague last weekend on a company trip. Fabulous city, cheap and decent beer. Culinary side of things wasn't that impressive, although it was better than Lisbon <a href="http://www.iki.fi/juri/blog/lisbon.html">where we were last year</a>. Main point of excitement on the trip was four people of our group getting their luggage stolen two hours or so after our plane landed, when they left them in our minibus and the driver wasn't guarding it. </p>
<p>We went to see <a href="http://www.praguepoint.cz/Eng/theatreblacklight-en.html">Faust</a> in a black light theatre. It's apparently a local speciality. As a co-worker put it, it was slightly worse than a piece of shit. We were all in agreement. Never, never again.</p>
<p>Also went to see the Czech - Germany world cup hockey match, which wasn't anything to write home about (not that it prevents me from blogging about it.) The result was clear from the start, even though the Czech team played a lazy game. They did win 8-2 or something like that, going on pure routine. It was the second time in my life I've been to a hockey game, and the commercialism was something painful, with multiple ad breaks in the middle of the periods. The tackiest moment of all was someone proposing (and apparently getting yes for an answer) to his girlfriend in the middle of the game, with the whole arena watching it on the screen. I thought that happened only on Friends.</p>
Straw's character set problems2004-07-07T14:01:00+00:002004-07-07T14:01:00+00:00Unknownhttps://juripakaste.fi/straw-character-set-problems/<p>Oh, and: If you are still experiencing character set troubles (like seeing ** (straw:14064): WARNING **: Invalid UTF8 string passed to pango_layout_set_text()) with Straw 0.24, you should probably try to remove the feed a subscribe to it again. There might be old data in your DB the new version can't do much about.</p>
Straw 0.242004-07-07T12:27:00+00:002004-07-07T12:27:00+00:00Unknownhttps://juripakaste.fi/straw-0-24/<p>Some pretty cool shit here.</p>
<p>Subscribe categories to OPML feeds: make <a href="http://www.gnome.org/~jdub/">Jeff</a> work for you! Create a category, make it subscribed to Planet GNOME's OPML feed, and you'll have an always up-to-date set of GNOME blogs in your aggregator. No support for FOAF blogrolls yet, though. Maybe in the next release.</p>
<p>Notification area thingy! If you have unread articles, there's a Straw icon in your notification area that tells you how many unread items you have. And click on it to bring Straw to front and to read those articles.</p>
<p>(Mostly) asynchronous subscription process! No longer will the subscribe dialog freeze Straw on you while it's working. Well, not very much at least. And there's less clicking involved, too.</p>
<p>No more feed information display in the main window screwing up the layout! Feed link is available in the article header, rest of the stuff in the feed properties dialog.</p>
<p>Prettier article view!</p>
<p>Overall less brokenness! Hopefully including a lot fewer locale/encoding problems.</p>
<p>Get it while it's hot from the <a href="http://savannah.nongnu.org/download/straw/">download area</a>.</p>
Images in the HIG2004-06-05T22:17:00+00:002004-06-05T22:17:00+00:00Unknownhttps://juripakaste.fi/images-hig/<p>Running <a href="http://www.catb.org/~esr/imgsizer/">imgsizer</a> on my local copy of the HIG made it <em>so</em> much nicer to read. Suddenly, the browser actually lands in the correct place when you click on a link in the table of contents.</p>
Yay, easter2004-04-09T13:12:00+00:002004-04-09T13:12:00+00:00Unknownhttps://juripakaste.fi/easter/<p>Four days off work does wonders. I hope. Been busy last couple of weeks. With mind numbed by J2EE/WebSphere pain, haven't had any energy to work on Straw or do much else with the computer, I've just spent most of my free time at the gym or cooking and doing home stuff. Or playing FFX2. I hope I'll squeeze off a few hours today and tomorrow for Straw. First thing on the agenda: sync up with Gnome CVS, last few times I tried I got a weird complaint about a conflict I couldn't locate. Second: make the configuration writer write into a temp file and copy that over the old configuration when it's done; I just lost my Straw config due to a crash. I wish I knew what piece of hardware is causing those. </p>
<p>At least my Gnome 2.6 setup is starting to work, slowly. I used to have a ~/.Xsession file that set up some locale variables and stuff, but apparently that isn't a good idea anymore. With it around, the normal Gnome session refused to work. Possibly I should have exec'd gnome-session or something there, I have no idea. Oh well, I nuked it, at least the thing starts now. I wonder what's the approved location for that kind of stuff… Another thing that still has to be figured out is why my ~/.Xmodmap no longer works, now I have to call xmodmap manually to get my keyboard configuration. Maybe one of these years I should really look into that new-fangled xkb thing. </p>
<p>But now, I think, is time to go out running, the weather is beautiful and relatively warm at +7 degrees celcius. Here's hoping it won't be as painful as the first time in the spring usually is. I at least <em>should</em> be in a slightly better condition than I usually am this time of year.</p>
Downtime brought to you by stupidity2004-03-19T20:37:00+00:002004-03-19T20:37:00+00:00Unknownhttps://juripakaste.fi/downtime/<p>I've had this problem with random crashes with my new-ish computer which I haven't managed (or bothered) to pinpoint. It's something hardware related, but that's all I know. Last saturday the thing crashed once again and after I rebooted it, it wouldn't start. There was, on the console, some complaint about missing modules for INPUT products and a last line with nothing on but "pci", then nothing happened. I pressed enter a couple of times, decided it was broken and shut it down frustrated. Reiserfs3 has gotten some files confused when the system has crashed a few times (hooray for journaling... I should really try ext3/XFS/JFS/Reiser4), so I thought that it had finally screwed up something vital. </p>
<p>At the beginning of the week I downloaded a CD image of Knoppix, burnt it to a CD and and booted my computer with it. Worked like a charm (first time I used KDE in a while. Still not to my taste.) I wasn't feeling much like doing forensics on the piece of junk so I didn't properly get to it before today; fscking my HDs found nothing strange so I just reinstalled the kernel and rebooted the machine. The same problem. Then I started reading the boot process output a bit better; among other things I noticed it had already loaded pretty much everything including sound card and network adapter drivers and the error messages were about hotplug. Oh boy. The first thought was to again reboot with Knoppix and tinker with the boot process; the second one was to press Ctrl C.</p>
<p>Next time I'll try that first, not a week afterwards.</p>
Lessons learned2004-01-31T15:59:00+00:002004-01-31T15:59:00+00:00Unknownhttps://juripakaste.fi/lessons-learned/<p>We're working on dropping the <code>mxDateTime</code> dependency from Straw for version 0.22. It's proving to be a bit more difficult than expected, and exposes two design blunders made early in the development. The other is easy to circumvent, the other one isn't.</p>
<p>First, you should store a version identifier in all files you create that you expect to read back. Including the data store. That's easy enough to deal with later on, luckily, just assume that in the case of absent version identifier, it's version 1.</p>
<p>The other is the result of too eager use of Python's <code>pickle</code> mechanism and a total brain-fart. For whatever reason, I've been storing <code>mxDateTime</code> instances in the database. Duh. Now to get rid of those, we can't drop the <code>mxDateTime</code> dependency completely. Luckily we can restrict it to the conversion function called, if necessary, on start up, but still, that's something I should have seen coming. It's not as if it's difficult to serialize a date object independently of the object format.</p>
<p>I'll just close this with a quote from Franklin P. Jones, whoever he is: <q>Experience is that marvelous thing that enables you recognize a mistake when you make it again.</q></p>
My thumb hurts2004-01-10T01:10:00+00:002004-01-10T01:10:00+00:00Unknownhttps://juripakaste.fi/jak2/<p>I bought myself yesterday <a href="http://www.naughtydog.com/jak2/index.html">Jak II</a>. Not only because it was developed using Lisp. It rocks. It's also at times maddeningly difficult. So I just spent an hour with the blow up the ammo mission, one of the first ones. Oh well. Last time I played a platformer, the hero featured in the state of the art title was a green rabbit.</p>
<p>No, it's not making Straw's progress towards 0.22 any faster.</p>
Christmas in Madeira2003-12-28T16:14:00+00:002003-12-28T16:14:00+00:00Unknownhttps://juripakaste.fi/madeira/<p>Spent the christmas in Funchal, Madeira. It would have been more fun to be home for christmas (apparently -11 degrees celsius and snow) and be at Madeira some other time (a week of listening to christmas carols gets pretty taxing, and half the time everything was closed.)</p>
<p>The island was in some places breathtakingly beautiful, with lovely flora and absolutely stunning views (or scary views, depending on what you think about a bouncy, narrow road with a low fence and a 200 metre drop right next to it.) Walking along the <a href="http://www.madeira-levada-walks.com/">levadas</a> was quite an experience, too: we only traced one suburban levada and a part of another easy one a bit west of Funchal, but it was a lot of fun and we got to see the nature — and the sheer bluffs — a lot better than through a car window. And the weather was a lot nicer than what weather.com would have had us believe: instead of constant rain and thunderstorms, it was mostly sunny. Some very nice colonial architecture, too. And when the shops were open, there was a surprisingly large number of them, for such a small town.</p>
<p>But then there was the food. And the restaurant scene in general. I wasn't expecting a culinary dream world, having <a href="http://www.iki.fi/juri/blog/lisbon.html">visited</a> Lisbon earlier in the fall, but still it was a disappointment. Probably 95% of the restaurants served similar cuisine, I guess it's what would be called Madeiran. And in all those restaurants, the menu was nearly identical. There were the four salads, the four soups, the six omelettes, a bunch of fish and red meat courses. As a vegetarian I didn't sample most of that — the tomato and onion soup with a boiled egg wasn't nothing to write home about, though — but a reliable source informed me that at least the fish was mostly mediocre (no <a href="http://gastronomia.oninet.pt/receita/receita.asp?codReceita=555">espada with banana</a> next time, I guess.)</p>
<p>And the wine... I know there are good portuguese wines, but when you can't remember the brands you are left wandering in a wilderness of barely drinkable dishwater. Occasionally, with luck, it was OK, but sometimes you just had to concentrate on water. And of course trying to find anything from, say, France, Italy or Spain was pretty much a lost cause. Mostly it was all Portugal all the time. Mind you, the local beer was very nice light lager, just the sort of stuff you want in hot weather.</p>
<p>To top off the wonderful dining experience, add totally ham-handed service. Hot tips for waiters everywhere: don't remember who ordered what. Don't remember who drinks what. Serve the dishes one at a time, not bothering to serve everyone in a table at the same time. Ignore the empty wine glasses of customers; when they start pouring wine themselves, come over, remove the bottle from the hand of the customer without a comment, and see if there are any empty glasses left, for example, in front of people who don't drink the wine in question. Or half-empty, so you can make them too full. You wouldn't want to be forced into a pouring intervention too soon again, would you? Oh yeah, and a really nice touch is to forcefully make the customer read the awful Finnish translation of the menu when she is reading the English version. I know tourists are probably tedious, but it would be nicer if they didn't treat us as children and try to speak the three words they know of our native language, especially as we have little trouble speaking English which will probably make communication a lot easier.</p>
<p>Mind you, there were exceptions. And I'm sure you could find more if you looked well enough. There was a decent indian restaurant there, and Villa Cipriani, the Italian restaurant of <a href="http://www.reidspalace.com/">Reid's Palace</a>, was fabulous, with flawless service, a good wine list and superb food. The quality was mirrored in the price, too, though.</p>
<p>I don't know, of course you can't avoid all the effects of an economy ran by tourism, but having people in front of the restaurants trying to get you in very aggressively is unpleasant when you are just trying to have a walk. Not to mention the taxi drivers who think they know better than you do where you want to go. No thanks, we've already seen the botanical gardens, we just want to get to the centre of the city. No, we're not going to Monte at this time, our plane is leaving in four hours. And no, no levadas either, just get us to the centre of the city, will you? </p>
<p>It was twenty years since the last time I was in Madeira, and I have to admit I don't remember too much. Maybe we'll go and see it again in another twenty years.</p>
Scheming for freecell2003-11-18T00:18:00+00:002003-11-18T00:18:00+00:00Unknownhttps://juripakaste.fi/freecell-supermove/<p>I didn't really feel like starting work on Straw's subscription categories today, so I was procrastinating playing freecell. Then I got once again annoyed by the fact that the Aisleriot freecell doesn't support <a href="http://home.earthlink.net/~fomalhaut/fcfaq.html#Supermove">supermove</a>, so I decided to <a href="http://bugzilla.gnome.org/show_bug.cgi?id=127235">fix</a> that instead. I think that was the first time I did anything in Scheme beyond a hello world (so the patch isn't big, but I needed a bit more code to create an initial deal to make testing things fast.) Gotta love Aisleriot, it was just so much nicer than it would have been had it been in C.</p>
PCL-CVS2003-10-16T23:07:00+00:002003-10-16T23:07:00+00:00Unknownhttps://juripakaste.fi/pcl-cvs/<p>Argh. PCL-CVS (2_9_9, comes with Debian's XEmacs package) is giving me Connection refused to <a href="http://savannah.gnu.org/">Savannah</a> when plain old CVS works just fine, even from a <em>shell</em> buffer. Can't figure out why.</p>
<p>...</p>
<p>Oh well, it works from GNU Emacs. Maybe time to switch Emacs families once again, I've been now three years or so with XEmacs, that's about as long as I usually stick with one of them.</p>
Lisbon2003-09-08T17:58:00+00:002003-09-08T17:58:00+00:00Unknownhttps://juripakaste.fi/lisbon/<p>Flew to Lisbon early Friday morning with coworkers, saw some sights, got plenty drunk and afterwards a two day long splitting headache caused by neck pain (luckily painkillers helped at least the first day), ate several servings of absolutely lousy food (you get the idea that the Portuguese aren't big on vegetarianism), thought the city was really beautiful, was extremely happy to get back home at midnight yesterday.</p>
Burden of history2003-08-21T23:02:00+00:002003-08-21T23:02:00+00:00Unknownhttps://juripakaste.fi/clipboard-2/<p>It's true that <a href="http://www.hole.fi/jajvirta/weblog/20030821T2201.html">users shouldn't have to know</a> how things like the X Window System clipboard/selection mechanism <a href="http://www.helsinki.fi/~pakaste/blog/clipboard.html">works</a>. But really, they don't have to: as the freedesktop.org documents points out, users are really expected to use the CLIPBOARD mechanism which works pretty much like clipboards on the competing platforms and live happily without the select/middle click thingy. Jarno, you only have this problem because even without an unix beard, you've been using these things long enough to know the PRIMARY selection mechanism.</p>
Clippings2003-08-21T17:27:00+00:002003-08-21T17:27:00+00:00Unknownhttps://juripakaste.fi/clipboard/<p>I expect Jarno has heard about this already, but just to nitpick a bit: in X Window System, selecting a piece of text does not normally copy it to the clipboard, despite this being <a href="http://www.hole.fi/jajvirta/weblog/20030820T0801.html">a widely-held belief</a>. Or well, it does, but it's not the only clipboard, and in X parlance, it's usually called the PRIMARY selection. There are three standard selections: PRIMARY, SECONDARY and CLIPBOARD.</p>
<p>To copy things to "the" clipboard — the CLIPBOARD selection, that is — you usually have to invoke an explicit copy operation, typically via the menu choice Edit -> Copy (or your localized equivalent) or via a keyboard shortcut, typically Ctrl-C. You might notice some similarities to certain other systems. And the same goes for the opposite actions: middle clicking only pastes the PRIMARY selection. To paste the contents of the clipboard, Edit -> Paste or Ctrl-V is typically the way.</p>
<p>For more information, see <a href="http://www.freedesktop.org/standards/clipboards-spec/clipboards.txt">the explanation at freedesktop.org</a>.</p>
Straw 0.192003-08-17T02:57:00+00:002003-08-17T02:57:00+00:00Unknownhttps://juripakaste.fi/straw-0-19/<p>Yow. Just released it. Finally. It's bloody late. Late as in it was supposed to be out three months ago, and late as in it was supposed to be out earlier today. Or yesterday, really, seeing what the clock says. It involved adventures with CVS (My First Branch) and stuff like that. Plenty of new stuff there, though mostly rather minor. BTW, have I ever mentioned that doing releases is way too much a hassle? I announce them on the <a href="http://www.nongnu.org/straw/news.html">Straw news page</a> (hasn't updated yet when writing this, but the updated pages are in CVS), Freshmeat and the GNOME Software Map (which I see still sports the old look), and it always feels like it takes ages.</p>
<p>I hope I didn't screw up too badly when fooling around with CVS.</p>
Straw and Dashboard2003-07-11T13:54:00+00:002003-07-11T13:54:00+00:00Unknownhttps://juripakaste.fi/straw-dashboard/<p>As you might have noticed if you follow these things, I actually did something about that <a href="http://www.iki.fi/juri/blog/dashboard.html">Dashboard thing</a> and now the changes to Straw are in CVS. I don't have screenshots of it myself, but Nat <a href="http://www.nat.org/dashboard/straw.png">has</a>.</p>
Groan like it's 19972003-07-08T22:45:00+00:002003-07-08T22:45:00+00:00Unknownhttps://juripakaste.fi/aix/<p>I used AIX today. The last time was in 1997. Except for the fact that the black POWER (tee hee) box packed quite a bit more punch than the horrible Apple-branded POS the size of a small fridge I had to use six years ago, it felt like nothing had changed. CDE. ksh. It all came rushing back.</p>
<p>Man, this old slow Linux box running GNOME feels <em>good</em>.</p>
UI infatuation2003-06-09T19:53:00+00:002003-06-09T19:53:00+00:00Unknownhttps://juripakaste.fi/firebird-and-radial/<p>As an excercise in pain, I've been installing quite a bit of software on a old IBM ThinkPad 600 running Windows 2000 today. I only have to use the machine for two weeks or so, but for that time I do need software. Like a sensible browser. And because W2K makes <a href="http://galeon.sourceforge.net/">Galeon</a> and <a href="http://epiphany.mozdev.org/">Epiphany</a> non-options, the next choice was <a href="http://www.mozilla.org/projects/firebird/">Mozilla Firebird</a>.</p>
<p>It's not too bad. It seems to fit in a lot better in the Windows UI than Seamonkey, the previous version of Mozilla's browser, ever did in GNOME. Maybe because my Windows looks the same as every other Windows out there, but my GNOME is running non-default theme (oh, I just checked: Firebird is better at mimicking GNOME than Seamonkey is. Cool.) Mozilla does manage to assimilate some of the customizations, but not nearly all. It's fast, too; it feels faster than IE6.</p>
<p>And, man, I stumbled upon the coolest thing. The <a href="http://www.gamemakers.de/mozilla/radialcontext/index.html#about">RadialContext</a> extension is <em>sexy</em>: good looking, fast, functional. I've heard of pie menus before, but never used one. Partly because all the screenshots always looked butt-ugly. Well, not with RadialContext. In fact, in real life it looks better than in the screen shots. And it feels good to use. It does mostly everything gestures do, but better. Fast and easy gestures accomplish all the operations in the context menus, and you get visual help while still learning where everything is.</p>
<p>Highly recommended.</p>
Mark Watson on automatic accessors for Java2003-05-05T15:15:00+00:002003-05-05T15:15:00+00:00Unknownhttps://juripakaste.fi/automatic-accessors-for-java/<p>Mark Watson <a href="http://radio.weblogs.com/0115954/2003/05/02.html#a114">proposes a change</a> to the Java language, in the form of making it possible to automatically generate accessor methods for class attributes. It would work a bit like the generic functions created by the <code>:reader</code>, <code>:writer</code> and <code>:accessor</code> slot options given to <code>defclass</code> in Common Lisp.</p>
<p>However, I think that if that part of the language is to be changed — and I agree it should be — the new design should go all the way to hiding the methods from the users of the class. It should be possible to optionally define the accessor methods you want and then have them accessible by simply using the attribute name. So object.attribute = value would invoke object.setAttribute(value) behind the scenes, if such a method was defined. For backwards compatibility, it would need to do the normal assignment if it was permitted and no method was defined. A close relative to look at for inspiration would be C#. This one of those places where the language is, simply put, better.</p>
<p>[UPDATE]</p>
<p>Reading Bruce Eckel's <a href="http://www.mindview.net/WebLog/">great blog</a>, I stumbled across <a href="http://mindview.net/WebLog/log-0021">this</a>: <q>The Java team is clearly not ignoring the threat of C# [...] along with the new features already on the list for "Tiger" (JDK 1.5), including true enumerations, autoboxing, and generics (templates), they are adding attributes, something (along with autoboxing) taken directly from C#, because it's a good idea.</q></p>
99 bottles of GOO2003-04-17T16:27:00+00:002003-04-17T16:27:00+00:00Unknownhttps://juripakaste.fi/goo-99/<p>Here's a version of <a href="http://99-bottles-of-beer.ls-la.net/">99 bottles</a> written in <a href="http://www.googoogaga.org/">GOO</a>. I have submitted it.</p></p>
<pre><code>
(df bottles (n)
(cat
(cond ((= n 0) "No more bottles")
((= n 1) "One bottle")
(#t (cat (num-to-str n) " bottles")))
" of beer"))
(df bow (n)
(cat (bottles n) " on the wall"))
(do
(fun (n)
(post (bow n))
(post "\n")
(post (bottles n))
(post "\nTake one down, pass it around\n")
(post (bow (1- n)))
(post "\n\n"))
(range-by 100 >= 1 1-))
</code></pre>
<p>Happy easter.
Berkeley DB XML2003-03-15T08:59:00+00:002003-03-15T08:59:00+00:00Unknownhttps://juripakaste.fi/berkeley-db-xml/<p>I just went surfing <a href="http://www.sleepycat.com/">Sleepycat Software</a> to read some documentation (don't ask me why, I just realized I have it all installed locally), when I noticed this: <a href="http://www.sleepycat.com/products/xml.shtml">Berkeley DB XML</a>. Apparently not yet released, but interesting anyway. I'm not big on XML DBs, but with Sleepycat doing it, it might be worth checking out. At least they have a great history of doing high-quality, simple, lightweight products that just do their thing with minimum fuss. And if they continue with their licensing policy, all the better. Then we just need some Python bindings...</p>
The pain of going back2003-03-10T14:13:00+00:002003-03-10T14:13:00+00:00Unknownhttps://juripakaste.fi/the-pain-of-going-back/<p>DTML is really arse after two months of ZPT.</p>
Content management tools fail, Plone marches on2003-03-04T12:33:00+00:002003-03-04T12:33:00+00:00Unknownhttps://juripakaste.fi/cms-plone/<p>A Jupiter study — I'm usually highly sceptical of these, but hey, maybe they're right for once — says businesses are often <a href="http://www.atnewyork.com/news/article.php/1690881" title="Study: Content Management Tools Fail">dissatisfied with their content management solutions</a>. Complexity, lock-in, overengineering, ridiculous prices abound.</p>
<p>Meanwhile, in Plone land: <a href="http://plone.org/development/current/plip/index_html/view" title="PLIP overview">Plone Improvement Process</a> (a formalized approach to improvements: this sort of thing seems to be pretty popular in the larger free software projects these days, have you noticed?) has a few interesting improvement proposals for an already fine system: <a href="http://plone.org/Members/arnia/myprojects/Oslo/proposal">Oslo Skin system</a> and <a href="http://plone.org/Members/geoff/plip_controller_and_validation">Navigation and validation rewrite</a>.</p>
<p>These coupled with improvements to and integration of <a href="http://plone.org/documentation/developer/CMFTypes/view">CMFTypes</a> and possibly the brand spanking new <a href="http://plone.org/Members/mrtopf/announces/ttwtype_announce_1">TTWType</a> promise a bright future. It's not perfect and probably not suitable to all situations (there's a clear community site bias, but it doesn't have to be used that way) but it is easy, powerful and has a lot of momentum. And clearly there's still lots of space in the CMS field to conquer. Zope 3 is coming, but this Zope 2 thing is nowhere near its end of life.</p>
Asynchronous fetching seems to be working2003-02-14T00:07:00+00:002003-02-14T00:07:00+00:00Unknownhttps://juripakaste.fi/asynchronous-images/<p>Coolness. Straw is actually successfully fetching the dozens of feeds I'm reading, asynchronously, without the troublesome network thread. DNS lookups with ADNS (ha, yes, another dependency), reading stuff off network via asyncore. And the UI is pretty responsive. It's still slower than it used to be, but I think that's a tradeoff I can make. And, as I said earlier, things can possibly be tuned to work out better.</p>
<p>However, one somewhat serious usability problem remains: the stuff is basically fetched off a queue, where things go like this: request an URL -> resolve the host asynchronously -> add the split URL + IP to the fetching queue -> fetch data asynchronously -> stuff the results into the internal datastructures. Now, when you press 'g' or when the timer triggers a polling run, we request all the RSS URLs, then when we have finally gotten something, we parse it, see if there are any images, and request those. See the problem? Unless name lookups are in some cases seriously slow, all the articles are fetched before any of the images. I suppose that's fine if you are interested only in the text, but if not, and you are sitting in front of your aggregator, waiting to read the words of the bloggerdom hot off their keyboards, you get to read the items without images.</p>
<p>So maybe I should prioritize the images higher. That is, stuff the image requests to the front of the queue. And I suppose that as long as we are talking data structures instead of, say, things you see in Helsinki in front of bars Friday and Saturday evenings, it won't be a queue any longer. Oh well. That's theory for you.</p>
Asynchronicity2003-02-11T11:48:00+00:002003-02-11T11:48:00+00:00Unknownhttps://juripakaste.fi/asynchronicity/<p>The asynchronous networking rework of Straw is progressing pretty nicely. I've already got the basics working: feeds are polled successfully and images are fetched. The only significant bit of code to be converted is the subscription tool / rssfinder.</p>
<p>It hasn't been totally easy. Converting an existing code base which depends on synchronous network operation to be asynchronous is a bit hairy in places. In fact, rssfinder is one beast which makes me consider leaving some synchronous networking in place, but we'll see about that.</p>
<p>Things aren't all peachy in the brand new asynchronous world, though; first of all fetching data over the network is, at the moment, noticeably slower than in 0.15. However, I suspect that tuning a few constants (timing and limits on open sockets) a bit might help here a lot. And Straw speaks HTTP 1.1, I could try using the same connections for multiple files, even though that would complicate the code a bit.</p>
<p>The second problem is that DNS lookups are still freezing the program. First thing to do, I guess, is to implement a name cache, so it'll be an one-time (per session) cost. But I suspect I'll have to do something about the lookup part itself. The options are, I guess, either using <a href="http://www.chiark.greenend.org.uk/~ian/adns/">ADNS</a> or forking a separate lookup process. I've been dependency-happy in the past, but I'll have to think what to do with this one; I have no idea how common ADNS is.</p>
<p>Oh, and a big THANK YOU to Fredrik Lundh for <a href="http://effbot.org/zone/effnews-1.htm">the series about building EffNews</a> on his web site. I have no prior asynchronous networking programming experience, and his examples have been extremely helpful.</p>
The Problem With XML2003-01-02T00:00:00+00:002003-01-02T00:00:00+00:00Unknownhttps://juripakaste.fi/the-problem-with-xml/<p>XML, despite still having immense momentum, is not without problems. These days it probably is the pragmatic solution for many jobs, thanks to the multitude of tools for various platforms for processing it. That doesn't mean it couldn't, and shouldn't, be better. Not surprisingly, people persist in discussing its faults and how it should be. </p>
<p>Erik Naggum, in his characteristic style, details his beef with XML with a very long <a href="http://groups.google.com/groups?selm=%3C3250033069468718%40naggum.no%3E">article</a> [via <a href="http://pythonowns.blogspot.com/2002_12_29_pythonowns_archive.html#86789293">Python Owns Us</a>] where he makes many good points about the evolution of data formats and gives his suggestions on what should be changed in the XML format to make it better. While I'm not totally convinced about the binary format bit, otherwise I do agree with his points. </p>
<p>However, I think Aaron Swartz <a href="http://www.aaronsw.com/weblog/000776">put it best</a> in one little sentence, even if he didn't suggest any improvements: "XML [...] encourages you to be conservative in what you accept and liberal in what you put out.", in direct contrast to the traditional — and sensible — ethos of protocol designers. That is the single thing that is most damning to XML. Thanks to Aaron for articulating it. </p>
Back to Work2002-12-30T01:00:00+00:002002-12-30T01:00:00+00:00Unknownhttps://juripakaste.fi/back-to-work/<p>Christmas is over; read two books — <a href="http://books.guardian.co.uk/reviews/generalfiction/0,6121,790947,00.html">Dead Air</a> by Iain Banks, a decent book if not among his best, and <a href="http://books.guardian.co.uk/reviews/biography/0,6121,560311,00.html">Red Dog</a> by Louis de Berniéres, a fun and fast read — did a couple of jigsaws with my gf, ate way too much, drank some very nice wines — like <a href="http://www.masi.it/UveAppassiteIng.htm#SerMezzanellaRecioto">Masi Mezzanella</a> and <a href="http://www.champagne-bollinger.fr/us/les_vins/spec_cuvee.html">Bollinger Special Cuvée</a> — and had a very good time in other respects, too. </p>
<p>Now it's back to work. It would probably have been a good idea to be on vacation for this week, too. </p>
Base642002-12-30T00:00:00+00:002002-12-30T00:00:00+00:00Unknownhttps://juripakaste.fi/base64-lisp/<p>I have this <a href="http://www.juripakaste.fi/dl/base64.lisp">base64 implementation</a> (both decoding and encoding) in Common Lisp available on my web page. I have had some very nice feedback about it, but I've been a lazy maintainer; <a href="http://www.b9.com/">KMR</a> sent me some patches for ACL compatibility many moons ago but I never got around to integrating them. Um. I think. Or maybe I did... Well, anyway, it took ages. </p>
<p>Now it turns out that a) <a href="http://www.cliki.net/Edi%20Weitz">Edi Weitz</a> created a page for it in <a href="http://www.cliki.net/">CLiki</a> and b) KMR forked base64, now calling it <a href="http://www.cliki.net/cl-base64">cl-base64</a>, giving it some undoubtedly much-needed optimizations (I never benchmarked my implementation and speed wasn't much of a concern when coding it) and adding some nice extra functionality. </p>
<p>Thanks, guys, both of you. </p>
Short gtkhtml2 tutorial2002-10-29T00:00:00+00:002002-10-29T00:00:00+00:00Unknownhttps://juripakaste.fi/art-of-gtkhtml2/<p>GtkHTML2 is a sorely undocumented widget. I still occasionally get trouble with it, and I know I'm not the only one. While its future may be in question — gtkhtml1 replacing it is still a possibility, because the Ximian hackers are committed to maintaining it — it's the HTML widget of choice for Gtk2/GNOME 2 hacking today. Few notes on it follow, examples in Python: </p>
<p>Initializing the widget. </p>
<p><code> # I assume there's a scrolledwindow widget called html_scrolled_window<br></br> # in a glade widget tree available<br></br> scrolled_window = glade_widget_tree.get_widget('html_scrolled_window')<br></br> htmlview = gtkhtml2.View()<br></br> document = gtkhtml2.Document()<br></br> scrolled_window.set_hadjustment(htmlview.get_hadjustment())<br></br> scrolled_window.set_vadjustment(htmlview.get_vadjustment())<br></br> document.clear()<br></br> htmlview.set_document(document)<br></br> scrolled_window.add(htmlview)<br></br> scrolled_window.show_all()<br></br> </code> </p>
<p>IIRC doing things in the wrong sequence could cause segfaults. </p>
<p>There are a few signals that are useful to connect to: </p>
<p><code> # The document object's link-clicked signal is emitted when the user<br></br> # clicks a link.<br></br> document.connect("link-clicked", link_clicked)<br></br> # The document emits request-url when it needs data from an url.<br></br> # This is used for at least image fetching.<br></br> document.connect("request-url", request_url)<br></br> # Htmlview emits on_url (yes, this is spelled with an underscore<br></br> # unlike the two others) when the mouse is on an url. This is useful for<br></br> # displaying the URL somewhere.<br></br> htmlview.connect("on_url", on_url)<br></br> </code> </p>
<p><code>link_clicked</code> receives the document object and link href as arguments. <code>request_url</code> receives the document, URL and a stream to write the data to. You should just fetch the data from the URL and write it to the stream. <code>on_url</code> receives the htmlview object and the url. </p>
<p>To display data in the widget, you need to write to the document object's stream. Something like this: </p>
<p><code> document.clear()<br></br> document.open_stream("text/html") # this is the only mime type recognized<br></br> document.write_stream("<html><head><title>title</title>" +<br></br> "</head><body>body</body></html>")<br></br> document.close_stream()<br></br> </code> </p>
<p>Note that this — <strong>I think</strong> — assumes ISO 8859-1 encoding. If you want something different, you have to specify that... wait for it, try to guess... with a meta header. There is apparently no API to do that; it must be specified in the document stream itself. So, for UTF-8, you want to add this to the head part: </p>
<p><code><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></code> </p>
<p>That's it. Now you can display HTML documents in your GNOME 2 applications. I'll probably write some more once I have something to tell about using CSS with gtkhtml2. </p>
Things we do for money2002-10-03T00:00:00+00:002002-10-03T00:00:00+00:00Unknownhttps://juripakaste.fi/things-we-do-for-money/<p>C#. .NET. And logistics automation. </p>
<p>I expect this will be a learning experience. </p>
<p>I suppose I should start investigating Mono now :-) </p>
Chihuly2002-09-30T19:25:00+00:002002-09-30T19:25:00+00:00Unknownhttps://juripakaste.fi/chihuly/<p>Some of <a href="http://www.chihuly.com/">Dale Chihuly</a>'s work is
on display at Tampere. We were there last weekend, and went to see the
exhibition.</p>
<p>It was incredible. It was beautiful.</p>
<p>Finland is a country of world-famous glass blowers (think Timo
Sarpaneva), but Chihuly's work something the kind of which I've never
seen before. The colours, the shapes, scale of the works were totally
different from what I'm used to. No nordic minimalism in sight. A
rowboat full of red glass objects shaped like tentacles, stars and
things totally alien, reaching out in every direction, is such an
full-on assault on your visual cortex that it leaves you breathless.</p>