Converting between NSBezierPath and CGPath

The macOS SDK ships with at least two graphics path types: NSBezierPath and CGPath. They are mostly used in different contexts but sometimes it would be useful to convert between them. On iOS UIBezierPath has tools for the conversion, but on macOS we have to do it manually.

Here are the two conversions, based on Stack Overflow answers (NSBezierPath to CGPath, CGPath to NSBezierPath), converted to Swift 5.5 and tested on macOS 11.4.

extension NSBezierPath {
    // https://stackoverflow.com/a/39385101
    var cgPath: CGPath {
        let path = CGMutablePath()
        var points = [CGPoint](repeating: .zero, count: 3)
        for i in 0 ..< self.elementCount {
            let type = self.element(at: i, associatedPoints: &points)
            switch type {
            case .moveTo: path.move(to: points[0])
            case .lineTo: path.addLine(to: points[0])
            case .curveTo: path.addCurve(to: points[2], control1: points[0], control2: points[1])
            case .closePath: path.closeSubpath()
            @unknown default: fatalError("Unknown element \(type)")
            }
        }
        return path
    }

    // https://stackoverflow.com/a/49011112
    convenience init(cgPath: CGPath) {
        self.init()
        cgPath.applyWithBlock { (elementPointer: UnsafePointer<CGPathElement>) in
            let element = elementPointer.pointee
            let points = element.points
            switch element.type {
            case .moveToPoint:
                self.move(to: points.pointee)
            case .addLineToPoint:
                self.line(to: points.pointee)
            case .addQuadCurveToPoint:
                let qp0 = self.currentPoint
                let qp1 = points.pointee
                let qp2 = points.successor().pointee
                let m = 2.0/3.0
                let cp1 = NSPoint(
                    x: qp0.x + ((qp1.x - qp0.x) * m),
                    y: qp0.y + ((qp1.y - qp0.y) * m)
                )
                let cp2 = NSPoint(
                    x: qp2.x + ((qp1.x - qp2.x) * m),
                    y: qp2.y + ((qp1.y - qp2.y) * m)
                )
                self.curve(to: qp2, controlPoint1: cp1, controlPoint2: cp2)
            case .addCurveToPoint:
                let cp1 = points.pointee
                let cp2 = points.advanced(by: 1).pointee
                let target = points.advanced(by: 2).pointee
                self.curve(to: target, controlPoint1: cp1, controlPoint2: cp2)
            case .closeSubpath:
                self.close()
            @unknown default:
                fatalError("Unknown type \(element.type)")
            }
        }
    }
}

Swift networking with AsyncHTTPClient

When you need to access resources over HTTP in Swift, in most cases the answer is URLSession from Foundation. On server side that's most probably not the right choice; there you are most likely running on SwiftNIO and you'll want something that integrates with it. On command line it's a toss up; on macOS URLSession is great, on other platforms… well, hope you don't run into any unimplemented corners.

I needed a command line tool that fetches a JSON file, parses it, downloads the files listed in the JSON file, and saves the JSON file too. I'm mostly on macOS so URLSession would have been fine, but I wanted to explore my options. SwiftNIO ships with a low-level HTTP client implementation, but that's not the right choice for a quick utility. The good news is there's also a higher-level implementation: AsyncHTTPClient. It's a lovely future based (while we wait for async/await in Swift) asynchronous HTTP client that makes this task a breeze.

The format of the JSON manifest file looks like this:

{
    "files": [
        {
            "file": "filename1",
            "bytes": 42,
            "sha256": "8e079926d7340822e6b4c501811af5d1edc47d796b97f56c1cbe3177b47d588b"
        },

        {
            "file": "filename2",
            "bytes": 10,
            "sha256": "4998ab8c155e03ebf843d4adf51d702b9560dc0fbafe35405c826d9a76460289"
        }
    ]
}

And so on. That's just a couple of structs:

struct ManifestFile: Codable {
    let file: String
}

struct Manifest: Codable {
    let files: [ManifestFile]
}

I'll just ignore bytes and sha256. They are relevant in other contexts, but here they don't matter.

Command line

Let's start by defining a command line interface. We'll use Swift Argument Parser:

import ArgumentParser
import Foundation

struct DownloadManifestFiles: ParsableCommand {
    static var configuration = CommandConfiguration(
        commandName: "manifestdl",
        abstract: "Download manifest files"
    )
    
    @Argument(help: "URL to the manifest")
    var url: String

    @Option(help: "Target directory, default to working directory")
    var directory: String = "."

    mutating func validate() throws {
        guard URL(string: self.url) != nil else {
            throw ValidationError("url \(self.url) was not valid")
        }
    }

    mutating func run() throws {
        guard let reallyURL = URL(string: self.url) else {
            throw ValidationError("URL \(self.url) was not valid")
        }
        let cwd = URL(fileURLWithPath: FileManager.default.currentDirectoryPath, isDirectory: true)
        let directory = URL(fileURLWithPath: self.directory, isDirectory: true, relativeTo: cwd)

        fatalError("TODO")
    }
}

DownloadManifestFiles.main()

Nice and easy. An interesting wrinkle is that ArgumentParser doesn't support URLs, probably because they're in Foundation and ArgumentParser uses only stdlib. And NSURL, which backs URL, is a class with far more responsibilities than is comfortable in a simple data wrapper.

Downloading…

Next we need the actual networking. Let's wrap it in a helper class:

import AsyncHTTPClient
import Foundation
import NIO
import NIOHTTP1

class Downloader {
    let httpClient = HTTPClient(eventLoopGroupProvider: .createNew)

    func syncShutdown() throws {
        try self.httpClient.syncShutdown()
    }

    func downloadListedFiles(url: URL, directory: URL) -> EventLoopFuture<Void> {
        fatalError("TODO")
    }
}

We don't really care about any details about the download, we just want to know if it succeeded or not, hence EventLoopFuture<Void>. EventLoopFuture uses untyped errors, so we don't need to include an error in the type signature. It makes a bit odd next to Result and Combine's Publisher, but it does help when integrating with Swift's exceptions.

Next let's implement the downloadListedFiles method.

class Downloader {
    /* … */
    func downloadListedFiles(url: URL, directory: URL) -> EventLoopFuture<Void> {
        let future = self.downloadManifest(url: manifestURL)
            .flatMap { manifest, data in
                self.downloadManifestContent(
                    manifest: manifest,
                    manifestURL: manifestURL,
                    directory: directory
                ).map { data }
            }
            .flatMap { data in
                self.saveManifest(directory: directory, data: data)
            }
        return future
    }

    func downloadManifest(url: URL) -> EventLoopFuture<(manifest: Manifest, data: Data)> {
        fatalError("TODO")
    }

    func downloadManifestContent(
        manifest: Manifest,
        manifestURL: URL,
        directory: URL
    ) -> EventLoopFuture<Void> {
        fatalError("TODO")
    }

    func saveManifest(directory: URL, data: Data) -> EventLoopFuture<Void> {
        fatalError("TODO")
    }
}

That looks like an acceptable outline for it. Give downloadListedFiles an URL to a listing file and a directory to download to and it'll download the file manifest, parse it, and then download the listed files, and finally save the manifest too. I'll fill in the blanks one by one.

Next let's look how downloadManifest should look.

func downloadManifest(url: URL) -> EventLoopFuture<(manifest: Manifest, data: Data)> {
    self.httpClient.get(url: url.absoluteString)
        .flatMapThrowing { response in
            guard var body = response.body,
                    let data = body.readData(length: body.readableBytes)
            else {
                throw MissingBodyError()
            }
            return (manifest: try JSONDecoder().decode(Manifest.self, from: data), data: data)
        }
}

As you can see, just like ArgumentParser, HTTPClient eschews URL as a Foundation type. Other than that, HTTPClient gives us a really easy interface. Just .get a String containing an URL, and then do whatever you need to do with a chaining method like .map, .flatMap, or as in this case, .flatMapThrowing.

Next we can tackle downloadManifestContent. It's the function that's responsible for downloading all the listed files.

func downloadManifestContent(
    manifest: Manifest,
    manifestURL: URL,
    directory: URL
) -> EventLoopFuture<Void> {
    let baseURL = manifestURL.deletingLastPathComponent()
    let requestFutures: [EventLoopFuture<Void>]
    do {
        requestFutures = try manifest.files.map { manifestFile in
            let localURL = directory.appendingPathComponent(manifestFile.file)
            try FileManager.default.createDirectory(
                at: localURL.deletingLastPathComponent(),
                withIntermediateDirectories: true,
                attributes: nil
            )
            let localPath = localURL.path
            let delegate = try FileDownloadDelegate(path: localPath, pool: self.threadPool)
            let request = try HTTPClient.Request(
                url: baseURL.appendingPathComponent(manifestFile.file).absoluteString
            )
            return self.httpClient.execute(request: request, delegate: delegate)
                .futureResult
                .map { _ in () }
        }
    } catch {
        return self.httpClient.eventLoopGroup.next().makeFailedFuture(error)
    }
    return EventLoopFuture.andAllSucceed(requestFutures, on: self.eventLoopGroup.next())
}

var eventLoopGroup: EventLoopGroup {
    self.httpClient.eventLoopGroup
}

This one's not quite as simple, but it's not too bad. For each listed file we create a future for downloading it. The download to disk, as opposed to memory, happens with the help of FileDownloadDelegate, a delegate included in AsyncHTTPClient that can write downloads to disk and report progress. Then once we have a list of futures, we smash them all together with andAllSucceeded. Again we don't care about anything else other than success, so Void is a perfectly fine value type.

One detail I need to point out here is the eventLoopGroup property. SwiftNIO works with EventLoops, and EventLoops are apparently usually threads. While we're working only with with networking code it's probably not a problem to ask HTTPClient for its EventLoopGroup instance.

File I/O

We have read the manifest and written the files listed in it to disk. One thing left to do: saving the manifest. Writing the file with SwiftNIO isn't quite as friendly as AsyncHTTPClient is, and if you were doing more of this you'd want to put a nicer façade on it, but here we just need it this once.

To prepare for this, lets first set up a bit of scaffolding. It feels cleaner to move the management of the EventLoopGroup to our own code now that we're using it for not just the HTTP client, and we'll also need a thread pool for the file I/O.

class Downloader {
    let eventLoopGroup: EventLoopGroup // replaces the computed property
    let httpClient: HTTPClient
    let threadPool: NIOThreadPool

    init() {
        self.eventLoopGroup = NIOTSEventLoopGroup()
        self.httpClient = HTTPClient(eventLoopGroupProvider: .shared(self.eventLoopGroup))
        self.threadPool = NIOThreadPool(numberOfThreads: 1)

        self.threadPool.start()
    }

    func syncShutdown() throws {
        try self.httpClient.syncShutdown()
        try self.threadPool.syncShutdownGracefully()
        try self.eventLoopGroup.syncShutdownGracefully()
    }

    /* … */
}

Forcing NIOTSEventLoopGroup here probably ties this code to macOS. For portability, there are other implementations. Here's what AsyncHTTPClient does when you ask it to create the event loop group itself:

#if canImport(Network)
    if #available(OSX 10.14, iOS 12.0, tvOS 12.0, watchOS 6.0, *) {
        self.eventLoopGroup = NIOTSEventLoopGroup()
    } else {
        self.eventLoopGroup = MultiThreadedEventLoopGroup(numberOfThreads: 1)
    }
#else
    self.eventLoopGroup = MultiThreadedEventLoopGroup(numberOfThreads: 1)
#endif

Doing something similar in your own code should help make this more cross platform.

With that setup done, we can dive into the file writing itself.

func saveManifest(directory: URL, data: Data) -> EventLoopFuture<Void> {
    let io = NonBlockingFileIO(threadPool: self.threadPool)
    let eventLoop = self.eventLoopGroup.next()
    let buffer = ByteBuffer(data: data)
    return io
        .openFile(
            path: directory.appendingPathComponent("manifest.json").path,
            mode: .write,
            flags: .allowFileCreation(),
            eventLoop: eventLoop
        ).flatMap { handle in
            io.write(fileHandle: handle, buffer: buffer, eventLoop: eventLoop)
                .map { handle }
        }.flatMapThrowing { handle in
            try handle.close()
        }
}

It's two async operations and one synchronous one in a pipeline. Open file asynchronously, write file asynchronously, close it synchronously. The flatMaps can feel a little daunting if you're not used to them, as always with future libraries. But once you get used to them, it's pretty OK. Async/await will hopefully help.

After all that work we're ready to loop back to our run method. We left it calling fatalError after processing the arguments. Now we can finish it up:

mutating func run() throws {
    guard let reallyURL = URL(string: self.url) else {
        throw ValidationError("URL \(self.url) was not valid")
    }
    let cwd = URL(fileURLWithPath: FileManager.default.currentDirectoryPath, isDirectory: true)
    let directory = URL(fileURLWithPath: self.directory, isDirectory: true, relativeTo: cwd)

    let downloader = Downloader()
    let dlFuture = downloader.downloadListedFiles(url: reallyURL, directory: directory)

    defer {
        do {
            try downloader.syncShutdown()
        } catch {
            print("Error shutting down: \(error)")
        }
    }
    try dlFuture.wait()
}

And that's it! Create a Downloader, call it to get a future, set up cleanup, then wait until the future is done.

Conclusion

SwiftNIO is a fantastic library that powers the Swift on the server ecosystem. It's also great with command line tooling. It can occasionally be a bit more involved than Foundation, but especially with HTTP requests the difference is negligible. You'd have had to bring in Combine too to make URLSessions composable.

The Foundation/standard library split is a bit awkward here, as it often is when working with Swift command line tools. It's not that Foundation doesn't work, but it's clear that often there's the Swift way and then there's the Foundation way. And Foundation's cross platform story has been a bit rocky.

As Swift's async story progresses a lot of this code can be simplified, I hope. In the ideal case the structure would stay pretty much as is, but those nested maps and flatMaps could be replaced with more straightforward code. However, I don't think you need to wait for async/await and all the related enhancements to arrive. This is already pretty great.

Database connections in Vapor 4

Version 4 of the Swift web framework Vapor was released a while ago. Vapor emphasizes their ORM, Fluent, and it seems that version 4 has changed how a database connection can be acquired if you prefer to write the SQL yourself. They've also skipped documenting it, so getting things working requires some digging. In this post I'll explain how to do it. I'm using PostgreSQL.

You need a connection pool. The right place to set it up is your app's configure(_:Application) method. Use an environment variable to feed a database URL to your app:

import Vapor

public func configure(_ app: Application) throws {
    guard let dbUrlString = Environment.get("DBURL") else {
        preconditionFailure("Missing DBURL")
    }
    /* … */

    // register routes
    try routes(app)
}

Now that you have a URL, import PostgresKit and set up the pool:

import PostgresKit
import Vapor

public func configure(_ app: Application) throws {
    guard let dbUrlString = Environment.get("DBURL") else {
        preconditionFailure("Missing DBURL")
    }
    
    let postgresConfiguration = PostgresConfiguration(url: dbUrlString)!
    let pool = EventLoopGroupConnectionPool(
        source: PostgresConnectionSource(configuration: postgresConfiguration),
        on: app.eventLoopGroup
    )
    /* … */

    // register routes
    try routes(app)
}

Next we need to make the pool available to our request handlers and make sure it's shut down correctly. Making it available to request handlers happens by inserting it into the Application object's storage. Shutdown requires implementing Vapor's LifecycleHandler and registering it with the Application.

First define a struct that wraps the pool:

struct DatabaseService {
    let pool: EventLoopGroupConnectionPool<PostgresConnectionSource>
}

To keep the service in Application.storage, we need a key type:

struct DatabaseServiceKey: StorageKey {
    typealias Value = DatabaseService
}

Add an Application extension property to make it easier to access the service in the storage:

extension Application {
    var databaseService: DatabaseService? {
        get { self.storage[DatabaseServiceKey.self] }
        set { self.storage[DatabaseServiceKey.self] = newValue }
    }
}

The lifecycle implementation looks like this:

extension DatabaseService: LifecycleHandler {
    func shutdown(_ application: Application) {
        self.pool.shutdown()
    }
}

Now you just have slot these pieces in place in configure:

import PostgresKit
import Vapor

public func configure(_ app: Application) throws {
    guard let dbUrlString = Environment.get("DBURL") else {
        preconditionFailure("Missing DBURL")
    }
    
    let postgresConfiguration = PostgresConfiguration(url: dbUrlString)!
    let pool = EventLoopGroupConnectionPool(
        source: PostgresConnectionSource(configuration: postgresConfiguration),
        on: app.eventLoopGroup
    )

    let dbService = DatabaseService(pool: pool)
    app.databaseService = dbService
    app.lifecycle.use(dbService)

    // register routes
    try routes(app)
}

Now you have a working database setup. If you want to run migrations, the configure method is probably a good place to do it, before you set up the routes. I keep the DB code in a domain specific DBClient type; you can use any division of responsibilities you like.

    app.lifecycle.use(dbService)

    let db = dbService.pool.database(logger: app.logger)
    let dbClient = DBClient(database: db)
    app.logger.info("Will run migrate on DB")
    _ = try dbClient.migrate().wait()
    app.logger.info("DB migration done")

    // register routes

When handling requests, you'll just have to get the database service from Request.application. I like to create a struct called RequestEnvironment that encapsulates acquisition of the service and creation of domain logic services. Something like this:

struct RequestEnvironment {
    var makeFooService: () -> FooService

    static func makeDefault(req: Request) -> RequestEnvironment {
        guard let dbService = req.application.databaseService else {
            fatalError("Missing DatabaseService")
        }
        let db = dbService.pool.database(logger: req.logger)
        let dbClient = DBClient(database: db)
        return RequestEnvironment(
            makeFooService: { FooService(dbClient: dbClient, request: req) }
        )
    }    
}

Now when your controller handles a request, create the RequestEnvironment object and use it to call your services with the database client:

struct FooController {
    func create(_ req: Request) -> EventLoopFuture<FooExternal> {
        let newFoo = try req.content.decode(NewFooIncoming.self)
        let env = RequestEnvironment.makeDefault(req: req)
        return env.makeFooService().makeFoo(newFoo)
    }
}

That's it! Go forth and SQL, swiftly.

© Juri Pakaste 2024