Binary QR Codes on iOS

Kristof Van Landschoot - May 25, 2020

Scanning QR codes has become easy with the API's we have at our disposal in the iOS operating system.

There is the CIDetector we can use on any image:

let ciImage = CIImage(cvPixelBuffer: pb)            
let d = CIDetector(
    ofType: CIDetectorTypeQRCode,
    context: nil,
    options: [CIDetectorAccuracy: CIDetectorAccuracyHigh])!
d.features(in: ciImage)
    .compactMap { $0 as? CIQRCodeFeature }
    .compactMap { $0.messageString }
    .forEach { print("qr code found: \($0)")}

Or if we are using the camera we can leverage the imaging pipeline to have it call us back when it found a QR code, using the meta data output delegate:

func metadataOutput(_ output: AVCaptureMetadataOutput,
                    didOutput metadataObjects: [AVMetadataObject],
                    from connection: AVCaptureConnection) {
    metadataObjects
        .compactMap { $0 as? AVMetadataMachineReadableCodeObject)
        .compactMap { $0.stringValue }
        .forEach { print("qr code found: \($0)")}
    }
}

This is all good and well, until we have to do with QR codes that contain binary data.

There seems to be a lot of misleading information on getting to the binary data. On stackoverflow somebody found a way of using the internals of AVMetadataMachineReadableCodeObject.

The solution, however, is hiding in plain sight: the CIQRCodeFeature has a symbolDescriptor which has a errorCorrectedPayload which contains the data. So if you pipe the complete camera output pixel buffer through a CIDetector we now have Apple-blessed access the the binary data!

The only problem remaining: how to interpret the data. Because this data is raw it contains the QR code's ECI headers. So to extract the data it is still necessary to interpret the bits. I found some sample code to do this on github, and was succesful in getting the binary data out of the QR code in the scenario where I need it.

Another option is to use Firebase's MLKit, where the QR code detected has two methods: rawValue and rawData. The first returns a string and the latter returns the raw data, without any ECI headers. Firebase definitely is easier to use here, but in my first tests it seems that the native solution described above works a bit more efficiently.