Reading raw bytes from an Aztec barcode via AVMetadataMachineReadableCodeObject yields unexpected result

Question

I have been working on an iOS reader app for a special kind of barcode used by Deutsche Bahn (german rail) for a while now. It is a variant of Aztec barcode that concatenates a DSA signature string and a zlib deflated payload.

I was stuck, when I learned that the AVMetadataMachineReadableCodeObject had no public method for reading the raw bytes contained in the barcode and the string methods always garbled the zlib-compressed data.

Luckily, this answer sent me in the right direction. The (private) bytes can be accessed with KVO, and as I don't currently expect to distribute the app on the App Store, that was perfect.

Despite my almost non-existent Swift and Objective-C knowledge I managed to get this to work, as you can see in the sample code. But the bytes from the barcode that are stored in NSData don't match the expected result! I suspected the zlib library I use (DeflateSwift) was not working, so I built a test case, which worked fine.

My question is: what am I doing wrong? Do I need to further process the raw bytes to get to the expected result (see below)? How raw exactly are the bytes that are stored in the AVMetadataMachineReadableCodeObject? Can anyone point me in the right direction? Any help is appreciated.

Here is my code (which is a sad mashup of Swift and Objective-C)

if let metadataObject = metadataObjects.first {
    let readableObject = metadataObject as! AVMetadataMachineReadableCodeObject;
    let rawReadableObject = readableObject.valueForKeyPath("_internal.basicDescriptor")!["BarcodeRawData"] as? NSData;

    if let rawBytes = rawReadableObject {
       let barcodeData = rawBytes; // or use testData instead

        let barcodeSplit:Int = 68;
        let barcodeLength:Int = barcodeData.length;
        let barcodeHeader:NSData = barcodeData.subdataWithRange(NSRange(location: 0, length: barcodeSplit))
        let barcodeZlibContent:NSData = barcodeData.subdataWithRange(NSRange(location: barcodeSplit, length: (barcodeLength-barcodeSplit)))

        let count = barcodeZlibContent.length / sizeof(UInt8)
        var array = [UInt8](count: count, repeatedValue: 0)
        barcodeZlibContent.getBytes(&array, length:count * sizeof(UInt8))

        print("\(barcodeLength)kb")
        print(barcodeHeader)
        print(barcodeZlibContent)

        var inflater = InflateStream()
        var (inflated, err) = inflater.write(array, flush: true)
        if err != nil{
            fatalError("\(err!)")
        }

        if let ticketString = String(bytes: inflated, encoding: NSUTF8StringEncoding) {
            print(ticketString)
        } else {
            print("not a valid UTF-8 sequence")
        }
    }
}

This is what I get

The bytes returned from AVMetadataMachineReadableCodeObject are

40 B4 FA 88 89 8A 88 88 98 E6 3E 20 09 10 0A 0E EF 25 ED AC DE C8 80 5A 6F 9D 21 9F 4A 6D 61 33 19 F3 12 10 8A 80 2B F0 C2 7C CE E0 AB 83 46 AF A6 42 79 FD E8 35 D4 8B 0B 00 00 00 00 03 13 3B A3 47 8E A9 C2 B4 DC 30 03 C2 89 32 8D A3 B0 D4 E6 2B 35 5B 7B 08 88 12 A0 AA A2 00 8E 22 20 31 95 10 1C 21 2A FF 78 2C BE 31 1B A2 12 B5 CF A3 87 9B 9B 59 EF 7B BC AC AE CA 88 C8 1C 02 E8 D2 B5 87 76 0D 93 77 8B FB 04 A2 B5 D1 F8 9A 67 D5 55 15 DA 61 13 91 EC 08 60 2D 9B 86 E1 94 35 C3 D8 A9 49 41 5B 3A 7C 59 A5 FD 9A E3 FE F8 3C 9F 3F 7B B2 59 DC 98 E3 5E 92 CC C0 21 11 EC AF BA D7 F4 5D DB FC BD A5 CA AF 99 08 28 E3 02 30 06 20 A8 00 88 43 8E A2 58 2D 87 24 33 40 18 C1 AE 50 04 08 91 7E 59 E1 F6 9B 87 E7 8A 67 AA 1B 3E FF FE EF 79 46 18 5A 23 03 B4 E9 1A 4F 2F 15 EA DC 46 F5 A9 67 AE B8 F7 16 0B F2 38 8B B3 96 35 34 AB D3 A6 0E 6C 77 9D 72 D5 85 7E 58 0B E0 25 69 2C AC 42 9C 13 0F 27 4F 13 72 4A 90 CB 1C ED 78 B3 60 F4 AD 4C FE 2B F4 51 A8 0D 60 CC DF 78 C7 65 78 CC E6 63 02 45 B1 F3 1F A8 ED 9E FE 63 00 00 00 00

Here is the sample barcode of a Deutsche Bahn ticket I used for testing.

This is what I need

When scanned with a proper barcode reader (I used bcTester 5) this yields the following bytes:

23 55 54 30 31 30 30 38 30 30 30 30 30 31 30 2C 02 14 1C 3D E9 2D CD 5E C4 C0 56 BD AE 61 3E 54 AD A1 B3 26 33 D2 02 14 40 75 03 D0 CF 9C C1 F5 70 58 BD 59 50 A7 AF C5 EB 0A F4 74 00 00 00 00 30 32 37 31 78 9C 65 50 CB 4E C3 30 10 E4 53 2C 71 43 4A D9 F5 2B 36 B7 84 04 52 01 55 51 40 1C 51 01 23 2A 42 0E 21 15 3F C7 8D 1F 63 36 11 52 2B 7C F1 78 76 76 66 BD F7 8F 4D 5D 54 C4 44 CE 10 05 D2 EB 78 5B AC 32 7B B4 77 C8 11 6B 62 C7 D6 79 AA EA AA 16 E1 B2 22 4D C4 01 AD 36 58 61 CA 6B 30 C6 E5 64 A0 B6 97 0F A6 A9 6F D6 71 DF C7 CF 3E 7F 37 93 66 8E C6 71 DE 92 4C C0 E1 22 0D FD 57 7A CB EE B6 CF EF 69 54 FD 66 44 05 31 D0 03 18 01 05 40 04 70 9C 51 46 AD 38 49 33 00 86 20 DD 42 88 04 22 5F A6 A1 DB F6 78 79 D4 79 95 76 1F 3F DF FD E7 98 86 16 B1 30 0B 65 D6 3C BD 2A 15 CE D8 AB E5 79 9D 47 7B DA 34 13 C7 34 73 5A 6B 0B 35 72 D9 5C 0D BB AE 53 AA E8 5F 86 B4 01 E9 25 8D 0D 50 8E 72 3C 39 3C B2 13 94 82 74 CE 2D C7 B3 41 8B ED 4C 9F F5 0B E2 85 6C 01 8C FE C7 B8 E9 87 8C D9 F1 90 28 A3 73 FE 05 6D DE 5F F1

As you can see, at offset 68 (78 9C) begins a valid zlib stream. If you split the data here and inflate the zlib data, it returns a string like this:

U_HEAD01005300802P9QAN-40501201514560DEDE0080ID0200180104840080BL020357031204GW3HEMP906012015060120151021193517S0010018Fernweh-Ticket natS00200012S0030001AS00900051-0-0S01200010S0140002S2S0150006BerlinS0160011NeumünsterS0210038B-Hbf 8:16 ICE794/HH-Hbf 10:16 IC2224S0230013Krull AndreaS026000213S0270019***************0484S0280013Andrea#Krull S031001006.01.2015S032001006.01.2015S035000511160S0360003271

Test NSData

If I build the byte array manually using the bytes returned from bcTester, everything works out as expected and the zlib data inflates correctly. Here is how I tested:

let testArray = [UInt8](arrayLiteral: 0x23, 0x55, 0x54, 0x30, 0x31, 0x30, 0x30, 0x38, 0x30, 0x30, 0x30, 0x30, 0x30, 0x31, 0x30, 0x2C, 0x02, 0x14, 0x1C, 0x3D, 0xE9, 0x2D, 0xCD, 0x5E, 0xC4, 0xC0, 0x56, 0xBD, 0xAE, 0x61, 0x3E, 0x54, 0xAD, 0xA1, 0xB3, 0x26, 0x33, 0xD2, 0x02, 0x14, 0x40, 0x75, 0x03, 0xD0, 0xCF, 0x9C, 0xC1, 0xF5, 0x70, 0x58, 0xBD, 0x59, 0x50, 0xA7, 0xAF, 0xC5, 0xEB, 0x0A, 0xF4, 0x74, 0x00, 0x00, 0x00, 0x00, 0x30, 0x32, 0x37, 0x31, 0x78, 0x9C, 0x65, 0x50, 0xCB, 0x4E, 0xC3, 0x30, 0x10, 0xE4, 0x53, 0x2C, 0x71, 0x43, 0x4A, 0xD9, 0xF5, 0x2B, 0x36, 0xB7, 0x84, 0x04, 0x52, 0x01, 0x55, 0x51, 0x40, 0x1C, 0x51, 0x01, 0x23, 0x2A, 0x42, 0x0E, 0x21, 0x15, 0x3F, 0xC7, 0x8D, 0x1F, 0x63, 0x36, 0x11, 0x52, 0x2B, 0x7C, 0xF1, 0x78, 0x76, 0x76, 0x66, 0xBD, 0xF7, 0x8F, 0x4D, 0x5D, 0x54, 0xC4, 0x44, 0xCE, 0x10, 0x05, 0xD2, 0xEB, 0x78, 0x5B, 0xAC, 0x32, 0x7B, 0xB4, 0x77, 0xC8, 0x11, 0x6B, 0x62, 0xC7, 0xD6, 0x79, 0xAA, 0xEA, 0xAA, 0x16, 0xE1, 0xB2, 0x22, 0x4D, 0xC4, 0x01, 0xAD, 0x36, 0x58, 0x61, 0xCA, 0x6B, 0x30, 0xC6, 0xE5, 0x64, 0xA0, 0xB6, 0x97, 0x0F, 0xA6, 0xA9, 0x6F, 0xD6, 0x71, 0xDF, 0xC7, 0xCF, 0x3E, 0x7F, 0x37, 0x93, 0x66, 0x8E, 0xC6, 0x71, 0xDE, 0x92, 0x4C, 0xC0, 0xE1, 0x22, 0x0D, 0xFD, 0x57, 0x7A, 0xCB, 0xEE, 0xB6, 0xCF, 0xEF, 0x69, 0x54, 0xFD, 0x66, 0x44, 0x05, 0x31, 0xD0, 0x03, 0x18, 0x01, 0x05, 0x40, 0x04, 0x70, 0x9C, 0x51, 0x46, 0xAD, 0x38, 0x49, 0x33, 0x00, 0x86, 0x20, 0xDD, 0x42, 0x88, 0x04, 0x22, 0x5F, 0xA6, 0xA1, 0xDB, 0xF6, 0x78, 0x79, 0xD4, 0x79, 0x95, 0x76, 0x1F, 0x3F, 0xDF, 0xFD, 0xE7, 0x98, 0x86, 0x16, 0xB1, 0x30, 0x0B, 0x65, 0xD6, 0x3C, 0xBD, 0x2A, 0x15, 0xCE, 0xD8, 0xAB, 0xE5, 0x79, 0x9D, 0x47, 0x7B, 0xDA, 0x34, 0x13, 0xC7, 0x34, 0x73, 0x5A, 0x6B, 0x0B, 0x35, 0x72, 0xD9, 0x5C, 0x0D, 0xBB, 0xAE, 0x53, 0xAA, 0xE8, 0x5F, 0x86, 0xB4, 0x01, 0xE9, 0x25, 0x8D, 0x0D, 0x50, 0x8E, 0x72, 0x3C, 0x39, 0x3C, 0xB2, 0x13, 0x94, 0x82, 0x74, 0xCE, 0x2D, 0xC7, 0xB3, 0x41, 0x8B, 0xED, 0x4C, 0x9F, 0xF5, 0x0B, 0xE2, 0x85, 0x6C, 0x01, 0x8C, 0xFE, 0xC7, 0xB8, 0xE9, 0x87, 0x8C, 0xD9, 0xF1, 0x90, 0x28, 0xA3, 0x73, 0xFE, 0x05, 0x6D, 0xDE, 0x5F, 0xF1)
let testData = NSData(bytes: testArray, length: testArray.count)

Answer 1

Although I had the same issue while working with Data Matrix code, I assume this answer will be helpful for those who encountered the same problem.

So, we managed to retrieve raw bytes from AVMetadataReadableMachineObject via KVO ("internalBasicDescriptor" "barcodeRawData") as Data and printed its description to console, as you did in example. The bytes are not what we expect to see and e.g. we cannot produce a readable string from it (or it is not even a text). The reason is that the code was generated using a particular encoding scheme.

In my case, Data Matrix use C40, X12, Edifact, Base256... encoding schemes. In case of Data Matrix codes, you can check the table from the official documentation https://www.gs1.org/docs/barcodes/GS1_DataMatrix_Guideline.pdf (p. 50). For instance, if we convert the first byte and get 231, it indicates that the code is encoded using base256.

QR, Aztec and other codes may use the same approach of encoding data (probably not the ones I mentioned). Thus, depending of your type you should search for encoding schemes (which are, in most cases, simple math formulas converting one integer to another). Consider, that a detailed documentation of encodation schemes may be increasingly limited thought the web.

So, the bytes returned by AVMetadataObject are correct, though encoded (you should actually try to understand how).

Lastly, from iOS 11.0 you can get raw bytes in a better way:

if let object = metadataObjects[0] as? AVMetadataMachineReadableCodeObject {
    if object.type == AVMetadataObject.ObjectType.qr {
         let descriptor = object.descriptor {
             let rawBytes = descriptor.errorPayload

use .dataMatrix, .aztec, etc. instead of .qr (or you can even skip this line).

Thus, descriptor is basically CIBarcodeDescriptor, which has child classes so you can get more precise results (such as CIDataMatrixCodeDescriptor, CIQRCodeDescriptor, etc.) You can check some information on Apple Developer Website as well.

Answer 2

I solved this some time ago in Xamarin/C#, but the idea is the same for Swift too. The encodedData and ReadCode methods are taken form ZXing lib. Hope it helps.

It works well for reading and decoding both "small" and "large" ticket codes, but the default Aztec reader in iOS SDK is not good enough, so in the end we went on with the reader from Manateeworks. I can see now that it did not get any better with iOS 10 SDK.

    public override void DidOutputMetadataObjects (AVCaptureMetadataOutput captureOutput, AVMetadataObject[] metadataObjects, AVCaptureConnection connection)
    {
        foreach (AVMetadataMachineReadableCodeObject metadata in metadataObjects) {
            var d1 = (metadata.ValueForKey ((NSString)"_internal"));
            var d2 = (d1.ValueForKey ((NSString)"basicDescriptor"));
            var data = (d2.ValueForKey ((NSString)"BarcodeRawData"));
            var str = data.ToString ().Trim ().Trim (new [] { '<', '>' }).Replace (" ", "");


            var bitarray = new bool[str.Length * 4];
            for (var i = 0; i < str.Length / 2; i++) {
                int value = Convert.ToInt32 (str.Substring (i * 2, 2), 16);
                bitarray [i * 8 + 0] = (value & 1) > 0;
                bitarray [i * 8 + 1] = (value & 2) > 0;
                bitarray [i * 8 + 2] = (value & 4) > 0;
                bitarray [i * 8 + 3] = (value & 8) > 0;
                bitarray [i * 8 + 4] = (value & 16) > 0;
                bitarray [i * 8 + 5] = (value & 32) > 0;
                bitarray [i * 8 + 6] = (value & 64) > 0;
                bitarray [i * 8 + 7] = (value & 128) > 0;
            }
            var pabData = encodedData (bitarray);

            parent.scanFinished (true, pabData);
        }
    }


    enum ZXAztecTable
    {
        ZXAztecTableUpper,
        ZXAztecTableBinary,
        ZXAztecTableDigit
    };

    public byte[] encodedData (bool[] bitArray)
    {
        var result = new List<byte> ();
        int endIndex = bitArray.Length;
        ZXAztecTable latchTable = ZXAztecTable.ZXAztecTableUpper; // table most recently latched to
        ZXAztecTable shiftTable = ZXAztecTable.ZXAztecTableUpper; // table to use for the next read
        int index = 0;
        while (index < endIndex) {
            if (shiftTable == ZXAztecTable.ZXAztecTableBinary) {
                if (endIndex - index < 5) {
                    break;
                }
                int length = ReadCode (bitArray, index, 5);
                index += 5;
                if (length == 0) {
                    if (endIndex - index < 11) {
                        break;
                    }

                    length = ReadCode (bitArray, index, 11) + 31;
                    index += 11;
                }
                for (int charCount = 0; charCount < length; charCount++) {
                    if (endIndex - index < 8) {
                        index = endIndex;  // Force outer loop to exit
                        break;
                    }

                    byte code = (byte)ReadCode (bitArray, index, 8);
                    result.Add (code);
                    index += 8;
                }
                // Go back to whatever mode we had been in
                shiftTable = latchTable;
            } else {
                int size = shiftTable == ZXAztecTable.ZXAztecTableDigit ? 4 : 5;
                if (endIndex - index < size) {
                    break;
                }
                ReadCode (bitArray, index, size);
                index += size;
                latchTable = shiftTable;
                shiftTable = ZXAztecTable.ZXAztecTableBinary;
            }
        }
        return result.ToArray ();
    }

    public int ReadCode (bool[] bitArray, int startIndex, int length)
    {
        int res = 0;
        for (int i = startIndex; i < startIndex + length; i++) {
            res <<= 1;
            if (bitArray [i]) {
                res |= 0x01;
            }
        }
        return res;
    }