C zlib crc32 and Python zlib crc32 doesn't match

Question

Im experimenting a bit with crc32 in Python and C but my results won't match.

C:
#include <stdio.h>
#include <stdlib.h>
#include <zlib.h>

#define NUM_BYTES 9

int
main(void)
{

  uint8_t bytes[NUM_BYTES] = {1, 2, 3, 4, 5, 6, 7, 8, 9};

  uint32_t crc = crc32(0L, Z_NULL, 0);

  for (int i = 0; i < NUM_BYTES; ++i) {
    crc = crc32(crc, bytes, 1);
  }

  printf("CRC32 value is: %" PRIu32 "\n", crc);
}

Gives the output CRC32 value is: 3136421207

Python

In [1]: import zlib
In [2]: int(zlib.crc32("123456789") + 2**32)
Out[2]: 3421780262

In python I'm adding with 2**32 to "cast" to unsigned int.

What am I missing here?

[edit 1]

Now I have tried with

In [8]: crc = 0;
In [9]: for i in xrange(1,10):
   ...:     crc = zlib.crc32(str(i), crc)
   ...:     
In [10]: crc
Out[10]: -873187034
In [11]: crc+2**32
Out[11]: 3421780262

and

int
main(void)
{

  uint32_t value = 123456789L;

  uint32_t crc = crc32(0L, Z_NULL, 0);

  crc = crc32(crc, &value, 4);

  printf("CRC32 value is: %" PRIu32 "\n", crc);
}

Still not the same result.

Answer 1

There were problems in your original C and Python code snippets. As for your second C snippet, I haven't tried to compile it, but it's not portable since byte order within an int is platform-dependant. So it will give different results depending on the endianness of the CPU.

One problem, as Serge Ballesta has mentioned, is the difference between {1, 2, 3, 4, 5, 6, 7, 8, 9} and {'1', '2', '3', '4', '5', '6', '7', '8', '9'}. Another problem is that the loop in your original C code didn't actually scan the data, since you didn't use i in the loop, as bav mentioned.

crctest.c

#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <zlib.h>

#define NUM_BYTES 9

// gcc -std=c99 -lz -o crctest test.c

void do_crc(uint8_t *bytes)
{
    uint32_t crc = crc32(0L, Z_NULL, 0);

    for (int i = 0; i < NUM_BYTES; ++i)
    {
        crc = crc32(crc, bytes + i, 1);
    }

    printf("CRC32 value is: %lu\n", crc);
}

int main(void)
{
    uint8_t bytes0[NUM_BYTES] = {1, 2, 3, 4, 5, 6, 7, 8, 9};
    uint8_t bytes1[NUM_BYTES] = {'1', '2', '3', '4', '5', '6', '7', '8', '9'};

    do_crc(bytes0);
    do_crc(bytes1);
}

output

CRC32 value is: 1089448862
CRC32 value is: 3421780262

crctest.py

#! /usr/bin/env python

import zlib

def do_crc(s):
    n = zlib.crc32(s)
    return n + (1<<32) if n < 0 else n

s = b'\x01\x02\x03\x04\x05\x06\x07\x08\x09'
print `s`, do_crc(s)

s = b'123456789'
print `s`, do_crc(s)

output

'\x01\x02\x03\x04\x05\x06\x07\x08\t' 1089448862
'123456789' 3421780262

edit

Here's a better way to handle the conversion in Python:

def do_crc(s):
    n = zlib.crc32(s)
    return n & 0xffffffff

See the answers here for more info on this topic: How to convert signed to unsigned integer in python.