Ethernet CRC32 calculation - software vs algorithmic result

Question

I'm trying to calculate the Frame Check Sequence (FCS) of an Ethernet packet byte by byte. The polynomial is 0x104C11DB7. I did follow the XOR-SHIFT algorithm seen here http://en.wikipedia.org/wiki/Cyclic_redundancy_check or here http://www.woodmann.com/fravia/crctut1.htm

Assume the information that is supposed have a CRC is only one byte. Let's say it is 0x03.

1. step: pad with 32 bits to the right

   0x0300000000

2. align the polynomial and the data at the left hand side with their first bit that is not zero and xor them

   0x300000000 xor 0x209823B6E = 0x109823b6e

3. take remainder align and xor again

   0x109823b6e xor 0x104C11DB7 = 0x0d4326d9

Since there are no more bit left the CRC32 of 0x03 should be 0x0d4326d9

Unfortunately all the software implementations tell me I'm wrong, but what did I do wrong or what are they doing differently?

Python tells me:

 "0x%08x" % binascii.crc32(chr(0x03))
 0x4b0bbe37

The online tool here http://www.lammertbies.nl/comm/info/crc-calculation.html#intr gets the same result. What is the difference between my hand calculation and the algorithm the mentioned software uses?

UPDATE:

Turns out there was a similar question already on stack overflow:

You find an answer here Python CRC-32 woes

Although this is not very intuitive. If you want a more formal description on how it is done for Ethernet frames you can look at the Ethernet Standard document 802.3 Part 3 - Chapter 3.2.9 Frame Check Sequence Field

Lets continue the example from above:

1. Reverse the bit order of your message. That represents the way they would come into the receiver bit by bit.

   0x03 therefore is 0xC0

2. Complement the first 32 bit of your message. Notice we pad the single byte with 32 bit again.

   0xC000000000 xor 0xFFFFFFFF = 0x3FFFFFFF00

3. Complete the Xor and shift method from above again. After about 6 step you get:

   0x13822f2d

4. The above bit sequense is then complemented.

   0x13822f2d xor 0xFFFFFFFF = 0xec7dd0d2

5. Remember that we reversed the bit order to get the representation on the Ethernet wire in step one. Now we have to reverse this step and we finally fulfill our quest.

   0x4b0bbe37

Whoever came up with this way of doing it should be ...

A lot of times you actually want to know it the message you received is correct. In order to achieve this you take your received message including the FCS and do the same step 1 through 5 as above. The result should be what they call residue. Which is a constant for a given polynomial. In this case it is 0xC704DD7B.

As mcdowella mentions you have to play around with your bits until you get it right, depending on the Application you are using.

Answer 1

This snippet writes the correct CRC for Ethernet.

Python 3

# write payload
for byte in data:
    f.write(f'{byte:02X}\n')
# write FCS
crc = zlib.crc32(data) & 0xFFFF_FFFF
for i in range(4):
    byte = (crc >> (8*i)) & 0xFF
    f.write(f'{byte:02X}\n')

Python 2

# write payload
for byte in data:
    f.write('%02X\n' % ord(byte))
# write FCS
crc = zlib.crc32(data) & 0xFFFFFFFF
for i in range(4):
    byte = (crc >> (8*i)) & 0xFF
    f.write('%02X\n' % byte)

Would have saved me some time if I found this here.

Answer 2

There is generally a bit of trial and error required to get CRC calculations to match, because you never end up reading exactly what has to be done. Sometimes you have to bit-reverse the input bytes or the polynomial, sometimes you have to start off with a non-zero value, and so on.

One way to bypass this is to look at the source of a program getting it right, such as http://sourceforge.net/projects/crcmod/files/ (at least it claims to match, and comes with a unit test for this).

Another is to play around with an implementation. For instance, if I use the calculator at http://www.lammertbies.nl/comm/info/crc-calculation.html#intr I can see that giving it 00000000 produces a CRC of 0x2144DF1C, but giving it FFFFFFFF produces FFFFFFFF - so it's not exactly the polynomial division you describe, for which 0 would have checksum 0

From a quick glance at the source code and these results I think you need to start with an CRC of 0xFFFFFFFF - but I could be wrong and you might end up debugging your code side by side with the implementation, using corresponding printfs to find out where the first differ, and fixing the differences one by one.