Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What does "int &= 0xFF" in a checksum do?

I implemented this checksum algorithm I found, and it works fine but I can't figure out what this "&= 0xFF" line is actually doing.

I looked up the bitwise & operator, and wikipedia claims it's a logical AND of all the bits in A with B. I also read that 0xFF is equivalent to 255 -- which should mean that all of the bits are 1. If you take any number & 0xFF, wouldn't that be the identity of the number? So A & 0xFF produces A, right?

So then I thought, wait a minute, checksum in the code below is a 32 bit Int, but 0xFF is 8bit. Does that mean that the result of checksum &= 0xFF is that 24 bits end up as zeros and only the remaining 8 bits are kept? In which case, checksum is truncated to 8 bits. Is that what's going on here?

    private int CalculateChecksum(byte[] dataToCalculate)
    {
        int checksum = 0;

        for(int i = 0; i < dataToCalculate.Length; i++)
        {
            checksum += dataToCalculate[i];
        }

        //What does this line actually do?
        checksum &= 0xff;

        return checksum;
    }

Also, if the result is getting truncated to 8 bits, is that because 32 bits is pointless in a checksum? Is it possible to have a situation where a 32 bit checksum catches corrupt data when 8 bit checksum doesn't?

like image 888
JamesHoux Avatar asked Nov 30 '22 14:11

JamesHoux


1 Answers

It is masking off the higher bytes, leaving only the lower byte.

checksum &= 0xFF;

Is syntactically short for:

checksum = checksum & 0xFF;

Which, since it is doing integer operations, the 0xFF gets expanded into an int:

checksum = checksum & 0x000000FF;

Which masks off the upper 3 bytes and returns the lower byte as an integer (not a byte).

To answer your other question: Since a 32-bit checksum is much wider than an 8-bit checksum, it can catch errors that an 8-bit checksum would not, but both sides need to use the same checksum calculations for that to work.

like image 81
Ron Beyer Avatar answered Dec 10 '22 05:12

Ron Beyer