BitTorrent protocol with Java - Bitfield after successful handshake

Question

Having successfully sent and received handshakes from multiple peers, the next step in the BitTorrent message chain is the bitfield message.

The bitfield message looks like below, where the top line explainins the byte-size of the protocol segments:

<4-bytes><1-byte><x-bytes>
<nrOfBits><identifier><bits>

The problem I have is that nearly all peers seem to be sending bitfield messages that differ from the above representation! The messages tend to look like this:

size: 332, [0, 0, 0, 112, 5, 127, -1, -1, -1, -1, -5, -1, -1, -1, -1, -1, -17...]

The first problem is that the majority of the messages I receive have length bytes:

 [0, 0, 0, 112]

even though the received message in this case contains 332 bytes in total, while at some other instances the message may only be 80 bytes or so.

The second problem is that the bits usually are repeating -1 or some other strange negative value..

I don't think this can be attributed to low-level encoding problems on my side however, since other messages works fine..

Answer 1

Problem #1:

TCP, which I assume you're using, is a steam protocol. Messages come in never-ending streams of bytes. You have to divide the stream into individual messages yourself. The fact that you've read 332 bytes from the socket in a single read doesn't mean that you've read a single message. Bit torrent clients often pipeline messages (sending multiple messages at a time, without waiting for the answer). If the length is [0,0,0,112] then the length of the message is 4 + 1 + 111 (4 bytes of length, one byte for the identifier, and 111 bitfield bytes). That's it. After these 116 bytes comes the next message.

Edit: the same is true about uTP, the uTorrent UDP-based transport protocol despite the fact that it's based on the datagram oriented UDP.

Problem #2:

What you're seeing is an array of bytes, which are ALWAYS unsigned in java (which is really annoying). Their range is always between -128 and 127, so when the first (most significant bit) is set, java considers the byte value to be negative. This way if a bit-field has a byte with all 8 bits set (binary 11111111), you'll get a byte value of -1 since binary 11111111 corresponds to -1 in two's complement. I suggest looking at these bytes in either binary form, or in hex (using something like Integer.toHexString(myByte & 0xff)).

Edit: On a different note, unless you're writing code just for the heck of it, I suggest using a ready-made Java BitTorrent library. There's very little sense in writing this kind of stuff from scratch yourself when there are existing, well tested, libraries that implement everything you need and then some.