Torrent related: tracker response on UDP protocol (Update #3 - working)

Question

Update #4: Demo java snippet added for working with UDP & sending announce msg (remember connect is first!) check own response bellow.

====================================================

Update #3: I managed to make it work, method doConnect() presented bellow is OK, more info in my own response bellow.

====================================================

I am mainly interested in how to download a tracker response when protocol of announce url is UDP.

Details: So these are some announce urls from a valid torrent file (first one is the main one)

http://tracker.torrentbox.com:2710/announce
udp://elbitz.net:80/announce.php?passkey=362fc69de3402e8ef5794c7ecf7c58d4
udp://tracker.podtropolis.com:2711/announce

If protocol is HTTP every thing goes well & this is how i work:

String fullUrl = announceURL + "?info_hash=" + this.m_TorrentInfoHashAsURL + .. // i add the params
URL url = new URL(fullUrl);
URLConnection connection = url.openConnection();
InputStream is = connection.getInputStream();
.. //reading the stream

If protocol is UDP, the URL constructor throws a "java.net.MalformedURLException: unknown protocol: udp"

So i guess problem can be resumed to the following: how do I download a resounce from a URL on UDP protocol? (hope it simple & i see no Datagrams stuff)

UPDATE #1:

I did some more investigations online & arrived at the following structure pasted below (should work..but doesnt, i mean locally it does, but not with real tracker)

link to specs: http://www.bittorrent.org/beps/bep_0015.html

Ex: This is how i create the socket, but on valid tracker i never receive nothing back as response so something aint working:

if full url: udp://elbitz.net:80/announce.php?passkey=362fc69de3402e8ef5794c7ecf7c58d4
this.m_TrackerHost: elbitz.net 
this.m_TrackerPort: 80

private DatagramSocket m_WorkingSocket;
    private DatagramSocket getWorkingSocket() {
        Logger.d(TAG, "getWorkingSocket()");

        if(this.m_WorkingSocket==null){
            Random rnd = new Random();
            for (int i = 0; i < 100; i++) {
                try {
                    int port = 15000 + rnd.nextInt(15000); // [15000-30000)
                    DatagramSocket result = new DatagramSocket(port);
                    InetAddress trackerAddress = InetAddress.getByName(this.m_TrackerHost);
                    result.connect(trackerAddress, this.m_TrackerPort);
                    this.m_WorkingSocket = result;
                } catch (SocketException se) {
                    Logger.w(TAG, "getWorkingSocket() - port is taken");
                } catch (SecurityException se) {
                    Logger.w(TAG, "getWorkingSocket() - port is blocked?");
                } catch (UnknownHostException e) {
                    Logger.w(TAG, "getWorkingSocket() - unkwnown host?");
                }
            }
        }

        return this.m_WorkingSocket;
    }

& now full code from doConnect which should be the first comunication phase (next is announce .. similar code there)

private boolean doConnect() throws IOException{
    Logger.d(TAG, "doConnect()");

    DatagramSocket workingSocket = this.getWorkingSocket();
    DatagramPacket sendPacket = null, receivePacket = null;

    byte[] sendData = null;
    byte[] receiveData = null;
    int round = 0;

    Logger.d(TAG, "doConnect(): first round, timeout: " + this.getTimeoutInMillis(ACTION_ID_CONNECT, round));
    while(true) {
        if(round==8){
            Logger.w(TAG, "doConnect() - failed to connect with tracker, consumed in vain all 8 rounds..");
            return false;
        }

        workingSocket.setSoTimeout(this.getTimeoutInMillis(ACTION_ID_CONNECT, round));

        if(receivePacket==null){
            /*
            Offset  Size            Name            Value
            0       32-bit integer  action          0 // connect
            4       32-bit integer  transaction_id
            8       64-bit integer  connection_id
            16  */
            receiveData = new byte[16]; //CONNECT: at least 16 bytes
            receivePacket = new DatagramPacket(receiveData, receiveData.length);

            sendData = this.getConnectRequest();//return byte[] with everything..just like in specs
            sendPacket = new DatagramPacket(sendData, sendData.length); 
        }

        try {
            Logger.d(TAG, "doConnect() - sending connect data: " + (Arrays.toString(sendData)));
            workingSocket.send(sendPacket);
            workingSocket.receive(receivePacket);
            break;
        } catch (SocketTimeoutException ste) {
            round ++;
            Logger.w(TAG, "doConnect() connect - new round: " + (round+1) + "|timeout: " + this.getTimeoutInMillis(ACTION_ID_CONNECT, round));
            continue;
        }
    }

    byte[] connectResponse = receivePacket.getData();
    int actionIdFromResponse = Utils.byteArrayToInt(Utils.subArray(connectResponse, 0, 4));
    int transactionIdFromResponse = Utils.byteArrayToInt(Utils.subArray(connectResponse, 4, 4));
    long connectionIdFromResponse = Utils.byteArrayToLong(Utils.subArray(connectResponse, 8, 8));

    if(transactionIdFromResponse!=this.m_TransactionId){
        Logger.w(TAG, "doConnect() - received different transactionId");
        return false;
    }

    if(actionIdFromResponse!=ACTION_ID_CONNECT){
        Logger.w(TAG, "doConnect() - didnt received ACTION_ID_CONNECT");
        return false;
    }

    //store connectionId
    this.m_ConnectionId = connectionIdFromResponse;
    return true;
}

Problem remains.. i never receive a response from tracker (tried with other url too) also New question: is it OK to create socket on elbitz.net, port: 80, when full url contains more info (ex: /announce) ?

Update #2

Code above seems to work OK.. i found on google a list of trackers that have implemented this spec & voila response happened (ex: "udp://tracker.openbittorrent.com:80")

New question & again spec is here: http://www.bittorrent.org/beps/bep_0015.html - i dont seem to see how do i get a list of peers ?? .. in the normal request to a torrent tracker (by http), there were 2 cases: normal response (a bencoded map) & compacted response (in binary form). So were is the list of peers now ?

• from specs this is the announce response:

/*
              Offset      Size            Name            Value
              0           32-bit integer  action          1 // announce
              4           32-bit integer  transaction_id
              8           32-bit integer  interval
              12          32-bit integer  leechers
              16          32-bit integer  seeders
              20 + 6 * n  32-bit integer  IP address
              24 + 6 * n  16-bit integer  TCP port
              20 + 6 * N  */

from my tests i allways receive same values for the fields: IP address & TCP port .. plus that i get one response per request ..so this CANT BE IT!.. i need a list of peers!

Please help! the only types of response message that i havent implemented yet are scrape & error... but no one contains info of interest to me (peer info: ip & port) ex: scrape

Offset      Size            Name            Value
0           32-bit integer  action          2 // scrape
4           32-bit integer  transaction_id
8 + 12 * n  32-bit integer  seeders
12 + 12 * n 32-bit integer  completed
16 + 12 * n 32-bit integer  leechers
8 + 12 * N

Answer 1

My responses to my own question.. well thanks, me!

• If it starts with udp:// it is valid
• If url is: udp://elbitz.net:80/announce.php?passkey=362fc69de3402e8ef5794c7ecf7c58d4 you create socket on udp://elbitz.net, using port 80 (/announce part is not used when creating UDP socket)
• Since it is UDP, response it is not guaranteed (no exception thrown, you just wait), specs mention the max 8 round rule...But that means to wait quite some time to realize tracker ain't alive.
• In case of communication by UDP you send 2 types of request (Connect & Announce, each as 1 UDP package) & receive 1 response (obviously just if you are lucky, & also as UDP package).. to extract usable info, you have to parse the response byteBuffer. Again the real info is here: http://www.bittorrent.org/beps/bep_0015.html;
• You can't Announce until you got a Connect response (you need a transactionId, read specs) my doConnect() is Ok, I changed very few lines from it (ex: I'm using max size in bytes for the byteBuffer used by DataGramPacket, meaning new byte[65508] instead of [16] - the min size. When receiving a response the rest will be padding bytes, meaning 0's).
• As you see in doConnect there is a while-loop (it shows the max 8 rounds rule);
• The socketTimeOut is continually increasing (using formula from specs, 30 sec, 90.. etc).
• After receiving connect response you have to do a announce(), method content is similar, request differs (again see specs), & again I use max size for buffer(new byte[65508]) but this time it has more sense, because list size of received peers is not hardcoded (it can be 1/8/10 peers...I always request 10)..
  (min size is 20 bytes, this happens when no peer info got added in msg). Again this response is just one packet (there is room in those max 65508 bytes...I think I get 70% padding)
• Everything is slow, due to blocking methods (receive()) & I have it on own thread (usually those trackers that respond don't need to consume more than 2 rounds.. that is nice).
• Not seen in above code is rule that 1 minute rule (to do connect again after 1 min).
• My opinion: this ugly code! working with UDP.. & threads, because everything is slow & blocking...

UPDATE***

Code snipped for working with udp

DatagramSocket workingSocket = ?;//
DatagramPacket sendPacket = null, receivePacket = null;
byte[] sendData = null;
byte[] receiveData = null;

receiveData = new byte[65508]; //NOTE: at least 16 bytes | 65508 is max size, unused bytes are 0
receivePacket = new DatagramPacket(receiveData, receiveData.length);

sendData = this.getRequestTypeAnnounce(announceUDPWrapper.a_TransactionId);
sendPacket = new DatagramPacket(sendData, sendData.length); 
workingSocket.send(sendPacket);
workingSocket.receive(receivePacket);
byte[] fullResponse = receivePacket.getData();

Code snipped for composing announce msg

private byte[] getRequestTypeAnnounce(AnnounceUDPWrapper announceUDPWrapper) {
    //long connectionId, int transactionId, int tcpPort
    ByteBuffer bBuffer = ByteBuffer.allocate(98);
    bBuffer.putLong(announceUDPWrapper.a_ConnectionId);
    bBuffer.putInt(ACTION_ID_ANNOUNCE);
    bBuffer.putInt(announceUDPWrapper.a_TransactionId);
    bBuffer.put(announceUDPWrapper.a_InfoHash);//<<<< what you asked.. adding the infoHash which is byte[]
    bBuffer.put(this.m_MyId);

    bBuffer.putLong(announceUDPWrapper.a_Downloaded);
    bBuffer.putLong(announceUDPWrapper.a_Uploaded);
    bBuffer.putLong(announceUDPWrapper.a_Left);
    bBuffer.putInt(announceUDPWrapper.a_EventType.ordinal());

    bBuffer.put(Utils.intToByteArray(0));// ip, 0 = default
    bBuffer.putInt(0);//key
    bBuffer.putInt(10);//num_want

    byte[] portAsBytes = Utils.intToByteArray(announceUDPWrapper.a_ListeningTCPPort);
    bBuffer.put(Utils.subArray(portAsBytes, 2, 2)); //port
    return bBuffer.array();
}