Implementing my own remote desktop in java

Question

I am trying to implement my own remote desktop solution in java. Using sockets and TCP/UDP. I know I could use VNC or anything else, but its an assignmentwork from school that I want to do.

So for moving the mouse and clicking I could use the Robot class. I have two questions about this:

1. What about sending the video? I know the Robot class can capture the screen too, so should I just send images in a sequence and display in order at the other side of the connection? Is this the best way to implement remote desktop?

2. Also should I use TCP or UDP? I think UDP would be harder to implement since I will have to figure out which image comes after the other.

Answer 1

What you are trying to do will work, but incredibly slow. The images must be compressed before you send them over the net. Before compressing, the number of colors should be reduced. Also, only the portions of the image which have changed since the last update should be sent.

When transferring mouse coordinates an update should only occur if the new mouse position is more than x pixels away from the last position away or a number of y seconds is over. Otherwise you spend so much traffic for the mouse position that there is no room for the images.

UDP will be the best solution here, since it is fastest for video streaming (which is what you are effectively doing).

Answer 2

About 2:

UDP would be much harder, since it's a datagram-based protocol there are limits on how much data you can send at a time; it's not very likely that you are going to be able to fit entire images into single datagrams. So, you're going to have to work with differential/partial updates, which becomes complicated pretty quickly.

TCP, however, is stream-based and only delivers data in-order. If a packet in the middle disappears and needs to be re-sent, all following packets need to wait, even if they've been received by the target machine. This creates lag, which is often very undesirable in interactive applications.

So UDP is probably your best choice, but you can't design it around the assumption that you can send whole images at a time, you need to come up with a way to send just parts of images.