j2me networking, threads and deadlocks

Question

The simple piece of midlet code (class Moo) below (after the excerpts) deadlocks (At least I assume it deadlocks after reading this post on threads here).

I have reproduced the relevant excerpts from the post :

    String url = ...
    Connection conn = null;

    try {
        conn = Connector.open( url );
        // do something here
    }
    catch( IOException e ){
        // error
    }

The root of the problem is the blocking nature of the open() call. On some platforms, the system does the actual connection under the covers, on the equivalent of a separate thread. The calling thread blocks until the connection thread makes the connection. At the same time, the security subsystem may require the user to confirm the connection, and the connection thread blocks until the event thread gets confirmation from the user. Deadlock occurs because the event thread is already waiting for the connection thread.

public class Moo extends MIDlet {

    protected void destroyApp(boolean arg0) throws MIDletStateChangeException {
        // TODO Auto-generated method stub

    }

    protected void pauseApp() {
    }

    protected void startApp() throws MIDletStateChangeException {
        Display display = Display.getDisplay(this);
        MyCanvas myCanvas = new MyCanvas();
        display.setCurrent(myCanvas);
        myCanvas.repaint();

    }

    class MyCanvas extends Canvas {

        protected void paint(Graphics graphics) {
                try {
                        Image bgImage = Image.createImage(getWidth(), getHeight());

                        HttpConnection httpConnection = (HttpConnection) Connector
                                        .open("http://stackoverflow.com/content/img/so/logo.png");
                        Image image = Image.createImage(httpConnection
                                        .openInputStream());
                        bgImage.getGraphics().drawImage(image, 0, 0, 0);
                        httpConnection.close();

                        graphics.drawImage(bgImage, 0, 0, 0);
                } catch (IOException e) {
                        e.printStackTrace();
                }
        }

    }

}

Can someone please tell me how the system thread invocation is done here (event and notification threads) and the sequence of events leading to the deadlock. I am not clear as to what are the thread involved here that lead to deadlock.

1. Is there any documentation on j2me thread model?
2. Where can I get the sources for j2me system classes (I want to check out the implementation of Connection classes)?

EDIT : In the above code I get the logic. But the below code should at least work right? This one also deadlocks where I am doing the network connection in a separate thread.

public class Foo extends MIDlet {

    protected void destroyApp(boolean arg0) throws MIDletStateChangeException {
        // TODO Auto-generated method stub
    }

    protected void pauseApp() {
        // TODO Auto-generated method stub
    }

    protected void startApp() throws MIDletStateChangeException {
        Display display = Display.getDisplay(this);
        MyCanvas myCanvas = new MyCanvas();
        display.setCurrent(myCanvas);
        myCanvas.repaint();
    }

    class MyCanvas extends Canvas {
        protected void paint(Graphics graphics) {
            try {
                Image bgImage = Image.createImage(getWidth(), getHeight());

                FetchImage fetchImage = new FetchImage();
                Thread thread = new Thread(fetchImage);
                thread.start();

                thread.join();

                bgImage.getGraphics().drawImage(fetchImage.image, 0, 0, 0);

                graphics.drawImage(bgImage, 0, 0, 0);
            } catch (Exception e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        }
    }

    public class FetchImage implements Runnable {
        public Image image;

        public void run() {
            HttpConnection httpConnection;
            try {
                httpConnection = (HttpConnection) Connector
                        .open("http://10.4.71.200/stage/images/front/car.png");
                image = Image.createImage(httpConnection.openInputStream());
                httpConnection.close();
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        }

    }
}

Answer 1

Where can I get the sources for j2me system classes (I want to check out the implementation of Connection classes)?

You cant. Its actually vendor dependent. The way in which Nokia handles this situation may be different from Motorola.

The lesson you have to learn is that don't do expensive computation in system callbacks as that might make the system unresponsive. So put the time consuming operation in a sepearate thread and always return from call backs as early as possible.

Even though you have created a separate thread in your second example, you wait for its completion in paint() and it eventually makes a result of no threading at all!

One thing you can do is

class MyCanvas extends Canvas {

    Image image;
    boolean imageFetchFailed;

    protected void paint(Graphics g) {
        if (image == null) {
            fetchImage();
            g.drawString("Fetching...", getWidth() >> 1, getHeight() >> 1, Graphics.HCENTER | Graphics.TOP)

        } else if (imageFetchFailed) {
            g.drawString("Failed to fetch image", getWidth() >> 1, getHeight() >> 1, Graphics.HCENTER | Graphics.TOP)
        } else {
            g.drawImage(image, 0, 0, 0);
        }
    }


    private void fetchImage() {
        new Thread(new Runnable() {
            public void run() {
                HttpConnection httpConnection = null;
                try {
                    httpConnection = (HttpConnection) Connector
                            .open("http://10.4.71.200/stage/images/front/car.png");
                    image = Image.createImage(httpConnection.openInputStream());
                } catch (IOException e) {
                    e.printStackTrace();
                    imageFetchFailed = true;
                }

                if (httpConnection != null) {
                    try {
                        httpConnection.close();
                    } catch (IOException ignored) {
                    }
                }

                // Following will trigger a paint call 
                // and this time image wont be null and will get painted on screen
                repaint();    
            }
        }).start();
    }
}

Answer 2

Canvas.paint() is a event delivery method, which means it is called by a system event thread.

Assume on a system, both Canvas.paint() calling and user confirmation event handling are implemented to be done by a UI event thread (UT).

Now, when UT is blocked inside Canvas.paint() by Connector.open(), UT sure not able to handle the next coming event, which in this case is the user confirmation event triggered by Connector.open(). It is not possible for UT to process another event when it's blocked inside your application code.

That's why the deadlock occur here, the connection thread is waiting for something will never happen, and blocks UT forever.

In general, you should not expect how will system event thread be implemented, and try to return from event handling method as quickly as possible. Otherwise, you might receive lower performance or deadlock like this.

Answer 3

Well, the basic problem is that some Java VM implementations use the same Java thread to do everything.

One of the first thing you need to figure out about the threading model of your VM is who developed it.

There is a list of J2ME licensees here: http://java.sun.com/javame/licensees/index.jsp

From that information, try to figure out how many native threads your VM is using. The 2 usual models are either run all bytecode interpretation into one native thread or run each java thread into its own native thread.

The next step is to gather information about how asynchronous the underlying operating system APIs are. When developing the VM, the licensees would have had to write native code to port the VM to the operating system. Any Inter-Process Communication or use of slow transmission medium (from flash card to GPS signal) might be implemented using a separate native thread that could allow the bytecode interpreter thread to keep running while the system waits for some data.

The next step is to realize how badly implemented the VM is. Typically, your problem happens when the VM uses only one internal java thread for all callbacks methods in the MIDP spec. So, you don't get a chance to react to keypad events until after your connection is opened if you try to open it in the wrong java thread.

Worse, you can actually prevent the screen from being refreshed because Canvas.paint() would be called in the same java thread as javax.microedition.media.PlayerListener.playerUpdate() for example.

From the VM implementation perspective, the golden rule is that any callback that you don't control (because it can end up in "user" code, like a listener) cannot be called from the same java thread you use unblock standard API calls. A lot of VM out there simply break that rule, hence the Sun recommendation that JavaME developers work around it.