Gesture segmentation in Android

Question

I am doing a sample work of handwritten letter detection through gestures of Android.It works well when I input 1 character at a time. That means when I write A on screen by gesture, the program recognizes it well(as I put it on gesture library earlier). As of now I code like this.

public void onGesturePerformed(GestureOverlayView overlay, Gesture gesture) {
    ArrayList<Prediction> predictions = gLib.recognize(gesture);

if (predictions.size() > 0 && predictions.get(0).score > 1.0) {

    String letter = predictions.get(0).name;  

    Toast.makeText(this, letter, Toast.LENGTH_SHORT).show();

    if(letter.contains("A"))  //when matches i print it to edittext
    edittext.setText("A");
    .
    .      //rest of stuff here like previous way
    .

    }
}

But my criteria isn't that. I want to recognize a word. I want to write a word at a time just like as.

And during writing a word for each successful match the corresponding letter should be printed on edittext just like as.

A,N,D,R,O,I,D

So my question is how can I gain it? Is it possible to segment gestures(segment the word while writing)?Any working code example or links would be appreciated.

Answer 1

If you write a word as separate letters (i.e. not cursive writing) as shown in the image given in the question. Then simply do this -

public void onGesturePerformed(GestureOverlayView overlay, Gesture gesture) {
    ArrayList<Prediction> predictions = gLib.recognize(gesture);

    if (predictions.size() > 0) {
        Prediction prediction = predictions.get(0);
        String letter = prediction.name;

        if (prediction.score > 1.0) {
            edittext.setText(edittext.getText().toString() + letter);
        }
    }
}

Which is essentially appending the new letter to the existing edittext string.

But if you are talking about cursive writing then it's a lot complicated. Here is some code which can track cursive writing.

public class MainActivity extends Activity {
    private Handler mHandler;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);
    }

    @Override
    protected void onResume() {
        super.onResume();
        Tracker t = new Tracker();
        t.start();
    }

    @Override
    protected void onPause() {
        if (mHandler != null) 
            mHandler.getLooper().quit();
        super.onPause();
    }   

    @Override
    public boolean onTouchEvent(MotionEvent event) {
        switch (event.getAction()) {
        case MotionEvent.ACTION_MOVE:
            if (mHandler != null) {
                Message msg = Message.obtain();
                msg.obj = event.getX() + "," + event.getY();
                mHandler.sendMessage(msg);
            }
            break;
        }
        return true;
    }

    private class Tracker extends Thread {
        private static final int LETTER_SIZE = 30;

        private GestureLibrary gLib;
        private ArrayList<GesturePoint> points;

        public Tracker() {
            points = new ArrayList<GesturePoint>();
            gLib = GestureLibraries.fromRawResource(MainActivity.this, R.raw.gestures);
            gLib.load();
        }

        @Override
        public void run() {
            Looper.prepare();
            mHandler = new Handler() {

                public void handleMessage(Message msg) {
                    String[] pos = String.valueOf(msg.obj).split(",");
                    points.add(new GesturePoint(Float.parseFloat(pos[0]), Float.parseFloat(pos[1]), System.currentTimeMillis()));

                    if (points.size() < LETTER_SIZE) return;

                    GestureStroke stroke = new GestureStroke(points);
                    Gesture gesture = new Gesture();
                    gesture.addStroke(stroke);

                    ArrayList<Prediction> predictions = gLib.recognize(gesture);
                    if (predictions.size() > 0) {
                        Prediction prediction = predictions.get(0);
                        String letter = prediction.name;

                        if (prediction.score > 1.0) {
                            Log.e("Found", letter);
                            points.clear();
                        }
                    }
                }
            };          
            Looper.loop();
        }
    }   
}

So basically we capture the touch positions and create a Gesture from it which pass to recognize() method of GestureLibrary. If a gesture is recognized then we print it and clear the touch positions so that a new letter can be recognized.

Sample Project: Cursive_eclipse_project.zip