androidgesture-recognitionhandwriting

Gesture segmentation in Android


I am doing a sample work of handwritten letter detection through gestures of Android.It works well when I input 1 character at a time. That means when I write A on screen by gesture, the program recognizes it well(as I put it on gesture library earlier). As of now I code like this.

public void onGesturePerformed(GestureOverlayView overlay, Gesture gesture) {
    ArrayList<Prediction> predictions = gLib.recognize(gesture);

if (predictions.size() > 0 && predictions.get(0).score > 1.0) {

    String letter = predictions.get(0).name;  

    Toast.makeText(this, letter, Toast.LENGTH_SHORT).show();

    if(letter.contains("A"))  //when matches i print it to edittext
    edittext.setText("A");
    .
    .      //rest of stuff here like previous way
    .

    }
}

But my criteria isn't that. I want to recognize a word. I want to write a word at a time just like as. example

And during writing a word for each successful match the corresponding letter should be printed on edittext just like as.

A,N,D,R,O,I,D

So my question is how can I gain it? Is it possible to segment gestures(segment the word while writing)?Any working code example or links would be appreciated.


Solution

  • If you write a word as separate letters (i.e. not cursive writing) as shown in the image given in the question. Then simply do this -

    public void onGesturePerformed(GestureOverlayView overlay, Gesture gesture) {
        ArrayList<Prediction> predictions = gLib.recognize(gesture);
    
        if (predictions.size() > 0) {
            Prediction prediction = predictions.get(0);
            String letter = prediction.name;
    
            if (prediction.score > 1.0) {
                edittext.setText(edittext.getText().toString() + letter);
            }
        }
    }
    

    Which is essentially appending the new letter to the existing edittext string.

    But if you are talking about cursive writing then it's a lot complicated. Here is some code which can track cursive writing.

    public class MainActivity extends Activity {
        private Handler mHandler;
    
        @Override
        protected void onCreate(Bundle savedInstanceState) {
            super.onCreate(savedInstanceState);
            setContentView(R.layout.main);
        }
    
        @Override
        protected void onResume() {
            super.onResume();
            Tracker t = new Tracker();
            t.start();
        }
    
        @Override
        protected void onPause() {
            if (mHandler != null) 
                mHandler.getLooper().quit();
            super.onPause();
        }   
    
        @Override
        public boolean onTouchEvent(MotionEvent event) {
            switch (event.getAction()) {
            case MotionEvent.ACTION_MOVE:
                if (mHandler != null) {
                    Message msg = Message.obtain();
                    msg.obj = event.getX() + "," + event.getY();
                    mHandler.sendMessage(msg);
                }
                break;
            }
            return true;
        }
    
        private class Tracker extends Thread {
            private static final int LETTER_SIZE = 30;
    
            private GestureLibrary gLib;
            private ArrayList<GesturePoint> points;
    
            public Tracker() {
                points = new ArrayList<GesturePoint>();
                gLib = GestureLibraries.fromRawResource(MainActivity.this, R.raw.gestures);
                gLib.load();
            }
    
            @Override
            public void run() {
                Looper.prepare();
                mHandler = new Handler() {
    
                    public void handleMessage(Message msg) {
                        String[] pos = String.valueOf(msg.obj).split(",");
                        points.add(new GesturePoint(Float.parseFloat(pos[0]), Float.parseFloat(pos[1]), System.currentTimeMillis()));
    
                        if (points.size() < LETTER_SIZE) return;
    
                        GestureStroke stroke = new GestureStroke(points);
                        Gesture gesture = new Gesture();
                        gesture.addStroke(stroke);
    
                        ArrayList<Prediction> predictions = gLib.recognize(gesture);
                        if (predictions.size() > 0) {
                            Prediction prediction = predictions.get(0);
                            String letter = prediction.name;
    
                            if (prediction.score > 1.0) {
                                Log.e("Found", letter);
                                points.clear();
                            }
                        }
                    }
                };          
                Looper.loop();
            }
        }   
    }
    

    So basically we capture the touch positions and create a Gesture from it which pass to recognize() method of GestureLibrary. If a gesture is recognized then we print it and clear the touch positions so that a new letter can be recognized.

    Sample Project: Cursive_eclipse_project.zip