I wanted to create a talking avatar, using tts Android and drawable frame anmation in Android. The lip sync images were stored in the drawable folder. And this is the piece of function that is executed when the speak button is pressed.
The gist of the function is, based on the letters occurring, each corresponding lip sync action is added to the animation. The rest of actions are based on the language selected to speak, the pitch and speech rate decided. Thread is used to run the animation and the voice parallel.
b1.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
//thread to add the animation
Thread avatarSp = new Thread(new Runnable() {
@Override
public void run() {
String toSpeak = ed1.getText().toString();
String[] words = toSpeak.split(" ");
for (String word : words) {
word = word.toLowerCase();
char[] letters = word.toCharArray();
for (int i = 0; i < letters.length; i++) {
if (letters[i] == 'a') {
if (sr1 == 1)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.a_i), 500);
else if (sr1 == 2)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.a_i), 750);
} else if (letters[i] == 'e') {
if (letters[i + 1] == 'i' || letters[i + 1] == 'a' || letters[i + 1] == 'e') {
i++;
if (sr1 == 1)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.e), 500);
else if (sr1 == 2)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.e), 750);
} else {
if (sr1 == 1)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.l), 500);
else if (sr1 == 2)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.l), 750);
}
} else if (letters[i] == 'i') {
if (sr1 == 1)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.c_d_g_k_n_r_s_th_y_z), 500);
else if (sr1 == 2)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.c_d_g_k_n_r_s_th_y_z), 750);
} else if (letters[i] == 'o') {
if (sr1 == 1)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.o), 500);
else if (sr1 == 2)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.o), 750);
} else if (letters[i] == 'u') {
if (sr1 == 1)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.u), 500);
else if (sr1 == 2)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.u), 750);
} else if (letters[i] == 'w' || letters[i] == 'q') {
if (sr1 == 1)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.w_q), 500);
else if (sr1 == 2)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.w_q), 750);
} else if (letters[i] == 'f' || letters[i] == 'v') {
if (sr1 == 1)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.f_v), 500);
else if (sr1 == 2)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.f_v), 750);
} else if (letters[i] == 'l') {
if (sr1 == 1)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.l), 500);
else if (sr1 == 2)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.l), 750);
} else if (letters[i] == 'm' || letters[i] == 'b' || letters[i] == 'p') {
if (sr1 == 1)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.m_b_p), 500);
else if (sr1 == 2)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.m_b_p), 750);
} else {
if (sr1 == 1)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.c_d_g_k_n_r_s_th_y_z), 500);
else if (sr1 == 2)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.c_d_g_k_n_r_s_th_y_z), 750);
}
}
if (sr1 == 1)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.rest), 500);
else if (sr1 == 2)
avatarSpeak.addFrame(getResources().getDrawable(R.drawable.rest), 750);
}
avatar.post(new Starter());
}
});
//start the thread
avatarSp.start();
tts1.setPitch(p1);
tts1.setSpeechRate(sr1);
String toSpeak = ed1.getText().toString();
lang = sp1.getSelectedItem().toString();
if (lang.equals("US")) {
System.out.print("Condition satisfied");
tts1.setLanguage(Locale.US);
} else if (lang.equals("UK"))
tts1.setLanguage(Locale.UK);
else if (lang.equals("Germany"))
tts1.setLanguage(Locale.GERMANY);
else if (lang.equals("Italy"))
tts1.setLanguage(Locale.ITALY);
else if (lang.equals("Japan"))
tts1.setLanguage(Locale.JAPAN);
else
tts1.setLanguage(Locale.CHINA);
Toast.makeText(getApplicationContext(), toSpeak, Toast.LENGTH_SHORT).show();
//speak
tts1.speak(toSpeak, TextToSpeech.QUEUE_FLUSH, null);
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
tts1.speak(toSpeak, TextToSpeech.QUEUE_FLUSH, null, null);
} else {
tts1.speak(toSpeak, TextToSpeech.QUEUE_FLUSH, null);
}
}
});
}
The API level in which the whole coded tested was Android 5.0 (21) And Android 6.0 (AVD 23). But when I press the speak button, the API crashes, both on the AVD and Phone. Can you please tell me:
1. Where the error could possibly be?
2. Is there a better way to do this and how?
Please bear with me if its novice, but I would really like to know better ways to do achieve functions, in my own code.
UPDATE 1
The log cat info is added.
10-29 09:02:26.245 1931-1931/com.bluesbegone.avatarspeak I/art﹕ Not late-enabling -Xcheck:jni (already on)
10-29 09:02:26.246 1931-1931/com.bluesbegone.avatarspeak I/art﹕ Late-enabling JIT
10-29 09:02:26.411 1931-1931/com.bluesbegone.avatarspeak I/art﹕ JIT created with code_cache_capacity=2MB compile_threshold=1000
10-29 09:02:27.282 1931-1931/com.bluesbegone.avatarspeak W/System﹕ ClassLoader referenced unknown path: /data/app/com.bluesbegone.avatarspeak-2/lib/x86
10-29 09:02:28.992 1931-1931/com.bluesbegone.avatarspeak I/TextToSpeech﹕ Sucessfully bound to com.svox.pico
10-29 09:02:29.026 1931-1956/com.bluesbegone.avatarspeak D/OpenGLRenderer﹕ Use EGL_SWAP_BEHAVIOR_PRESERVED: true
10-29 09:02:29.056 1931-1931/com.bluesbegone.avatarspeak D/﹕ HostConnection::get() New Host Connection established 0xa3fff460, tid 1931
10-29 09:02:29.300 1931-1938/com.bluesbegone.avatarspeak W/art﹕ Suspending all threads took: 37.049ms
10-29 09:02:29.459 1931-1956/com.bluesbegone.avatarspeak D/﹕ HostConnection::get() New Host Connection established 0xa3fff830, tid 1956
10-29 09:02:29.509 1931-1956/com.bluesbegone.avatarspeak I/OpenGLRenderer﹕ Initialized EGL, version 1.4
10-29 09:02:29.671 1931-1956/com.bluesbegone.avatarspeak W/EGL_emulation﹕ eglSurfaceAttrib not implemented
10-29 09:02:29.672 1931-1956/com.bluesbegone.avatarspeak W/OpenGLRenderer﹕ Failed to set EGL_SWAP_BEHAVIOR on surface 0xad77ac60, error=EGL_SUCCESS
10-29 09:02:29.912 1931-1931/com.bluesbegone.avatarspeak I/Choreographer﹕ Skipped 41 frames! The application may be doing too much work on its main thread.
10-29 09:02:31.082 1931-1931/com.bluesbegone.avatarspeak I/Choreographer﹕ Skipped 69 frames! The application may be doing too much work on its main thread.
10-29 09:02:31.297 1931-1931/com.bluesbegone.avatarspeak I/TextToSpeech﹕ Connected to ComponentInfo{com.svox.pico/com.svox.pico.PicoService}
10-29 09:02:31.822 1931-1968/com.bluesbegone.avatarspeak I/TextToSpeech﹕ Set up connection to ComponentInfo{com.svox.pico/com.svox.pico.PicoService}
10-29 09:02:32.280 1931-1938/com.bluesbegone.avatarspeak W/art﹕ Suspending all threads took: 288.632ms
10-29 09:02:35.320 1931-1938/com.bluesbegone.avatarspeak W/art﹕ Suspending all threads took: 12.137ms
10-29 09:02:43.335 1931-2173/com.bluesbegone.avatarspeak E/AndroidRuntime﹕ FATAL EXCEPTION: Thread-83
Process: com.bluesbegone.avatarspeak, PID: 1931
android.view.ViewRootImpl$CalledFromWrongThreadException: Only the original thread that created a view hierarchy can touch its views.
at android.view.ViewRootImpl.checkThread(ViewRootImpl.java:6556)
at android.view.ViewRootImpl.invalidateChildInParent(ViewRootImpl.java:942)
at android.view.ViewGroup.invalidateChild(ViewGroup.java:5081)
at android.view.View.invalidateInternal(View.java:12713)
at android.view.View.invalidate(View.java:12649)
at android.view.View.invalidateDrawable(View.java:16788)
at android.widget.ImageView.invalidateDrawable(ImageView.java:248)
at android.graphics.drawable.DrawableContainer.invalidateDrawable(DrawableContainer.java:377)
at android.graphics.drawable.Drawable.invalidateSelf(Drawable.java:385)
at android.graphics.drawable.Drawable.setVisible(Drawable.java:764)
at android.graphics.drawable.DrawableContainer.initializeDrawableForDisplay(DrawableContainer.java:510)
at android.graphics.drawable.DrawableContainer.selectDrawable(DrawableContainer.java:459)
at android.graphics.drawable.AnimationDrawable.setFrame(AnimationDrawable.java:274)
at android.graphics.drawable.AnimationDrawable.addFrame(AnimationDrawable.java:251)
at com.bluesbegone.avatarspeak.MainActivity$4$1.run(MainActivity.java:190)
at java.lang.Thread.run(Thread.java:818)
10-29 09:02:44.093 1931-1942/com.bluesbegone.avatarspeak I/art﹕ Background sticky concurrent mark sweep GC freed 10741(781KB) AllocSpace objects, 0(0B) LOS objects, 39% free, 2MB/3MB, paused 1.490ms total 313.129ms
10-29 09:02:44.122 1931-1942/com.bluesbegone.avatarspeak W/art﹕ Suspending all threads took: 26.447ms
10-29 09:02:44.278 1931-1938/com.bluesbegone.avatarspeak W/art﹕ Suspending all threads took: 146.147ms
10-29 09:02:45.073 1931-1931/com.bluesbegone.avatarspeak I/Choreographer﹕ Skipped 77 frames! The application may be doing too much work on its main thread.
10-29 09:02:45.160 1931-1956/com.bluesbegone.avatarspeak W/EGL_emulation﹕ eglSurfaceAttrib not implemented
10-29 09:02:45.161 1931-1956/com.bluesbegone.avatarspeak W/OpenGLRenderer﹕ Failed to set EGL_SWAP_BEHAVIOR on surface 0xa29586e0, error=EGL_SUCCESS
10-29 09:02:45.231 1931-1956/com.bluesbegone.avatarspeak E/Surface﹕ getSlotFromBufferLocked: unknown buffer: 0xab81f1e0
10-29 09:02:47.005 1931-1956/com.bluesbegone.avatarspeak E/Surface﹕ getSlotFromBufferLocked: unknown buffer: 0xab81f1e0
10-29 09:02:48.334 1931-2173/com.bluesbegone.avatarspeak I/Process﹕ Sending signal. PID: 1931 SIG: 9
You're attempting to animate a view from a background thread, but only the main thread can touch views. Get rid of the thread and just put the code in the main onClick method.