androidspeech-recognitionandroid-audiorecordgoogle-now

How to get audio amplitude with speech recognizer?

I'm trying to make a custom dialog while speech recognizing instead of using the official one. I got that part, but when then I decided to show the amplitude of the sound while recognizing, in order to make it more fancy, like Google Now search bar does (its a circle around the microphone that grows if voice its louder):

googlenow

Then I started to code how to obtain the amplitude of the sound, and finally I got it with AudioRecord Class.

The problem comes when I try to mix both (SpeechRecognizer and AudioRecord), because seems like they are not able to share microphone, or something like that...

In logcat I have this error:

03-03 21:16:07.461: E/ListenerAdapter(23359): onError
03-03 21:16:07.461: E/ListenerAdapter(23359): com.google.android.speech.embedded.Greco3RecognitionEngine$EmbeddedRecognizerUnavailableException: Embedded recognizer unavailable
03-03 21:16:07.461: E/ListenerAdapter(23359):   at com.google.android.speech.embedded.Greco3RecognitionEngine.startRecognition(Greco3RecognitionEngine.java:108)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at java.lang.reflect.Method.invokeNative(Native Method)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at java.lang.reflect.Method.invoke(Method.java:511)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at com.google.android.searchcommon.utils.ThreadChanger$1$1.run(ThreadChanger.java:77)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:390)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at java.util.concurrent.FutureTask.run(FutureTask.java:234)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:153)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1080)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:573)
03-03 21:16:07.461: E/ListenerAdapter(23359):   at com.google.android.searchcommon.utils.ConcurrentUtils$2$1.run(ConcurrentUtils.java:112)

and some other times i have this:

03-03 21:47:13.344: E/ListenerAdapter(23359): onError
03-03 21:47:13.344: E/ListenerAdapter(23359): com.google.android.speech.exception.AudioRecognizeException: Audio error
03-03 21:47:13.344: E/ListenerAdapter(23359):   at com.google.android.speech.embedded.Greco3Recognizer.read(Greco3Recognizer.java:107)
03-03 21:47:13.344: E/ListenerAdapter(23359):   at dalvik.system.NativeStart.run(Native Method)
03-03 21:47:13.344: E/ListenerAdapter(23359): Caused by: java.io.IOException: couldn't start recording, state is:1
03-03 21:47:13.344: E/ListenerAdapter(23359):   at com.google.android.speech.audio.MicrophoneInputStream.ensureStartedLocked(MicrophoneInputStream.java:119)
03-03 21:47:13.344: E/ListenerAdapter(23359):   at com.google.android.speech.audio.MicrophoneInputStream.read(MicrophoneInputStream.java:159)
03-03 21:47:13.344: E/ListenerAdapter(23359):   at com.google.common.io.ByteStreams.read(ByteStreams.java:806)
03-03 21:47:13.344: E/ListenerAdapter(23359):   at com.google.android.speech.audio.Tee.readFromDelegate(Tee.java:374)
03-03 21:47:13.344: E/ListenerAdapter(23359):   at com.google.android.speech.audio.Tee.readLeader(Tee.java:267)
03-03 21:47:13.344: E/ListenerAdapter(23359):   at com.google.android.speech.audio.Tee$TeeLeaderInputStream.read(Tee.java:464)
03-03 21:47:13.344: E/ListenerAdapter(23359):   at java.io.InputStream.read(InputStream.java:163)
03-03 21:47:13.344: E/ListenerAdapter(23359):   at com.google.android.speech.audio.AudioSource$CaptureThread.run(AudioSource.java:193)

And this is how i launch both:

//previously in constructor
speechrec = SpeechRecognizer.createSpeechRecognizer(getActivity());
speechrec.setRecognitionListener(this);
//

public void launchListening()
{       
    if (speechrec.isRecognitionAvailable(getActivity()))
    {
        Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
        speechrec.startListening(intent);       
    }

    bufferSize = AudioRecord.getMinBufferSize(sampleRate, AudioFormat.CHANNEL_CONFIGURATION_MONO, AudioFormat.ENCODING_PCM_16BIT);// * bufferSizeFactor;
    audio = new AudioRecord(MediaRecorder.AudioSource.MIC, sampleRate, AudioFormat.CHANNEL_CONFIGURATION_MONO, AudioFormat.ENCODING_PCM_16BIT, bufferSize);
    audio.startRecording();

    captureThread = new Thread(new Runnable()
    {
        public void run()
        {
            //calculate amplitude here
        }
    });
    captureThread.start();
}

Any ideas on how creating a custom dialog for speech recognition, where I can show amplitude based on the noise, like Google does?

Solution

  • The way to do it is to register a listener with the SpeechRecognizer and visualize the output of onRmsChanged. Note however that:

    There is no guarantee that this method will be called.

    So the speech recognizer that you are using needs to support this method. Note that the return value of SpeechRecognizer.createSpeechRecognizer(getActivity()) depends on the user's device's configuration.

    (You cannot start an AudioRecord while the SpeechRecognizer is recording and vice versa.)