HMS ASR cannot start recording

I'm trying to add HMS automatic speech recognition (ASR) to my app. I already have SpeechRecognizer implemented, but it requires GMS to work.

The current HMS implementation works on a non-huawei device with HMS core installed, but does not work on my Huawei Mediapad T5.

Things I've tried

The methods are called from different threads (main thread and graphics thread), so I've tried synchronizing the methods on a lock or posting a Runnable to the activity handler, without making much a difference. I.E., wrapping the functions in synchronized(lock) or activity.post.

Code:

• fun init(activity: Activity)

speechRecognizer = MLAsrRecognizer.createAsrRecognizer(activity)
speechIntent = Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH)
    .putExtra(
        MLAsrCaptureConstants.LANGUAGE,
        "en-US"
    )
    .putExtra(MLAsrConstants.FEATURE, MLAsrConstants.FEATURE_ALLINONE)
speechRecognizer?.setAsrListener(listener)

• fun startListening()

speechRecognizer?.startRecognizing(speechIntent)

• fun destroy()

speechRecognizer?.destroy()

Logs

4945-4945 W/HmsSpeechRecognitionHolder@c1cafe: init on Thread[main,5,main]
4945-4945 W/InputMethodManager: startInputReason = 1
4945-4945 W/HmsSpeechRecognitionHolder@c1cafe: startListening on Thread[main,5,main]
4945-4945 E/HaLogProvider: forbiddenHiLog openHa = false
4945-4945 E/HaLogProvider: forbiddenHiLog.getVenderCountry=ca
4945-4945 E/HaLogProvider: forbiddenHiLog openHa = false
4945-4945 E/MLASR_HaAdapter_MLKitAsr: mEventsToBeReported: has no response event isInfoGatherStart:falsemsg: 0
4945-4945 E/HwCustAudioRecordImpl: isOpenEC : false
634-985 E/HuaweiProcessing: ProcessingLib_Create: the algo have already been created.
634-985 W/EffectsFactory: EffectCreate() library huawei_processing: could not create fx Huawei Audio Preprocessing Effect, error -22
634-985 E/EffectFactoryHAL: Error creating effect e707d040-6b79-11e2-b16a-0002a5d5c51b: Invalid argument
721-8060 E/AudioEffect: set(): AudioFlinger could not create effect, status: -19
721-8060 W/AudioPolicyEffects: addInputEffects(): failed to create Fx huawei_pre_processing on source 1
634-8049 E/baidu_asr_interface: asr_baidu_set_parameters_data-not baidu asr
634-7993 W/DeviceHAL: Device 0x78c2d00000 get_mic_mute: Function not implemented
634-987 W/DeviceHAL: Device 0x78c2dc4680 get_mic_mute: Function not implemented
721-8009 W/HuaweiAudioFlinger: soundtrigger is now disable or not support, pls enable it first from setting
721-8009 W/APM_AudioPolicyManager: startInput(78) failed: other input 70 already started
721-8009 E/AudioFlinger: RecordThread::start error,setCallingAppName -1
4945-4945 E/AudioRecord: start() status -38
4945-4945 E/MLASR_A: getVendorCountry=ca
500-8480 W/libc: Set property "hw.wifi.dns_stat" to "99,14,14044,1,34759"
1472-1472 W/HwKeyguardDragHelper: AnimationBlocked
4945-5079 W/libEGL: EGLNativeWindowType 0x79e0317010 disconnect failed
1140-2290 E/WindowManager: win=Window{d80c651 u0 ProjectActivity} destroySurfaces: appStopped=true win.mWindowRemovalAllowed=false win.mRemoveOnExit=false
767-767 E/wificond: Failed to get NL80211_RATE_INFO_NOISE
767-767 E/wificond: Failed to get NL80211_RATE_INFO_SNR
767-767 E/wificond: Failed to get NL80211_STA_INFO_CNAHLOAD
1140-1316 E/WificondControl: Noise: 0, Snr: -1, Chload: -1
767-767 E/wificond: Failed to get NL80211_RATE_INFO_SNR
767-767 E/wificond: Failed to get NL80211_STA_INFO_CNAHLOAD
767-767 E/wificond: Failed to get NL80211_RATE_INFO_NOISE
767-767 E/wificond: Failed to get NL80211_RATE_INFO_SNR
767-767 E/wificond: Failed to get NL80211_STA_INFO_CNAHLOAD
1140-1316 E/WificondControl: Noise: 0, Snr: -1, Chload: -1
761-8466 W/ACodec: forcing OMX state to Idle when received shutdown in ExecutingState
769-8467 W/SimpleSoftOMXComponent: onChangeState mState= 3, mTargetState = 3, state = 2
769-8467 W/SimpleSoftOMXComponent: checkTransitions port buf count is not zero
769-1826 W/SimpleSoftOMXComponent: checkTransitions port buf count is not zero
769-1826 W/SimpleSoftOMXComponent: checkTransitions port buf count is not zero
769-1826 W/SimpleSoftOMXComponent: checkTransitions port buf count is not zero
769-1826 W/SimpleSoftOMXComponent: checkTransitions port buf count is not zero
769-1826 W/SimpleSoftOMXComponent: checkTransitions mState = 2, mTargetState = 1
721-8060 W/HuaweiAudioFlinger: soundtrigger is now disable or not support, pls enable it first from setting
1900-3437 E/HSM: BMNCaller:is not PermissionEnabled.
721-6695 W/AudioFlinger::EffectModule: EffectModule 0x7ba4f22a00 destructor called with unreleased interface
634-941 E/audio_hw_primary: in_remove_audio_effect error effect is null
634-941 W/StreamHAL: Error from HAL stream in function remove_audio_effect: Function not implemented
721-6695 E/AudioFlinger::EffectModule: Error when removing effect: -38
721-6695 W/AudioFlinger::EffectHandle: disconnect Effect handle 0x7ba4e45800 disconnected after thread destruction
1640-1796 W/AudioState: session release  and not found  sessionId: 81
4945-4945 W/HmsSpeechRecognitionHolder@c1cafe: destroy on Thread[main,5,main]
4945-8481 E/HwCustAudioRecordImpl: isOpenEC : false
4945-8481 E/HwCustAudioRecordImpl: isOpenEC : false

Things I found suspicious in the logs

634-985 E/HuaweiProcessing: ProcessingLib_Create: the algo have already been created.
634-985 W/EffectsFactory: EffectCreate() library huawei_processing: could not create fx Huawei Audio Preprocessing Effect, error -22
634-985 E/EffectFactoryHAL: Error creating effect e707d040-6b79-11e2-b16a-0002a5d5c51b: Invalid argument
721-8060 E/AudioEffect: set(): AudioFlinger could not create effect, status: -19
721-8060 W/AudioPolicyEffects: addInputEffects(): failed to create Fx huawei_pre_processing on source 1
721-8009 W/APM_AudioPolicyManager: startInput(78) failed: other input 70 already started
721-8009 E/AudioFlinger: RecordThread::start error,setCallingAppName -1
4945-4945 E/AudioRecord: start() status -38

Note: the HMS demo apps I've tried works correctly on my Mediapad T5.

Update: After some fixes pointed out by @shirley, ASR seems to be working reliably on a P30Lite. But still facing the same issue on an older Mediapad T5.

According to the logs you provided, the voice of the user is not detected. The meanings of logs and status codes are as follows:

solution

1. It is recommended that you add logs to the callback method of the MLAsrListener listener to view the speech recognition process.

2. You are advised to check mSpeechRecognizer.destroy(). Check whether this method is invoked prematurely and has ended before it starts.

3. Check whether the device is faulty or whether the microphone of the device is invalid. Replace the device and perform the test.

---

After reviewing your logs, the following errors were found:

The reason for this error is:The Languagecode for speech recognition exceeds 10.

You can view the code here:

Ensure that the speech recognition Languagecode does not exceed 10.

---

11203 ,subError code: 3002,errorMessage: Service unavailable

The cause of this error is that the app_id information is not found in the project.

You are advised to check whether the agconnect-services.json file exists in the project, as shown in the following

If the file does not exist, you need to add it to the project. If the file exists, ensure that app_id is correct.

For details, see the following Docs.

---

1. Check whether Automatic Speech Recognition fails to be enabled.

   If Automatic Speech Recognition fails to be enabled, you can obtain the cause by using the onError(int error, String errorMessage) method of the MLAsrListener class, as shown in the following figure.

You can add the above method to the listener's class:

2.If speech recognition is enabled successfully but the speech recognition result is not obtained:

The MLAsrConstants.FEATURE parameter is set to FUNCTION_ALLINONE. Therefore, you need to obtain the speech recognition result in the onResults(Bundle results) method, as shown in the following figure.