botframeworkspeech-recognitiondirect-line-botframework

Add support for various accents in Directline speech for Azure chat bot


We are working on a requirement where we need to support various accents for azure chat bot. Currently we have Directline speech enabled for the speech as below.

(async function () {
    var speechServicesTokenRes = await fetch(
       'https://eastus.api.cognitive.microsoft.com/sts/v1.0/issuetoken',
        {
           method: 'POST',
           headers: {
               'Ocp-Apim-Subscription-Key': '***************'
           }
        });

    if (speechServicesTokenRes.status === 200) {
        authorizationToken = await speechServicesTokenRes.text();

        var webSpeechPonyfillFactory = await window.WebChat.createCognitiveServicesSpeechServicesPonyfillFactory({
            authorizationToken: authorizationToken,
            region: 'eastus'
        });
        window.WebChat.renderWebChat({
            directLine: createDirectLine({
                secret: '********************'
            }),

            webSpeechPonyfillFactory: webSpeechPonyfillFactory
        }, document.getElementById('webchat'));
    }
})().catch(err => console.error(err));

Can anyone guide me if there is any way to customize Directline speech for accents.


Solution

  • Unfortunately, Direct Line Speech does not support accents. The full list of supported languages, including regional variations (for example Spanish (Honduras) vs Spanish (Panama)) can be found here.

    Direct Line Speech does support SSML (Speech Synthesis Markup Language) which has a rich set of associated features. Some options that may help you are:

    One last option is to create a custom neural voice which

    lets you create a one-of-a-kind customized synthetic voice for your applications.

    In this case, you would provide audio and/or text samples in order to train the voice for use in customizing it.

    Hope of help!