.netvb.netspeech-synthesisspeechsynthesizer

Where is the stream number in the event info for System.Speech.Synthesis?


One can let the SpeechSynthesizer speak text in an asynchronous way, for example like this:

Private WithEvents _Synth As New SpeechSynthesizer

Private Sub TextBox1_KeyUp(sender As Object, e As KeyEventArgs) Handles TextBox1.KeyUp
    If e.KeyCode = Keys.Enter Then
        _Synth.SpeakAsync(New Prompt(Me.TextBox1.Text))
    End If
End Sub

The events that SpeechSynthesizer generates enables us to tell what the computer voice is just speaking.

For example, you may visualize the speech output by selecting the characters like this:

Private Sub _Synth_SpeakProgress(sender As Object, e As SpeakProgressEventArgs) Handles _Synth.SpeakProgress

    Me.TextBox1.SelectionStart = e.CharacterPosition
    Me.TextBox1.SelectionLength = e.CharacterCount

End Sub

However, when SpeakAsync is called repeatedly (for example when we tell the SpeechSyntesizer to speak the same text while it's currently just speaking), the speech requests are queued, and the SpeechSynthesizer plays them one by one.

However, I haven't been able to find out which request the synthesizer is currently speaking. The SpeakProgressEventArgs don't reveal this:

Using SAPI5, the events provided a StreamNumber:

Parameters
StreamNumber
    The stream number which generated the event. When a voice enqueues more than one stream by speaking asynchronously, the stream number is necessary to associate an event with the appropriate stream.

Using this StreamNumber, you could always tell what the SpeechSynthesizer is just playing / speaking.

The System.Speech.Synthesis implementation is a modern version of the SAPI5 implementation.

However, I just don't find a StreamNumber indiciator or similiar information.

System.Speech.Synthesis provides information about just everything that is just happening, so it's highly unlikely that it doesn't provide the information which of the requests it's just processing.

How could this be retrieved?


Solution

  • To clarify my comment about using the Prompt Class to hold any identifying state you need, consider the following where the Prompt holds a reference to the source TextBox.

    Imports System.Speech.Synthesis
    Public Class MyPrompt : Inherits Prompt
        Private tbRef As WeakReference(Of TextBox)
    
        Public Sub New(textBox As TextBox)
            MyBase.New(textBox.Text)
            ' only hold a weak reference to the TextBox
            ' to avoid any disposal issues
            tbRef = New WeakReference(Of TextBox)(textBox)
        End Sub
    
        Public ReadOnly Property SourceTextBox As TextBox
            Get
                Dim ret As TextBox = Nothing
                tbRef.TryGetTarget(ret)
                Return ret
            End Get
        End Property
    End Class
    

    Now your original code could be written as:

    Imports System.Speech.Synthesis
    
    Public Class Form1
        Private WithEvents _Synth As New SpeechSynthesizer
    
        Private Sub TextBox1_KeyUp(sender As Object, e As KeyEventArgs) Handles TextBox1.KeyUp
            If e.KeyCode = Keys.Enter Then
                ' use a custom prompt to store the TextBox
                _Synth.SpeakAsync(New MyPrompt(Me.TextBox1))
            End If
        End Sub
    
        Private Sub _Synth_SpeakProgress(sender As Object, e As SpeakProgressEventArgs) Handles _Synth.SpeakProgress
            Dim mp As MyPrompt = TryCast(e.Prompt, MyPrompt)
            If mp IsNot Nothing Then
                Dim tb As TextBox = mp.SourceTextBox
                If tb IsNot Nothing Then
                    ' set the selection in the source TextBox
                    tb.SelectionStart = e.CharacterPosition
                    tb.SelectionLength = e.CharacterCount
                End If
            End If
        End Sub
    
    End Class
    

    Edit:

    The OP wants to use this with the SpeakSsmlAsync method. That in itself is not possible as that method creates a base Prompt using the Prompt(String, SynthesisTextFormat) Constructor and returns the created Prompt after calling SpeechSynthesizer.SpeakAsync(created_prompt).

    Below is a derived Prompt class that accepts either a string of ssml or a PromptBuilder instance along with an integer identifier. A new version of MyPrompt to use ssml and an integer identifer.

    Imports System.Speech.Synthesis
    
    Public Class MyPromptV2 : Inherits Prompt
        Public Sub New(ssml As String, identifier As Int32)
            MyBase.New(ssml, SynthesisTextFormat.Ssml)
            Me.Identifier = identifier
        End Sub
    
        Public Sub New(builder As PromptBuilder, identifier As Int32)
            MyBase.New(builder)
            Me.Identifier = identifier
        End Sub
    
        Public ReadOnly Property Identifier As Int32
    End Class
    

    ...

    Imports System.Speech.Synthesis
    
    Public Class Form1
        Private WithEvents _Synth As New SpeechSynthesizer
    
        Private Sub TextBox1_KeyUp(sender As Object, e As KeyEventArgs) Handles TextBox1.KeyUp
            If e.KeyCode = Keys.Enter Then
                ' build some ssml from the text
                Dim pb As New PromptBuilder
                pb.AppendText(TextBox1.Text)
                ' use ssml and and integer
                _Synth.SpeakAsync(New MyPrompt(pb.ToXml, 10))
                ' or 
                '_Synth.SpeakAsync(New MyPrompt(pb, 10))
            End If
        End Sub
    
        Private Sub _Synth_SpeakProgress(sender As Object, e As SpeakProgressEventArgs) Handles _Synth.SpeakProgress
            Dim mp As MyPromptV2 = TryCast(e.Prompt, MyPromptV2)
            If mp IsNot Nothing Then
                Select Case mp.Identifier
                    Case 10
                        TextBox1.SelectionStart = e.CharacterPosition
                        TextBox1.SelectionLength = e.CharacterCount
                End Select
            End If
        End Sub
    End Class