voice-recognitiongoogle-homegoogle-smart-home

Is it possible to add custom voice commands to my home network of google home/minis?


I would like to write custom voice commands that can be used on any of the google listening devices in my home network (home/minis) and then return a specific response for those commands using the same google voice as all the default commands/queries you can ask it.

Is there a way to do this?

The end goal for the commands would be to hit an URL (local network) that would return a message to be replied from the google home/minis.

EDIT: I found a solution.
Google may change how it works so be aware of the time of this posting.
I used the google actions console. https://console.actions.google.com/u/0/ You can find a few tutorials on this that aren't too bad. Basically I created a default project (dont use any of the presets), created a scene, created a custom intent for each action I wanted to do and had a bunch of training phrases listed under that intent for the best chance of the correct voice capture.
Then under the scene I added each intent to the "user intent handling" spot.
For each of these I set the "intent" dropdown to one of my custom intents and for the "when intent is matched" section I set it to "call your webhook" and give it relevant name. I set mine to the same name as the intent for consistency. I just repeated this for all my custom intents.
Now I went to the "Webhook" tab on the left and made sure my fulfillment method was set to HTTPS endpoint.
This is the part that took me awhile. There is very little documentation on how to do this and some of it is actually wrong. For my case, I made a .net core 5.0 API as I am familiar with it, im sure you could use anything with an HTTPS endpoint. There are 2 things I learned here.

  1. some docs say that the handler name you give the trigger will be added to the endpoint. it DOES NOT.
  2. You can only actually have 1 endpoint. So if you want to many things you will have to just filter out the request object in your API and then route it to whatever its meant to do.
    The simplest object I found (that actually worked) for just returning text to google to be spoken out of your home/mini is here: https://developers.google.com/assistant/conversational/prompts-simple#json_1

Whatever it is you have your endpoint set to, just put it into the box for HTTPS endpoint and hit save.
I also learned that for .net this doesnt play nice with localhost. Im not an expert on networking and https, but after some googling an easy fix I found for testing in local host is to use NGROK. run it, pass it a command like "ngrok http https://localhost: -host-header=localhost:" and it should spit out 2 randomly generated .ngrok.io addresses. one http and one https. you can use the HTTPS one as the root for your API to pass to google and it should work!
You won't need to use the "deploy" option on google actions unless you want the public to have it. It ties to your account, so if your account is on your google home/minis it will be usable right away.
So far I have 5 phrases I can ask to my google products and it will route through my API, generate a response with a random number in it, return it and speak it out to me.
Once this is working you can tie into/talk to whatever you want with your API. its completely up to you at that point.

Feel free to add comments or message me with any questions or clarifications.


Solution

  • Answered in edit of the initial question.