I’ve had a Google Home up and running at the house for about two month and finally decided to create a Google Assistant Action for it. I already have vistaicm-server running and connected to my Honeywell Vista 20P alarm panel so I thought I would create an Action so I could arm and disarm my house alarm system with my voice.

The Actions Overview points you to API.AI because using it makes Action development very easy. I started building the Action out in the API.AI interface but I ran into 2 issues: - Once deployed for “Preview”, the Action is only available for 30 minutes. That wasn’t going to work. I ended up figuring out how to get around this (and posted the solution on StackOverflow). - API.AI abstracts some of the details of the Conversation API which was a bit frustrating sometimes because I wanted to learn it and have more control over it.

Somewhere along the way I realized I didn’t need to go through API.AI and it would be a bit more of a learning experience to not do so. It turns out that writing the Action by hand is not terribly hard.

Below is what I learned and how I got it all setup.

Overview

A Google Assistant Action has 2 main components:

  1. Action Package - a JSON manifest that describes all the metadata about your action in addition to how users invoke and provide input to your action.
  2. Fulfillment - a service that you provide which conforms to the Conversation API](https://developers.google.com/actions/reference/conversation) and ultimately fulfills the user’s request.

Action Package

The Action Package I ended up with is fairly simple.

{
  "name": "tiger",
  "versionLabel": "tiger",
  "agentInfo": {
    "languageCode": "en-US",
    "projectId": "08f738659b0b6368493346964c66bb01"
  },
  "actions": [
    {
      "description": "Default Welcome Intent",
      "initialTrigger": {
        "intent": "assistant.intent.action.MAIN",
        "queryPatterns": []
      },
      "httpExecution": { "url": "https://[fulfillment_service_endpoint_url_here.com]" }
    },
    {
      "description": "alarm",
      "initialTrigger": {
        "intent": "ALARM",
        "queryPatterns": [
            { "queryPattern": "to $alarm-status:alarm-status" },
            { "queryPattern": "to $alarm-status:alarm-status the alarm" },
            { "queryPattern": "$alarm-status:alarm-status the alarm" }
          ]
      },
      "httpExecution": { "url": "https://[fulfillment_service_endpoint_url_here.com]" }
    },
  ],
  "customTypes": [
    {
      "name": "$alarm-status",
      "items": [
          {"key": "arm", "synonyms": ["arm"]},
          {"key": "disarm", "synonyms": ["disarm"]}
      ]
    }
    [...]

Let’s break down the important parts:

  • name: This is the name of the Action (“tiger”) but does not define the “invocation name” (keyword that Google Assistant uses to launch your Action). The invocation name is defined at the time the Action package is deployed.
  • intent: This defines the ID for the intent, used by the fulfillment endpoint to identify which action has been triggered.
  • queryPatterns: These are the speech patterns that Google Assistant will recognize.
  • httpExecution: This is the endpoint URI for the fulfillment service which implements the Conversation API.
  • customTypes: These are word variables (if you will) and allow you to define a list of words that can be used in a slot. You can reference a customType in your queryPatterns.

With this setup, I can say any of the following:

  • Hey Google, tell tiger to arm (via the to $alarm-status:alarm-status queryPattern)
  • Hey Google, tell tiger to arm the alarm (via the to $alarm-status:alarm-status the alarm queryPattern)
  • Hey Google, tell tiger arm the alarm (via the $alarm-status:alarm-status the alarm queryPattern)

Fulfillment

There is a npm package called actions-on-google that is the client library for implementing the Conversation API in Node. This client makes it super easy to create a fulfillment service.

Here is what my service looks like:

var ActionsSdkAssistant = require('actions-on-google').ActionsSdkAssistant;
var assistant = new ActionsSdkAssistant({ request: assistantRequest, response: assistantResponse });
var actionMap = new Map();

function mainIntent(assistant) {
  assistant.tell("You can say something like tell tiger to arm the alarm or tell tiger to open the left garage door.");
}

function alarmIntent(assistant) {
  var status = assistant.getArgument("alarm-status"); // alarm-status is a customType
  var tellSpeech = null;
  var commandUrlPrefix = "http://vistaicmHost:2945/execute?command=";
  var commandUrl = "";

  if (status == "arm") {
      commandUrl = commandUrlPrefix + "arm";
      tellSpeech = "All secure!";
  } else if (status == "disarm") {
      commandUrl = commandUrlPrefix + "disarm"
      tellSpeech = "Disarmed!";
  }

  console.log("GET " + commandUrl);
  http.get(commandUrl, (response) => {
      assistant.tell(tellSpeech);
  }).on('error', (e) => {
      console.log(`Error: ${e.message}`);
      assistant.tell("Sorry, there was an error when trying to communicate with the house.");
  });
}

// Action map: For each intent defined in the Action Package, define the handler for it.
actionMap.set(assistant.StandardIntents.MAIN, mainIntent);
actionMap.set("ALARM", alarmIntent);

assistant.handleRequest(actionMap);

google-action-tiger

After some refining, the final result is a project called google-action-tiger. This project contains all the necessary parts for the Action: The action package, fulfillment code, and provisioning (i.e. “deploy”) script. I decided to host the fulfillment service on AWS Lambda because it is really suitable for this type of type of application, especially since HTTPS is baked in.

Usage

In a gist, to get up and running all you need to do is:

  1. Install dependencies per the README (including AWS and gactions CLI)
  2. Create an AWS Lambda function
  3. Create a config file, using config.example as a template
  4. Run ./deploy.sh

The detailed instructions are located on the README.

I really like how everything is in a single repository and updates are deployed simply by running ./deploy.

deploy.sh

I decided to use a bash script for the deploy.sh (rather than node) script because the primary interaction is with the AWS and gactions CLI and doing that from node would have ended up just being a bunch of spawning of child processes. Wanting to use the config file for all config but also update the actions.json file with the httpExecution.url from config without creating a node script was a little tricky but I settled on passing straight JavsScript code to node using the -e parameter here. I am quite happy with how the deploy.sh file turned out.