botframeworkazure-language-understanding

Retrieve complete LUIS DateTimeV2 entity from recognizer


I'm using the Microsoft Bot Framework v4 to do some fairly basic date and date range recognition.

The most straightforward use of the LUIS recognizer that most examples use and is used in the enterprise template doesn't return the complete DateTimeV2 structure. The example below is for "last week" corresponding to a daterange. These results are in recognizer.result.entities:

{
  "$instance": {
    "datetime": [
      {
        "startIndex": 8,
        "endIndex": 17,
        "text": "last week",
        "type": "builtin.datetimeV2.daterange"
      }
    ]
  },
  "datetime": [
    {
      "type": "daterange",
      "timex": [
        "2018-W43"
      ]
    }
  ]
}

Notice there are no start / end tags for the date range. This seems like an incomplete version of the DateTime2 specification. https://learn.microsoft.com/en-us/azure/cognitive-services/LUIS/luis-reference-prebuilt-datetimev2

If I use the specific DateTime resolver, I get the more detailed start and end information that I want.

[timex, 2018-W43]
[type, daterange]
[start, 2018-10-22]
[end, 2018-10-29]

It seems like these steps are nearly redundant as the more general approach returns the Timex formatted string that contains start and end information but not in an easily used format.

I believe I'm missing something fundamental about configuring LUIS and recognizers.

The code snippet below is modified version of the csharp_dotnetcore / 12.nlp-with-luis example (https://github.com/Microsoft/BotBuilder-Samples/tree/master/samples/csharp_dotnetcore/12.nlp-with-luis)

I added single prebuilt DateTimeV2 entity within luis.ai.

public async Task OnTurnAsync(ITurnContext turnContext, CancellationToken cancellationToken = default(CancellationToken))
{
    IList<Dictionary<string, string>> resolutionValues = null;

    if (turnContext.Activity.Type == ActivityTypes.Message)
    {

        // Use the more generic approach.
        var recognizerResult = await _services.LuisServices[LuisKey].RecognizeAsync(turnContext, cancellationToken);
        var topIntent = recognizerResult?.GetTopScoringIntent();
        if (topIntent != null && topIntent.HasValue && topIntent.Value.intent != "None")
        {
            await turnContext.SendActivityAsync($"==>LUIS Top Scoring Intent: {topIntent.Value.intent}, Score: {topIntent.Value.score}\n");
            if (recognizerResult.Entities != null)
            {
                await turnContext.SendActivityAsync($"==>LUIS Entities Found: {recognizerResult.Entities.ToString()}\n");
            }
        }
        else
        {
            var msg = @"No LUIS intents were found. Try show me last week.";
            await turnContext.SendActivityAsync(msg);
        }

        // Check LUIS model using specific DateTime Recognizer.
        var culture = Culture.English;
        var r = DateTimeRecognizer.RecognizeDateTime(turnContext.Activity.Text, culture);
        if (r.Count > 0 && r.First().TypeName.StartsWith("datetimeV2"))
        {
            var first = r.First();
            resolutionValues = (IList<Dictionary<string, string>>)first.Resolution["values"];
            var asString = string.Join(";", resolutionValues[0]);
            await turnContext.SendActivityAsync($"==>LUIS: resolutions values: {asString}\n");
            var subType = first.TypeName.Split('.').Last();
            if (subType.Contains("date") && !subType.Contains("range"))
            {
                // a date (or date & time) or multiple
                var moment = resolutionValues.Select(v => DateTime.Parse(v["value"])).FirstOrDefault();
                await turnContext.SendActivityAsync($"==>LUIS DateTime Moment: {moment}\n");
            }
            else if (subType.Contains("date") && subType.Contains("range"))
            {
                // range
                var from = DateTime.Parse(resolutionValues.First()["start"]);
                var to = DateTime.Parse(resolutionValues.First()["end"]);
                await turnContext.SendActivityAsync($"==>LUIS DateTime Range: from: {from} to: {to}\n");
            }
        }
    }
    else if (turnContext.Activity.Type == ActivityTypes.ConversationUpdate)
    {
        // Send a welcome message to the user and tell them what actions they may perform to use this bot
        await SendWelcomeMessageAsync(turnContext, cancellationToken);
    }
    else
    {
        await turnContext.SendActivityAsync($"{turnContext.Activity.Type} event detected", cancellationToken: cancellationToken);
    }
}

Is there a straightforward way to get the more complete DateTimeV2 entity out of the more general recognizer? Possibly chaining recognizers?


Solution

  • In order to see the entire API result there is an option to set when creating the Luis Recognizer. includeApiResults defaults to false, so setting that parameter to true causes the entire API result to be included in the luis result.

    So, in BotServices.cs when creating the Luis Service it will look like this:

    case ServiceTypes.Luis:
         {
            var luis = service as LuisService;
            var luisApp = new LuisApplication(luis.AppId, luis.SubscriptionKey, luis.GetEndpoint());
            LuisServices.Add(service.Id, new TelemetryLuisRecognizer(luisApp, includeApiResults:true));
            break;
        }
    

    So now the properties in the luis object contains both the sentiment analysis and the entire luis response.

    {{
      "query": "show me orders for last week",
      "alteredQuery": null,
      "topScoringIntent": {
        "intent": "Shopping.CheckOrderStatus",
        "score": 0.619710267
      },
      "intents": null,
      "entities": [
        {
          "entity": "last week",
          "type": "builtin.datetimeV2.daterange",
          "startIndex": 19,
          "endIndex": 27,
          "resolution": {
            "values": [
              {
                "timex": "2018-W43",
                "type": "daterange",
                "start": "2018-10-22",
                "end": "2018-10-29"
              }
            ]
          }
        }
      ],
      "compositeEntities": null,
      "sentimentAnalysis": {
        "label": "negative",
        "score": 0.241115928
      }
    }}