Implementation: Entities & Training
Building Chatbots with Google Dialogflow
This course is the second part of our series on building chatbots using Google Dialogflow. In this course, we'll be taking a hands-on approach to implementing a Dialogflow chatbot, specifically focusing on the graphical user interface. You'll follow along as we cover the example of building an online banking customer service chatbot using Dialogflow.
We'll also discuss the use of databases and other knowledge sources in order to provide meaningful responses that give you the option to serve real-world information to your user.
- Learn what intents are and how to train your chatbot to look for them
- Set up entities in Dialogflow
- Understand how chatbots interact with users and how to test your chatbot
- Learn how to connect your chatbot to a data source
- Understand the user interfaces available with Dialogflow
- Learn how to use the Knowledge service to build chatbots quickly and with less configuration
- Anyone looking to build chatbots using Google Dialogflow
Before taking this course, make sure you've done Part One first. To get the most out of this course, you should also have a basic understanding of:
- Computer science techniques
- REST APIs and SQL
- Google Cloud Platform
So we need to put a quick pause on the intent creation and switch over to entities. You'll also see these on the left right below intents. And when you click on it, you'll see create first one. You'll also see the options for customers system. We'll discuss that in a second. But what we really need to start doing is create custom entities by either clicking the create entity at the top or in the middle of the screen. And we have to start naming them much like you would with an intent.
Now don't get too fancy with names. Try to keep them safe and lowercase because much like intents they might get passed to the backend as well. So we're going to need three entities in total. We need a card number, which is used in both of our intents, a dollar amount, which is the currency, and the foil, the account ID, the piece of data that somebody might throw us that we absolutely don't want to get confused with card number, and we need to be able to dismiss quickly.
Actually I want to pause here for a second and highlight that we're making a pretty big assumption. The users are going to be entering all of these entities in their message directly. Now, this isn't always the case though. When an external service is used, particularly for a service like Twilio when handling text messages, or Slack, or WeChat, you're going to get entities like account ID being passed in from that service's plugin. So you're actually gonna have some of them filled in, which will massively decrease the burden on the user.
So back to creating the entity, when defining these they're going to fall into one of three types. The first one is the pre-defined system entity type. This is something that Dialogflow comes out of the box with. The good news is, for the entity dollar amount, that's already in Dialogflow for currency value. Dialogflow comes programmed to detect tons of currencies and times and dates and many of the common variables. So be sure to check the functionality that you want and that it doesn't come out of the box already.
And the second type is developer entities, aka custom entities. These are the ones that you're going to create yourself for stuff that is specific to your use case. In this case, Dialogflow has no idea what a payment card or account ID look like. So we're going to need to program it so it's able to recognize that.
And the third type is session entities. We're not gonna dive too much into these but this is where that previously mentioned built-in stuff comes from, from Twilio or the web interface client that gets passed in, where the data is coming from another service or a previous comment related to the user's current chatting.
So on the create entity screen, underneath customer entity or developer entity, whichever you're calling it, you need to train it how to detect a card number. Now, card numbers, just for clarification, are always going to follow the same pattern of four numbers followed by a letter of the alphabet.
So within the entity creation screen, we're presented with four main options: defined synonyms, regex expression or regular expression, allow automated expansion, and fuzzy matching. Basically we need to train the system to identify the number and what traditional settings to allow.
So to just cut straight to the chase, we're going to use regex. This is a regular expression if you're familiar with it. It's been around for a very long time. And it's standard across most languages and platforms. The reason regex is really good is because in the case of card number, where it follows a regular well-defined pattern, we can easily express this. There's no weirdness, there's no fuzziness, there's no deviation allowed in the characters. We're always going to have four numbers followed by a letter.
So regex in its greatest strength, and frankly greatest weakness, is rigid. Regex is really good for detecting the same pattern again and again because you can set it to either match or not match. And we also have some added value that we can target intents where there might be additional confusing stuff.
So in intents, such as where they might throw account number in there by mistake, regex's rigidity allows us to automatically discount that with very little overlap while still capturing the card number with a high degree of reliability. However, it's really worth emphasizing that while rigidity is its strength, it's also its weakness. There's some options we can have to mitigate this weakness. So let's dive into those, but know that regex is a very strict way of defining your entity.
So one of the other options is to find synonyms, which is actually the default entity type. And in this one, you're simply going to create a list of potential values for the parameters, such as if you're gonna create a shopping list for an entity of bread, you might want to include alternative bread products, which would roll up into bread. For example, when the user says bread synonyms might be bagel, baguette, English muffin, ciabatta, Italian bread, or flatbread. And all of those will roll up into the parent bread definition.
However, this sometimes isn't enough. You can't name all the types of bread. And this is where automated expansion comes in. Automated expansion allows Google to attempt to interpret the intent of the entity and add values on to it. So if you're training the entity to identify food items, it actually can expand those food items and attempt to make best estimates.
So in this case, you're giving up some of the direct control, but you're also capturing more of what the user might asking. Now, a step beyond this is fuzzy matching. Fuzzy matching allows things to be remixed even when the entity isn't perfect.
Imagine another example where we're defining a red ball or maybe we're defining a small ball or maybe it could also be expressed as a large red ball. What about a small red ball? Would that be allowed? Well, if you have fuzzy matching on, you're able to remix all of those terms and phrases into something that would match. So fuzzy matching allows you to have an even more comprehensive option.
So a lot of the entity definition, just to recap, is about how tight we control it versus how much of loose control do you want on what an entity is. You can go all the way down to make the exact characters and exact word, or it can go all the way back out and say, Hey, remix these words and letters around to make a best effort guess. And now with entities defined, let's go back to the intent and enter and touch up the training phrases.
So back on the intent screen, if you've already entered some intents, you might actually see that incorrectly identified some entities. So if you happen to enter check my card balance for number 1234A and then you put it down like that, you'll actually see that it probably identified the card number as a phone number type.
Now, it's really cool that Dialogflow is making a best effort to identify entities, but that's not what we need. We need to change it. So depending on your system, you either need to click, double click or highlight the segment of the part that is the card number. And once you do that, you're able to select the correct entity type from the dropdown.
So let's do highlight it or double-click it if it's been misidentified or re-highlight it. You can select that entity type and assign it to card number, which now means you're correctly identifying it. And you should see on the right of the menu what it's resolved as. And that's especially useful. So as you're going through multiple training phrases you can actually see what it's detecting and resolving and actually test it in real time.
So let's be creative. Don't use the same card number every time. Create a few representative phrases so you don't need to have the entity parameters slotted in the same order. Put one of them before the other and get really good coverage on the permutations. And this is important because at the next step, we're going to actually test it with a conversation.
Calculated Systems was founded by experts in Hadoop, Google Cloud and AWS. Calculated Systems enables code-free capture, mapping and transformation of data in the cloud based on Apache NiFi, an open source project originally developed within the NSA. Calculated Systems accelerates time to market for new innovations while maintaining data integrity. With cloud automation tools, deep industry expertise, and experience productionalizing workloads development cycles are cut down to a fraction of their normal time. The ability to quickly develop large scale data ingestion and processing decreases the risk companies face in long development cycles. Calculated Systems is one of the industry leaders in Big Data transformation and education of these complex technologies.