Creating a Cognitive Search Service
Enhance Cognitive Search Solution
This course focuses on the skills necessary to implement a knowledge-mining solution with a focus on the Cognitive Search solution. The course will walk through how to create a Cognitive Search solution and how to set up the process for importing data. Once the data sources have been set up properly, the course will teach you how to create a search index and then how to configure it to provide the best results possible.
- Create a Cognitive Search solution
- Import from data sources
- Create, configure, and test indexes
- Configure AutoComplete and AutoSuggest
- Improve results based on relevance
- Implement synonyms
- Developers who want to include full-text search in their applications
- Data engineers focused on providing better accessibility to organizational data
- AI engineers that provide AI combined with search functionality in their solutions
To get the most out of this course, you should:
- Have a strong understanding of data sources and how data will be needed by users consuming a Cognitive Search solution
- Be able to use REST-based APIs and SDKs to build knowledge-mining solutions on Azure
Hi there, in the next few videos we're gonna be talking about how to enrich your Azure Cognitive Search solution. And we're gonna be doing this based off of the Azure search service, the indexes, the data sources, and indexers that we created in the previous videos related to this course. In the next few videos, we'll talk about how to configure Auto Complete and Auto Suggest which will allow you to provide better search querying capabilities. And we'll talk about how to improve results based on relevancy, and we'll also talk about how to implement synonyms. But in this video we're talking about Auto Complete and Auto Suggest.
Auto Complete finishes a term or phrase based on partial input completing for example, micro with Microsoft. Now, the majority of this is done inside of the user interface that you're providing for your search capability. Within our Azure Portal user interface, we only have access to the search query API and we'll talk about how to actually do that using an API as well.
Auto Suggest is a short list of matching documents such as returning book titles with an ID so that you can link to a detail page about that particular book. There are some very specific requirements for implementing Auto Complete and Auto Suggest. And the requirements are actually the same. The first is that you need to create a suggester that's embedded in the schema, in the index schema. And then when you actually perform a query, you're going to specify either the Auto Complete or Auto Suggest APIs when making the necessary rest API call. And then lastly, if you wanna make it available via your web application is you need to have a UI control to handle search as you type interactions in your client app.
Next is if the field is part of a complex type, for example, a city field within address, make sure to include the parent in the field path when defining the actual suggester or completion. Next, use the default Standard Lucene Analyzer or a Microsoft Language Analyzer. Now I'm gonna show you a demonstration of this using the Azure Portal just to clarify this one, but this should be fairly straightforward if you've created enough indexes and understand the field mapping properties. And then lastly, you cannot add pre-existing fields to a suggester because it requires a rebuild.
Now I say you cannot. The Azure Portal will not allow you to do it. However, you can absolutely do it using the API, but it will require you to go through a rebuild process after you've redefined your existing schema to have a suggester. It's just something to keep in mind. Your best bet is always to add the suggester at the time of index creation.
So let's take a quick look at how to do this inside of the portal, and then we'll wrap up the video by taking a look at some query options. Here we are in my import data wizard in my Azure search service. I'm going through the process of creating a brand new index and using a different table from the adventure works database. So here are all of my fields that have been pulled from the customer table. In order for us to actually define a suggester related to say the company named field, we actually have to define the name of the suggester first.
Once we do that, that then immediately turns on these check boxes here in the suggester column. So we can actually say that this should be a value or a field that is used in Auto Complete or Auto Suggest capabilities when performing a query. Now, again, this is a string based field. So we would set it as being retrievable and searchable, and then we would want to define a specific analyzer. The recommendation is either Standard Lucene or one of the language based Microsoft analyzers.
Now here's the thing. The portal will not stop you from using one of the language-based Lucene analyzers. So keep in mind that these are the requirements. So I'm gonna just use the Standard Lucene, and then specify suggester. Now because of that, the suggester will of course be created and it will be attached to this particular field for both Auto Complete and Auto Suggest capabilities. Now let's go back into the slide deck real quick.
When performing a query against your index, this is what a simple query looks like from a REST API perspective. You're going to do a post call against a set of indexes. In this particular case the index name is hotels-sample-index. You're then going to call the search function specifying an API version and then you're gonna pass an object for what is actually contained in that request.
In this particular case, it's search with a text that would be your full text search. What type of query it is. This is a simple query. And then which fields are you searching against, and then which ones do you want to be returned. So the search fields are what you're going against. The select is what you're gonna be retrieving. And then lastly, do you want to display account.
Now, if we wanna add Auto Complete and Auto Suggest, this is what an Auto Suggest would look like. You'll notice at the end of the docs, we've called the hotels index but we're now calling the suggest API rather than the search API. And then passing it only three fields, the search what's the value being searched for, how many should be returned in the suggestion and what is the suggester name that we should actually be using to produce that list. Auto Complete is very similar, we're going to, again, in this case go up against an insurance index but we're gonna use the Auto Complete API. We're gonna search for Washington medic and then we're gonna specify an Auto Complete mode.
In this particular case, it's gonna be two terms. What do we wanna filter on? How many should we return? And which suggester are we gonna actually use? Now, if you want more information about what all of these attributes are that you would be passing into your request object, you can absolutely take a look at the Azure documentation for further information. But this is the basics for what you're gonna need to perform either an Auto Complete or Auto Suggest capability within the scope of your search service and your search solution. That's all that there is. Hopefully, I'll see you in the next video.
Brian has been working in the Cloud space for more than a decade as both a Cloud Architect and Cloud Engineer. He has experience building Application Development, Infrastructure, and AI-based architectures using many different OSS and Non-OSS based technologies. In addition to his work at Cloud Academy, he is always trying to educate customers about how to get started in the cloud with his many blogs and videos. He is currently working as a Lead Azure Engineer in the Public Sector space.