Amazon CloudSearch: Search-as-a-Service in AWS part 1

Amazon CloudSearch offers an industrial strength search engine for your website that’s fast, reliable, and fully integrated with other AWS services.

As different as large web sites can be from each other, there is one thing you will find just about everywhere: the search box. Without search, many of the resources you’ve worked so hard to create will be virtually inaccessible. In other words, your search box – and the search engine that powers it – is among your web site’s most important tools…which makes selecting a search engine a most critical design consideration. Amazon CloudSearch is definitely an option to consider.

There are some very popular search engines and frameworks available but the most popular, and probably the best recognized, is Apache Lucene. Lucene, an open source library for information retrieval (IR), powers most websites and also enables some other well-known technologies like Solr, Elasticsearch, and Hibernate Search. Amazon CloudSearch incorporates Lucene, and also uses Solr as its underlying search engine.

Amazon CloudSearch:

Amazon CloudSearch is a fully-managed service in the AWS Cloud. It’s simple to set up, manage, and scale, and is a cost-effective search solution for a website or application. Like all managed AWS services, CloudSearch offers easy configuration and auto scaling for data and traffic, self-healing clusters and, when with Multi-AZ is enabled, high availability.

Amazon CloudSearch aims to provide high throughput and low latency in cloud environments. It also supports a rich set of features such as free text search, faceted search, geospatial search, customizable relevance ranking, highlighting, autocomplete, and support for 34 languages. It can be configured for user-provided scaling and availability options and can index documents in various formats and return results in JSON or XML format.

It’s no surprise that Amazon’s own products and sites are powered by CloudSearch for its reliability and performance.

Before going for a deep dive into CloudSearch, we should review some IR terminology:

Indexing

An IR library or search engine pre-processes documents and texts to make them searchable at a very fast pace. Instead of searching through the whole data store in response to each new request, information is retrieved from an index where the search terms are stored. Indexes lie at the heart of any search engine.

Document

Documents are indexed – stored and made searchable. In search engine terms, a document is the basic unit of information and contains data describing something. A document about an employee, for example, might contain information like the employee ID, name, department, job role, manager information, and city. A document about a book could contain the title, author, year of publication, and number of pages.

Documents are composed of fields, which contain more specific pieces of information, like employee ID or address. When you add a document, the information in the document’s fields is added to an index. When you make a query, the index is queried and the matching documents returned.

Domain

A domain has one or more search instances – each with resources such as RAM, CPU allocated to it – and storage for indexing data and processing requests. The number of search instances depends upon the volume and complexity of the documents you want to process.

Amazon CloudSearch features

Let’s explore some key CloudSearch features.

Managed

A managed AWS service takes care of all the intricacies of low level provisioning, error handling, fault tolerance, monitoring, and various management activities. Amazon CloudSearch does all that, leaving you with nothing more to do than create and configure a search domain and upload the data you want indexed. Once that’s done, you and your users will be free to search the data from your website.

But that’s not the whole story. As a managed service, CloudSearch scales up or down automatically according to the amount of data or index size. If your configured search instance becomes inadequate, CloudSearch automatically upgrades itself to the next larger instance type. And when the capacity goes beyond the largest available instance type, the index is partitioned to multiple instances. Moreover you don’t need to worry about indexing, query parsing, query processing, and results handling: that’s all taken care by CloudSearch.

Integrated with AWS Services

Documents for CloudSearch can be inexpensively stored on S3. CloudSearch also works with records in RDS or DynamoDB databases, is integrated with Amazon CloudWatch and supports index field statistics, and is fully integrated with IAM.
Amazon CloudSearch publishes the following four metrics into Amazon CloudWatch:

  • SuccessfulRequests. Number of search requests successfully processed by the search instance.
  • SearchableDocuments. Number of documents available in the search index.
  • IndexUtilization. Index storage utilization rate of the search instance.
  • Partitions. Number of partitions available in the search index.

Highly Available:

CloudSearch is highly available and, if Multi-AZ is configured, a search domain will span two Availability Zones in same region. Updates are automatically pushed to the instances in both AZs.

Security Features:

Integration with IAM means fine-grained control over the creation and deletion of domains, indexing and re-indexing, and user access. Users can use both HTTP and HTTPS connections to send and search data.

CloudSearch Architecture

Users can interact with CloudSearch through any one of three different services:

  • Configuration service, to create and configure search domains.
  • Document service, to upload documents.
  • Search service, to submit search requests.

CloudSearch supported Instance Types:

Currently CloudSearch supports these five AWS EC2 instance types: search.m1.small, search.m3.medium, search.m3.large, search.m3.xlarge, search.m3.2xlarge. If your needs outgrow the capacity of a single search.m3.2xlarge instance, CloudSearch will automatically partition your service across multiple search instances. A search index can be split across as many as ten partitions.

Amazon CloudSearch pricing

Amazon CloudSearch pricing for the various configurations they offer isn’t all that complicated:

Search instances

The cost of single search instances will vary by region.

Amazon CloudSearch - types

When Multi-AZ is enabled, the cost for redundant search instances is also added. When partitioning occurs, the cost of each new search instance in each AZ added to the cost.

Document batch uploads

$0.10 per 1,000 Batch Upload Requests (the maximum size for each batch is 5 MB).

Index Documents requests

Re-indexing is required for indexes when a new field is added. The charge for a re-indexing request is $0.98 per GB of data stored in your search domain.

Data transfer

Data transfer in is free between Amazon CloudSearch and other AWS Services. Here’s the cost for data transfer out:

Amazon CloudSearch - costs

Data transferred between Amazon CloudSearch and AWS services in different regions will be charged as Internet Data Transfers at both ends.

For traffic sent between Amazon CloudSearch and Amazon EC2 instances in the same region, you are only charged for the Data Transfer in and out of the Amazon EC2 instances. Standard Amazon EC2 Regional Data Transfer charges apply.

Conclusion

With Amazon CloudSearch, users can add low-cost search capabilities to their website without bothering with provisioning, managing and handling indexing, data partitioning, and monitoring. In a coming blog post, we will provide hands-on exercises to illustrate how to work with CloudSearch for real-world applications. Cloud Academy offers quizzes on Amazon CloudSearch that you can take in study mode or test mode depending on your goals. The quizzes are a terrific tool both for assessing your knowledge level and for raising it. The feedback from users is that our quizzes work. Cloud Academy has a 7-day free trial so you can try out our courses, labs, quizzes and learning paths.

Got something to say? Add a comment below.

Avatar

Written by

Chandan Patra

Cloud Computing and Big Data professional with 10 years of experience in pre-sales, architecture, design, build and troubleshooting with best engineering practices. Specialities: Cloud Computing - AWS, DevOps(Chef), Hadoop Ecosystem, Storm & Kafka, ELK Stack, NoSQL, Java, Spring, Hibernate, Web Service

Related Posts

Avatar
Jeremy Cook
— September 17, 2019

Cloud Migration Risks & Benefits

If you’re like most businesses, you already have at least one workload running in the cloud. However, that doesn’t mean that cloud migration is right for everyone. While cloud environments are generally scalable, reliable, and highly available, those won’t be the only considerations dri...

Read more
  • AWS
  • Azure
  • Cloud Migration
Joe Nemer
Joe Nemer
— September 12, 2019

Real-Time Application Monitoring with Amazon Kinesis

Amazon Kinesis is a real-time data streaming service that makes it easy to collect, process, and analyze data so you can get quick insights and react as fast as possible to new information.  With Amazon Kinesis you can ingest real-time data such as application logs, website clickstre...

Read more
  • amazon kinesis
  • AWS
  • Stream Analytics
  • Streaming data
Joe Nemer
Joe Nemer
— September 6, 2019

Google Cloud Functions vs. AWS Lambda: The Fight for Serverless Cloud Domination

Serverless computing: What is it and why is it important? A quick background The general concept of serverless computing was introduced to the market by Amazon Web Services (AWS) around 2014 with the release of AWS Lambda. As we know, cloud computing has made it possible for users to ...

Read more
  • AWS
  • Azure
  • Google Cloud Platform
Joe Nemer
Joe Nemer
— September 3, 2019

Google Vision vs. Amazon Rekognition: A Vendor-Neutral Comparison

Google Cloud Vision and Amazon Rekognition offer a broad spectrum of solutions, some of which are comparable in terms of functional details, quality, performance, and costs. This post is a fact-based comparative analysis on Google Vision vs. Amazon Rekognition and will focus on the tech...

Read more
  • Amazon Rekognition
  • AWS
  • Google Cloud Platform
  • Google Vision
Alisha Reyes
Alisha Reyes
— August 30, 2019

New on Cloud Academy: CISSP, AWS, Azure, & DevOps Labs, Python for Beginners, and more…

As Hurricane Dorian intensifies, it looks like Floridians across the entire state might have to hunker down for another big one. If you've gone through a hurricane, you know that preparing for one is no joke. You'll need a survival kit with plenty of water, flashlights, batteries, and n...

Read more
  • AWS
  • Azure
  • Google Cloud Platform
  • New content
  • Product Feature
  • Python programming
Joe Nemer
Joe Nemer
— August 27, 2019

Amazon Route 53: Why You Should Consider DNS Migration

What Amazon Route 53 brings to the DNS table Amazon Route 53 is a highly available and scalable Domain Name System (DNS) service offered by AWS. It is named by the TCP or UDP port 53, which is where DNS server requests are addressed. Like any DNS service, Route 53 handles domain regist...

Read more
  • Amazon
  • AWS
  • Cloud Migration
  • DNS
  • Route 53
Alisha Reyes
Alisha Reyes
— August 22, 2019

How to Unlock Complimentary Access to Cloud Academy

Are you looking to get trained or certified on AWS, Azure, Google Cloud Platform, DevOps, Cloud Security, Python, Java, or another technical skill? Then you'll want to mark your calendars for August 23, 2019. Starting Friday at 12:00 a.m. PDT (3:00 a.m. EDT), Cloud Academy is offering c...

Read more
  • AWS
  • Azure
  • cloud academy content
  • complimentary access
  • GCP
  • on the house
Avatar
Michael Sheehy
— August 19, 2019

What Exactly Is a Cloud Architect and How Do You Become One?

One of the buzzwords surrounding the cloud that I'm sure you've heard is "Cloud Architect." In this article, I will outline my understanding of what a cloud architect does and I'll analyze the skills and certifications necessary to become one. I will also list some of the types of jobs ...

Read more
  • AWS
  • Cloud Computing
Avatar
Nitheesh Poojary
— August 19, 2019

Boto: Using Python to Automate AWS Services

Boto allows you to write scripts to automate things like starting AWS EC2 instances Boto is a Python package that provides programmatic connectivity to Amazon Web Services (AWS). AWS offers a range of services for dynamically scaling servers including the core compute service, Elastic...

Read more
  • Automated AWS Services
  • AWS
  • Boto
  • Python
Avatar
Andrew Larkin
— August 13, 2019

Content Roadmap: AZ-500, ITIL 4, MS-100, Google Cloud Associate Engineer, and More

Last month, Cloud Academy joined forces with QA, the UK’s largest B2B skills provider, and it put us in an excellent position to solve a massive skills gap problem. As a result of this collaboration, you will see our training library grow with additions from QA’s massive catalog of 500+...

Read more
  • AWS
  • Azure
  • content roadmap
  • Google Cloud Platform
Avatar
Adam Hawkins
— August 9, 2019

DevSecOps: How to Secure DevOps Environments

Security has been a friction point when discussing DevOps. This stems from the assumption that DevOps teams move too fast to handle security concerns. This makes sense if Information Security (InfoSec) is separate from the DevOps value stream, or if development velocity exceeds the band...

Read more
  • AWS
  • cloud security
  • DevOps
  • DevSecOps
  • Security
Avatar
Stefano Giacone
— August 8, 2019

Test Your Cloud Knowledge on AWS, Azure, or Google Cloud Platform

Cloud skills are in demand | In today's digital era, employers are constantly seeking skilled professionals with working knowledge of AWS, Azure, and Google Cloud Platform. According to the 2019 Trends in Cloud Transformation report by 451 Research: Business and IT transformations re...

Read more
  • AWS
  • Cloud skills
  • Google Cloud
  • Microsoft Azure