Amazon CloudSearch: Search-as-a-Service in AWS part 1

Amazon CloudSearch offers an industrial strength search engine for your website that’s fast, reliable, and fully integrated with other AWS services.

As different as large web sites can be from each other, there is one thing you will find just about everywhere: the search box. Without search, many of the resources you’ve worked so hard to create will be virtually inaccessible. In other words, your search box – and the search engine that powers it – is among your web site’s most important tools…which makes selecting a search engine a most critical design consideration. Amazon CloudSearch is definitely an option to consider.

There are some very popular search engines and frameworks available but the most popular, and probably the best recognized, is Apache Lucene. Lucene, an open source library for information retrieval (IR), powers most websites and also enables some other well-known technologies like Solr, Elasticsearch, and Hibernate Search. Amazon CloudSearch incorporates Lucene, and also uses Solr as its underlying search engine.

Amazon CloudSearch:

Amazon CloudSearch is a fully-managed service in the AWS Cloud. It’s simple to set up, manage, and scale, and is a cost-effective search solution for a website or application. Like all managed AWS services, CloudSearch offers easy configuration and auto scaling for data and traffic, self-healing clusters and, when with Multi-AZ is enabled, high availability.

Amazon CloudSearch aims to provide high throughput and low latency in cloud environments. It also supports a rich set of features such as free text search, faceted search, geospatial search, customizable relevance ranking, highlighting, autocomplete, and support for 34 languages. It can be configured for user-provided scaling and availability options and can index documents in various formats and return results in JSON or XML format.

It’s no surprise that Amazon’s own products and sites are powered by CloudSearch for its reliability and performance.

Before going for a deep dive into CloudSearch, we should review some IR terminology:

Indexing

An IR library or search engine pre-processes documents and texts to make them searchable at a very fast pace. Instead of searching through the whole data store in response to each new request, information is retrieved from an index where the search terms are stored. Indexes lie at the heart of any search engine.

Document

Documents are indexed – stored and made searchable. In search engine terms, a document is the basic unit of information and contains data describing something. A document about an employee, for example, might contain information like the employee ID, name, department, job role, manager information, and city. A document about a book could contain the title, author, year of publication, and number of pages.

Documents are composed of fields, which contain more specific pieces of information, like employee ID or address. When you add a document, the information in the document’s fields is added to an index. When you make a query, the index is queried and the matching documents returned.

Domain

A domain has one or more search instances – each with resources such as RAM, CPU allocated to it – and storage for indexing data and processing requests. The number of search instances depends upon the volume and complexity of the documents you want to process.

Amazon CloudSearch features

Let’s explore some key CloudSearch features.

Managed

A managed AWS service takes care of all the intricacies of low level provisioning, error handling, fault tolerance, monitoring, and various management activities. Amazon CloudSearch does all that, leaving you with nothing more to do than create and configure a search domain and upload the data you want indexed. Once that’s done, you and your users will be free to search the data from your website.

But that’s not the whole story. As a managed service, CloudSearch scales up or down automatically according to the amount of data or index size. If your configured search instance becomes inadequate, CloudSearch automatically upgrades itself to the next larger instance type. And when the capacity goes beyond the largest available instance type, the index is partitioned to multiple instances. Moreover you don’t need to worry about indexing, query parsing, query processing, and results handling: that’s all taken care by CloudSearch.

Integrated with AWS Services

Documents for CloudSearch can be inexpensively stored on S3. CloudSearch also works with records in RDS or DynamoDB databases, is integrated with Amazon CloudWatch and supports index field statistics, and is fully integrated with IAM.
Amazon CloudSearch publishes the following four metrics into Amazon CloudWatch:

  • SuccessfulRequests. Number of search requests successfully processed by the search instance.
  • SearchableDocuments. Number of documents available in the search index.
  • IndexUtilization. Index storage utilization rate of the search instance.
  • Partitions. Number of partitions available in the search index.

Highly Available:

CloudSearch is highly available and, if Multi-AZ is configured, a search domain will span two Availability Zones in same region. Updates are automatically pushed to the instances in both AZs.

Security Features:

Integration with IAM means fine-grained control over the creation and deletion of domains, indexing and re-indexing, and user access. Users can use both HTTP and HTTPS connections to send and search data.

CloudSearch Architecture

Users can interact with CloudSearch through any one of three different services:

  • Configuration service, to create and configure search domains.
  • Document service, to upload documents.
  • Search service, to submit search requests.

CloudSearch supported Instance Types:

Currently CloudSearch supports these five AWS EC2 instance types: search.m1.small, search.m3.medium, search.m3.large, search.m3.xlarge, search.m3.2xlarge. If your needs outgrow the capacity of a single search.m3.2xlarge instance, CloudSearch will automatically partition your service across multiple search instances. A search index can be split across as many as ten partitions.

Amazon CloudSearch pricing

Amazon CloudSearch pricing for the various configurations they offer isn’t all that complicated:

Search instances

The cost of single search instances will vary by region.

Amazon CloudSearch - types

When Multi-AZ is enabled, the cost for redundant search instances is also added. When partitioning occurs, the cost of each new search instance in each AZ added to the cost.

Document batch uploads

$0.10 per 1,000 Batch Upload Requests (the maximum size for each batch is 5 MB).

Index Documents requests

Re-indexing is required for indexes when a new field is added. The charge for a re-indexing request is $0.98 per GB of data stored in your search domain.

Data transfer

Data transfer in is free between Amazon CloudSearch and other AWS Services. Here’s the cost for data transfer out:

Amazon CloudSearch - costs

Data transferred between Amazon CloudSearch and AWS services in different regions will be charged as Internet Data Transfers at both ends.

For traffic sent between Amazon CloudSearch and Amazon EC2 instances in the same region, you are only charged for the Data Transfer in and out of the Amazon EC2 instances. Standard Amazon EC2 Regional Data Transfer charges apply.

Conclusion

With Amazon CloudSearch, users can add low-cost search capabilities to their website without bothering with provisioning, managing and handling indexing, data partitioning, and monitoring. In a coming blog post, we will provide hands-on exercises to illustrate how to work with CloudSearch for real-world applications. Cloud Academy offers quizzes on Amazon CloudSearch that you can take in study mode or test mode depending on your goals. The quizzes are a terrific tool both for assessing your knowledge level and for raising it. The feedback from users is that our quizzes work. Cloud Academy has a 7-day free trial so you can try out our courses, labs, quizzes and learning paths.

Got something to say? Add a comment below.

Avatar

Written by

Chandan Patra

Cloud Computing and Big Data professional with 10 years of experience in pre-sales, architecture, design, build and troubleshooting with best engineering practices. Specialities: Cloud Computing - AWS, DevOps(Chef), Hadoop Ecosystem, Storm & Kafka, ELK Stack, NoSQL, Java, Spring, Hibernate, Web Service


Related Posts

Amanda Cross
Amanda Cross
— February 12, 2021

New Content: Get Ready for the CISM Cert Exam & Learn About Alibaba, Plus All the AWS, GCP, and Azure Courses You Know You Can Count On

This month our team of intrepid certification specialists released five learning paths, seven courses, 19 hands-on labs, and three lab challenges!  One particularly interesting new learning path is Certified Information Security Manager (CISM) Foundations. After completing this learn...

Read more
  • alibaba
  • AWS
  • Azure
  • cism
  • DevOps
  • Google Cloud Platform
  • programming
Avatar
Cloud Academy Team
— January 31, 2021

Which Certifications Should I Get?

The old AWS slogan, “Cloud is the new normal” is indeed a reality today. Really, cloud has been the new normal for a while now and getting credentials has become an increasingly effective way to quickly showcase your abilities to recruiters and companies. With all that in mind, the s...

Read more
  • AWS
  • Azure
  • Certifications
  • Cloud Computing
  • Google Cloud Platform
Avatar
Andrew Larkin
— January 31, 2021

The 12 AWS Certifications: Which is Right for You and Your Team?

As companies increasingly shift workloads to the public cloud, cloud computing has moved from a nice-to-have to a core competency in the enterprise. This shift requires a new set of skills to design, deploy, and manage applications in cloud computing. As the market leader and most ma...

Read more
  • AWS
  • AWS Certifications
Avatar
Stuart Scott
— January 29, 2021

AWS Certified Solutions Architect Associate: A Study Guide

Want to take a really impactful step in your technical career? Explore the AWS Solutions Architect Associate certificate. Its new version (SAA-C02) was released on March 23, 2020. The AWS Solutions Architect - Associate Certification (or Sol Arch Associate for short) offers some ...

Read more
  • AWS
  • AWS Certifications
  • AWS Certified Solutions Architect Associate
Amanda Cross
Amanda Cross
— January 7, 2021

New Content: AWS Terraform, Java Programming Lab Challenges, Azure DP-900 & DP-300 Certification Exam Prep, Plus Plenty More Amazon, Google, Microsoft, and Big Data Courses

This month our Content Team continues building the catalog of courses for everyone learning about AWS, GCP, and Microsoft Azure. In addition, this month’s updates include several Java programming lab challenges and a couple of courses on big data. In total, we released five new learning...

Read more
  • AWS
  • Azure
  • DevOps
  • Google Cloud Platform
  • Machine Learning
  • programming
Avatar
Stuart Scott
— December 17, 2020

Where Should You Be Focusing Your AWS Security Efforts?

Another day, another re:Invent session! This time I listened to Stephen Schmidt’s session, “AWS Security: Where we've been, where we're going.” Amongst covering the highlights of AWS security during 2020, a number of newly added AWS features/services were discussed, including: AWS Audit...

Read more
  • AWS
  • AWS re:Invent
  • cloud security
Joe Nemer
Joe Nemer
— December 4, 2020

AWS re:Invent: 2020 Keynote Top Highlights and More

We’ve gotten through the first five days of the special all-virtual 2020 edition of AWS re:Invent. It’s always a really exciting time for practitioners in the field to see what features and services AWS has cooked up for the year ahead.  This year’s conference is a marathon and not a...

Read more
  • AWS
  • AWS Glue Elastic Views
  • AWS re:Invent
Bryony Harrower
Bryony Harrower
— November 6, 2020

WARNING: Great Cloud Content Ahead

At Cloud Academy, content is at the heart of what we do. We work with the world’s leading cloud and operations teams to develop video courses and learning paths that accelerate teams and drive digital transformation. First and foremost, we listen to our customers’ needs and we stay ahea...

Read more
  • AWS
  • Azure
  • content roadmap
  • GCP
Joe Nemer
Joe Nemer
— October 25, 2020

Excelling in AWS, Azure, and Beyond – How Danut Prisacaru Prepares for the Future

Meet Danut Prisacaru. Danut has been a Software Architect for the past 10 years and has been involved in Software Engineering for 30 years. He’s passionate about software and learning, and jokes that coding is basically the only thing he can do well (!). We think his enthusiasm shines t...

Read more
  • AWS
  • careers
  • champions
  • upskilling
Joe Nemer
Joe Nemer
— October 14, 2020

New Content: AWS Data Analytics – Specialty Certification, Azure AI-900 Certification, Plus New Learning Paths, Courses, Labs, and More

This month our Content Team released two big certification Learning Paths: the AWS Certified Data Analytics - Speciality, and the Azure AI Fundamentals AI-900. In total, we released four new Learning Paths, 16 courses, 24 assessments, and 11 labs.  New content on Cloud Academy At any ...

Read more
  • AWS
  • Azure
  • DevOps
  • Google Cloud Platform
  • Machine Learning
  • programming
Joe Nemer
Joe Nemer
— September 15, 2020

New Content: Azure DP-100 Certification, Alibaba Cloud Certified Associate Prep, 13 Security Labs, and Much More

This past month our Content Team served up a heaping spoonful of new and updated content. Not only did our experts release the brand new Azure DP-100 Certification Learning Path, but they also created 18 new hands-on labs — and so much more! New content on Cloud Academy At any time, y...

Read more
  • AWS
  • Azure
  • DevOps
  • Google Cloud Platform
  • Machine Learning
  • programming
Joe Nemer
Joe Nemer
— August 28, 2020

AWS Certification Practice Exam: What to Expect from Test Questions

If you’re building applications on the AWS cloud or looking to get started in cloud computing, certification is a way to build deep knowledge in key services unique to the AWS platform. AWS currently offers 12 certifications that cover major cloud roles including Solutions Architect, De...

Read more
  • AWS
  • AWS Certifications