image
Key Value
Start course
Difficulty
Beginner
Duration
50m
Students
139
Ratings
5/5
Description

In this course, we will introduce you to some common types of databases. Different data problems can be solved in a wide variety of ways, which is why so many different types of databases exist. After learning about a few different types of databases, you’ll be ready to explore specific implementations further.

Learning Objectives

During this course, I’ll introduce you to the following:

  • SQL databases
  • Key-value databases
  • Document-databases
  • Graph databases

Intended Audience

This course is for novice:

  • Software engineers
  • Data engineers
  • DevOps engineers
  • Site reliability engineers

Prerequisites 

  • You should have at least a conceptual understanding of programming and be comfortable with data structures, data types, etc.
Transcript

Most programming languages include a data structure referred to as a dictionary or hashmap. Dictionaries are one of the most generally useful data structures in computer science. They’re a collection of key-value pairs of data. Where some key is used to access the associated value data. 

In this lesson, I’ll introduce you to key-value databases. Which are databases that are based on the same underlying abstract data structure as dictionaries. Dictionaries are examples of associative arrays. Associative arrays are collections of key-value pairs. Keys are used as means of accessing the value associated with the key. Associative arrays can be compared to a coat check. 

Imagine that you arrive at an event and a member of the event staff offers to take your coat. You hand them your coat and they return you a ticket with an ID which represents your coat. The coat is stored in some closet until you need it. To get your coat back you provide a staff member with your ticket so that they can identify which coat to retrieve from the closet. 

To overlay the concept of an associative array; the closet where the coats reside is our array filled with coat values. And the tickets are the key used to identify which coat to retrieve. As a less illustrative and more technical example, imagine using a key consisting of GPS coordinates to represent metadata about that specific location. Keys inside an associative array are guaranteed to be unique. Which means if you attempt to set the same key multiple times, it will use the last value set. 

Having distinct keys is a valuable feature of the underlying data structure. It allows developers to ensure that there won’t be any duplicated data. Key-value databases are examples of associative arrays. At their core, they’re quite similar. They can: get, set, and delete data rather quickly. They’re basically giant lookup tables with some additional functionality built on top. The additional functionality depends on the implementation of the database.

For example: 

  • Some are intended to be embedded in an application

  • Some are intended to be used at a large scale and can be partitioned

  • Some support features such as:

    • bulk data insertion

    • key expiration dates

    • multiple data types for keys and or values

    • single data type for keys and or values

    • storing data on disk for long-term storage

    • in-memory data only

Interacting with key-value databases is different from interacting with SQL databases. SQL databases use a domain-specific language to interact with data. However, key-value databases don’t tend to include their own programming language. Rather, they tend to provide some other interface. Such as basic network-based command protocol. Where commands are sent directly to the database to perform data operations. Commonly key-value databases include some form of client. Which might be command line based, or a programming language specific library.

Each implementation determines which data types are allowed for keys and values. Some have few types and might store everything as a string. Others may allow different data types such as integers, floating point numbers, etc. While getting data back from a key-value database will be similar across different implementations, there is variation. For example, some implementations might support some form of key querying such wildcards. 

While conceptually basic, there’s a lot of computer science effort that goes into specific implementations of abstract data structures. Key-value databases have existed for decades and continue to grow and evolve. During that time different contributors have invested countless hours towards the continued development of these databases. 

Modern key-value databases take advantage of modern hardware. By efficiently storing data in memory they can access data rather quickly. Being able to quickly lookup value data for a key is useful in a wide range of applications. 

Let’s talk about some of the different use cases for key-value databases. Developers often use key-value databases to cache slow running queries or commonly requested data. Allowing it to be accessed much faster than going directly to data’s canonical source. When using a key-value database as a cache, typically, data only exists in memory. In this scenario the cache is used to provide a more responsive user experience; however, should the cache become unavailable the application remains functional.

Other use cases include embedding a key-value database into an application such as a video game, or mobile application. Where the application uses the functionality of the database towards some purpose. For example, a video game developer might store a player’s checkpoint data. Database types such as SQL provide such general purpose usability that they’re commonly used by a wide range of users. Software engineers, data engineers, data analysts across different industries, and many non-engineering roles. 

Key-value databases are a bit less popular across such a wide user audience. The typical user audience for key-value databases consists of different types of engineers. For example, software engineers might introduce an embedded key-value database to an application to cache slow operations. 

A product team might introduce a key-value database into a web application to cache commonly requested data thereby reducing demand on the data’s canonical service. There are many different key-value databases available which all follow their own implementation. Choosing one over another is about considering the needs of the application and finding the key-value database with acceptable trade-offs.

Computing is all about trade-offs. Every design choice brings with it a set of trade-offs. Perhaps one implementation is optimized to quickly write records to disk. Yet another might be optimized for efficient key queries. There are quite a few different key-value databases. Though, I’m going to call out some standouts which are so widely used that they’re worth knowing about. The first option I want to mention is an open-source database called memcached. 

Memcached describes itself as a:

“Free & open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load. Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.” 

Memcache was released in 2003 and is still actively developed today. It’s used by companies such as Youtube, Twitter, Reddit, Google, etc. It’s quite fast and reasonably easy to interact with in code. Making an option worth remembering. The next option I want to mention is an open-source database called Redis. Here’s what Redis says about itself: 

“Redis is an in-memory database that persists on disk. The data model is key-value, but many different kinds of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes, Streams, HyperLogLogs, Bitmaps.”

Redis was released in 2009 and has continued to evolve. It’s widely used due to its minimalist design and fast operations. Redis includes a command line interface which allows commands to be issued directly to the database. There are also many programming language specific libraries used to interact with Redis. This is one of the more commonly used key-value databases. The final option I want to mention is another open source database called etcd. Here’s what ectd says about itself:

“etcd is a strongly consistent, distributed key-value store that provides a reliable way to store data that needs to be accessed by a distributed system or cluster of machines. It gracefully handles leader elections during network partitions and can tolerate machine failure, even in the leader node.”

Etcd was released in 2013 to be a key-value database used for mission critical applications. Since then it has become a core component for larger projects such as Kubernetes - a popular container orchestration tool.  Its use inside these higher-level systems has expanded ectd’s chances to prove its efficacy as a production quality database.  

Okay, this seems like a natural stopping point. Here are your key takeaways for this lesson:

  • Associative arrays are collections of key-value pairs

    • Where each key is mapped to its value data

  • Key-value databases are implementations of associative arrays

    • Which means they can get, set, and remove data.

    • Additional functionality depends on the purpose of the databases

  • Key-value databases commonly provide programming language specific libraries used to interact with data

  • Key-value databases range in complexity from simple to complex

  • Three common key-value databases include:

    • memcached

    • redis

    • ectd

Okay, that's going to be all for this lesson. Thanks so much for watching. And I’ll see you in another lesson!

About the Author
Students
101042
Labs
37
Courses
44
Learning Paths
58

Ben Lambert is a software engineer and was previously the lead author for DevOps and Microsoft Azure training content at Cloud Academy. His courses and learning paths covered Cloud Ecosystem technologies such as DC/OS, configuration management tools, and containers. As a software engineer, Ben’s experience includes building highly available web and mobile apps. When he’s not building software, he’s hiking, camping, or creating video games.

Covered Topics