As I said in the introduction, query structure is definitely not the whole story to better query performance. The biggest bottleneck to efficiency and speed, as with most computer systems, is accessing data from the hard drive. This bottleneck is particularly acute with databases due to their massive size.

Table structure and in particular indexes have the greatest bearing on how quickly you can pull the data you want. I don't think it's an exaggeration to say that without a good structure and indexes, finding a particular record in a 20-gigabyte table would be like you were asked to find Alec Baldwin and the only information you had to go on, that he was in New York. With no other clues, you would drive up to the outskirts of New York and then just start knocking on doors and asking, "Are you Alec Baldwin?" On average, it will take a very long time to find him.

When put in those terms, it seems like an outrageous task, yet routinely computers are asked to do a similar thing in poorly structured databases. It's only because electrons move at the speed of light that it doesn't seem like a big ask to us humans. But when the database and query are fully optimized, it's like you are being given the address of Alec Baldwin's apartment building. It's literally orders of magnitude improvement in efficiency, and we can expect to see the same improvement with a database.

Obviously, with the database, you aren't always looking for people, but products or dates and times. In fact, you might want to find any piece of data that is stored. More often than not, you don't know ahead of time what you are wanting to find.

Very broadly speaking, the search strategy comes in two parts. Firstly, indexes, which are maps or directories of your data, and secondly, the strategy of divide and conquer. This is essentially breaking down a large amount of data into smaller pieces that can be searched more quickly. I know in a sense that is how indexes can be viewed, but this idea of segmenting or partitioning data can be applied to all structural levels of the database, from files to tables and indexes.

To be honest, I did struggle a bit with how to start this course. I could start at the macro level with structuring the database's files, or at the beginning of building a database with the table design. But I decided that to really appreciate the performance benefits of correctly doing those first two things is to be able to assess the difference between good and bad design. So with that in mind, let's start by looking at the tools we can use to measure query performance.