Start course

This course takes you through a case study of a real-world scenario in a financial setting. We'll go through how data gets processed, stored, and presented for an insurance dashboard. This dashboard displays all different types of insurance and is powered by an underlying database. So over this course, you'll both see some specifics for the industry vertical of insurance and some specifics on how to process data.

This course is a little less technical than some of our other database-related content, so if you don't have Python or SQL skills, that's fine. This course is really ideal for those who want to learn a little bit more about how all of the pieces go together to understand more of where to dive in deeper.

If you have any feedback relating to this course, please let us know at

Learning Objectives

  • Learn how to put together a data processing solution for an insurance company
  • Plan the project
  • Build the front-end interface, the back-end database, and the middleware that connects the two
  • Put the whole dashboard solution together

Intended Audience

  • Data engineers and database administrators
  • Anyone who wants to gain insights into how a data-handling solution is built in a real-world environment.


To get the most out of this course, you should have some basic experience with databases and building IT solutions.


All right, so now that we have an idea of our high-level process, and we understand what the basic architecture is, we can start going into each one of these steps. So the first step, as I mentioned, is planning, and it's really important to have a clear plan and to build a development roadmap. So the first thing you want to do is you want to meet with your business users and understand their needs. You want to know who is going to be using this product, what level of technology they're comfortable with, what their expectations are, and what type of data and visualizations they expect to get out of the product. It's also important at this stage to understand what is most important to the users, as you may not be able to create a product that does everything they want right away. What requirements are must have versus what's just a nice to have? All these things are super important to lay out at the beginning, so that everybody's on the same page before you start the development process. 

You also want to meet with subject matter experts to understand what's currently available in terms of both data and technology. So what claims information is even available? Where is that stored? What technology is currently being used for the claims adjusters to access the historical claims data? Because what you don't want to have happen is the business users come to you and they say we absolutely need to know the date of every claim. And then you find out that that information is just not available anywhere. You want to know that up front, you don't want to have that be a surprise three months into the process.

The next thing is to create a roadmap for development with clear milestones. You want to make sure that everybody understands what you're trying to achieve and when you're trying to achieve it. 

And then make sure that you review this roadmap with all parties, the business people, your subject matter experts, and your technology team, and make sure that everybody is in agreement before you start the project because you don't want to be developing something and working towards something that's not what the user wants. You also want to make sure that the user understands that the first version might not have everything that they asked for in there. It's really important that everyone's on the same page before the project starts and understands both the roadmap and the timeline and that possibly this might be an iterative process. 

So now let's talk about the planning step in terms of our Acme Insurance Company claims scenario. If you are running this project at Acme, the first thing you do is to call a meeting between your team, the people who are in your technology division, who will be building this product, and the leaders of the claims department to better understand their needs. So what are their needs? 

They want to be able to view claims activities over various time periods. They also want to be able to see how many claims there were in a given week, month, or year. So, for example, how many claims did we have in the last week? How many claims do we have in the last month? Etcetera. 

They also want to understand claims activity broken down by claim type. So how many flood claims did we have versus how many fire claims versus how many auto claims, and they want to be able to drill down into a claim and get additional information. So let's say there's a claim that they want to investigate. They want to be able to click on it and drill down and get the policy information, maybe the policyholder, maybe how many other claims that policyholder has had during their current policy period or during the lifetime of the policy that they've had with Acme. So these are the main requirements that the claims department has come to us and told us that they want in their dashboard. So now we understand what their expectations are.


Introduction - Case Study Scenario - Building the Database - Front-End Design - Middleware Design - Claims Dashboard: Putting it Together

About the Author
Learning Paths

Calculated Systems was founded by experts in Hadoop, Google Cloud and AWS. Calculated Systems enables code-free capture, mapping and transformation of data in the cloud based on Apache NiFi, an open source project originally developed within the NSA. Calculated Systems accelerates time to market for new innovations while maintaining data integrity.  With cloud automation tools, deep industry expertise, and experience productionalizing workloads development cycles are cut down to a fraction of their normal time. The ability to quickly develop large scale data ingestion and processing  decreases the risk companies face in long development cycles. Calculated Systems is one of the industry leaders in Big Data transformation and education of these complex technologies.