This course takes you through a case study of a real-world scenario in a financial setting. We'll go through how data gets processed, stored, and presented for an insurance dashboard. This dashboard displays all different types of insurance and is powered by an underlying database. So over this course, you'll both see some specifics for the industry vertical of insurance and some specifics on how to process data.
This course is a little less technical than some of our other database-related content, so if you don't have Python or SQL skills, that's fine. This course is really ideal for those who want to learn a little bit more about how all of the pieces go together to understand more of where to dive in deeper.
If you have any feedback relating to this course, please let us know at support@cloudacademy.com.
Learning Objectives
- Learn how to put together a data processing solution for an insurance company
- Plan the project
- Build the front-end interface, the back-end database, and the middleware that connects the two
- Put the whole dashboard solution together
Intended Audience
- Data engineers and database administrators
- Anyone who wants to gain insights into how a data-handling solution is built in a real-world environment.
Prerequisites
To get the most out of this course, you should have some basic experience with databases and building IT solutions.
Thanks, Chris, for that great introduction. So let's start right in with the case study scenario.
Imagine you work for the technology division at Acme Insurance Company. The claims department is tired of their scattered approach to historical claims analysis and they come to you for help. They want to be able to view their historical claims data all in one place.
This course is going to take you through a step-by-step guide of how you would approach this project and at every step, we're going to return to this claims dashboard scenario. So for each step in the process, we're going to take a step back, talk generally about how you would think about approaching that step, and then we're always going to bring it back to the scenario example.
So let's start by going through a high-level overview of the development process. The first step in the process is planning. You can't start building something until you understand what the requirements are. The most important part in planning is to talk to the users and make sure that you know what they want, what they need, and what their expectations are. This is going to set your project up for success.
The next step is to start looking at the database, and there's three pieces to this. First, you need to consider your technology selection. What type of database are you going to use? How are you going to store your data? What's the design of the database? And don't worry if you haven't done this before, we're going to talk a lot more in a lot more detail about this later on in the course. So once you've selected your technology solution, next you need to consider the data pipeline. How are you going to get the data from whatever disparate sources it's currently stored in into your data warehouse, the nice database that you've just designed?
The next step is to start working on the front end. How are you going to design things? What's the front-end technology that you're going to use to build your user interface? And what's the design of the dashboard? What's the UI gonna look like and what's the user experience going to be? These are all really important things that you need to make sure that you get right so that your users are happy.
Finally, we need the glue that holds the front end and the back end together, which is the middleware. We have our front-end dashboard. We have our back-end database solution. But now we need to figure out what's the technology that's gonna allow the front end to talk to the back end and vice versa so that you can actually serve data to your claims dashboard.
So this diagram is showing the architecture of our dashboard, and I think this is really key to understanding the project from start to finish and how you actually build all the pieces and how the pieces fit together to build what your user wants. So if we start looking at this diagram, all the way on the left-hand side, you have your user. That's your claims adjuster. The person who has come to us with this project and who's going to ultimately be using the product. They're the ones that are gonna be touching the front-end application, which is our dashboard.
That front end application, which is slightly to the right of our user on this diagram, is what's gonna make a request for data every time the user clicks a button or tries to update a visualization. So when they open a page or when they update a chart, they're actually going to be requesting data from our database in real-time.
This data request will be handled by the middleware, which we're gonna use an API for and the API will send the data request to the back-end database. And then that database will take the data request and then send back the appropriate response to the middleware, which will then serve that response to the front-end dashboard.
So basically, what happens is the dashboard says, Hey, middleware, I need some data, and the middleware says okay and sends that request on to the database. The database understands this request because the middleware has phrased it in a way that the database understands whether that's in SQL or some other language that the database understands. The database says Okay, here you go. Here's what you asked for, and the middleware then turns around and serves that data to the front-end UI dashboard so that the user can see it.
Lectures
Introduction - Planning - Building the Database - Front-End Design - Middleware Design - Claims Dashboard: Putting it Together
Calculated Systems was founded by experts in Hadoop, Google Cloud and AWS. Calculated Systems enables code-free capture, mapping and transformation of data in the cloud based on Apache NiFi, an open source project originally developed within the NSA. Calculated Systems accelerates time to market for new innovations while maintaining data integrity. With cloud automation tools, deep industry expertise, and experience productionalizing workloads development cycles are cut down to a fraction of their normal time. The ability to quickly develop large scale data ingestion and processing decreases the risk companies face in long development cycles. Calculated Systems is one of the industry leaders in Big Data transformation and education of these complex technologies.