Hello and welcome. Let's use this lesson to talk about what it is that we're building. We're building a web application for the purpose of helping us to visualize our dataset.

In a previous course, we created a data ingestion process that extracted named entities from articles, published by different publications. The data is stored in Google CloudsFirestore. The structure is roughly this. 

We have a collection of publications. Each publication document contains a count property which represents the number of articles processed for the given publication. Publication documents also contain an entities collection named ent. Each entity has a property for count and word. Word contains the extracted named entity, and the count is how many times that was mentioned for the given publication.

If I sort this by count, we can see the types of phrases that we should expect at the top of the list. So this is our data set that we'll use to visualize word counts. Since we're dealing with visualizing the frequency of words, I wanna create a word cloud. The library I'm using for this is aptly named wordcloud and it's built on top of the Python Imaging Library, PIL. Let's check out the user interface for the app.

This app is a bit basic, it's a single page app with some cards showing a word cloud, publication name and article count. Each card has a button that opens up a sidebar listing off the word counts and we can page these with the left and right arrows.

All right, so we have our data in Firestore and we have our front end. Let's use these two sources to kind of reverse engineering and figure out what our data access layer should look like. Looking at the sidebar, we know that we need a function that returns a list of words and counts.

Looking at the cards, we can see that we need a function that returns a list of publication names and article counts. Looking at the API for word clouds, it shows that we can use a fit_words method which accepts a dictionary and we'll use that to build the images. The dictionary's keys are going to represent the word and the value stores the count.

So basically we need a method that returns publications, another that returns entities for a given publication and a method that transforms the results into the format that we need for word cloud. This is roughly what this looks like in Python.

Publications will be a generator that yields publication models, the word counts method returns a generator yielding word counts and frequencies returns a dictionary with a string value for the key and an integer for the count. So these are roughly speaking what we need to build out our data access code. And that's what we'll check out in the next lesson.

So whenever you're ready to keep going, I will see you in the next lesson.

Lectures

Course Introduction - Quality of Life for Developers - Exploring the Data Access Layer - The Web Server Gateway Interface - Exploring the Web Application Layer - Exploring the Front End Code - Running the Web App - Summary / Next Steps