Welcome back! In this lesson, we're going to put Twitter out of business with our amazing Twitter replacement. OK, not really, though what we are going to do is set up a Twitter clone called Tweeter, that uses Cassandra, Kafka, Zeppelin, the Marathon Load Balancer, virtual IP's, persistent volumes, Docker containers, and more.

Here's the source code for the application that we'll be using. It's only maintained by Mesosphere. We're going to use the Twitter clone named Tweetter to serve as the web UI. Part of the same repo is a post tweet container that will start posting Shakespeare quotes, and we're going to use the Zeppelin package from the catalog to analyze the data.

In order to use this app first, we need to install the pre-reqs. First up is the Marathon Load Balancer. And we're going to use the default, so we can install this with relative ease by typing DCOS package, install, Marathon LB. And will say yes to the prompt, and there it is. Next will add Cassandra, so I'll just edit the previous command, and I will say yes.

All right. Perfect. Now let's install Kafka. OK. Great. All right, these are going to, do their thing and deploy. However, while that's trudging along, let's take a look at the Tweeter services deployment file. I have the app already cloned here, and I've already added the IP address to the V host, so the load balancer will work.

You can see that there's a health check, and we have some environment variables that the app requires. Check out this here, this port mapping. Checkout line 19, where it sets up this VIP. This means that an internal service can interact with this app on the IP address of 1. 1. 1. 1 over port 30,000. This is how the tweet bots are going to know where to find this service.

This makes for a great way to find internal services, and it serves as both a layer for a load balancer and service discovery mechanisms. So that's really cool. So this app will start a Rails::Application that runs the UI. It also sets up a VIP that allows internal services to interact with it, via IP address of 1.

1. 1. 1. Next let's create this. OK. Perfect. And let's head over to the UI, and there's the app, and it's deploying now. Now this could take a few minutes, so I'm going to fast forward. OK, now that the app is deployed, let's test it out, and we can try out the user interface. So here it is. You can see that there's a spot to set a handle, and then a message.

I'll just say, "Hello. " And there's the message. And it looks like I don't need to add the at symbol for myself, so that's good to know. So this is great. The app is running and it works. We can use the UI, it's persisting data, which means Cassandra's running. And, now it's time to try and analyze the tweets.

However, we don't have enough data, so we need to do something to generate some tweets. And luckily Mesosphere has given us a container for that. So let's check out this post tweets app definition. It's going to use the Tweeter Docker container, and it's going to run to the tweet command, and notice that it passes the VIP of the Tweeter app.

This shows the value of VIPs, because, you don't need to know where the service is running, all you need to know is the virtual IP address and the port, and you're able to interact with your service. Let's create this app, so that it will start spamming out tweets to our Twitter clone. OK, perfect. Back in the UI, if I refresh this.

OK look at all of these tweets. So it is working. So we have our Tweeter app running. It's storing tweets and Cassandra. It streaming incoming tweets with Kafka, and now it's time to analyze the tweets with Zeppelin. So, first we actually need to install Zeppelin. So ,over in the catalog, I'm going to search for it.

And there it is, and let's do an advanced installation by clicking on configure. Let's set the max cores to eight, and let's install this. And I'm going to fast forward just a bit until this is done. OK here we are. So if you're not familiar with Zeppelin, it's a web-based notebook that's used for data analysis.

It allows you to easily execute code, visualize the results, and share both the code and the results. Now it's outside the scope of this course. However it's a really cool product, and I think you should check it out if you're going to need to do any sort of data analysis. So let's open up the app by clicking on the link next to the service name.

For this demo we're going to use the code provided by Mesosphere. So let's import that notebook. This is in the Tweeter analytics JSON file. And importing it doesn't look like it did much. But what it did is at this notebook here, so let's open that up. And you can see that there are different sections here called paragraphs that allow you to break out your code, And for this demo let's use the run all paragraphs option.

What this is doing is getting all the tweets and analyzing them to determine the bot that tweets the most. It's going to take a little while, so I'm going to fast forward just a bit. OK. Notice here this last paragraph threw an error. It's complaining because the tweets table doesn't exist. Running this paragraph again should solve that problem.

So let's do that. And there we have it. We have a little chart that shows the top tweeting bots. All right. So far a lot has happened in this lesson, whether it really looks like it or not. So let's recap. In order to see how all the different functionality comes together, we installed an app that uses several different services together.

Some of the services were found in the catalog. Others were created using Docker containers. We created a service called Tweeter, that was a Rails::Application running in Docker. It served as the user interface that store the tweets to Cassandra. We used another Docker container to run a binary that generated tweets based on Shakespeare quotes by a bunch of different bots.

And this generated enough data to analyze. The incoming tweets streamed with Kafka, and then Zeppelin analyzed all that data to determine which bots tweet the most. Okay, let's wrap up this lesson here. In the next lesson, let's wrap up this course and talk about next steps.