The course is part of this learning path
This course focuses on machine learning. We're going to discuss what machine learning is and how we can leverage it to create intelligent and engaging apps. You'll also follow along as we create an image recognition app.
Hi. Within this lecture we're going to complete our image recognition app by writing our handler so that we can fire up this request and get the observations out of this Core ML model. As you can see there is a sample code for creating the handler and before we go on and do that I'm just going to focus on this line. So, DispatchQueue.global, add userInitiated global and this will work asynchronously. So, we have never worked with this kind of DispatchQueue before. We have done the main I think thing and we have understood why it was necessary to do that, but what is this? Why we are using this global and why are we using this async at all? So, let me go over to Google and let me try to understand this a little bit. So, let me write something like DispatchQueue main versus global, okay? So, the first result is the Stack Overflow, of course, because this is the best friend of a developer, so this is Stack Overflow and we can find all the answers in the world regarding to development. So, let me write swift in order to be more explicit. And as you can see we have, again the Stack Overflow over here and let me just try to go one of these results and let me try to understand the difference between this DispatchQueue types. And as you can see, one of the guys over here has the same question in mind regarding to DispatchQueue. So, it says that, there is a main thread; there is a main and there is a global and what are the differences between those two? And let me just look at the first result, the first answer in here. So, this guy is trying to explain the differences between them and in here we have something like, sync put tasks in queue and wait until it finishes. The async doesn't actually wait until it finishes, so it works asynchronously and we already know that. We actually used it before and we understand what it takes to do something asynchronously and why is it necessary because we have done this a lot, like we have been downloading data from the Internet and everything and we have, we need actually a more clear answer. And by the way, you don't have to do that, obviously. I'm just trying to make a case here showing that you can find your answers easily online by Googling and looking over the Stack Overflow and here I believe we have our answer. So, as you can see, it says that, you can use this global queue and it's a very high priority background queue. So, if we want to make something high priority and if we want to work this on a background thread, then we have to use this global queue, but it's not generally used, so Swift doesn't actually want us to be involved in the priorities too often. And as you can see in other answers they mentioned that, this is rarely used and if you want to do something very fast in the background you can use this. So, we didn't need that before, but as Apple suggests in this case, we have to use this because there's a lot of things going on in the background when we try to recognize an image. It just goes into the model and follows the language algorithm and it just tries to do a lot of stuff in the background. We have to do this fast, we have to do this prioritized so that we can get the results immediately. And that's what we're going to do actually. I'm going to create the handler in here and I'm going to say VNImageRequestHandler. This is the thing that I'm looking for; if I open the parentheses, this handler will ask me for a CIImage. And this is not a CIImage, I believe this is a CGImage. Yeah, this is the one that we are looking for, CIImage and it has some options parameter as well but I believe this is optional, so we can just delete this and go along with the CIImage. So, let me delete these options. And in the CIImage, I'm going to give the image as an input because remember we are getting this inside of our function here as an input so we can just give it as an input here as well. And I'm going to do what Swift told me to do, but Apple told me to do actually I'm going to use this global and in this one I'm going to go for userInteractive. And then I can say .async and it will create a coding block for me. And here I'm going to use my handler to perform the requests. So, there are two things to remember here. As you can see it asks for an array of requests and it throws an error, so we're going to do this within do, try, catch and we're going to have to create an array here and put our requests inside of that array. So, far so good, but we're getting an error in here. So, let's see it couldn't understand what the request is. I believe I have misplaced this, so let me put it inside of our model here after the creation of request so that it can see the request. Now it sees and even though we're just working with a single request, it asks us to create an array so that's what we are doing. So, let me create a do, try, catch block in here. So, do, try, catch okay. And if we get an error, I'm just going to print one, okay. I'm just going to print "error" and that's it. Actually, this will fire up the requests and in the request I'm getting this image processed in my model. I'm getting the observations out of that model and I'm displaying the first result to the user. So, let's try this. Let's open our photo library and we don't have some good images in here, but I'm just going to choose one and as you can see, we got the result back and this is actually a fountain I believe, and it understood that this is a fountain, but as you can see we have too much decimals in here. So, that's what I have been talking about. So, rather than saying 26.7975, I'm just going to say something like 27% this is a fountain. And I believe we don't have to change anything else, so this is working fine. All we have to do is just round this up. And in order to do that, I'm going to create another thing called rounded. I'm going to convert my confidence level to an integer and let me stop this simulator because it's getting a little bit lagging here. So, let's say confidence level. So, let me just multiply this by 100 and divide this by 100 so it will run this up. And of course if we don't put this in our resultLabel, it won't matter much. So, rather than using confidence level, I'm going to use rounded in here. Now, this is really the time to test this app and see if we can get the good results back from our model. So, let me change this picture. Let me choose the fountain. And as you can see it's 26% fountain. So, this is finding the image content very well, actually this is indeed a fountain. So, maybe you can do something like a resultLabel.text founding or trying to find, okay so that we won't have a blank label in the process of finding the image. So, this will be much more elegant. So, let me try this. Let me go over here to change and let me choose one of these pictures. Let me go for a fountain. It found it very fast, so I couldn't see it's finding. So, as you can see it finds this content. This is a cliff. So, this is 45% cliff. So, let me go over here to this flowers maybe or do we have anything specific. No, lets go for these flowers. It says that it's coral reef and I don't know if this is coral reef. I don't understand very much about flowers. So, in fact, let me go to Safari here. Let me close this app and try to find some images to test this model. Let me go to Google and let me just search for monkey. So, far we have been talking about this monkey image. Let me find some monkey image, and here you go. If we click over here and long press, we can actually add this to the photo library of the user. Not the user actually, in this case we are the user. So let's go for Lion this time. And this brought up the Lion King I believe, but it doesn't matter. It's still a lion, so it has to be recognized by our model, right? So, let me bring this in and let's go for a car for example. So, this is bugging me. This is asking for my location so, let me just give it anyway and open one of the car images. And let me try to get this one as well. And let's find the 4th one. Maybe we can go for something like a plane, okay, or you can just search for airplane, it will be much more explicit. Let me find one of the plane images and try to save this to my photo library. And this doesn't work for some reason, let me choose another one. Let me long press over here and this is how we add these two photos; so far so good. So, let me close this down and go to my machine learning app and let me open the monkey image. So, as you can see this, I believe this is a macaque rather than a monkey knows it better than us. So, it's working. So, let me choose the lion and here you go. So, this is the lion and it founded as well. Let me search for a car. Here you go, it's not even a car, it's a race car as you can see, it's pretty good I believe. So, let me go for the final one and yeah 94% confidence level, this is an airliner. So, this is working very good and the reason this is working very good because the model is good because the model is trained very well. And we can integrate these models into our app, thanks to Core ML framework, thanks to Vision framework. Very easily as you can see, we have just written this much of code; we have just written like something like maybe 20 line, or 50 lines of code and we can actually understand the content of an image. This was very hard to do back in old days. So, this is now the truth, this is now the reality. Thanks to the Core ML model. So, I hope you enjoyed this section. We're going to stop here and continue within our next section.
Atil is an instructor at Bogazici University, where he graduated back in 2010. He is also co-founder of Academy Club, which provides training, and Pera Games, which operates in the mobile gaming industry.