Logarithms and exponents
Practical Machine Learning
The course is part of this learning path
To design effective machine learning, you’ll need a firm grasp of the mathematics that support it. This course is part one of the module on maths for machine learning. It will introduce you to the mathematics of machine learning, before jumping into common functions and useful algebra, the quadratic model, and logarithms and exponents. After this, we’ll move onto linear regression, calculus, and notation, including how to provide a general analysis using notation.
Part two of this module can be found here and covers linear regression in multiple dimensions, interpreting data structures from the geometrical perspective of linear regression, vector subtraction, visualized vectors, matrices, and multidimensional linear regression.
If you have any feedback relating to this course, please contact us at email@example.com.
The last kind of model or function or the Algebra that I want to talk about is to do with logarithms and exponents or exponentials. So let's have a look at the idea behind exponentials first. So what's an exponential? Well, it has to do with multiplication and how often something is multiplied. So let's take an example. So if I have a function here, f of x, and I'm gonna say that my function here is 2 to the power of x. So the way we read this form here is saying, you know, so let's just say how do we interpret this. That's going to be 2 multiplied by 2 x times, right? So let's put some example values in here for x and see what we get. So if I put in here, zero. Let's try that. And then 1, 2, 3, and see what happens. Now, by definition, any number, any number to the power of zero is just kind of defined to be 1. Just because it makes them convenient so if you put 2 to the power of zero that's going to be 1. 2 to the power of 2 is 4. 2 squared is 4. 2 to the power of 3 is 8. 2 times 2 times 2 is 8. And then times 2 once more you get 16, and you can see that's going to go 32, 64, 128, 256, 512, 1024, right? So you can see here that this gets big quite quickly. So each of these steps are increasing in size, and they're doubling. But the rate itself is going at a pretty quick pace so this is going 32. This is going 64. That's going 122, 256 is gonna be 128. Is that how I got that right? Well, yes. What you can see here is that the rate of the thing, the pace at which it's going bigger is itself exponential. And that's one of the key features of exponentials is that the pace of their own change has this multiplicative affect. So the pace is itself getting quicker and quicker and quicker. Whereas you know, especially like ordinary sorts of functions that we're interested in. We tend to have this linear intuition for lots of stuff. Don't we? Where, sort of, the pace is always the same. Well this is where the pace can be slow at the beginning. Well it appears slow at the beginning but the pace itself is getting faster and faster and faster. What we see with exponential functions, functions that have this form, is that they tend to have this kind of explosion behavior, so they, you know, for a long time the actual output is small. So the actual output is say small. But because the pace is getting bigger underneath the surface if you'd like then within not very long, the output starts getting very large. So you get this sort of small gradual, and then it shoots up like that. So, okay good. of the function that says its, you know, this is f of x, and this is x and that's what an exponential looks like. So, okay good. Now, one really interesting fact about exponentials is that you can kind of change the base without changing the function. Let's just think about what that means there for a second. So the base is this number here. So when you put a base of 2, that gives you x can be interpreted as a rate of doubling, right? So, or something like that. Like, if x is like a day and this is the spread of the disease, then on day 1 or what, on day 0, you have one case which makes a lot of sense, on day 0 maybe you have 1 case. And then by the time you get to day 3, you have 16 cases, maybe say we make it very gradual so this can be maybe 1, 100, 200, 300, 400, 500, 600, 700, 800, 900, 1000. Let's make that 1000. So, you know, the actual movement early on from 32 to 68 to 148 Can't really see it, you know? You're getting to maybe 64 around day 4 is 32, day 5 is 64, so maybe this is sort of like day 5. By the time you get to day 10, you've gone to 1064 so there's day 5. And then there's, well, on this scale this would be day 10, so you're going all the way to, very quickly, getting this huge explosion. Now, so x here could be, you know, days and the y on the vertical function output here could be the number of people infected so that would be, you know, so if this was a model, it could say that the thing we're trying to predict is the number infected, so number of people infected. See, by example. Spread of disease. Now, what should we say about this? Well, as I said, as I said, it turns out you can change the base. So this 2, when you put in an x here you can interpret the, kind of the output of the function and it's something like rated doubling where x is how much you're doubling. But actually you can get the same shape, same behavior by having 3 to the power of something, 4 to the power of something, 5 to the power of something. Doesn't even matter what the base is. You can get the same numbers. You can get 4, 8, 16, 4, 1, 2, 3, so you can get 1 input of 1 input of 2, input of 3. You can get 4, 8, and 16, and you can get that either by doing 2 to the power of 2, 2 to the power of 3, 2 to the power of 4, or there's actually a number that you could say 3 to the power of. You get the same output. No it's not gonna be a very pleasant number. It's gonna be a bit weird, a weird number. You could say 3 to the power of some 2 point something, something, something. You know, or whatever. 1 point something. It would be probably a little bit less than 2. 1 point something, something, something. And here it would be a bit less than 3. 2 point something, something. And that would give you 8. So let's just maybe bring up a calculator and see if you can figure out what that number's going to be. So if I try 3 to the power of 2.5, I get 15.5. That's very - so 2 to the power of 2.5 so over here this would go up to 2.5. So to get 16, we would need to put about 2.5, maybe even try 2.6 to get exactly there. You're not going to be exactly precise, but 3 to the power of 2.6. 17. So it's about 2.5 over here cause these numbers are obviously much smaller than this. All right, so the idea is actually you can choose any base that you like, and all you need to do is if we were to have an x here we'd have to multiply x by something, call it a. And if we multiplied it by a, that a could sort of deal with the issue of, you know, converting it into a multiplying, a doubling rate, if you wanted to think of it that way. So you have kind of like, fixing things so that the rate is the same. Now, so what the key idea there is, the key idea there is that you can kind of you know, you can, any exponential process can be understood as a doubling or a tripling or a quadrupling, but it would just have a different number of days at which it triples, so if it doubles at, you know, every day, it's gonna triple at every just day and a little bit. If it triples every day and a little bit, it will, you know, quintuple every few days or something, right? So you can see it doubling here every day. Well, how long does it take to, you know, get 4 to 12. Well let's, if I wanted to know when does it triple. It triples somewhere between days 2 and 3. So it's not every day. It's every sort of 2 point something days. So you can phrase any exponential process in terms of a doubling, a tripling, a whatever. Now by convention, and this is a strange convention, if you don't know why, but by convention, the rate that we use as the base convention in mathematics that we, kind of standard, it's always used, it 2.7182818 duh duh duh duh. All right. I believe, if I remember the number rightly. And you think well why on earth would that be the number that you would use as your basic question. Like how many times did they multiply by 2.718 duh duh duh duh. It's a strange rate to choose as your base, as you know, the thing that you're understanding the things in terms of. And the reason for that is that when this number multiplies by itself, which is what this rate is. If I, I'm gonna call this number e. I'll call this number e, otherwise it's difficult to write it out every time. But whenever e multiplies by itself like this over time. So say we take x days of that. It turns out, that the gap - so it turns out that if, you know, if this is, you know, 1, 2, 3, 4, so it's x equals 4. If we go to 5, go to 6, you know, so times e on that. Times e, times e. So the rate's increasing. So that's 5 and this is 6. So it turns out, that when that's the base, the rate at which the output is changing is that rate. So that's quite interesting. So the rate of change of the process is the same process. So we're picking up speed at the same rate that we're rising. So the higher we go, the faster we go. And we go as fast as are we are high so on a graph like that, you see. When you're here, you're gonna increase proportional to where you are. Right? So this little increase here is small because you're in some early days. And as you get higher, as you get later on, this increase here, as if it's trying to increase it from this point to this point. So x low, x high, say. Moving this way. Then you're increase is actually proportional to where you are over here. If we use this as the rate, it turns out that when we ask what the rate or pace of this change is, it's just exactly what you're, exactly where you are. It's just exactly the same answer. So that's really helpful. That's really helpful. So this little e here. E to the x. That's called the exponential function. And that's, you know, you think of, approximately in your mind you can think of e as being about 2. Closer to 3. If you think of e as 2, then that x would be a rate of doubling. And since it is a little bit more than 2, then x is not really a rate of doubling, it's almost a rate of tripling. If you think of it. So you can give it 2 or 3, think of a 3, then x is essentially the rate at which the thing triples. But since it's a little less than that, it's actually the rate at which it, you know, it multiplies by 2.71 something. So, it's the rate at which it es. Now, so that's exponentials. Let's talk about reversing that process or trying to understand the inverse of exponential, which is the logarithm. Log, I'll put log. The logarithm, now what the logarithm tells you is how many times a number has been multiplied. So rather than what number will be if multiplied, how many times it has been multiplied. So if we think of, if I say f of x is 2 to the x. We'll go back to that example. And I put in here 10. Right, so 2 to the power of 10, which I'm pretty sure is 1024. Let's just confirm that. 2 to the power of 10. 1024. There we go. Now let's try and do the opposite thing. Suppose I wanna say, how many times has 2 been multiplied to give me 1024. So this other function, let's call this other function g, and what I want to do is, I want to put in 1024 and I want the answer 10. So rather than putting in 10 and getting 1024, I'm gonna put in 1024 and get 10. So it's the opposite direction. And the formula for this, or the notation for this, is just log to the base 2 of 10, or in general g of x can be log to the base 2 of x. And then this 2 here, so you know this little 2 here, is the question how many times has 2 been multiplied to get this number. And just like you can express a rate of increase with any base. You can say how often did it double. How often did it triple? You know, it's the same process and you can ask the same question. Likewise, for any output, for any number of people being infected, you can say well how many days have passed or how many 2 days have passed or how many 3 days have passed. How many weeks have passed? So if I say log to the base 2, that's how many times has it doubled. That's how many times has it doubled. If I say log to the base 3, that's how many times has it tripled. And so if I say log to the base e, that's how many times it's e-d. And generally the shorthand for this, though it's not a hundred percent important to understand to learn this, the shorthand for saying log to the base e is ln. So when you have e to the x, if I say, you know, log of e, e to the x, that's just going to be x, so these things cancel each other out so how many times have you multiplied, multiply it, and I think it just gives you x. Just like, you know, log to the base 2 of 2 to the x. These cancel out. This gives you x. Okay? So what does a logarithm look like? So g of x is log, could be anything. Log 2 of x. Sort of has this gradual pace like that. Gradual pace of change. Okay, but I think the key idea I want to get out of this section is not so much the logarithm as a model. It's a sort of unusual model in machine learning. What we're gonna just think about here, what I'm trying to think about here is really just first of all, the exponential function. I think it's a good idea to have a visual intuition for how this works so, you know, the idea that it's gonna explode like this and being able to draw that curve and think about that curve and have it work. And then I think for the logarithm bit here, it's just about understanding that it's the opposite process. That if you have a number of infected, you can work out how many times it's doubled by taking the logarithm of it, and if you have a number of days you can work out the number of infected by taking the exponent of it or taking it to a power. Okay? All right.
About the Author
Michael began programming as a young child, and after freelancing as a teenager, he joined and ran a web start-up during university. Around studying physics and after graduating, he worked as an IT contractor: first in telecoms in 2011 on a cloud digital transformation project; then variously as an interim CTO, Technical Project Manager, Technical Architect and Developer for agile start-ups and multinationals.
His academic work on Machine Learning and Quantum Computation furthered an interest he now pursues as QA's Principal Technologist for Machine Learning. Joining QA in 2015, he authors and teaches programmes on computer science, mathematics and artificial intelligence; and co-owns the data science curriculum at QA.