Multiple Linear Regression: Background
00:36 In this case, the model takes the following form. We’ve got the intercept, b₀, and then we’re multiplying each component of the input vector by an unknown coefficient. So in this case, the number of unknowns or coefficients is going to be r plus 1 plus that intercept. As in simple linear regression, in order to build this model, we’re given n observations.
01:14 And just to make sure you’re remembering that each of these input observations, they are r-dimensional arrays or r-dimensional vectors. And so, for example, if we’re looking at the ith input observation, that ith input observation has r components.
01:32 And so we’re going to need two indices. The first index corresponds to i, so i would be referring to the ith observation. And then the second index tells us the component of that ith observation.
01:54 The problem is the same as in simple linear regression. We want to find that model, and to find that model, we need to find the coefficients. And one way to do that is to minimize the residual sum of squares function.
02:31 In that case, there were only two coefficients. I want to do the same thing in the case of multiple linear regression, but to do so, I have to introduce a little bit of matrix notation. Now, if you’ve never taken a course, say, on linear algebra, or maybe it’s been a long time since you’ve worked with matrices, then just come along for the ride, take a look at the nice formulas that we’ll display and just let scikit-learn do all the computations for you. All right.
02:59 So this is what we’ll do. The coefficients we’re going to stack them up in a vector. There are r plus 1 coefficients that are unknown. And so we’re going to denote that vector by the bold letter b.
03:48 We opened up a can of worms. So these input vectors, they are really r-dimensional. And so we stack those inputs as rows in a matrix, but then we also add this column of ones, just so that the mathematics is easy to do, and we get nice formulas at the end.
04:15 So using some calculus and a little bit of matrix notation, there’s actually a nice closed form expression for those coefficients. It involves some transposes and some inverses, but this is sort of a theoretical formula.
Become a Member to join the conversation.