For more information, please see full course syllabus of Statistics

For more information, please see full course syllabus of Statistics

### Least Squares Regression

Lecture Slides are screen-captured images of important points in the lecture. Students can download and print out these lecture slide images to do practice problems as well as take notes while watching the lecture.

- Intro
- Roadmap
- Best Fit
- Sum of Squared Errors (SSE)
- Why Squared?
- Quantitative Properties of Regression Line
- So How do we Find Such a Line?
- How Do We Find Slope (b1)
- Hoe Do We Find Intercept
- Example 1: Which of These Equations Fit the Above Data Best?
- Example 2: Find the Regression Line for These Data Points and Interpret It
- Example 3: Summarize the Scatterplot and Find the Regression Line.
- Example 4: Examine the Mean of Residuals

- Intro 0:00
- Roadmap 0:13
- Roadmap
- Best Fit 0:47
- Best Fit
- Sum of Squared Errors (SSE) 1:50
- Sum of Squared Errors (SSE)
- Why Squared? 3:38
- Why Squared?
- Quantitative Properties of Regression Line 4:51
- Quantitative Properties of Regression Line
- So How do we Find Such a Line? 6:49
- SSEs of Different Line Equations & Lowest SSE
- Carl Gauss' Method
- How Do We Find Slope (b1) 11:00
- How Do We Find Slope (b1)
- Hoe Do We Find Intercept 15:11
- Hoe Do We Find Intercept
- Example 1: Which of These Equations Fit the Above Data Best? 17:18
- Example 2: Find the Regression Line for These Data Points and Interpret It 26:31
- Example 3: Summarize the Scatterplot and Find the Regression Line. 34:31
- Example 4: Examine the Mean of Residuals 43:52

### General Statistics Online Course

### Transcription: Least Squares Regression

*Hi and welcome to www.educator.com.*0000

*In the previous lesson we learned about conceptually the idea of regression.*0003

*In this lesson of squares regression we are going to talk about how to actually calculate a regression line and find it.*0007

*Here is the roadmap and we are going to talk about what it means to best fit the data, and what does it mean for a line to best fit the data.*0016

*We are going to talk about sum of squared errors and y that conflict is important for regression.*0025

*We are going to talk about sum quantitative properties of the regression line. *0030

*We know conceptually what it means but once we do have a regression line there are sum rules that the regression line conforms to.*0034

*We are going to talk about how to actually find the slope and the intercept of the regression line, *0042

*What does it mean to best fit the data?*0050

*Well you can think about it like this, there are any number of lines that you could drop through a set of data.*0053

*We could draw that one, we could draw this one, we can draw this one, we could draw that one.*0060

*There are an infinite number of possible ones, but our goal is a regression line that is in the middle of all of these data points.*0067

*When it is in the middle that is what we mean by best fitting line.*0077

*You can think of this fit as roughly being equal to the concept of in the middle and the difference between all of these lines *0082

*and the true regression line is that the best fitting line is roughly in the middle. *0106

*How do we find the best fitting line?*0114

*Quantitatively what it means to best fit the line means that this line had the lowest sum of squared errors.*0117

*Because of that the regression line is also called the least squares line.*0128

*That is y it is called the least squares method. *0135

*Even though in the middle and best fit are good conceptual ideas they are not quantitative ideas.*0138

*This is the quantitative definition of what the best fitting line is.*0146

*Let us talk about what error is.*0151

*We had a particular word for error and that word is the residual.*0154

*And that residual is the difference between y and the predicted y from our best fitting line.*0160

*Having the lowest sum of squared errors is having the lowest SSE is really having a sum of all the squared residuals.*0170

*Residuals square and another way to write that in y – y hat ^{2}.*0185

*This is our quantitative measure of how good our line is. *0199

*Now there is one of a Catch-22 here. *0206

*We have to have the line before we could figure out whether it has the lowest SSE but the question is how do we find that line?*0208

*First, before we go on let us talk about y we need to square these residuals?*0220

*Remember when we talked about what it means to be in the middle?*0226

*It means that the distances on the positive side or the point above the line and the points below the line, *0228

*the negative distances and these should all balance out.*0235

*If you have a bunch of positive and a bunch of negative and you add them together you should get 0.*0239

*Here is the tricky part that sum of the residuals, y – y hat.*0246

*The sum of the residual period that should be roughly equal to 0 for the best fitting line.*0255

*Because of that we want to square these distances.*0263

*I will write the squared in red because that means that this value, the sum of squared errors should be greater than 0.*0272

*We definitely want to square it.*0283

*These other mathematical properties that will be able to take these seventh feature.*0285

*We know what it means to quantitatively be the regression line. *0295

*It means having the lowest sum of squared errors but there is other quantitative properties that come along. *0299

*One important property to note is that this line, the regression line also contains point of averages.*0304

*The average of all your x and the average of all your y.*0313

*The average of variable 1 and the average of variable 2.*0317

*This point is often also called the center of mass.*0321

*It is really easy to find this point you just take the average of your x and take the average of your y.*0325

*x bar and y bar is your point of average. *0333

*You can also think of it as the center of mass because if we think of all your points, the scatter plot as long like a object, this is the center of that mass.*0339

*We already know that this line has the lowest SSE of any other line that also contains point of averages.*0352

*The sum of the residuals when you do not square it that should be approximately 0.*0362

*And because the sum of the residual is 0, the mean of the residuals is also 0 because the mean is the sum divided by the number of points.*0371

*If the sum is 0 it means it will be 0 and the variation of the residual is as small as possible.*0379

*It is smaller than other lines.*0389

*One way to quantify variation is something like standard deviation.*0391

*The residual have the smallest standard deviation than any other line. *0396

*Those are very important quantitative properties that we need to know.*0404

*This sounds like a wonderful, magical line.*0411

*How do we find such a line?*0413

*You might be thinking that this is sounding pretty hard and maybe we have to find the SSE *0416

*for a whole bunch of different line equations and find the one with the lowest SSE.*0424

*That is actually problematic. *0431

*It is a good idea.*0433

*It is a good conceptual idea but it is problematic, and here is y.*0435

*There are an infinite number of lines.*0439

*You can just change the y intercept by .0001 and get a totally different line.*0447

*You can change the slope but a tiny, tiny amount and get a totally different line.*0454

*There is an infinite number of lines that we would have to test.*0459

*Infinite number of potential lines.*0465

*We can find the SSE of infinite number of line.*0473

*That is just not an option for us.*0479

*Thank you to our hero Carl Gauss he was a mathematician and all kinds of German guy, and he helped us out a lot in statistics.*0483

*Carl Gauss invented this method called the method of least squares and through Carl method we could easily find the slope.*0498

*Here is how we do it.*0508

*The slope is going to be a ratio.*0510

*Slopes are always ratios.*0512

*Rise/run but ratio change of y over change of x.*0514

*Through this methodology has a similar line to it.*0520

*Here is how Carl Gauss find slope. *0524

*Remember slope is not z sub 0, that is intercept.*0527

*It is z sub n.*0532

*B sub 1.*0535

*We call Gauss’s method you want to add up, take the sum of all your x deviation, so the deviation between x and the mean.*0536

*X – x bar and multiply that to all deviations of y.*0552

*Notice that we are not using x hat or y hat because we do not have the line but we do have the center of mass.*0566

*We are using that.*0576

*We are finding the deviations from sort of the center of mass right.*0577

*And that sum over the sum of x - x bar ^{2}.*0582

*It is sort of think about this, as this as the 2 variation over the x variation squared.*0595

*When you think of rise/run you think of the y/x and you will see that here.*0604

*There is the change of y and there is the changes of x.*0612

*There is two changes of x and here is the change of y.*0617

*This method will give you the slope of your regression line.*0623

*And just as a review, remember that when we have this x here, we really mean x sub I and we mean x sub i.*0632

*I goes from 1 all the way up to n.*0642

*However many data points we have in our sample.*0645

*This often goes without saying that this we want to do this for every single data point that you have.*0648

*That is Carl Gauss’s method.*0659

*In order to find slope we need to use that function.*0663

*It is the change of x, the deviations of x multiplied by the deviations of y all added up over the deviations of x ^{2}.*0669

*The sum of the deviations of x ^{2}.*0683

*Let us actually do a little example here. *0685

*If we had a whole bunch of x and a whole bunch of y I just put a few here.*0693

*X = 1, 1.*0698

*The first point is 1, 1 and the second point is 0 and third point is -1, -1.*0702

*A very easy line.*0709

*We already know that the line equations should be something like Y = x.*0711

*Let us see if we could use Carl Gauss’s method in order to find slope and often find it useful and that is where we are going.*0721

*The deviations of x and the deviations of y so the sum of the deviations of x times the deviations of y and the ratio of that sum to the deviations of x ^{2}.*0733

*In this way we have to find X bar and y bar and easily we can tell here if we take the X bar and we just add it up to 0, so the average of 0 adding this up 0.*0752

*We already know X bar and y bar.*0777

*In order to do this I’m going to have to find x - x bar.*0783

*I’m going to have draw this in a different color to make it easier.*0792

*X - x bar and y - y bar. *0797

*Not only that I’m going to need to know x.*0805

*I need to know X - X bar × Y - y bar and I'm going to need to know x – x bar ^{2}.*0812

*Let me draw some lines here.*0828

*Let us get started, because my x bar and y bar is 0, 0 this makes this easy for me.*0838

*Let us find this difference for x deviation × y deviation.*0849

*I will just multiply it across that is y across 0 and that is y across 1.*0857

*X – x bar ^{2}.*0867

*This is y ^{2}.*0870

*That is 1, 0, and 1.*0870

*I need to find the sum here and take that and put it over this one.*0873

*This sum is 2 and this sum is 2 and I’m going to put that in here my b1 = 2/2 which is 1.*0884

*We found our slope.*0895

*Our slope is just 1.*0897

*Since I already knew that slope of a regression line here should be y = x we know that y = 1 × x which is y = x.*0899

*Now we know how to find slope but how do we find intercept once we have our slope.*0913

*Let us see our previous example b1 = 1.*0923

*We know 1 is a point that falls under our regression line already.*0931

*X bar / y bar which is 0, 0.*0937

*If we know all of those things we could find our intercept just by plugging it in.*0945

*Our equations have a line in statistics is y = b knot + b sub 1 × x.*0955

*All you have to do is plug in our numbers and substitute in order to find the sum.*0966

*That is what we are looking for.*0973

*Here is an example y.*0976

*B sub knot, b sub 0 + 1 × 0.*0979

*Here I will get b = 0.*0986

*This is definitely easy.*0991

*This is just finding our missing value just by having our example y, x, and having this slope.*0995

*We could just derive this linear so that in the future we will know what exactly to plug in.*1006

*Instead of trying to solve for y we could just slip around these things in order to solve for b sub 0.*1010

*All we have to do is move this over to that side so that is y – b sub 1 × x.*1022

*That is how to find b sub 0 / y intercept.*1031

*Let us do some more examples.*1039

*Here is example 1.*1042

*Pretend that this is 3 different kind of pizzas.*1044

*Let us say this is medium size pizza.*1049

*Let us say that this is giant size pizza. *1054

*It has 100 grams of fat per pizza but the cost is $17.50.*1059

*The double size let us say is 110 grams of fat per pizza but the cost is $18.00.*1068

*Pizza x has 120 grams of fat but the cost is $20.00.*1074

*Maybe we would have a feeling that fat makes the taste better or the cost.*1082

*The question is which of these following equations fit this data the best.*1092

*In order to solve this problem we have to find the sum of squared errors for each of these equations.*1102

*We are not sure if any of these equations is the regression line.*1109

*We are just trying to find the best equation out of the 3 that we have.*1114

*Which of these set above data the best which equation has lowest error or sum of square error?*1120

*You can put up your examples right for you in the x so far and click on example 1 are already in the data right here.*1138

*Here is our 3 pizzas of fat as well as the cost.*1149

*It seems just from borrowing it that bar is a positive trend.*1154

*As fat goes up the cost goes up.*1159

*Let us go ahead and try our first equation that we are given.*1165

*The equation sub y = 4.45 that is the intercept + .1x.*1183

*I separated that in order to the intercept as well as the slope because we are going to need those numbers.*1192

*Here is the fat, here is the cost.*1200

*Let us find the predicted cost or y hat.*1202

*In order to find y hat all we have to do is plug in our x into our y equation.*1206

*That would be these values .75 and that will change.*1216

*I’m going to lock it in place and add that to b1 × x.*1223

*B1 is not going to change either so I’m going to lock that in place.*1236

*We do want b12 to keep changing.*1250

*I’m going to take that predicted cost and I’m just copying and pasting.*1254

*These predicted cost are always a little bit less than the actual cost.*1264

*Here all I have are residuals are going to be.*1273

*The residuals are the actual cost – the predicted cost.*1281

*All of our residuals are going to be positive.*1286

*That is the case where all of our actual data are above our prediction line and so because of that we know that this is not quite as good is not a great regression line.*1292

*Maybe it has the best smallest SSC.*1306

*We have our residuals and what I’m going to do is take this residual and square it.*1310

*You can find all my squared residuals and then in order to get the sum of squared residuals I will just add them all up and so I get 23.1875 as my sum of squared errors.*1318

*Who knows, maybe that is the lowest one, we will see.*1331

*Here I put in the data for the next equation.*1343

*It is y equals 8 +.025x.*1351

*I separated out into the intercept versus the slope and let us find the sum of squared error.*1361

*To find the predicted cost I need to add this, take my intercept, lock in place and add that to my slope × x.*1369

*I’m going to lock my slope in place as well. *1391

*And so right now we are a little bit low, still low, still really low.*1401

*I could see that because our predicted costs are more off than our predicted cost I’m going to guess the sum of squared errors is going to be considerably larger.*1408

*Let us find the residual.*1420

*The residual is the data minus the predicted.*1424

*The data minus the predicted and then all I do is square that residual and then sum them all up *1428

*because all of our predicted costs were more off than the predicted cost. *1447

*This equation is much better than this equation. *1454

*Now let us test out the third one.*1458

*I hope we did not see those answers and let us see what the predicted costs look like.*1471

*We want to add our intercept with our slope like that in place × the x.*1477

*Excel will automatically do order of operations, so I do not have to put parentheses around the multiplication first.*1499

*Let us say that this is actually close to the costs.*1508

*If all of that is off by 20% but just below.*1511

*Let us say that is the next one.*1518

*This one is off in the opposite direction.*1520

*It is off in the negative direction. *1523

*This one is off in the negative direction.*1527

*This seems like pretty good prediction where we are getting pretty close to the cost. *1530

*Let us find out what the residual is.*1535

*Here we should have a mix of residuals.*1538

*Some positive and some are negative. *1540

*So costs - the predicted.*1543

*We have 2 positive ones and one -1 and in order to balance each other out quite nicely because the positive ones are smaller, *1547

*but the negative one is a little bit bigger.*1557

*Let us square this.*1560

*Here if we sum that up we get .375 and that is considerably smaller error than 23 and 182.*1566

*I can say that the third equation is the best fitting line. *1582

*This one is the best one.*1590

*Here is example 2.*1593

*Now it give us the same data and x find the regression line for these data point and them interpret it.*1595

*If we go back to our Excel file and click on example 2 then you will see the data here for you.*1605

*First thing we probably want to do is figure out all the different things we would like to get *1614

*and I’m just going to use a little bit of a shorthand instead of writing x – x bar.*1623

*I’m going to write deviations of x.*1633

*The deviations of x and I'm also going to need deviations of y and then I'm going to need to multiply the deviations of x × the deviations of y.*1635

*I’m also going to find deviations of x ^{2}.*1649

*These are the four things I need.*1655

*In order to get these, I need X bar.*1657

*Here I’m going to put averages and you need to find X bar and y bar.*1666

*And that is right here.*1673

*Here I’m going to put average and find my X bar which is that and also just copy and paste that over to find y bar, the average cost.*1675

*My point of averages is 110 and 18.5. *1688

*Let us find all the deviations of x in order to find slope.*1695

*The deviations of x is x- my x bar.*1701

*Here I’m going to lock my X bar in place and then I can just copy and paste all the way down. *1708

*Let us also find the deviations of y which is costs minus the average cost.*1731

*And then I could just copy and paste that all the way down as well. *1745

*Notice that my deviations of x and deviations of y they are like helping us toward that lowering *1748

*of the residual idea because the deviations of x if you look at all of them they are very balanced.*1757

*Half of them are one side of the average and half of them are the other.*1766

*The definition that is what average means and so are my deviations of y half of them are on the negative side *1772

*and half of them on the positive side and they balance one another up.*1779

*Now let us multiply the deviations of x by the deviations of y and noticed them doing this for every data point.*1784

*Here I know I need to find sum.*1793

*I will sum them here.*1799

*That is my sum.*1809

*Actually color these the different colors so that we do not get confused.*1811

*Let us also find our deviations of x ^{2} and let us find the sum of those.*1817

*Here are two sum and what we need to find in order to find the b sub 1.*1828

*Finding b sub 1 we need to find the ratio between this and that.*1839

*Our b sub 1 equals .125.*1850

*Now that we know b sub 1 we can easily find the b sub 0.*1855

*Now actually color these the different color and remember the formula for b sub 0 is just y – b sub 1 × x.*1862

*I already have an X and Y, my point of averages.*1875

*I forgot to put equal sign.*1882

*y – b sub 1 × X and I get 4.75.*1884

*In order to find my equation for the line all we do is take the two values and put them into my actual line equation. *1901

*In order to find my predicted y I would take 4.75 and add that to .125 × x.*1911

*That is my regression line for this set of data.*1924

*The previous example of this would actually choice c.*1931

*It actually happened to be the regression line as well.*1934

*Here is the kicker though we need to interpret this.*1940

*It is not good enough for us to just have this, we need to know what this means.*1945

*In order to get y, we are changing everything from that into costs.*1952

*You can think of the Y intercept as a base cost.*1960

*4.75 seems to be the base cost for these pizza and then for every gram of fat you add 12 ½ cents.*1965

*If you have 1 g of fat presumably, then you would just add 12 ½ cents to this pizza and perhaps that pizza would taste very good.*1977

*It would be probably a lot healthier for you.*1987

*If you add 100 grams of fat so hundred grams of fat and each of those grams of fat is worth .12 then you have to multiply that in order to add that to your base cost.*1991

*In some ways these base cost and there is sort of acting like giving you an idea of how much every gram of fat cost.*2011

*Because notice that as grams of fat goes up, the cost goes up.*2035

*This data is actually wrong.*2040

*This would be very cheap pizza.*2047

*This equation is actually helping us to get an idea of how much each gram of fat is costing and exactly what the relationship is between grams of fat and the cost.*2058

*That is the goal of the regression line.*2070

*For these 40 data points summarize the scatter plot then find the regression line.*2072

*Presumably these data points are in the Excel file and remember how to summarize the scatter plot we are going to be doing that.*2078

*We have to bring them up that Excel file and click on example 3 that have at the bottom.*2087

*This data looks sort of familiar to us, but now they are giving us a different label for the these variables.*2096

*Here it says student faculty ratio on the x-axis and cost per unit on the y axis.*2107

*I'm presuming that each of these cases are something likes schools, maybe universities.*2115

*When the student faculty ratio is very high, then it is cheap to enroll at the schools.*2125

*It is cheap to take units there. *2131

*But when the student faculty ratio is very low then it is more expensive.*2133

*This sort what it looks like.*2137

*Number 1.*2141

*What are our cases?*2142

*Our cases particular, probably something likes schools or universities.*2143

*Our variables are the student faculty ratio and cost per unit.*2147

*Number two in summarizing the scatter plot it seems as the general shape is linear roughly so we can just stick with that.*2152

*Number 3 the trend seems to be a negative trend where as one goes up, as ratio goes up the cost goes down.*2168

*As ratio was down, the cost goes up.*2178

*Number 4, what does this sort of strange look like?*2184

*A sort of like maybe small to medium.*2193

*That is harder to add up and number 5 potential explanations. *2198

*Well, it might be that in order to provide more faculty per students or a better student faculty ratio you need more faculty or you need less students.*2205

*More faculty cost for many less students it costs more for each student.*2218

*That makes sense but it could be when you have a high cost you want to keep the student faculty ratio low.*2223

*Or maybe some of the third variable like prestige that keeps this relationship going. *2232

*We summarize the scatter plot that I think now we have to find the regression line.*2241

*In order to find the regression line we do not really need this chart very much. *2246

*I’m just going to make it feel small and put it over here.*2254

*It is useful to look at later just to eyeball whether our regression line makes sense.*2258

*But let us go ahead and take our steps to find Carl Gauss’s method of finding b sub 1.*2266

*I'm going to write here X deviations, Y deviations, X deviations × Y deviations and then X deviations ^{2}.*2276

*And this is when Excel comes in real handy because it would be really sort of crazy in order to do all of these.*2300

*Just make life easier for, let us go ahead and find X bar and y bar.*2309

*It does not matter where you find this.*2321

*It is somewhere easy for you to keep track of.*2324

*I’m going to find the average of my x and just use my student faculty ratio as my x.*2327

*The average student faculty ratio is about 20 students per faculty and just a copy that over our average cost is about $366 per unit. *2336

*Let us find the X deviations, so that would be my x - x bar and I want that to just locked in place and then I’m also going to find my Y deviations.*2353

*Y - Y bar lock that in place and multiply my x deviation and y deviations.*2382

*I’m also going to find c deviations ^{2}.*2402

*Once I have this I can actually just copy and paste all four of these values all the way down for all 40 data points. *2406

*If you take a look half of the X deviations should be negative and approximately half are positive.*2418

*And same with the Y deviations some are positive and then some are negative to balance that out.*2426

*We know we need to find the sum.*2437

*We need to find the sum of our x deviations × y deviations and just to help us out I’m going to pull down this little bar here.*2442

*You see in this corner there is a little sandwich looking thing I pulled it down in order to lock that row in place and so that row does not move.*2453

*Move that down and I know what column I am in.*2464

*I want to sum of all of these together and then I'm also going to sum all of these together and I'm just going to color all of this in a different color so we know.*2468

*Let us find b sub 1.*2497

*B sub 1 is the ratio of this sum over this sum.*2501

*Our slope is a negative slope and that makes sense because we had a negative trend and that -21.51.*2510

*Given that let us find b sub 0.*2522

*We know in order to find b sub 0 we need to use x bar and y bar as our example point.*2527

*I’m going to take y, my y – b sub 1 × x.*2534

*Again Y intercept is 795.21.*2555

*I’m just going to pull this over hold us over on this side and here I can now talk about the regression line.*2566

*The regression line would be Y equals and we put the intercept first 795.21.*2578

*Instead of plus, we could just put a minus because our slope is -21.51 × x.*2589

*This is our regression line and if you want to interpreted the idea is that sort of the base cost is around $800 *2601

*and for whatever the student faculty ratio is with each increment you get to the detection of about 20 to 21.50.*2613

* As the ratio goes up and up and up you get a little deduction every time.*2625

*Here is example 4.*2634

*Remember that the regression line must past through the point of averages.*2636

*That is one of the quantitative features of regression lines and the residual should be equal to 0 approximately.*2642

*One of these actually causes the other.*2653

*It is either that the passing through the point averages automatically makes the mean of the residual 0 *2658

*or that the mean of this residuals been 0 causes the point averages to be positive.*2664

*This problem is going supposed to be basically to explore which one causes the other.*2672

*Examine the mean of residuals for the regression line, which definitely passes through the point of average.*2678

*An example line that did not pass through the point of average and we should try to see in that case is the mean of residuals still 0.*2685

*Or an example line that does pass through the point of averages, but had the wrong slope.*2697

*For any slope of the line that passes through the point of averages that is not the regression line.*2704

*And then finally we want to discuss the question is that going to find the regression line as the line that makes the sum or mean of the residual 0.*2711

*Let us see. *2721

*If you click on example 4, I put back the pizza example that we covered at the very beginning.*2723

*Here I put in our regression line which have $4.75 as the base rate and 12 ½ cent increase for every gram of fat.*2733

*I already calculated for you the predicted costs, the residuals, and the squared residuals because we actually already did this in the first problem.*2748

*The only that I have changed is i also provided for you the sum of the residual.*2761

*Here we find that the sum of the residuals is 0.*2767

*Let us think about this regression line. *2771

*It definitely passes through the point of averages and the sum of residuals of 0.*2773

*This regression line definitely fits our quantitative definition for regression line and it has a very low sum of squared residual.*2778

*Now given this point let us think about a line that does not pass through the point of averages. *2792

*Now, if we take our line or lines, and which is slightly up or down in either direction it won't pass through the point of averages because of parallel lines never intercept.*2800

*We can keep the same slope .125, but we just change our b sub 0 very slightly.*2817

*We could just change the intercept very slightly.*2829

*Maybe would not move a line just a little bit so when we get 4.8 instead of 4.75 and here our y is y = 4.8 +.125 × x.*2831

*Let us find the squared residuals and all that stuff.*2853

*The predicted costs would be b sub 0 + B sub 1 and lock that in place × x.*2856

*Noticed that are our predicted costs are very, very close because our line is not that far off.*2880

*Let us calculate the residual.*2889

*The actual cost minus the predicted costs and let us also calculate the squared residuals. *2891

*Just squaring each of my residual and are being added up down here. *2904

*Notice that although these sum of squared errors are very close to slightly this one is just bigger than this one.*2909

*It is slightly worse fit than this one.*2917

*This one is a better fit but let us check and see whether our residual at up to 0.*2920

*It does not.*2927

*It has been close to 0, but it does not quite add up to 0.*2928

*These lines that do not quite pass through the point of averages, even though they are only a little bit off these do not add that the sum of the residuals do not add up to 0.*2934

*Now that we have all this we can actually just change it. *2948

*Let us move the regression line down just a little bit.*2952

*Let us just move it down slightly and make this 4.5 instead of 4.75.*2956

*What if we do that?*2963

*Well again it is not that far off. *2965

*It is still pretty low sum of squared error, but the regression line is still the lowest and the residuals still does not add up to 0.*2968

*If it does not pass through the point of averages then it is off by a little bit. *2982

*The other thing we could do is we could keep the intercept the same and instead we could change the slope by a little bit.*2988

*If we do that, then we know it does not pass through the point of averages.*2999

*When we do that what we find once again is that the sum of squared residual is more off than our regression line.*3003

*Our sum of residuals still does not add up to 0. *3015

*Although we try a couple of lines if it does not pass through the point of averages, we see that the residual does not add up to 0. *3018

*Now let us talk about the flip side.*3029

*A line that does pass through the point of averages, but it is still not the regression line. *3032

*Well, in order to find one that passes through the point of averages, but had the wrong slope.*3038

*It is nice to figure out from our actual point, a line that passes through there but had just a different slope.*3047

*You can pick any slope you want.*3058

*I will pick the slope of 5.*3059

*Y is 5/1.*3061

*Let us find b sub 0.*3066

*We could just use that same formula we have use and plug in our values for the point of averages.*3069

*That would be y – x × b1.*3080

*Our B1 is right next to it.*3090

*This is the point that definitely passes through the point of averages, but obviously has the wrong slope.*3093

*Let us find the predicted costs. *3101

*I remember this is the line that it is totally made up.*3104

*Predicted costs might be very off.*3106

*Predicted costs would be the intercept and lock that in place + b1 × x and then lock b1 in place.*3109

*We see that the costs are fairly off.*3136

*$-31 this is pretty close to 18 but this was pretty far off 68.5.*3141

*Now let us find the residual.*3148

*The actual cost minus the predicted cost.*3150

*and finally, let us find the squared residuals.*3156

*Notice that the sum of squared residuals is very very off 4,753.*3163

*It is pretty off.*3172

*We know that this is not a great line. *3173

*It is not a well fitting line. *3175

*These other lines actually fits better, but let us check that sum of the residual. *3176

*What does that add up to be?*3181

*That has a sum of the residual is 0.*3185

*Just because this line passes through the point of average.*3195

*Remember in order to calculate residual always using is x bar and y bar.*3201

*It actually makes sense that as long as it passes through that point of averages the sum of residual is going to be 0.*3209

*Now that we have all of this setup with all our nice formulas we can actually put in any slope.*3220

*Let us put it -.1. *3228

*It will find the B sub 1 and this line perfectly passes through the point of averages.*3231

*Even though our sum of squared residuals have improved, our residual still add up to 0.*3238

*0 even though it is not the line of regression and let us try another one -.00035.*3246

*Excel will do this just because it too many small points for it to show you, but still, you get the idea.*3259

*Although it looks sort of crazy number this means that you need to move the decimal point to the left 18 times.*3266

*That is very, very close to 0.*3275

*Let us try another one 500.*3279

*Once again we see that the sum of the residual is 0.*3284

*These are obviously not very good lines they are not very good regression lines because the squared residuals are terribly, terribly off.*3290

*The sum of the residual is 0 as long as the line passes through the point of averages. *3301

*Let us go back to example 4.*3310

*Here we have seen the mean of the residuals or the sum of the residual similar idea for the regression line, and so the mean of residual equals 0. *3314

*An example line that does not pass through the point of averages, mean of residual is not equal 0. *3328

*An example line that does pass through the point of averages that has the wrong slope.*3338

*Here we find the mean of residuals once again equal 0.*3345

*Is it good enough to define the regression line as the line that makes the sum or mean of the residual 0?*3353

*No, that is not good enough because any line that passes through the point of averages will have the sum or mean of the residual as 0.*3359

*This one really causes that one.*3370

*We also need to have all those other rules.*3374

*For instant the other rules being the sum of squared errors is the lowest in a regression line that definitely has to be there.*3379

*That is it for calculating regressions using the least squares method.*3389

*See you next time on www.educator.com.*3393

1 answer

Last reply by: Professor Son

Wed Nov 5, 2014 12:47 PM

Post by IBRAHIM FORNA on November 5, 2014

HOW DO YOU HAVE THE 14.75

0 answers

Post by Professor Son on October 10, 2014

Sorry everyone but the table in example #2 should read 17.50, 18, and 20 dollars. It's correct in the excel file but not on the slide!

1 answer

Last reply by: Professor Son

Fri Oct 10, 2014 1:39 PM

Post by Kambiz Khosrowshahi on April 1, 2013

To find the intercept, how did you come up with y=0?