For more information, please see full course syllabus of Linear Algebra

For more information, please see full course syllabus of Linear Algebra

## Discussion

## Download Lecture Slides

## Table of Contents

## Transcription

## Related Books

### Orthonormal Bases in n-Space

Lecture Slides are screen-captured images of important points in the lecture. Students can download and print out these lecture slide images to do practice problems as well as take notes while watching the lecture.

- Intro 0:00
- Orthonormal Bases in n-Space 1:02
- Orthonormal Bases in n-Space: Definition
- Example 1
- Theorem 1
- Theorem 2
- Theorem 3
- Example 2
- Theorem 2
- Procedure for Constructing an O/N Basis
- Example 3

### Linear Algebra Online Course

### Transcription: Orthonormal Bases in n-Space

*Welcome back to Educator.com, welcome back to linear algebra.*0000

*In the previous lesson, we talked about transition matrices for a particular space, where we have 2 or more bases for that space.*0004

*There we said that one basis is as good as another. As it turns out that is true.*0015

*One basis is not necessarily better than another intrinsically, however as it turns out, there are certain bases... one particular basis in particular that is better computationally.*0019

*It just makes our life a lot easier. That is called an orthonormal basis.*0029

*So, that is what we are going to talk about today. We are going to introduce the concept, and then we will talk about how to take a given basis and turn it into an orthonormal basis by something called the Gram Schmidt orthonormalization process.*0034

*It can be a little bit computationally intensive, and notationally intensive, but there is nothing particularly strange about it.*0047

*It is all things you have seen. Essentially it is all just arithmetic, and a little bit of the dot product.*0055

*So, let us just jump in and see what we can do.*0059

*Okay. Let us talk about the standard basis in R2 and R3, the basis that we are accustomed to.*0063

*If I take R2, I can have a basis (1,0), that is one vector, and the other vector is (0,1)... okay?*0071

*Also known as i and j, also known as e1 and e2, again just different symbols for the same thing.*0084

*R3 is the same thing. R3... we have... (1,0,0), (0,1,0), and (0,0,1), as a basis.*0095

*Three vectors, three dimensional vector space, you also know them as i,j, k, and we have also referred to them as e1, e2, e3.*0108

*Now, what is interesting about these particular bases, notice, let us just deal with R3... vector a and vector 2 are orthogonal meaning that their dot product is 0, perpendicular.*0122

*1 and 3 are orthogonal, 2 and 3 are orthogonal. Not only that, they are not just orthogonal, mutually orthogonal, but each of these has a length of 1.*0136

*So, this is what we call orthonormal, that the vectors are mutually orthogonal, and they have a length of 1.*0146

*As it turns out, this so called natural basis works out really, really, well computationally.*0154

*We want to find a procedure... is there a way where given some random basis, or several random bases, can we choose among them and turn that basis into something that is orthonormal, where all of the vectors have a length of 1 and all of the vectors are mutually orthogonal.*0160

*As it turns out, there is. The Gram Schmidt orthonormalization process. A beautiful process, and we will go through it in just a minute.*0175

*Let us just start off with some formal definitions first.*0183

*So, we have a set s, which is the vectors u1, u2... all the way to uN is orthogonal, if any two vectors in s are orthogonal.*0188

*What that means mathematically is that u _{i} · u_{j}, so u_{1} · u_{3} = 0.*0222

*The dot product of those two vectors is equal to 0. That is the definition of orthogonality. Perpendicularity if you will.*0231

*Now, the set is orthonormal if each vector also has a length norm of 1.*0239

*Mathematically, that means u _{i} dotted with itself gives me 1.*0262

*Let us just do an example. Let us take u _{1} = the vector (1,0,2), u_{2} = the vector (-2,0,1), and u_{3} = the vector (0,1,0).*0269

*Okay. So, as it turns out, the set u1, u2, u3, well if I do the dot product of u1, u2... u1, u3... u2, u3, I get 0 for the dot product.*0293

*So, this set is orthogonal.*0311

*Now, let us calculate some norms. Well, the norm of u _{1} is equal to this squared + this squared + this squared under the radical sign.... is sqrt(5).*0316

*The norm for u2, that is our symbol for norm, is equal to this squared + this squared + this squared... also sqrt(5).*0334

*And... the norm for u3 is 1. So, this one is already a unit vector, these two are not.*0348

*So, since I have the norm, how do I create a vector that is length 1. I take the vector and I multiply it by the reciprocal of its norm, or I divide it by its norm, essentially.*0353

*So, we get the following. If I have the set u1/sqrt(5), u2/sqrt(5), and u3... this set is orthonormal.*0369

*Again, radical sign in the denominator, it is not a problem, it is perfectly good math. In fact, I think it is better math than simplification.*0388

*What they call rationalizing the denominator, or something like that... I like to see where my numbers are.*0399

*If there is a radical 5 in the denominator, I want it to be there. I do not want it to be u2sqrt(5)/5, that makes no sense to me, personally, but it is up to you though.*0405

*Okay. Quick little theorem here.*0416

*If s is an orthogonal set, or orthonormal, then s is linearly independent.*0428

*So, if I have a set that I do not know is a basis, I know this theorem says that they are linearly independent.*0452

*So, the particular space that they span, it is a basis for that space.*0460

*Okay. Let us go ahead and do this. Like we said before, using an orthonormal basis can actually improve the computation.*0466

*It just makes the computational effort a little bit less painful.*0476

*So, now if we have s, u1, u2, all the way to u _{n}...*0481

*If this set is a basis for some vector space v and u is some random vector in v, then we know that we can express u as linear combination... c1u1 + c2u2 + cNuN.*0498

*u's, v's, w's, all kinds of stuff going on. Okay, so what we did before was we just solved this linear system.*0528

*We found c1, c2, all the way to cN, you now, Gauss-Jordan elimination, reduced row echelon form.*0536

*Well, here is what is interesting. If s is orthonormal, well still if it is orthonormal, when it is orthogonal it is still a basis so you still get this property.*0545

* You know -- u is still that, but it is a really, really simple way to find the cN without finding the linear system.*0562

*What you end up with is the following. Each c _{i} is equal to the vector u dotted with u_{i}.*0570

*For example, if I wanted to find the second coefficient, I would just take the vector u and I would dot it with the second vector in the basis.*0582

*That is fantastic. It is just a simple dot product. For vectors that are -- you know -- maybe 2-space, 3-space, 4-space, maybe even 5-space, there is no solution, you do not have to worry about it.*0592

*You can just do the multiplication and the addition in your head. The dot product is really, really easy to find.*0600

*So, let us do an example of this. We will let s equal... okay... it is going to be a little strange because we are going to have some fractions here...*0608

*(2/3, -2/3, 1/3, 2/3, 1/3, -2/3, 1/3, 2/3, 2/3,), so this is our set s.*0625

*Okay, well, we want to be able to write some random vector v, which is equal to let us say (3,4,5) as a linear combo of the vectors in s.*0650

*So, we want c1, let us call it v1, the vectors in s, let us just call them v1, v2, v3... + c2v2 + c3v3.*0675

*So, we want to find c1, c2, c3. Well, as it turns out that basis, even though it looks a little strange, it actually ends up being orthonormal.*0691

*The length of each of those vectors is 1, and the mutual dot product of each of those is 0.*0700

*So, it is an orthonormal set. Therefore, c1 = v · v1.*0708

*Well, v is just 3, 4, 5, okay? I am going to dot it with v1, which was 2/3, -2/3, 1/3, and I get 1.*0718

*When I do c2, well that is just equal to v · v2. When I do that, I get 0.*0737

*If I do c3, that is just v, again, 3, 4, 5, dotted with v3, which was v3 up there in s, and this ends up being 7.*0746

*Therefore, v is equal to, well, 1 × v1 + 0 × v2 + 7v3.*0759

*There we go. I did not have to solve the linear system. I just did simple dot product. Simple multiplication and addition. Very, very nice.*0771

*One of the benefits of having an orthonormal basis. There are many benefits, trust me on this one.*0781

*Okay. So, now let us come down to the procedure of actually constructing our orthonormal basis.*0786

*So we are going to go through this very, very carefully. There is going to be a lot of notation, but again, the notation is not strange. You just have to make sure to know what is where.*0792

*The indices are going to be very, very important... so, take a look at it here, take some time to actually stare at the Gram Schmidt orthogonalization process in your textbook.*0802

*That is really the best way to sort of wrap your mind around it. Of course, doing examples, which we will do when you do problems, but just staring at something is really a fantastic way to understand what it is that is going on.*0815

*In mathematics, it is the details, the indices, the order of things. Okay.*0826

*Let us see. So, let us write it out as a theorem first.*0835

*You know, let me... let me go to a black ink... theorem...*0841

*Let w be a non-zero subspace. So again, when we can speak of a basis we can speak of a base for the entire space, it does not really matter.*0857

*In this case we are just going to express this theorem as a subspace.*0870

*Again, the whole space itself is a subspace of itself, so this theorem is perfectly valid in this case.*0875

*... be a subspace of rN with basis s = u1, u2, uN. N vectors, N space.*0883

*Then, there exists an orthonormal basis t, which is equal to... let us call it w1, w2, all the way to wN for w.*0906

*So, this theorem says that if I am given a basis for this subspace or a space itself, I can find... there exists an orthonormal basis.*0934

*Well, not only does one exist, as it turns out, this particular procedure constructs it for us. So, the proof of this theorem is the construction of that orthonormal basis itself.*0945

*That is the Gram Schmidt orthonormalization process. They call it the orthogonalization process, which is really what we are doing.*0955

*We are finding orthogonal vectors, but we know how to change vectors that are orthogonal to vectors that are orthonormal.*0961

*We just divide by their norms... nice and simple. Okay.*0967

*So, let us list the procedure so that we have some protocol to follow.*0973

*Procedure... okay... procedure for constructing an orthonormal basis t, which we said is going to be w1, w2... all the way to wN.*0979

*Constructing an orthonormal basis t... from basis s = u1, u2, all the way to uN.*1014

*I am going to change something here. I am going to not use w, I think I am going to use v instead.*1038

*I used u for s, so I am going to go back and choose... let me call them v so that we stay reasonably consistent... to vN.*1047

*Okay. So, we are given a basis s, we want to construct an orthonormal basis t from s, here is how we do it.*1058

*First things first. We let v1, the first vector in our basis t, our orthonormal, we let it equal u1.*1068

*We just take it straight from there. The first vector is the first vector.*1076

*Okay. Two. This is where the notation gets kind of intensive.*1082

*v _{i} = u_{i} - u_{i} · 1_{1}/v_{1} · v_{1} × the vector v_{1} - u_{i} · v_{2}/v_{2} · v_{2} × the vector v_{2}... and so on.*1086

*Until we get to u _{i} · v_{i-1}/v_{i-1} · v_{i-1} × v_{i-1}. Okay.*1134

*Do not worry about this, it will... when we do the example, this will make sense. Again, this is just mathematical formulism to make sure that everything is complete*1158

*When we do the example, you will see what all of these i's and i-1's and v's mean.*1165

*Three. t*, when we have collected our v _{1}, v_{2}, that we go from the first two steps... is orthogonal.*1173

*We have created an orthogonal set.*1194

*Now, we want to take -- that is fine, we will go ahead and we will take for every v _{i}, we are going to divide it by its norm.*1199

*So, for each of these v _{1}, v_{1}, in this set which is orthogonal, we are going to divide each of these vectors by the norm of that vector.*1231

*Then, of course, what you get is the final set t, which is v _{1}/norm(v_{1}), v_{2}/norm(v_{2}), and so on and so forth, all the way to v_{n}, not v_{i}.../norm(v_{n})/*1242

*This set is orthonormal. Let us just do an example and it will all make sense.*1287

*So, let us start here. Let us do our example in blue.*1299

*We will let s = u1, u2, u3 = 1, 1, 1, -1, 0, -1, -1, 2, 3.*1308

*This is our set. We want to transform s into an orthonormal basis, for R3. This is a basis for R3.*1342

*These are linearly independent. They span R3. Three vectors. We want to change this into an orthonormal basis.*1353

*We want each of the vectors in our basis to be orthogonal, mutually orthogonal, and we want them to have a length of 1. So, we will run through our procedure.*1361

*Okay. First thing we do, let v1 = u1. So the first thing I am going to do is I am going to choose by vector (1,1,1).*1371

*That is my first vector in my orthogonal set. Nice, we got that out of the way. Okay.*1380

*Two. Go back to the previous slide and check to see that number two thing with all of the indices going on. Here is what it looks like based on this number of vectors.*1387

*v _{2} = u_{2} - u_{2} · v_{1}, which is this thing /v_{1} · v_{1} × vector v_{1}.*1400

*That is equal to... well, u _{2} is (-1, 0, -1).*1430

*Now, when I take u _{2}, which is (-1, 0, -1), and I dot it with v_{1}, which is (1, 1, 1), I get -2.*1440

*When I take v _{1} dotted with v_{1}, I get three. So it is -2/3 × v_{1}, which is the (1,1,1).*1453

*When I do that, I get -1/3, 2/3, -1/3... okay.*1471

*The next one v3, I have v _{1}, I have v_{2}, which is here. I need v_{3}, right? because I need 3 vectors.*1489

*So, v _{3} = well, it is equal to -- go back to that number 2 in our procedure -- it is equal to u_{3} - u_{3} · v_{1}/v_{1} · v_{1} × v_{1} - u_{3} · v_{2}/v_{2} · v_{2}, all × v_{2}.*1498

*Well if you remember that last entry in that number 2, it said v ^{i-1}.*1543

*Well, since we are calculating v _{3}, 3 - 1 is 2, so that is it. We stop here. We do not have to go anymore.*1547

*That is what that symbolism meant, it tells us how many of these we get.*1557

*If we are calculating v _{4}, well, 4 - 1 is... so that is v_{i}, i is 4. It is 4 - 1, that means we go all the way up to this last entry, which is 3. So we would have three of these.*1563

*That is all this means. That is all that symbolism meant. Just follow the symbolism, and everything will be just fine.*1578

*okay. This actually ends up equaling... well u _{3} is (-1,2,3).*1584

*When I do u _{3} · v_{1}, which is (-1,2,3) · (1,1,1) over (1,1,1) · (1,1,1), I am going to end up with 4/3 × (1,1,1) - ... when I do u_{3} · v_{2}, which is u_{3} · v_{2}/v_{2} · v_{2}.*1597

*I am going to get -2/6 × v _{2} - (1,2,1)... okay?*1627

*I am going to end up with (-2,0,2). So, let me go to red.*1644

* (1,1,1), that... and that... so let us write our what we have got.*1657

*Our t*, our orthogonal set is this sub... interesting... 1, 1, 1, -1/3, 2/3, -1/3, and -2, 0, 2.*1678

*This set is orthogonal. Now, let us take a look at v _{2} real quickly here.*1706

*v _{2} = (-1/3, 2/3, -1/3), well, let me pull out the third... that is equal to 1/3 × (-1, 2, -1).*1721

*These vectors, if I just take the vector (-2, 2, 1), and if I take the vector 1/3 × that, which is this vector, they are vectors going in the same direction.*1741

*They are just different lengths of each other. So, because they are vectors going in the same direction, I do now need the fractional version of it. I can just drop the denominator from that, because again, we are going to be normalizing this.*1754

*We are going to be reducing it to a length of 1, so it does not matter whether I take this vector or this vector.*1766

*They are just different vectors in the same direction. Does that make sense?*1772

*So, I can rewrite my t*, my orthogonal set, as (1,1,1), (-1,2,-1), and (-2,0,2).*1776

*They are still orthogonal. This, the dot product of this and this is going to be 0. They dot product of this and this is going to be 0, it does not matter. They are in the same direction.*1795

*They are just different lengths, we are going to be normalizing it anyway. So, we want to make life easier for ourselves, so that is our orthogonal set.*1806

*Now, what we want to do is calculate the norms.*1820

*So, the norm of v _{1} = sqrt(3). The norm of v_{2} = sqrt(6), and the norm of v_{3}, right here, is equal to sqrt(8).*1827

*Therefore, our final orthonormal... we have 1/sqrt(3), 1/sqrt(3), 1/sqrt(3)... -1/sqrt(6), 2/sqrt(6), -1/sqrt(6) and... -2/sqrt(8), 0, 2/sqrt(8)...*1855

*Again, I may have missed some minus signs or plus signs... it is procedure that is important.*1897

*This is orthonormal. In this case, the length matters. In the previous, they were orthogonal and so we dropped the denominator from the vectors that we found because again, they are vectors in the same direction.*1907

*The same direction, they are still going to be orthogonal. So, I can make it easy for myself by not having to deal with fractions. But, in this case, we are normalizing it. We are taking that orthogonal set and we are dividing each of those vectors by its own norm to create vectors.*1925

*Each of these has a length of 1. Any two of these are mutually orthogonal. Their dot product equals 0. This is an orthonormal basis for our three.*1940

*It is just as good of a basis as our standard basis, the i, j, k. Computations are easy, and who knows, there might be some problem where this basis is actually the best basis to work with.*1953

*Again, it just depends on frames of reference.*1964

*Okay. Thank you for joining us at Educator.com.*1968

2 answers

Last reply by: Elias Assaf

Thu Nov 14, 2013 3:00 AM

Post by Elias Assaf on November 13, 2013

Hi, you're videos are absolutely top, top, top class.

Just a little thing though, in you're first example, if you take the dot product of U1 and U2 you get

(-2,0,2) which is not orthogonal. Or have I missunderstood anything?

Thanks.

Elias

2 answers

Last reply by: Josh Winfield

Thu Apr 3, 2014 11:15 PM

Post by Christian Fischer on November 5, 2013

Hi Raffi, Great lecture - it helped me a lot!

I might be mistaking but at 27.15 (minutes.seconds) when you write v2=(-1,2,1) Did you really mean ((-1/3),(2/3),(-1/3))?

P.S. I got all year replies on multivariable calculus. Thank you so much! I'll get back to you later with respect to that :)

Have a great day

Christian

0 answers

Post by Manfred Berger on June 20, 2013

I have actually never seen the techniques you used in example 2 before. I'm quite sure this could be modified to solve arbitrary homogeneos systems.