WEBVTT mathematics/linear-algebra/hovasapian
00:00:00.000 --> 00:00:04.000
Welcome back to Educator.com, welcome back to linear algebra.
00:00:04.000 --> 00:00:15.000
In the previous lesson, we talked about transition matrices for a particular space, where we have 2 or more bases for that space.
00:00:15.000 --> 00:00:19.000
There we said that one basis is as good as another. As it turns out that is true.
00:00:19.000 --> 00:00:29.000
One basis is not necessarily better than another intrinsically, however as it turns out, there are certain bases... one particular basis in particular that is better computationally.
00:00:29.000 --> 00:00:34.000
It just makes our life a lot easier. That is called an orthonormal basis.
00:00:34.000 --> 00:00:47.000
So, that is what we are going to talk about today. We are going to introduce the concept, and then we will talk about how to take a given basis and turn it into an orthonormal basis by something called the Gram Schmidt orthonormalization process.
00:00:47.000 --> 00:00:55.000
It can be a little bit computationally intensive, and notationally intensive, but there is nothing particularly strange about it.
00:00:55.000 --> 00:00:59.000
It is all things you have seen. Essentially it is all just arithmetic, and a little bit of the dot product.
00:00:59.000 --> 00:01:03.000
So, let us just jump in and see what we can do.
00:01:03.000 --> 00:01:11.000
Okay. Let us talk about the standard basis in R2 and R3, the basis that we are accustomed to.
00:01:11.000 --> 00:01:24.000
If I take R2, I can have a basis (1,0), that is one vector, and the other vector is (0,1)... okay?
00:01:24.000 --> 00:01:35.000
Also known as i and j, also known as e1 and e2, again just different symbols for the same thing.
00:01:35.000 --> 00:01:48.000
R3 is the same thing. R3... we have... (1,0,0), (0,1,0), and (0,0,1), as a basis.
00:01:48.000 --> 00:02:02.000
Three vectors, three dimensional vector space, you also know them as i,j, k, and we have also referred to them as e1, e2, e3.
00:02:02.000 --> 00:02:16.000
Now, what is interesting about these particular bases, notice, let us just deal with R3... vector a and vector 2 are orthogonal meaning that their dot product is 0, perpendicular.
00:02:16.000 --> 00:02:26.000
1 and 3 are orthogonal, 2 and 3 are orthogonal. Not only that, they are not just orthogonal, mutually orthogonal, but each of these has a length of 1.
00:02:26.000 --> 00:02:34.000
So, this is what we call orthonormal, that the vectors are mutually orthogonal, and they have a length of 1.
00:02:34.000 --> 00:02:40.000
As it turns out, this so called natural basis works out really, really, well computationally.
00:02:40.000 --> 00:02:55.000
We want to find a procedure... is there a way where given some random basis, or several random bases, can we choose among them and turn that basis into something that is orthonormal, where all of the vectors have a length of 1 and all of the vectors are mutually orthogonal.
00:02:55.000 --> 00:03:03.000
As it turns out, there is. The Gram Schmidt orthonormalization process. A beautiful process, and we will go through it in just a minute.
00:03:03.000 --> 00:03:08.000
Let us just start off with some formal definitions first.
00:03:08.000 --> 00:03:42.000
So, we have a set s, which is the vectors u1, u2... all the way to uN is orthogonal, if any two vectors in s are orthogonal.
00:03:42.000 --> 00:03:51.000
What that means mathematically is that u < font size="-6" > i < /font > ⋅ u < font size="-6" > j < /font > , so u₁ ⋅ u₃ = 0.
00:03:51.000 --> 00:03:59.000
The dot product of those two vectors is equal to 0. That is the definition of orthogonality. Perpendicularity if you will.
00:03:59.000 --> 00:04:22.000
Now, the set is orthonormal if each vector also has a length norm of 1.
00:04:22.000 --> 00:04:29.000
Mathematically, that means u < font size="-6" > i < /font > dotted with itself gives me 1.
00:04:29.000 --> 00:04:53.000
Let us just do an example. Let us take u₁ = the vector (1,0,2), u₂ = the vector (-2,0,1), and u₃ = the vector (0,1,0).
00:04:53.000 --> 00:05:11.000
Okay. So, as it turns out, the set u1, u2, u3, well if I do the dot product of u1, u2... u1, u3... u2, u3, I get 0 for the dot product.
00:05:11.000 --> 00:05:16.000
So, this set is orthogonal.
00:05:16.000 --> 00:05:34.000
Now, let us calculate some norms. Well, the norm of u₁ is equal to this squared + this squared + this squared under the radical sign.... is sqrt(5).
00:05:34.000 --> 00:05:48.000
The norm for u2, that is our symbol for norm, is equal to this squared + this squared + this squared... also sqrt(5).
00:05:48.000 --> 00:05:53.000
And... the norm for u3 is 1. So, this one is already a unit vector, these two are not.
00:05:53.000 --> 00:06:09.000
So, since I have the norm, how do I create a vector that is length 1. I take the vector and I multiply it by the reciprocal of its norm, or I divide it by its norm, essentially.
00:06:09.000 --> 00:06:28.000
So, we get the following. If I have the set u1/sqrt(5), u2/sqrt(5), and u3... this set is orthonormal.
00:06:28.000 --> 00:06:39.000
Again, radical sign in the denominator, it is not a problem, it is perfectly good math. In fact, I think it is better math than simplification.
00:06:39.000 --> 00:06:45.000
What they call rationalizing the denominator, or something like that... I like to see where my numbers are.
00:06:45.000 --> 00:06:56.000
If there is a radical 5 in the denominator, I want it to be there. I do not want it to be u2sqrt(5)/5, that makes no sense to me, personally, but it is up to you though.
00:06:56.000 --> 00:07:08.000
Okay. Quick little theorem here.
00:07:08.000 --> 00:07:32.000
If s is an orthogonal set, or orthonormal, then s is linearly independent.
00:07:32.000 --> 00:07:40.000
So, if I have a set that I do not know is a basis, I know this theorem says that they are linearly independent.
00:07:40.000 --> 00:07:46.000
So, the particular space that they span, it is a basis for that space.
00:07:46.000 --> 00:07:56.000
Okay. Let us go ahead and do this. Like we said before, using an orthonormal basis can actually improve the computation.
00:07:56.000 --> 00:08:01.000
It just makes the computational effort a little bit less painful.
00:08:01.000 --> 00:08:18.000
So, now if we have s, u1, u2, all the way to u < font size="-6" > n < /font > ...
00:08:18.000 --> 00:08:48.000
If this set is a basis for some vector space v and u is some random vector in v, then we know that we can express u as linear combination... c1u1 + c2u2 + cNuN.
00:08:48.000 --> 00:08:56.000
u's, v's, w's, all kinds of stuff going on. Okay, so what we did before was we just solved this linear system.
00:08:56.000 --> 00:09:05.000
We found c1, c2, all the way to cN, you now, Gauss-Jordan elimination, reduced row echelon form.
00:09:05.000 --> 00:09:22.000
Well, here is what is interesting. If s is orthonormal, well still if it is orthonormal, when it is orthogonal it is still a basis so you still get this property.
00:09:22.000 --> 00:09:30.000
You know -- u is still that, but it is a really, really simple way to find the cN without finding the linear system.
00:09:30.000 --> 00:09:42.000
What you end up with is the following. Each c < font size="-6" > i < /font > is equal to the vector u dotted with u < font size="-6" > i < /font > .
00:09:42.000 --> 00:09:52.000
For example, if I wanted to find the second coefficient, I would just take the vector u and I would dot it with the second vector in the basis.
00:09:52.000 --> 00:10:00.000
That is fantastic. It is just a simple dot product. For vectors that are -- you know -- maybe 2-space, 3-space, 4-space, maybe even 5-space, there is no solution, you do not have to worry about it.
00:10:00.000 --> 00:10:08.000
You can just do the multiplication and the addition in your head. The dot product is really, really easy to find.
00:10:08.000 --> 00:10:25.000
So, let us do an example of this. We will let s equal... okay... it is going to be a little strange because we are going to have some fractions here...
00:10:25.000 --> 00:10:50.000
(2/3, -2/3, 1/3, 2/3, 1/3, -2/3, 1/3, 2/3, 2/3,), so this is our set s.
00:10:50.000 --> 00:11:15.000
Okay, well, we want to be able to write some random vector v, which is equal to let us say (3,4,5) as a linear combo of the vectors in s.
00:11:15.000 --> 00:11:31.000
So, we want c1, let us call it v1, the vectors in s, let us just call them v1, v2, v3... + c2v2 + c3v3.
00:11:31.000 --> 00:11:40.000
So, we want to find c1, c2, c3. Well, as it turns out that basis, even though it looks a little strange, it actually ends up being orthonormal.
00:11:40.000 --> 00:11:48.000
The length of each of those vectors is 1, and the mutual dot product of each of those is 0.
00:11:48.000 --> 00:11:58.000
So, it is an orthonormal set. Therefore, c1 = v ⋅ v1.
00:11:58.000 --> 00:12:17.000
Well, v is just 3, 4, 5, okay? I am going to dot it with v1, which was 2/3, -2/3, 1/3, and I get 1.
00:12:17.000 --> 00:12:26.000
When I do c2, well that is just equal to v ⋅ v2. When I do that, I get 0.
00:12:26.000 --> 00:12:39.000
If I do c3, that is just v, again, 3, 4, 5, dotted with v3, which was v3 up there in s, and this ends up being 7.
00:12:39.000 --> 00:12:51.000
Therefore, v is equal to, well, 1 × v1 + 0 × v2 + 7v3.
00:12:51.000 --> 00:13:01.000
There we go. I did not have to solve the linear system. I just did simple dot product. Simple multiplication and addition. Very, very nice.
00:13:01.000 --> 00:13:06.000
One of the benefits of having an orthonormal basis. There are many benefits, trust me on this one.
00:13:06.000 --> 00:13:12.000
Okay. So, now let us come down to the procedure of actually constructing our orthonormal basis.
00:13:12.000 --> 00:13:22.000
So we are going to go through this very, very carefully. There is going to be a lot of notation, but again, the notation is not strange. You just have to make sure to know what is where.
00:13:22.000 --> 00:13:35.000
The indices are going to be very, very important... so, take a look at it here, take some time to actually stare at the Gram Schmidt orthogonalization process in your textbook.
00:13:35.000 --> 00:13:46.000
That is really the best way to sort of wrap your mind around it. Of course, doing examples, which we will do when you do problems, but just staring at something is really a fantastic way to understand what it is that is going on.
00:13:46.000 --> 00:13:55.000
In mathematics, it is the details, the indices, the order of things. Okay.
00:13:55.000 --> 00:14:01.000
Let us see. So, let us write it out as a theorem first.
00:14:01.000 --> 00:14:17.000
You know, let me... let me go to a black ink... theorem...
00:14:17.000 --> 00:14:30.000
Let w be a non-zero subspace. So again, when we can speak of a basis we can speak of a base for the entire space, it does not really matter.
00:14:30.000 --> 00:14:35.000
In this case we are just going to express this theorem as a subspace.
00:14:35.000 --> 00:14:43.000
Again, the whole space itself is a subspace of itself, so this theorem is perfectly valid in this case.
00:14:43.000 --> 00:15:06.000
... be a subspace of rN with basis s = u1, u2, uN. N vectors, N space.
00:15:06.000 --> 00:15:34.000
Then, there exists an orthonormal basis t, which is equal to... let us call it w1, w2, all the way to wN for w.
00:15:34.000 --> 00:15:45.000
So, this theorem says that if I am given a basis for this subspace or a space itself, I can find... there exists an orthonormal basis.
00:15:45.000 --> 00:15:55.000
Well, not only does one exist, as it turns out, this particular procedure constructs it for us. So, the proof of this theorem is the construction of that orthonormal basis itself.
00:15:55.000 --> 00:16:01.000
That is the Gram Schmidt orthonormalization process. They call it the orthogonalization process, which is really what we are doing.
00:16:01.000 --> 00:16:07.000
We are finding orthogonal vectors, but we know how to change vectors that are orthogonal to vectors that are orthonormal.
00:16:07.000 --> 00:16:13.000
We just divide by their norms... nice and simple. Okay.
00:16:13.000 --> 00:16:19.000
So, let us list the procedure so that we have some protocol to follow.
00:16:19.000 --> 00:16:54.000
Procedure... okay... procedure for constructing an orthonormal basis t, which we said is going to be w1, w2... all the way to wN.
00:16:54.000 --> 00:17:18.000
Constructing an orthonormal basis t... from basis s = u1, u2, all the way to uN.
00:17:18.000 --> 00:17:27.000
I am going to change something here. I am going to not use w, I think I am going to use v instead.
00:17:27.000 --> 00:17:38.000
I used u for s, so I am going to go back and choose... let me call them v so that we stay reasonably consistent... to vN.
00:17:38.000 --> 00:17:48.000
Okay. So, we are given a basis s, we want to construct an orthonormal basis t from s, here is how we do it.
00:17:48.000 --> 00:17:56.000
First things first. We let v1, the first vector in our basis t, our orthonormal, we let it equal u1.
00:17:56.000 --> 00:18:02.000
We just take it straight from there. The first vector is the first vector.
00:18:02.000 --> 00:18:06.000
Okay. Two. This is where the notation gets kind of intensive.
00:18:06.000 --> 00:18:54.000
v < font size="-6" > i < /font > = u < font size="-6" > i < /font > - u < font size="-6" > i < /font > ⋅ 1₁/v₁ ⋅ v₁ × the vector v₁ - u < font size="-6" > i < /font > ⋅ v₂/v₂ ⋅ v₂ × the vector v₂... and so on.
00:18:54.000 --> 00:19:18.000
Until we get to u < font size="-6" > i < /font > ⋅ v < font size="-6" > i-1 < /font > /v < font size="-6" > i-1 < /font > ⋅ v < font size="-6" > i-1 < /font > × v < font size="-6" > i-1 < /font > . Okay.
00:19:18.000 --> 00:19:25.000
Do not worry about this, it will... when we do the example, this will make sense. Again, this is just mathematical formulism to make sure that everything is complete
00:19:25.000 --> 00:19:33.000
When we do the example, you will see what all of these i's and i-1's and v's mean.
00:19:33.000 --> 00:19:54.000
Three. t*, when we have collected our v₁, v₂, that we go from the first two steps... is orthogonal.
00:19:54.000 --> 00:19:59.000
We have created an orthogonal set.
00:19:59.000 --> 00:20:31.000
Now, we want to take -- that is fine, we will go ahead and we will take for every v < font size="-6" > i < /font > , we are going to divide it by its norm.
00:20:31.000 --> 00:20:42.000
So, for each of these v₁, v₁, in this set which is orthogonal, we are going to divide each of these vectors by the norm of that vector.
00:20:42.000 --> 00:21:27.000
Then, of course, what you get is the final set t, which is v₁/norm(v₁), v₂/norm(v₂), and so on and so forth, all the way to v < font size="-6" > n < /font > , not v < font size="-6" > i < /font > .../norm(v < font size="-6" > n < /font > )/
00:21:27.000 --> 00:21:39.000
This set is orthonormal. Let us just do an example and it will all make sense.
00:21:39.000 --> 00:21:48.000
So, let us start here. Let us do our example in blue.
00:21:48.000 --> 00:22:22.000
We will let s = u1, u2, u3 = 1, 1, 1, -1, 0, -1, -1, 2, 3.
00:22:22.000 --> 00:22:33.000
This is our set. We want to transform s into an orthonormal basis, for R3. This is a basis for R3.
00:22:33.000 --> 00:22:41.000
These are linearly independent. They span R3. Three vectors. We want to change this into an orthonormal basis.
00:22:41.000 --> 00:22:51.000
We want each of the vectors in our basis to be orthogonal, mutually orthogonal, and we want them to have a length of 1. So, we will run through our procedure.
00:22:51.000 --> 00:23:00.000
Okay. First thing we do, let v1 = u1. So the first thing I am going to do is I am going to choose by vector (1,1,1).
00:23:00.000 --> 00:23:07.000
That is my first vector in my orthogonal set. Nice, we got that out of the way. Okay.
00:23:07.000 --> 00:23:20.000
Two. Go back to the previous slide and check to see that number two thing with all of the indices going on. Here is what it looks like based on this number of vectors.
00:23:20.000 --> 00:23:50.000
v₂ = u₂ - u₂ ⋅ v₁, which is this thing /v₁ ⋅ v₁ × vector v₁.
00:23:50.000 --> 00:24:00.000
That is equal to... well, u₂ is (-1, 0, -1).
00:24:00.000 --> 00:24:13.000
Now, when I take u₂, which is (-1, 0, -1), and I dot it with v₁, which is (1, 1, 1), I get -2.
00:24:13.000 --> 00:24:31.000
When I take v₁ dotted with v₁, I get three. So it is -2/3 × v₁, which is the (1,1,1).
00:24:31.000 --> 00:24:49.000
When I do that, I get -1/3, 2/3, -1/3... okay.
00:24:49.000 --> 00:24:58.000
The next one v3, I have v₁, I have v₂, which is here. I need v₃, right? because I need 3 vectors.
00:24:58.000 --> 00:25:43.000
So, v₃ = well, it is equal to -- go back to that number 2 in our procedure -- it is equal to u₃ - u₃ ⋅ v₁/v₁ ⋅ v₁ × v₁ - u₃ ⋅ v₂/v₂ ⋅ v₂, all × v₂.
00:25:43.000 --> 00:25:47.000
Well if you remember that last entry in that number 2, it said v^i-1.
00:25:47.000 --> 00:25:57.000
Well, since we are calculating v₃, 3 - 1 is 2, so that is it. We stop here. We do not have to go anymore.
00:25:57.000 --> 00:26:03.000
That is what that symbolism meant, it tells us how many of these we get.
00:26:03.000 --> 00:26:18.000
If we are calculating v₄, well, 4 - 1 is... so that is v < font size="-6" > i < /font > , i is 4. It is 4 - 1, that means we go all the way up to this last entry, which is 3. So we would have three of these.
00:26:18.000 --> 00:26:24.000
That is all this means. That is all that symbolism meant. Just follow the symbolism, and everything will be just fine.
00:26:24.000 --> 00:26:37.000
okay. This actually ends up equaling... well u₃ is (-1,2,3).
00:26:37.000 --> 00:27:07.000
When I do u₃ ⋅ v₁, which is (-1,2,3) ⋅ (1,1,1) over (1,1,1) ⋅ (1,1,1), I am going to end up with 4/3 × (1,1,1) - ... when I do u₃ ⋅ v₂, which is u₃ ⋅ v₂/v₂ ⋅ v₂.
00:27:07.000 --> 00:27:24.000
I am going to get -2/6 × v₂ - (1,2,1)... okay?
00:27:24.000 --> 00:27:37.000
I am going to end up with (-2,0,2). So, let me go to red.
00:27:37.000 --> 00:27:58.000
(1,1,1), that... and that... so let us write our what we have got.
00:27:58.000 --> 00:28:26.000
Our t*, our orthogonal set is this sub... interesting... 1, 1, 1, -1/3, 2/3, -1/3, and -2, 0, 2.
00:28:26.000 --> 00:28:41.000
This set is orthogonal. Now, let us take a look at v₂ real quickly here.
00:28:41.000 --> 00:29:01.000
v₂ = (-1/3, 2/3, -1/3), well, let me pull out the third... that is equal to 1/3 × (-1, 2, -1).
00:29:01.000 --> 00:29:14.000
These vectors, if I just take the vector (-2, 2, 1), and if I take the vector 1/3 × that, which is this vector, they are vectors going in the same direction.
00:29:14.000 --> 00:29:26.000
They are just different lengths of each other. So, because they are vectors going in the same direction, I do now need the fractional version of it. I can just drop the denominator from that, because again, we are going to be normalizing this.
00:29:26.000 --> 00:29:32.000
We are going to be reducing it to a length of 1, so it does not matter whether I take this vector or this vector.
00:29:32.000 --> 00:29:36.000
They are just different vectors in the same direction. Does that make sense?
00:29:36.000 --> 00:29:55.000
So, I can rewrite my t*, my orthogonal set, as (1,1,1), (-1,2,-1), and (-2,0,2).
00:29:55.000 --> 00:30:06.000
They are still orthogonal. This, the dot product of this and this is going to be 0. They dot product of this and this is going to be 0, it does not matter. They are in the same direction.
00:30:06.000 --> 00:30:20.000
They are just different lengths, we are going to be normalizing it anyway. So, we want to make life easier for ourselves, so that is our orthogonal set.
00:30:20.000 --> 00:30:27.000
Now, what we want to do is calculate the norms.
00:30:27.000 --> 00:30:55.000
So, the norm of v₁ = sqrt(3). The norm of v₂ = sqrt(6), and the norm of v₃, right here, is equal to sqrt(8).
00:30:55.000 --> 00:31:37.000
Therefore, our final orthonormal... we have 1/sqrt(3), 1/sqrt(3), 1/sqrt(3)... -1/sqrt(6), 2/sqrt(6), -1/sqrt(6) and... -2/sqrt(8), 0, 2/sqrt(8)...
00:31:37.000 --> 00:31:47.000
Again, I may have missed some minus signs or plus signs... it is procedure that is important.
00:31:47.000 --> 00:32:05.000
This is orthonormal. In this case, the length matters. In the previous, they were orthogonal and so we dropped the denominator from the vectors that we found because again, they are vectors in the same direction.
00:32:05.000 --> 00:32:20.000
The same direction, they are still going to be orthogonal. So, I can make it easy for myself by not having to deal with fractions. But, in this case, we are normalizing it. We are taking that orthogonal set and we are dividing each of those vectors by its own norm to create vectors.
00:32:20.000 --> 00:32:33.000
Each of these has a length of 1. Any two of these are mutually orthogonal. Their dot product equals 0. This is an orthonormal basis for our three.
00:32:33.000 --> 00:32:44.000
It is just as good of a basis as our standard basis, the i, j, k. Computations are easy, and who knows, there might be some problem where this basis is actually the best basis to work with.
00:32:44.000 --> 00:32:48.000
Again, it just depends on frames of reference.
00:32:48.000 --> 00:32:53.000
Okay. Thank you for joining us at Educator.com.