WEBVTT mathematics/linear-algebra/hovasapian
00:00:00.000 --> 00:00:04.000
Welcome back to Educator.com and welcome back to linear algebra.
00:00:04.000 --> 00:00:09.000
In our previous lesson, we introduced the notion of Eigenvector and Eigenvalue.
00:00:09.000 --> 00:00:15.000
Again, very, very profoundly profoundly important concepts throughout mathematics and science.
00:00:15.000 --> 00:00:25.000
Today, we are going to dig a little bit deeper and we are going to introduce the notion of similar matrices and the idea of diagonalization.
00:00:25.000 --> 00:00:36.000
So, let us jump right on in. Let us start with a definition... Let me go to blue ink here.
00:00:36.000 --> 00:01:23.000
Okay. A matrix b is said to be similar to matrix a if there is a non-singular matrix, p, such that... let us see what this definition says.
00:01:23.000 --> 00:01:42.000
If I have some matrix a, and I find some other matrix p, and if I multiply on the left of a by p inverse, and on the right by p, so if I take p inverse a × p, the matrix that I get, b, I say that b is similar to a.
00:01:42.000 --> 00:01:49.000
So, there is a relationship that exists between, if I can actually sandwich this matrix a between some matrix p and the inverse of p.
00:01:49.000 --> 00:02:00.000
In the course of this lesson, we are going to talk to you actually about how to find this matrix p, and about what this matrix b looks like. Really, really quite beautiful.
00:02:00.000 --> 00:02:29.000
Okay. A quick example just so you see what this looks like in real life. So, if I let the matrix a equal (1,1,-2,4), just a little 2 by 2... and if I say I had p which is (1,1,1,2), well, if I calculate the inverse of p, that is going to equal (2,-1,-1,1)... okay?
00:02:29.000 --> 00:02:50.000
Now, as it turns out, if I take b, if I actually do p inverse × a × p so I multiply that by that, then by that, I end up with the following... I end up with the matrix (2,0,0,3).
00:02:50.000 --> 00:03:03.000
Now, what you might want to do is take a look at the previous lesson. This matrix a, you remember, and I will discuss this in a minute... just to let you know what is coming up, what is coming ahead, this matrix a, we have dealt with this matrix before in the previous lesson.
00:03:03.000 --> 00:03:09.000
We found the Eigenvalues for it. The Eigenvalues were 2 and 3. Well, notice what we did.
00:03:09.000 --> 00:03:23.000
We found this matrix p... and we took the inverse of that, we multiplied p inverse a p, and we end up with a diagonal matrix, where the entries on the diagonal are exactly the Eigenvalues of a. That is what is going to end up being so beautiful.
00:03:23.000 --> 00:03:32.000
Here is what is even better. If you remember the two Eigenvectors that we found for the two Eigenvalues 2 and 3 were exactly of the form (1,1,1,2).
00:03:32.000 --> 00:03:43.000
So, this matrix p is going to end up being made up of the actual Eigenvectors for the Eigenvalues. We will just... a little preview of what is coming.
00:03:43.000 --> 00:04:04.000
Okay. Just some quick properties of similarity. So, the first property is a is similar to a, of course... intuitively clear.
00:04:04.000 --> 00:04:22.000
If b is similar to a, then a is similar to b. That just means that... we can always multiply on the left by p, here, and p × p inverse, this goes away, and we multiply on the right by p inverse so this is just the same.
00:04:22.000 --> 00:04:53.000
And three... If a is similar to b, and b is similar to c, then, by transitivity, c is similar to a... I am sorry, a is similar to c, which means that c is similar to a by property too... then a is similar to c.
00:04:53.000 --> 00:05:44.000
So, standard properties... we will be using those in a second. Another definition. We say matrix a is diagonalizable... did I spell that correct?... diagonalizable... if it is similar to a diagonal matrix.
00:05:44.000 --> 00:06:07.000
In this case, we say a can be diagonalized -- put a comma there, in this case we say... a can be diagonalized.
00:06:07.000 --> 00:06:27.000
Okay. Now, let us see what we have got. Alright. Profoundly, profoundly, profoundly important theorem.
00:06:27.000 --> 00:07:03.000
An n by n matrix is diagonalizable if, and only if, it has n linearly independent Eigenvectors.
00:07:03.000 --> 00:07:48.000
In this case, a is similar to a diagonal matrix d, where d is equal to p inverse a... p... and the diagonal elements of p, whose diagonal elements are the Eigenvalues of a.
00:07:48.000 --> 00:08:01.000
It is like we did before. It is similar to the diagonal matrix b, and the entries on that diagonal are precisely the Eigenvalues of a.
00:08:01.000 --> 00:08:32.000
Well, p is the matrix whose columns respectively... it means respective to the individual Eigenvalues... so Eigenvalue λ1 gives column 1, λ2 gives column 2, λ3 gives column 3... so on and so forth, respectively.
00:08:32.000 --> 00:08:49.000
Each of these columns, respectively, are the n linearly independent Eigenvectors of a.
00:08:49.000 --> 00:09:02.000
So, again, this is what we did in our example. So now we state it as a theorem... an n by n matrix is diagonalizable if and only if it has n linearly independent Eigenvectors.
00:09:02.000 --> 00:09:08.000
So, if I have a 3 by 3, I need 3 Eigenvectors. If I have a 5 by 5, I need 5 linearly independent Eigenvectors.
00:09:08.000 --> 00:09:30.000
In this case, the a is similar to a diagonal matrix d, whose diagonal elements are precisely the Eigenvalues of a, and the matrix p in this relation here, is the matrix made up of the columns that are the n independent, linearly independent Eigenvectors of a.
00:09:30.000 --> 00:09:37.000
So, from a, I can derive the matrix d, I can derive the matrix p, and the relationship is precisely this.
00:09:37.000 --> 00:09:54.000
Let us just do an example here. Okay. We will let a equal the matrix (1,2,3)... (0,1,0)... (2,1,2).
00:09:54.000 --> 00:10:02.000
I am not going to go through the entire process, again I use mathematical software to do this... to find the Eigenvalues and to find the Eigenvectors.
00:10:02.000 --> 00:10:19.000
Here is how it works out. As it turns out, one of the Eigenvalues is 4. 4 generates the following Eigenvector, when I do solve the homogenous system, I get (1,0,1).
00:10:19.000 --> 00:10:51.000
A second Eigenvalue is -1. Also real. It generates the Eigenvalue... the Eigenvector (3,0,2). The third Eigenvalue is equal to 1, all distinct, all real, and it generates 1 - 6 and 4, when I solve the homogeneous system.
00:10:51.000 --> 00:11:07.000
Therefore, p = (1,0,1), (-3,0,2), (1,-6,4).
00:11:07.000 --> 00:11:54.000
If I want to define p inverse, which I can, it is not a problem -- you know what I will go ahead and write it out here... it is not going to be too big of an issue -- I have (2/5, -1/5, 0), (7/15, 1/10, -1/6), (3/5, -1/5, 0), and of course my diagonal matrix d is going to end up being (4,0,0), (0,-1,0), (0,0,1).
00:11:54.000 --> 00:12:33.000
If I were to confirm... yes, I would find out that d does in fact equal p inverse × a × p. Excuse me... found the Eigenvalues... found the associated Eigenvector...put those Eigenvectors... these are linearly independent, put them as columns in p... if I take p, multiply and find p inverse, if I multiply a by p inverse on the left, p on the right, I end up with a diagonal matrix where the entries on the main diagonal are precisely the Eigenvalues (4,-1,1), (4,-1,1).
00:12:33.000 --> 00:12:41.000
So, 4 is the first, that is why its Eigenvector is the first column, that is what we meant by respectively.
00:12:41.000 --> 00:13:32.000
Okay. Now, if all the roots of the characteristic polynomial, which is what we solve to find the Eigenvalues, of a are real and distinct... if the roots of the characteristic polynomial are real and distinct... in other words if the Eigenvalues of the matrix are real and distinct, then a is diagonalizable. Always.
00:13:32.000 --> 00:13:42.000
Note how we wrote this. If all of the roots of the characteristic polynomial are real and distinct, then a is diagonalizable. There is no if and only if here.
00:13:42.000 --> 00:13:49.000
That does not mean that if a is diagonalizable, that the roots of the characteristic polynomial, the Eigenvalues are real and distinct.
00:13:49.000 --> 00:14:06.000
It is possible for a to be diagonalizable and have roots that are not distinct. You might have an Eigenvalue, you might have a 4 by 4, and you might have an Eigenvalue 4, and then (1,1,1). That 1 might be an Eigenvalue 3 times over, but it will still be diagonalizable.
00:14:06.000 --> 00:14:15.000
So, again. If then does not mean that it works the same backwards. It is not the same. It is not the same as if and only... if and only if means it goes both ways.
00:14:15.000 --> 00:14:26.000
So, if the roots of the polynomial are real and distinct. If it is diagonalizable, the Eigenvalues may or may not be real and distinct.
00:14:26.000 --> 00:14:50.000
Okay. Now, the characteristic polynomial... so the characteristic poly for non-distinct Eigenvalues looks like this.
00:14:50.000 --> 00:14:59.000
Well, we know we are dealing with some characteristic polynomial... λ³ + something λ², λ;^n.
00:14:59.000 --> 00:15:10.000
Well, every time we find a value, we can factor... that is what the fundamental theorem of algebra says... every polynomial can be factored into linear factors.
00:15:10.000 --> 00:15:17.000
For the non-distinct, we end up with something like this... λ - λ < font size="-6" > i < /font > to the k < font size="-6" > i < /font > power.
00:15:17.000 --> 00:15:39.000
So, if some root ends up showing up 5 times, that means that I have 5 factors for that root. λ - 1, λ - 1, λ - 1, λ - 1, λ - 1, well, that factor is λ - λ < font size="-6" > i < /font > ... you know, to this power. Okay.
00:15:39.000 --> 00:16:14.000
This is λ1, this is λ 2, in the case of 2, in the case of i... Okay.
00:16:14.000 --> 00:16:17.000
Well, this k < font size="-6" > i < /font > is called multiplicity of the Eigenvalue... λ < font size="-6" > i < /font >
00:16:17.000 --> 00:16:59.000
It can be shown that if the Eigenvalues of a are all real and distinct, in this case we already dealt with the fact they are distinct, we know they are diagonalizable if they are all real and distinct.
00:16:59.000 --> 00:17:12.000
So, we can also show, now that we have introduced this idea of multiplicity... so if our characteristic polynomial has multiple roots, so if I have a fourth degree equation, one of the roots is 4, the other root is 3, and the other roots are 2 and 2.
00:17:12.000 --> 00:17:18.000
Well, 4 has a multiplicity 1, 3 has a multiplicity 1, 2, because it shows up twice, has a multiplicity 2.
00:17:18.000 --> 00:18:08.000
It can be shown that if the Eigenvalues of a are all real, then a can be diagonalized, it can still be diagonalized if and only if for each λ < font size="-6" > i < /font > of multiplicity k < font size="-6" > i < /font > , we can find... if we can find k < font size="-6" > i < /font > linearly independent Eigenvectors.
00:18:08.000 --> 00:18:38.000
This means that the null space for that thing that we solved, L < font size="-6" > i < /font > , i < font size="-6" > n < /font > - a, × x = 0, this means that the null space of that equation that we solved to find the Eigenvectors has dimension k < font size="-6" > i < /font > .
00:18:38.000 --> 00:18:48.000
In other words, if I have a root of the characteristic polynomial, an Eigenvalue that has a multiplicity 3... let us say its 1 shows up 3 times.
00:18:48.000 --> 00:19:05.000
Well, when I put it into the equation, this homogeneous equation, if I can actually find 3 linearly independent vectors... if I can find that the dimension of that null space is 3, I can diagonalize that matrix.
00:19:05.000 --> 00:19:10.000
If not, I cannot diagonalize that matrix. Okay.
00:19:10.000 --> 00:19:29.000
So, let us see what we have got. You will let a = (0,0,1), (0,1,2), (0,0,1).
00:19:29.000 --> 00:19:33.000
That is interesting... okay.
00:19:33.000 --> 00:19:55.000
When we take the characteristic polynomial of this, we end up with the following. λ × (λ - 1)²... so we have λ1 = 0, λ2 = 1, so one has a multiplicity of 2, and λ3 is also equal to 1.
00:19:55.000 --> 00:20:14.000
Well, let us just deal with this multiplicity of 1 Eigenvalue. When we solved the homogeneous system, we end up with the following. We end up finding that the Eigenvector x is this... (0,r,0).
00:20:14.000 --> 00:20:40.000
Well, this is only 1 vector. Here we have 2 Eigenvalues, multiplicity is 2. Well, in order for this matrix to be diagonalizable, I have to... when I solve that homogeneous system to find the actual Eigenvectors, I need 2 vectors, not just the 1. This is only one vector so this is not diagonalizable.
00:20:40.000 --> 00:20:55.000
Now, mind you, it still has Eigenvalues, and it still has an associated Eigenvector, but it is not diagonalizable. I cannot find some matrix that satisfies that other property.
00:20:55.000 --> 00:21:12.000
Okay. So, now, let us try this one. Let us let a equal to (0,0,0), (0,1,0), (1,0,1).
00:21:12.000 --> 00:21:20.000
Well, as it turns out, this characteristic polynomial is also λ × (λ - 1)².
00:21:20.000 --> 00:21:34.000
So, again we have λ = 0, λ2 = 1, λ3 = 1, so our Eigenvalue 1 has a multiplicity of 2, so we want to find 2 Eigenvectors.
00:21:34.000 --> 00:21:42.000
We want to find the dimension of this null space associated with this Eigenvalue if it has a dimension 2, we are good. We can actually diagonalize this.
00:21:42.000 --> 00:22:06.000
Let us see what we have. Let us see. When we solve the associated homogeneous system, we end up with (1,0,0,0), (0,0,0,0), (-1,0,0,0).
00:22:06.000 --> 00:22:55.000
When we subject this to reduced row echelon form, it becomes (1,0,0,0), (0,0,0), (0,0,0,0)... so we have... there we go... we have x1 = 0, we have x2 = let us see, this is x2, x3, 3 parameters equals r... x3 = s, and we can rewrite this as equal to r × (0,1,0) + s × (0,0,1), so there you go.
00:22:55.000 --> 00:23:12.000
We have two Eigenvectors, a basis... our basis for that null space has 2 Eigenvectors. It is of dimension 2. It matches the multiplicity of the Eigenvalue, therefore this can be diagonalized.
00:23:12.000 --> 00:23:37.000
So, let us go ahead and actually finish the diagonalization process... when I go back and solve for the Eigenvalue λ1 = 0, for that Eigenvalue, I get the following vector... r0 - r, as a general, and its specific would be -- let us see -- (1,0,-1).
00:23:37.000 --> 00:23:48.000
Therefore our matrix p would be (1,0,-1), (0,1,0), (0,0,1). This is our matrix p.
00:23:48.000 --> 00:24:18.000
It is of course diagonalizable. Matrix d is going to end up being (0,0,0), (0,1,0), (0,0,1). These along the main diagonal are the Eigenvalues of our matrix. This is our matrix p. We can find the inverse for it, and when we multiply we will find that d, in fact, equals p inverse, a p.
00:24:18.000 --> 00:25:02.000
So, now, procedure for diagonalizing matrix a, this is going to be our recap. First thing we want to do is form the characteristic polynomial, which is symbolized also with f(λ) equals the determinant of λ × i < font size="-6" > n < /font > ... λ is our variable... λ times the identity matrix minus a.
00:25:02.000 --> 00:25:17.000
That is our polynomial. Then, we find the roots of the characteristic polynomial... okay?
00:25:17.000 --> 00:25:37.000
If not all real, if they are not all real, then you can stop... it cannot be diagonalized.
00:25:37.000 --> 00:26:21.000
Okay. Three. For each Eigenvalue, λ < font size="-6" > i < /font > of multiplicity k < font size="-6" > i < /font > ... find a basis for the null space of λ < font size="-6" > i < /font > , i < font size="-6" > n < /font > - a × x = 0.
00:26:21.000 --> 00:26:34.000
So, for each Eigenvalue λ < font size="-6" > i < /font > of multiplicity k < font size="-6" > i < /font > , find a basis for the null space... also called the Eigenspace.
00:26:34.000 --> 00:27:02.000
If the dimension... the dimension that we just found... the dimension of this null space that we just found is less than the k < font size="-6" > i < /font > , then you can stop. a cannot be diagonalized.
00:27:02.000 --> 00:28:17.000
If so, well, then, let p be the matrix whose columns are the n linearly independent Eigenvectors... found above... then, the inverse a × p = d, where d has λ < font size="-6" > i < /font > along the diagonals. The diagonal entries are the Eigenvalues.
00:28:17.000 --> 00:28:24.000
Okay. So, let us recap. We want to form the characteristic polynomial once we are given a matrix a, n by n.
00:28:24.000 --> 00:28:33.000
Once we find the characteristic polynomial, we want to find its roots. If the roots are not all real, if there is... they have to all be real.
00:28:33.000 --> 00:28:38.000
If they are not all real, then you can stop. You cannot diagonalize the matrix.
00:28:38.000 --> 00:28:55.000
If they are real, well, for each Eigenvalue of multiplicity... you know k < font size="-6" > i < /font > , for each one that has a multiplicity, you want to find a basis for the null space of that equation... we solve the homogeneous system.
00:28:55.000 --> 00:29:02.000
Well, if the dimension of that null space is equal to k < font size="-6" > i < /font > , we can continue. If not, we can stop, we cannot diagonalize the matrix.
00:29:02.000 --> 00:29:17.000
But, if we can, and for each distinct Eigenvalue, we are going to have 1 Eigenvector. We will let... take those n linearly independent Eigenvectors that we just found, we will arrange them as columns respectively.
00:29:17.000 --> 00:29:40.000
So λ1, column 1, λ2, column 2... that is going to be our matrix p. When we take p inverse × a × p, that actually is equal to our diagonal matrix d, the entries of which are the respective Eigenvalues of our original matrix a.
00:29:40.000 --> 00:29:51.000
So, we will be dealing more with Eigenvalues and Eigenvectors in our next lesson, so we are not quite finished with this, it gets a little bit deeper. We will have a little more exposure with it.
00:29:51.000 --> 00:29:55.000
So, until then, thank you for joining us at Educator.com, and we will see you next time.