WEBVTT mathematics/statistics/son
00:00:00.000 --> 00:00:01.600
Hi and welcome to www.educator.com.
00:00:01.600 --> 00:00:06.300
Today we are going to talk more in-depth about type 1 and type 2 errors.
00:00:06.300 --> 00:00:13.600
If you want to know more about power and effect size it is good to go through this lesson
00:00:13.600 --> 00:00:17.600
because it is going to help you understand some of the pictures that we are going to draw in the future.
00:00:17.600 --> 00:00:21.300
Here is the roadmap for today.
00:00:21.300 --> 00:00:33.300
We need to know about these type 1 and type 2 errors, but we also need to know when we make those errors in relationship to hypothesis testing.
00:00:33.300 --> 00:00:37.400
So far we only used t test as our hypothesis test.
00:00:37.400 --> 00:00:48.100
We have shown these errors and their relationship to hypothesis testing before as a box, but frequently in hypothesis testing we draw distributions.
00:00:48.100 --> 00:00:50.700
The SDOM to be more specific.
00:00:50.700 --> 00:00:58.200
What I want to show you how the errors fit on this distribution picture.
00:00:58.200 --> 00:01:04.900
We are going to show you how the box and the distributions fit together because these two things actually relationship to each other.
00:01:04.900 --> 00:01:07.600
They refer to the same concept.
00:01:07.600 --> 00:01:11.100
There are just 2 different ways of showing you that same concept.
00:01:11.100 --> 00:01:25.900
We go through hypothesis testing, but in the real world there is some reality that either the null hypotheses is just true or the null hypothesis is false.
00:01:25.900 --> 00:01:32.300
Although we do not know this reality, all we know is the result of our hypothesis testing.
00:01:32.300 --> 00:01:35.600
There are two kinds of ways we can make errors.
00:01:35.600 --> 00:01:39.600
We can make an incorrect decision by false alarming.
00:01:39.600 --> 00:01:46.400
We reject the null, but we should not have rejected the null.
00:01:46.400 --> 00:01:50.600
That is called the false alarm or a type 1 error.
00:01:50.600 --> 00:01:56.100
I used to get confused between which one is type 1 and type 2, these are arbitrate.
00:01:56.100 --> 00:02:06.800
I like to think of this as the more serious error when you successfully reject the null hypothesis that is a more extreme thing that you do.
00:02:06.800 --> 00:02:11.200
This is actually more dangerous than this miss.
00:02:11.200 --> 00:02:16.500
That is not much of an error but actually false alarming.
00:02:16.500 --> 00:02:22.000
That is how I remember the number 1 error you should look out for.
00:02:22.000 --> 00:02:31.500
The type 1 error is often also called the likelihood of false alarming.
00:02:31.500 --> 00:02:37.500
The probability of false alarming and that is referred to as α.
00:02:37.500 --> 00:02:50.800
If the reality that we do not know is that this null hypothesis is true we have a probability of false alarming with the rate of α.
00:02:50.800 --> 00:03:05.700
We have the probability of failing to reject when we should have rejected, a correct failure your probability is 1-α.
00:03:05.700 --> 00:03:08.800
These two things add up to 1.
00:03:08.800 --> 00:03:19.600
The probability of false alarming + the probability of making a correct failure =1.
00:03:19.600 --> 00:03:30.000
On the flipside, let us say that null hypothesis is false that is not a true picture or model of the world.
00:03:30.000 --> 00:03:33.200
Then we really should have reject it.
00:03:33.200 --> 00:03:46.000
It is not true, we should reject it, that would be a correct decision and that is called the hit where we are rejecting the null when we should have rejected it.
00:03:46.000 --> 00:03:50.400
That gives us the probability of hits.
00:03:50.400 --> 00:04:02.100
We could be incorrect and fail to reject when we should have rejected that is also another incorrect decision.
00:04:02.100 --> 00:04:04.100
That is the type 2 error.
00:04:04.100 --> 00:04:08.500
It is a miss and the probability of miss is given as β.
00:04:08.500 --> 00:04:14.000
β + 1 –β = 1.
00:04:14.000 --> 00:04:22.600
The probability of misses + probability of hits =1.
00:04:22.600 --> 00:04:34.300
In which of these boxes is the sample statistic statistically significant?
00:04:34.300 --> 00:04:40.500
In which of these boxes is our p value less than .05 or whatever our α level is.
00:04:40.500 --> 00:04:42.400
Let us think about that.
00:04:42.400 --> 00:04:53.900
When we reject the null hypothesis that means our test statistic in this case t is extreme.
00:04:53.900 --> 00:05:03.800
Our p value is significant and remember we mean significant as it stands out.
00:05:03.800 --> 00:05:06.700
It is very weird.
00:05:06.700 --> 00:05:15.500
In this case, these two quadrants up here is what we should worry about.
00:05:15.500 --> 00:05:21.900
This is the decision we need to worry about when we reject the null hypothesis.
00:05:21.900 --> 00:05:34.400
The other possibility is that when we reject the null hypotheses and our p is significant we made a correct decision.
00:05:34.400 --> 00:05:43.500
These are our two choices if we know that p is less than α or if our test statistic is extreme.
00:05:43.500 --> 00:05:49.600
Here p is not significant.
00:05:49.600 --> 00:05:58.100
It is not too weird and because of that we will fail to reject and we can be correct in failing to reject
00:05:58.100 --> 00:06:02.800
or when we fail to reject we could be wrong by making a type 2 error.
00:06:02.800 --> 00:06:05.100
Here is what I want you to know.
00:06:05.100 --> 00:06:12.300
Let us say we carry out hypothesis testing and I think I have a really low p value.
00:06:12.300 --> 00:06:15.600
I am going to reject my null hypotheses.
00:06:15.600 --> 00:06:21.600
Which error am I likely to make, a false alarm or a missed?
00:06:21.600 --> 00:06:37.300
Since I rejected my null, the only error I can possibly make is this one where I reject the null and get wrong.
00:06:37.300 --> 00:06:45.600
Let us say I go through my hypothesis testing and I get p=.4.
00:06:45.600 --> 00:06:48.100
Let us say I do not reject my null.
00:06:48.100 --> 00:06:51.400
What mistake or what error could I have possibly made?
00:06:51.400 --> 00:06:54.200
The only error I can make is the missed error.
00:06:54.200 --> 00:06:58.200
Here I fail to reject and I could be wrong in doing it.
00:06:58.200 --> 00:07:05.700
Let us talk about distributions and how errors fit in here.
00:07:05.700 --> 00:07:11.200
We have a one sample t test we set up some null population.
00:07:11.200 --> 00:07:23.100
This is our null hypotheses population and our hypothesized μ might be 230.
00:07:23.100 --> 00:07:34.400
We do not know whether our sample is part of this or it is part of some other population, not the null population.
00:07:34.400 --> 00:07:39.300
We can hypothesize maybe it comes from some other population like this one.
00:07:39.300 --> 00:07:56.300
When we set our α levels and create critical t and zones of rejection and all of that stuff what we are doing is recreating the line.
00:07:56.300 --> 00:08:11.700
If our sample t is outside here then we are going to reject the null.
00:08:11.700 --> 00:08:28.000
So far we have only colored in this part, but we really mean this part as well as all of this part.
00:08:28.000 --> 00:08:33.700
That is our reject the null zone, this entire area.
00:08:33.700 --> 00:08:48.300
In order to find out whether we should reject the null or not we also need to look past the raw score.
00:08:48.300 --> 00:08:56.100
We need to look past the raw score and we need to look at it in terms of the critical t.
00:08:56.100 --> 00:09:03.800
The critical t might be whatever like -2. Something .
00:09:03.800 --> 00:09:09.500
We need to find out this t value and so I am just going to make one.
00:09:09.500 --> 00:09:19.800
Let us say this t value is 5.5 and if our t value is sufficiently extreme then we reject are null hypothesis.
00:09:19.800 --> 00:09:33.700
This would be our critical t and this is our sample x bar, but this is our sample t.
00:09:33.700 --> 00:09:37.000
And that is how it looks out here.
00:09:37.000 --> 00:09:46.700
Our possibility of making an error is this little gray spot that I have colored in red.
00:09:46.700 --> 00:09:56.600
Just in case my sample really does come from these areas, I should not have rejected the null.
00:09:56.600 --> 00:10:12.800
If it happens by chance rule 50 heads in a row it is very unlikely but it is still possible.
00:10:12.800 --> 00:10:21.400
It is still possible that I got this x bar even though this is the true population distribution.
00:10:21.400 --> 00:10:27.800
This is my possibility of making a type 1 error.
00:10:27.800 --> 00:10:40.100
We actually have to add this side up to this side type 1 error.
00:10:40.100 --> 00:10:46.500
We know that this is α=.05.
00:10:46.500 --> 00:10:59.700
This part is 1 – α which is .95 and that is our possibility of not rejecting given that the null hypothesis is true.
00:10:59.700 --> 00:11:06.300
That is the example of one sample hypothesis testing.
00:11:06.300 --> 00:11:09.300
This is the same picture as before.
00:11:09.300 --> 00:11:16.400
I just written it more neatly for you by typing it out and you can think of this test statistic as just t.
00:11:16.400 --> 00:11:22.600
I have just written the generic word test statistic to think of this as critical t and sample t.
00:11:22.600 --> 00:11:25.200
Here is the important thing to realize.
00:11:25.200 --> 00:11:36.300
This gray distribution here represents an SDOM and that is why this is μ sub x bar and there is also an x bar here as a sample.
00:11:36.300 --> 00:11:46.200
This SDOM actually represents the probability where the null hypothesis is true and that probability equal to 1.
00:11:46.200 --> 00:11:53.000
Remember we talked about that before when we said the area underneath the normal distribution equal to 1.
00:11:53.000 --> 00:12:07.600
This represents the possibility that this may not be true and that there exists some other population that our sample really came from.
00:12:07.600 --> 00:12:10.700
We do not just know what that population is.
00:12:10.700 --> 00:12:16.200
That is the probability that the null hypothesis is false.
00:12:16.200 --> 00:12:24.300
That normal distribution also has an area =1.
00:12:24.300 --> 00:12:38.900
What we can additionally find out is when we create the zones of rejection and we say anything outside of this critical t reject it.
00:12:38.900 --> 00:12:40.800
We color in this area here.
00:12:40.800 --> 00:12:56.900
What we are saying is this is the probability of rejecting given that the null hypothesis is true.
00:12:56.900 --> 00:13:04.100
This is the area where we fail to reject.
00:13:04.100 --> 00:13:27.300
This probability right here represents the conditional probability of failing to reject given that h knot or null hypothesis is true.
00:13:27.300 --> 00:13:35.200
And that equals 1 – α because this one equal α.
00:13:35.200 --> 00:13:38.900
Those are the important things to remember.
00:13:38.900 --> 00:13:45.800
These are all conditional probabilities as we learned about previously in probability lessons.
00:13:45.800 --> 00:13:50.600
Let us talk about a two sample t test.
00:13:50.600 --> 00:13:58.100
The idea behind the two sample t test is almost exactly the same except there are just a couple of exceptions now.
00:13:58.100 --> 00:14:06.300
Instead of a raw score we have difference of scores and we still have a test statistic.
00:14:06.300 --> 00:14:21.700
Here our mean hypothesized difference between our non college sample and our college sample is going to be 0 because that means they are the same.
00:14:21.700 --> 00:14:35.200
Remember, these are SDOD (Sampling distributions of differences of means).
00:14:35.200 --> 00:14:47.500
This is 0 and this might be our actual sample difference x bar – y bar, the actual difference between the samples.
00:14:47.500 --> 00:15:02.300
Same thing down here, we have this as our critical test statistic and this is our sample t.
00:15:02.300 --> 00:15:10.600
We want to know whether our sample t is way far out, more extreme than our critical t.
00:15:10.600 --> 00:15:22.800
Here this represents the probability that the null hypothesis, that there is no difference is true and that is =1.
00:15:22.800 --> 00:15:33.200
Same thing here, the probability that the null hypothesis is false and actually there some other distribution we just do not know what that is.
00:15:33.200 --> 00:15:35.800
We will draw it like a ghost with blue.
00:15:35.800 --> 00:15:46.600
It is important to know that this μ is μ sub x bar - y bar because we are talking about SDOD.
00:15:46.600 --> 00:15:49.700
That is why it is a difference of means.
00:15:49.700 --> 00:15:58.500
Once we know this, now what we need to do is figure out what these probabilities mean.
00:15:58.500 --> 00:16:17.300
Here, let me draw the cut off again, here we have our rejection zone and our fail to reject zone.
00:16:17.300 --> 00:16:21.500
Once again we can find those conditional probabilities.
00:16:21.500 --> 00:16:31.800
What is the probability of rejecting given this thing is true, inside of this space where the null hypothesis is true?
00:16:31.800 --> 00:16:39.500
What is the probability of failing to reject given that the null hypothesis is true?
00:16:39.500 --> 00:16:45.200
That is the conditions that we are working under.
00:16:45.200 --> 00:16:48.500
It is still the same.
00:16:48.500 --> 00:16:58.000
Here we see α and here we see 1 – α.
00:16:58.000 --> 00:17:07.300
Ideally when we have these differences between distributions what we really would like is that
00:17:07.300 --> 00:17:11.100
there was very little overlap between these two distributions.
00:17:11.100 --> 00:17:15.900
The null distribution and the like real one that we do not know anything about.
00:17:15.900 --> 00:17:18.300
It will be nice if there was very little overlap.
00:17:18.300 --> 00:17:23.400
But in real life, there is usually a lot of overlap.
00:17:23.400 --> 00:17:34.800
The real world is noisy and the real population might be very, very different.
00:17:34.800 --> 00:17:51.600
The real population might be very similar to the null population.
00:17:51.600 --> 00:17:56.900
If that is the case, there is some overlap between their distributions.
00:17:56.900 --> 00:18:06.300
There are some chances that we might get a score over here and it could be part of the real population or part of the null population.
00:18:06.300 --> 00:18:17.800
If this is the case and we need to understand these conditional probabilities in anyway.
00:18:17.800 --> 00:18:20.500
Get ready here is the deal.
00:18:20.500 --> 00:18:32.500
Instead of writing real population, I am going to say not null population because we do not know what it is.
00:18:32.500 --> 00:18:34.700
It is just not the null population.
00:18:34.700 --> 00:18:41.600
I am going to take this picture, this great curve and I will draw up here in two ways.
00:18:41.600 --> 00:18:43.600
I am going to split it up into two parts.
00:18:43.600 --> 00:19:04.100
One part is going to be this blue part, this fail to reject region and that is that whole part.
00:19:04.100 --> 00:19:07.500
Here I am also going to draw the red part.
00:19:07.500 --> 00:19:18.100
I just draw it separated from each other so that you can see.
00:19:18.100 --> 00:19:27.100
Here we have this little part and that is red and it is red because we have rejected it.
00:19:27.100 --> 00:19:31.100
This is the case where we are actually wrong.
00:19:31.100 --> 00:19:33.700
This is the case where we are actually right.
00:19:33.700 --> 00:19:39.700
Here we are wrong because we rejected the null hypotheses that we should not have rejected.
00:19:39.700 --> 00:19:45.600
Here we are correct, because we fail to reject and truly we should not have rejected it.
00:19:45.600 --> 00:19:53.600
Now that is the case if the null hypothesis population is true.
00:19:53.600 --> 00:20:00.200
What happens in a case where it is not true?
00:20:00.200 --> 00:20:03.500
The null hypothesis is false.
00:20:03.500 --> 00:20:06.600
What happens here?
00:20:06.600 --> 00:20:18.400
Here I am going to draw a different looking picture because I'm going to draw this curve but this curve split up.
00:20:18.400 --> 00:20:27.600
Here I am going to split this curve up like this.
00:20:27.600 --> 00:20:52.900
On this side of the line I am going to draw this little section and draw just this little section.
00:20:52.900 --> 00:20:57.300
That part of it I have failed to reject.
00:20:57.300 --> 00:21:13.800
That is wrong so I am going to color it in red because we should have rejected it but we fail to reject it.
00:21:13.800 --> 00:21:19.300
On the other side, I am going to try the other part of this curve.
00:21:19.300 --> 00:21:31.800
It is this part and here I am going to color that in blue because although we rejected it we should have rejected it.
00:21:31.800 --> 00:21:48.300
Here we rejected the null hypothesis and you are right we should have rejected it because we are in this new unknown population.
00:21:48.300 --> 00:21:50.500
You should have rejected it.
00:21:50.500 --> 00:21:54.000
Let us look at the places where we are correct.
00:21:54.000 --> 00:21:59.600
We are correct here and this is called a correct failure.
00:21:59.600 --> 00:22:11.200
Here we are also correct and this is called a hit.
00:22:11.200 --> 00:22:24.200
Here we are incorrect and that is called a false alarm.
00:22:24.200 --> 00:22:32.200
Here we are also incorrect and this is called a miss.
00:22:32.200 --> 00:22:36.800
It is a miss because we have failed to reject it.
00:22:36.800 --> 00:22:41.000
We failed to hit the target when we should have hit the target.
00:22:41.000 --> 00:22:49.600
Given that, let us see how the distributions and the box go together.
00:22:49.600 --> 00:22:53.100
The false alarm is really that place.
00:22:53.100 --> 00:22:58.400
Remember when the hypothesis is true I am going to draw it in black.
00:22:58.400 --> 00:23:13.000
The correct decision is going to be this whole section where we fail to reject, but that is okay we are in this fail to reject zone.
00:23:13.000 --> 00:23:14.700
You are good to go.
00:23:14.700 --> 00:23:20.700
Here is the other part of this part.
00:23:20.700 --> 00:23:36.300
Here this is an error because we have rejected when we should not have rejected because it is actually true.
00:23:36.300 --> 00:23:38.300
This is our false alarm.
00:23:38.300 --> 00:23:50.200
Now, in the case of a correct decision where you actually hit it, this means you rejected it and it is good
00:23:50.200 --> 00:23:57.900
that you rejected it because actually a different population is true, not this null population.
00:23:57.900 --> 00:24:14.600
That is going to be the area where you reject, all rejections are going one on the right side of this line.
00:24:14.600 --> 00:24:20.900
You should have rejected it because you are in a different population.
00:24:20.900 --> 00:24:22.600
You are not in the null population.
00:24:22.600 --> 00:24:26.600
This is a good thing for you.
00:24:26.600 --> 00:24:28.800
You should have rejected it.
00:24:28.800 --> 00:24:34.100
The other part of that, the other piece of that is down here.
00:24:34.100 --> 00:24:37.000
It is this little piece down here.
00:24:37.000 --> 00:24:50.900
Here it is incorrect, because although you are part of a different population, not the null population, you did not reject it.
00:24:50.900 --> 00:24:53.100
You fail to reject.
00:24:53.100 --> 00:25:00.000
I want you to notice something here.
00:25:00.000 --> 00:25:08.000
All the fail to reject are always on this side of the line because these are values that are less extreme than the mean.
00:25:08.000 --> 00:25:11.600
And the rejection ones are all in this side of the line.
00:25:11.600 --> 00:25:16.800
I could also drawn it two tailed and also showing you the side but I'm showing you one tailed.
00:25:16.800 --> 00:25:24.100
It is all outside of the line, on the outer boundaries of this line, more extreme than the hypothesized mean.
00:25:24.100 --> 00:25:27.200
This is less extreme than the hypothesized mean.
00:25:27.200 --> 00:25:33.600
My hypothesized mean is somewhere here, less extreme than that.
00:25:33.600 --> 00:25:39.100
It is relative to the hypothesized mean.
00:25:39.100 --> 00:25:43.100
That is how these four pictures fit together.
00:25:43.100 --> 00:25:48.600
When you see those two distributions drawn, do not get confused you already know it.
00:25:48.600 --> 00:25:52.700
You just have to break it apart in slightly different ways.
00:25:52.700 --> 00:25:56.300
Let us go on to some examples.
00:25:56.300 --> 00:26:13.000
On the basis of results from a large sample of students from a university, a professor reports the mean high from my sample is not significantly below 60.
00:26:13.000 --> 00:26:15.900
That means he did not reject.
00:26:15.900 --> 00:26:20.700
This is fail to reject.
00:26:20.700 --> 00:26:26.200
If he said significantly that would be rejecting the null.
00:26:26.200 --> 00:26:29.800
Which type of error will this professor worry about?
00:26:29.800 --> 00:26:32.900
He failed to reject, that is important to know.
00:26:32.900 --> 00:26:35.900
What is the only error you can make if you fail to reject?
00:26:35.900 --> 00:26:56.700
Well if you fail to reject, but you should have rejected it, the null hypotheses is false, what kind of error is that?
00:26:56.700 --> 00:27:03.700
That is a missed and a type 2 error.
00:27:03.700 --> 00:27:10.800
The error rates are given by α and β and this is actually β so these are wrong.
00:27:10.800 --> 00:27:22.200
These are both correct rates instead of the error rates and this is nonsense having a non significant results are all error statistically.
00:27:22.200 --> 00:27:23.500
It is never the case.
00:27:23.500 --> 00:27:25.000
You are damned if you do and damned if you do not.
00:27:25.000 --> 00:27:29.000
There is always a way you can make an error either type 1 or type 2.
00:27:29.000 --> 00:27:35.600
Example 2, a researcher worries about trying incorrect conclusion.
00:27:35.600 --> 00:27:42.000
The researcher plan to select a sample of size 20 and to use the .01 level of significance.
00:27:42.000 --> 00:27:43.700
Here α is .01.
00:27:43.700 --> 00:27:56.600
In a two tailed test of the null hypothesis the critical t should be + or - because it is a two tailed test.
00:27:56.600 --> 00:28:00.700
It is + or -2.86.
00:28:00.700 --> 00:28:15.400
If he obtains the t of 2.8 which type of error would he be worried about and why?
00:28:15.400 --> 00:28:23.800
Well, you definitely know that he is not going to reject.
00:28:23.800 --> 00:28:37.600
Fail to reject because this is less extreme than this.
00:28:37.600 --> 00:28:41.700
This is less extreme so he fail to reject.
00:28:41.700 --> 00:28:49.200
The only error you can have when you fail to reject is if you fail to reject given the null hypothesis is false.
00:28:49.200 --> 00:28:53.200
What kind of error is that?
00:28:53.200 --> 00:29:03.700
That is a missed or type 2.
00:29:03.700 --> 00:29:12.000
What if he obtains a t of 2.869 which type of error would he be worried about?
00:29:12.000 --> 00:29:14.300
That is more extreme than this.
00:29:14.300 --> 00:29:16.700
In this case he would reject the null.
00:29:16.700 --> 00:29:20.000
When is he wrong when he rejects?
00:29:20.000 --> 00:29:25.500
When he should have not rejected it because the null hypothesis is actually true.
00:29:25.500 --> 00:29:28.200
What kind of error is that?
00:29:28.200 --> 00:29:36.600
That is a false alarm or type 1 error.
00:29:36.600 --> 00:29:42.300
Example 3, what is the danger of the type 1 error?
00:29:42.300 --> 00:29:45.000
This is a more conceptual question.
00:29:45.000 --> 00:29:54.500
The danger is mistakenly concluding that there is no significant difference between the obtained mean and the hypothetical population mean.
00:29:54.500 --> 00:30:07.900
When you make a type 1 error you have rejected the null but null hypothesis is true.
00:30:07.900 --> 00:30:13.900
Mistakenly concluding that there is no significant difference but that is not true
00:30:13.900 --> 00:30:17.900
because you concluded that there is a significant difference that is why you rejected the null.
00:30:17.900 --> 00:30:26.300
Mistakenly concluding that there is a significant difference between the obtained mean and the hypothetical population mean.
00:30:26.300 --> 00:30:28.800
That is true.
00:30:28.800 --> 00:30:38.500
You mistakenly rejected the null and said there is a significant difference but you should not have done that.
00:30:38.500 --> 00:30:42.700
Mistakenly being alarmed about a hypothesis when you should become.
00:30:42.700 --> 00:30:43.800
That is non sense.
00:30:43.800 --> 00:30:46.800
Mistakenly calculating the wrong test score.
00:30:46.800 --> 00:30:51.500
These errors are not errors that you can actually avoid.
00:30:51.500 --> 00:30:54.400
These are not errors because we were sloppy.
00:30:54.400 --> 00:31:00.100
These are errors that are made because we do not know the real nature of the world.
00:31:00.100 --> 00:31:04.900
This is actually not what we are talking about when we are talking about type 1 or 2 errors.
00:31:04.900 --> 00:31:12.300
Mistakenly choosing the wrong population standard deviation to calculate standard error, that is not it either.
00:31:12.300 --> 00:31:18.000
These two are just regular old mistakes or errors in calculation.
00:31:18.000 --> 00:31:21.500
They are not type 1 and 2 errors of hypothesis testing.
00:31:21.500 --> 00:31:25.400
That is it for type 1 and 2 errors.
00:31:25.400 --> 00:31:27.000
Thank you for using www.educator.com.