Section 7.1: What are inferential statistics
From Research Methods in Psychology
Contents |
[edit] HOW WILL YOU BE ASSESSED ON THIS
You are unlikely to be assessed directly on this material. You need to understand the material that is presented in this chapter if you want to have a thorough grasp of the information in the next chapter, and to have a better understanding of the way that inferential statistics are used in psychological research. This understanding will help you when it comes to evaluating research that you read about, and research that you carry out yourself. I should warn you that the material in this chapter is quite difficult, and it is possible to survive without a good grasp of this chapter.
[edit] What are Inferential Statistics
IN THIS SECTION, WE INTRODUCE SOME OF THE IDEAS INVOLVED IN INFERENTIAL STATISTICS. WE WILL HAVE A LOOK AT PROBABILITY, AND SEE WHAT IS MEANT BY PROBABILITY, BECAUSE THIS IS THE FOUNDATION OF INFERENTIAL STATISTICS. THEN WE LOOK AT STATISTICAL SIGNIFICANCE, AND SEE HOW STATISTICAL SIGNIFICANCE RELATES TO THE NULL HYPOTHESIS, WHICH WE LOOKED AT IN CHAPTER 1.
In the last chapter, we looked at descriptive, or exploratory, statistical analysis. We use exploratory data analysis to explore and describe the data that we collected, using (measures of central tendency (such as the mean) dispersion )(such as the standard deviation) and appropriate graphs. If we were only interested in the people that we studied, our sample, then they would constitute our whole population. We would not need or want to do any inferential statistics; we could stop there. However, we are interested in much more than just our sample - we want to be able to generalise our sample to the population from which we obtained our sample.
In other words, we want to know if the results that we found in our sample are likely to be true in the population. This process is called inferring.
| Crucial Concept: To infer is to derive a conclusion from facts - if we see smoke, we can infer that there is a fire. |
Of course, we can never be sure that there is a fire, there may be a smoke making machine, or we may have seen steam and mistaken it for smoke. Similarly, in psychological research we can never know for sure that a result for the sample will also be true for the population. We can never know absolutely for sure, but we can make an educated guess about whether the sample result will be true of the population we were sampling, and this is where we need inferential statistics.
Inferential statistical analysis is all about probability. We cannot, as mentioned above, speak in terms of absolute truth or falsehood when we are speaking about the natural world. Instead, we can say that something is likely to be true, or probably true. We can also say that something is likely to be false, or that it is probably false.
Therefore, when you use inferential statistics, you are talking about the likelihood of things – how likely they are to happen. To do this, we need to understand a little bit about probability theory.
[edit] A Bit On Probability
There are many different ways of expressing a probability: people tend to say things like “10%” or “1 in 10” chance.
|
Crucial Concept: In statistics, probabilities are expressed as numbers between 0 and 1: 10% or 1 in 10 chance would be expressed as a probability of 0.1. |
It is a good idea to get used to understanding the relationship between these different ways of saying the same thing. As a practice, have a go at filling in the blanks in the following table (answers over the page). It is not a good idea to express probabilities in terms of odds, e.g. 10 to 1, because this can be confusing and ambiguous (see Note for Gamblers).
| Probability | Percentage | Proportion |
| 0.25 | 25% | 1/4 |
| 0.40 | 40% | 2/5 |
| 0.80 | 80% | 4/5 |
| 0.75 | 75% | 3/4 |
| 0.70 | 70% | 7/10 |
|
Note for Gamblers Betting shops tend to express the probability of an event in terms of odds. This makes it easy to work out how much you will be paid, but can be confusing in terms of probabilities. What are the odds of a horse winning, which has a probability of winning of 0.25 (i.e. one in four)? If you said 4:1, you were wrong. The answer is 3:1. This is because when you say “1 in 4” you mean that in four races, the horse will win once. The betting shop says that it will lose three races and win one. The same thing, but rather a different way of putting it. |
If an event has a probability of 0, it will not happen. No matter what. If an event has a probability of 1, it will happen. No matter what. An event cannot have a probability less than 0, or greater than 1. The probability of an event occurring is written as p(event). It is calculated using:
p(event) = (number of events which satisfy criterion)/(total number of events)
When it comes to tossing a coin, there is one event that satisfies our outcome (heads) and one that does not (tails), making two possible events. We know that the probability of tossing a coin and it landing heads is equal to 0.5 or 1/2 because heads is one outcome out of two possible outcomes.
|
WARNING: This section that follows is fairly tricky. |
[edit] Statistical Significance
|
Crucial Concept: A statistically significant result is a result which would be unlikely to occur, if the null hypothesis were true. |
Some words used in everyday life have a specific technical meaning in psychology. One of the most common of these is the word significant.
When we have carried out a study, and collected our data, we analyse the data initially using the descriptive statistical techniques that were described in Chapter 7, which we use to describe the data.
We may want to know whether these results are generally true, or whether we have just been lucky: we want to know if we can generalise these results to the population. What we want to know is, are our results statistically significant. Before we go on to discuss this, there are two points to note about statistical significance.
- First, people shorten this phrase and just write ‘significant’ when they mean ‘statistically significant.’ Doing this is slightly dangerous; because results may be ‘significant’ without being ‘statistically significant,’ or results may be ‘insignificant,’ despite being ‘statistically significant.’
- Second, a result that is not statistically significant is not ‘insignificant,’ it is either non-significant, or better, it is ‘not statistically significant.’
[edit] Statistical Significance and the Null Hypothesis
We looked at hypotheses in Chapter 2, when we considered the scientific method. We will have a quick look again in this section, but if you have not read Chapter 2, you might find it easier if you read the relevant section before tackling this section. When psychologists want to find something out, they set up a hypothesis, which reflects their theory, and a corresponding null hypothesis, which contradicts their theory and can be tested.
Let’s say that you decide to study the hypothesis ‘Psychology students who work harder at statistics have fuller, happier lives.’
You sample a group of psychology students (see Chapter 6 for discussion of how you might do this) and divide them into two groups. The first group, the High Work Group spend more than 5 hours per week working on their statistics. The second, the Low Work Group spends less than 5 hours per week working on statistics. You then measure how full and happy the students’ lives are.
Using the techniques the descriptive techniques and graphs described in Chapter 7, you can compare the two groups. Let’s say you your findings are the statistics shown in Table 1. They tell us that the students in our sample who spent more time working on statistics had fuller and happier lives (according to our measure). However, we are not just interested in our sample – if our results are only true for our sample, that is not much of a result. We want to know if this result is generally true – is it true for the population that we drew our sample from? Here are the means:
| Mean Score | SD | N | |
| High Work Group | 20.0 | 10.0 | 25 |
| Low Work Group | 10.0 | 10.0 | 10 |
To find out if this result will be likely to be true in the population, you use inferential statistics. We carry out a statistical test (more on that later on, and in the next chapter). In this case you would carry out a t-test (don’t worry about why for now), and you would find that t=5.34.
This result does not mean anything to you. Students often don’t like that idea – they insist that it must mean something and that they should understand what it means. You shouldn’t and you won’t.
Other people (again, those people who are cleverer than we are) have looked at the distribution of t under the null hypothesis. They have calculated how likely different values of t will occur if the null hypothesis is correct. We don’t need to understand what they did, or how they did it. We want to know how likely it is that values of t as large as 5.34 (our t score) will appear if the null hypothesis is true. The t distribution looks a little like the normal distribution, but unlike the normal distribution, which is always the same shape, the shape of the t distribution depends on something called the degrees of freedom (df), which we will encounter later on.
You can either by looking it up in a table, or if you use a computer to calculate t, it is likely that the computer will tell you the probability associated with values of t that large.
When you have found the value of t, you will find the associated probability. This will tell you the probability of getting a value of t that large, if the null hypothesis is correct. As we will see later on, this is not the same thing as the probability of the null hypothesis being correct. The probability value associated with the statistical test we carried out is very small – less than 0.0001. I know this probability from either looking it up in a book, with the appropriate table in it, or because the computer program that I used to calculate t also told me the probability associated with it.
Before we do the study, we should decide upon a probability that will be low enough for us to reject the null hypothesis. This value is known as alpha (the Greek equivalent of the letter a, written as α) If the probability is below alpha – usually less than 0.05, then we can reject the null hypothesis. We can say that it is unlikely that we would have found these results if the null hypothesis were correct. In short, a statistically significant result is a rare result – a result we would not expect to get, if the null hypothesis were true.
To re-emphasise, the probability value (p value, for short), or significance value, is the result of most statistical tests. It is often what we are most interested in, because it tells us whether our theory was supported (if p is less than 0.05) or not (if p is greater than 0.05).
[edit] Directional and Non-Directional Hypotheses
|
Crucial Concept: A one-tailed hypothesis (or directional hypothesis) says that there will be an effect, and specifies the direction of the effect. A two-tailed hypothesis (or non-directional hypothesis) says that there will be an effect, but that effect could go in either direction. |
Hypotheses can be of two kinds. A hypothesis can specify that an effect will happen (i.e. that there will be a difference between two groups) without specifying the direction that difference will be in. This is known as a non-directional, or a two-tailed hypothesis. Alternatively, a hypothesis can specify both the effect and the direction of that effect. This is known as a directional, or a one-tailed hypothesis.
Example of a one tailed hypothesis: Participants who are given alcohol will have better performance at a task than participants given a placebo.
Example of a two-tailed hypothesis: Participants who are given alcohol will have different performance at a task than participants given a placebo.
You should only see one tailed hypotheses rarely. If you propose a one tailed hypothesis, and the result is in the opposite direction to that which you expected, you must still fail to reject your null hypothesis. In the example above, if you have a one-tailed hypothesis, and you find that alcohol improves performance, you must treat that the same as if you found no effect.
[edit] Conditional Probabilities
To understand what we mean by the probability value associated with a statistical test, we need to understand something called conditional probabilities.
A probability value associated with a statistical test tells us the probability of that result given that the null hypothesis was true. What we did not consider was the probability that the null hypothesis was true given that result. This seems like a bit of a strange thing - it may sound as though I wrote the same thing twice, and then I said that the two things were different. So let’s have a closer look.
Here is a concrete example:
There is a club frequented by students, called Henrietta’s. Monday night is student night, and alcoholic drinks are cheap. Therefore, most people who are in the club on a Monday night are drunk.
We can say that the chance or probability of a person in Henrietta’s being drunk are quite high. We could say the same thing in another way: that the probability of someone being drunk, given that they are in Henrietta’s, is high. If we wanted to write this more mathematically, we could use symbols, to show where people were, and what condition they were in. So:
D means Drunk
H means Henrietta’s.
| is a mathematical symbol which means ‘given that.’
p is short for probability
If 80% of the people in Henrietta’s are drunk, the chances are 0.8 that any one person in Henrietta’s is drunk (because 0.8 is another way of writing 80%).
We could write
p(D|H)=0.8
- which is a quick way of saying that the probability of someone being drunk, given that they are in Henrietta’s is quite high.
However, there are many other places where people get drunk on a Monday night - there are other clubs, pubs, and people get drunk at home. So just because someone is drunk, it doesn’t mean that they are in Henrietta’s. In fact, probably only 1%, or 1 in 100 of the people in the city who are drunk on a Monday night are in Henrietta’s. So the probability that someone is in Henrietta’s, given that they are drunk is only 0.01.
Using the same symbols as we used before we can say that:
p(H|D) = 0.01.
This means that the two probabilities are not the same - if you turn the conditions around, they are not equal. We can write this as:
p(H|D) ≠ p(D|H)
(≠ is a symbol which means ‘not equal to.’) So what has this got to do with us? When we have done a statistical test, we get a probability associated with our null hypothesis. It is tempting to think of this probability as a probability of H0 being true, given have the data that we have collected. We could write this as:
p(H|D)
It would be very nice if this were the case, because it would allow us to make very strong statements about our theory, and our hypothesis. Unfortunately, we can’t make statements like p(H|D). What we can say is that we have a probability of having the data that we have collected, given that the null hypothesis is true. We can write this as
P(D|H)
And just like before:
p(H|D) ≠ p(D|H)
If we were able to talk about p(H|D) we would be able to make strong statements about the null hypothesis. We would be talking about the probability of the null hypothesis being true. However, we are not talking directly about the probability of the null hypothesis. We are only talking about the probability of the data, which (as we have seen) is not the same thing, and does not allow us to make such a strong case.
[edit] Summary
In this section we have introduced some of the ideas behind statistical significance. First, we considered probability, which is the likelihood of an event, and ranges from 0 to 1. Then we considered what is meant by statistical significance. Statistical significance is the probability of the result we have occurring, if the null hypothesis (in the population) is true. A statistically significant result would be rare, if the null hypothesis (in the population) were correct.
We looked at two issues in statistical significance: we considered that a statistically significant result can relate to a one-tailed (directional) hypothesis or a two-tailed (non-directional) hypothesis. A two-tailed hypothesis is usually preferable. We then looked at conditional probabilities, and we considered that a statistically significant result does not tell us about the probability of our null hypothesis being correct, as we might be tempted to believe, rather it tells us about the probability of the result, given that the null hypothesis is correct. This is a difficult concept to understand, but means that we should be careful not to say that we have proved, or disproved a hypothesis.