Background

At the heart of most psychological research is the basic knowledge that interpreting data is frought with difficulties. One of the difficulties is the finding that in most situation, the findings are not clear. Is there a relationhip? Is there a difference between conditions? There is usually so much variability in the responses that we need help. That is the role of inferential statistics. It is important to note how this sentences was said. Inferential statistics help. Researchers are still ultimately responsible for making decisions about their study.

When running an inferential statistical test, you run it against the null (H0) hypothesis. The null hypothesis is the hypothesis that all the variation of the data in your study is due to chance and chance only. Ugh, nothing happened. Using your data, you can estimate the possibility of getting different statistical values under this random, noisy situation. At the end of the statistical test, you are presented a p value which is the probability that the null hypothesis is true. In other words, the p value is the chance that your data is just noise and random variations. Meaningless. So, you want the p value to be small. We don't want to say that meaningless data has a relationship or difference do we? But how low is low? Before you start the study, you need to set a criterion value or cuttoff value, called the alpha value. Traditionally, we set the alpha level to 0.05. To use the alpha level, if your p value is less than your alpha (p < alpha), the researcher rejects the null hypothesis. Otherwise, the null hypothesis is not rejected. Read those last few sentences very carefully and notice the language.

So, what do you do if you reject the null hypothesis? It all seems a bit up in the air. Well, there is a second statistical hypothesis, the alternative hypothesis (Ha) which is that there is a statistically significant relationship or difference in your data. Your data is not due to chance, but is interpretable. This outcome is the preferable one as it means you can interpret your data.

To review, there are two very general hypotheses, the null hypothesis that your data is due to chance and the alternative hypothesis that your data is not due to chance and is interpretable. Only one of the hypotheses is true and your p value will let you either reject the null hypothesis (and accept the alternative) or not reject the null hypothesis. The table below will review these situations.

Table 1 Possible outcomes of Inferential Statistics
		The Null Hypothesis (H0) is :
		False	True
You Decide H(0):	p less alpha Reject	Reject False H(0)	Reject True H(0) Type I Error
	p not less Alpha Not Reject	Not Reject False H(0) Type II Error	Not Reject True H(0)

From this table, you can observe that there are two correct possibilities: reject false H0 and not reject true H0. There are also two errors: Reject a true H0, a type I error, and not reject a false H0, a type II error. When the Null hypothesis (H0) is true, the alpha value you have selected gives you the rate of a type I error. It is much more complicated to estimate the rate of a type II error if the alternative hypothesis is true. In this activity, you will be able to see different rates of type II errors given different possible rates of data that could result if the alternative hypothesis is true.

In this activity, you can run imaginary studies that will give you the p value for the sample. Without your knowing it, either the null or the alternative hypothesis is true. Using the p value and the alpha level that you can set, the null hypothesis will either be rejected or not. It is imporant to see that the p value will never be 0 so it is always possible that the null hypothesis is true. This possibility is one reason that it is important to repeat a study, to do replication. You can repeat the study easily as many times as you wish in this experiment and the app will count the number of times that you reject or not reject the null hypothesis. When you have done enough repleicaton, you will, unlike reality, be able to see whether the null or alternative hypothesis was true. See what happens.

Instructions

You can collect data in this model. Whether the null or alternative hypothesis is true is rancomly selected.

Definitions

Just to review, the following terms below are important to know:

Alpha level: the pre-determined level that if the p values is less than this value the research will reject the null hypothesis.
Alternative Hypothesis: the statistical hypothesis that the data obtained in an experiment is not due to chance and is therefore interpretable.
Null Hypothesis: the statistical hypothesis that the data obtained in an experiment is due to chance.
p value: the probability that the null hypothesis is true.
Type I Error: rejecting a true null hypothesis (H0). If the null hypothesis is true, the probability of a Type I Error is the alpha level.
Type II Error: not rejecting a false null hypothesis (H0).

Illustration Tab

Settings

Below is a list of the interaction elements of the model. The settings include the following:

Collect Sample: run a 'study' and statistic. The obtained p value will be reported below the interaction elements.
New Experiment: restart the "data collection." Whether the null or alternative hypothsis is true will be randomly selected.
Effect Size: Effect size is a measure of how different the null and alternative hypotheses are from each other in what p values will be obtained. The larger the effect size is, the more different the data from the alternative hypothesis is from the null hypothesis. Chaning the effect size resets the experiment.
Alpha Level: chose the alpha level for your statistic. You can use the buttons below the slider for some predetermined alpha levels. Changing the alpha level resets the experiment.
Show: Null (H0), if selected the possible values for the statistic from the Null hypothesis will be shown with the higher the curve, the greater the probability of that value. Alternative (Ha), if selected (and not always possible) will show the range of possible statistic values from the alternative hypothesis.
Show Effect Size: if both null and alternative hypothesis curves are displayed, selecting this option will draw a line between the two curves to show the effect size.
Show True Hypothesis: select to show the true hypothesis, null or alternative. This option cannot be selected until several samples have been collected.
The checkboxes in the table when selected will highight the region associated with that outcome using the color indicated in the table.

Reset

Pressing this button restores the settings to their default values.