Choosing the right test

I often get asked to explain how to choose the right statistical test. Being able to explain why a test is appropriate for a given set of data is a skill that it is essential to learn for PSYA4 Research Methods. This handy guide should help....

Choosing a statistical test
There are several factors that must be considered before a statistical test can be chosen. Firstly the research design, aim and level of measurement must be identified:

Research design
Data can be either related or unrelated. Related data is produced from repeated measures and matched pairs designs. Unrelated data is produced from independent groups designs.

Research aim
Is the aim of the research to investigate a significant difference or a significant association? If, for example, there are 2 groups of participants, each in a different condition of the independent variable (e.g. in an experiment), then the aim is to test for a significant difference. If the aim is to test for a correlation between two variables, then the aim is to test for a significant association.

Level of measurement
Data can be produced at nominal, ordinal and interval levels:
  • Nominal data is the most basic level of measurement. An example is a frequency count of a distinct category, such as the number of aggressive and non-aggressive acts in an observation.
  • Ordinal data consists of a list of data that can be ranked in order, but not data that would fit to an interval scale. An example is the subjective rating of happiness (on a scale form 1 to 10) that participants may score themselves as on a questionnaire. A happiness rating of 10 is higher than 5, but it is not twice as happy as 5 or 5 times as happy as 2.
  • Interval data is measured on a scale in which each interval is exactly the same size. Time is interval data because each second is the same duration, and 10 seconds are twice as long as 5 seconds.

Statistical tests
Once the design, aim and level of measurement have been identified, the correct inferential test can be chosen.

Spearman’s rho
Spearman’s rho is a test for significant association, and produces a correlation coefficient. The level of measurement must be either ordinal or interval. The research design can be either related or unrelated.

Wilcoxon signed ranks test
Wilcoxon signed ranks is a test of significant difference for related data. The research design must produce related data (e.g. repeated measures or matched pairs). The level of measurement can be either ordinal or interval.

Mann-Whitney U test
Mann-Whitney U is a test of significant difference for unrelated data. The research design must produce unrelated data (e.g. independent measures). The level of measurement can be either ordinal or interval.

Chi-square test
Chi-square tests for difference when the data is nominal and unrelated. The research design must produce unrelated data (e.g. independent measures). The level of measurement must be nominal (e.g. categories).

inferential-tests-table-2

blog comments powered by Disqus