Comparing Two Means

Setup

Open up the Body Compsosition.omv file from previous weeks and clear your Results viewer of any previous results, so we can start fresh. Make sure you’ve saved the additions we did in Week 1, with PartID 53.

Independent measures Student’s t-test

For this first test we are going to go back to the Weight and Height data. We have 50 participants in our data set, and you can see from Gender we have two levels, 1 (Male) and 2 (Female). We are interested in these students to see: who is heavier, male or female?

At its very simplest, examining the mean and standard deviation will give us ‘an’ answer to this question. Click Analyses, then Exploration, then Descriptives. Move Weight into variables, and split by Gender. Go into the Plots option and turn on Box plot too.

It should be clear from the boxplot that the females are heavier than males, but notice that the spread of females is greater than males, so simply using the mean (or any other measure of central tendency) does not give the full story.

Assuming we have parametric data here, let’s do the independent samples t-test. In Analyses, click T-Tests, then Independent Samples T-Test. Move Weight into the dependent variables box and Gender into the grouping variable selection box.

Under Tests under the variables selection boxes, make sure Student’s is ticked.

Underneath Tests is the Hypothesis section. We don’t have a prediction of which group might be heavier yet, so make sure Group 1 ≠ Group 2 is ticked. This just means that we are testing is Group 1 (Male) is not equal to Group 2 (Female), but it doesn’t matter which one is heavier than the other, only that they are different.

While we’re here, under Additional statistics tick Mean difference and Effect size boxes, both with the Confidence interval box ticked (leave both at 95%), and Descriptives.

Under Assumption checks tick Homogeneity test.

Let’s look at our Results. Starting with the Group Statistics table, you can see the same basic information that we found in the Descriptives from before (SD - standard deviation, SE - standard error of the mean).

Under the Assumptions heading, we can also see our Levene’s test results for homogeneity of variance.

The Independent Samples T-Test table in our Results viewer shows us the outcome of our t-test. Reading from the top line, we are mostly interested in the p value. Here the p value is 0.015, so there is strong evidence to suggest that the difference in weight between male and female is > 0 kg. The mean difference shows that the average difference in weight between the groups in our sample is 12.7 kg.

Finally, we can look at the effect size. We can calculate Pearson’s correlation coefficient (r) for t-tests, similar to correlation, but the more common reported effect size for t-tests is Cohen’s d, and this is what Jamovi reports. We can interpret the values for Cohen’s d from the Results viewer based on the below table of commonly used criteria:

Now you try: Repeat the independent measures t-test for Height across gender using this data set.

Mann-Whitney U test

Continuing with the Body Composition.sav dataset, we are going to look at a comparison of non-parametric data. We are still comparing two independent groups (males and females), but this time we want to see if there is a difference in the self-perception of fitness between males and females, where the self-perception of fitness is measured on a Likert scale between 1-5 where 1 is completely unfit, and 5 is very fit.

For the non-parametric test, we do not have the strict parametric assumptions of the independent measures t-test. But, as with the parametric data, it is possible to perform a little exploratory analysis. In Analyses, go back into our Descriptives menu.

As this is a nominal variable, we can’t use some of our usual data points but we can still usefully use some. Click Frequency tables, N, Missing, Mode, Minimum and Maximum. Under Plots, click Histogram.

Let’s look at these in the Results viewer.

We can see in the Descriptives that we have one Female that did not provide a fitness perception (missing = 1). We can see that for both groups, we can see with the mode that the most common response was 3 (“Neither fit or unfit”). The minimum value of 2 for Males means that no Male rated themselves as “Very unfit”, and from the maximum that both groups had people who rated themselves “Very fit”. Finally, the frequency tables and histogram both show us how the groups break down across the fitness levels.

Let’s do the comparison test. Go back to the Independent Samples T-Test menu from before. Now, simply untick Students, and tick Mann-Whitney. Turn on Mean difference, Effect size, and both the Assumption tests too.

The table for our independent samples test results can be read in much the same way as before. Instead, we only really care about the p value. Here, p < 0.05, so we can conclude there is a significant difference between the overall fitness perceptions of each group.

The mean difference is 1, and the two means are in the Descriptives section. We can see from the means that Males have an average higher perception of their fitness than Females.

For an effect size, as our data is non-parametric, we can’t use Cohen’s d. Instead, Jamovi uses Rank biserial correlation. Rank biserial correlation is similar to r. They’re both correlations, and like r it can run from 0 to 1. It’s less easy to interpret, but you can treat it like r - the larger number, the better.

Now you try: Repeat the non-parametric Mann-Whitney U for self-perception of fitness. This time, we want to see if there is a difference in the perception of fitness between the therapists (dance and sport) versus the sports scientists (both sports science and sports psyc, coaching and PE).

As we have 4 groups, you will first need to create a new variable called PerceptionGroup. We can use the Transform menu to create a new variable that gives participants a 1 if they are on a therapy course or a 2 if they are on another course, and label 1 ‘Therapy’ and 2 ‘Other’.

Click Data, then Transform. Call the variable PerceptionGroup and give it an appropriate description. Set the Source variable to the Course variable, then click the using transform dropdown menu and click Create new transform.

In this new menu, we need to create a formula to tell Jamovi how to create our new variable, similar to the Compute menu from the Descriptives worksheet. Call the new transform something useful and give it an appropriate description. Then, click Add recode condition.

Set our recode condition (the top box) to say that if the Course ID is less than or equal to 2 (remember Sport Therapy is 1 and Sport and Dance Therapy is 2), use the word ‘Therapy’ in the transformed variable. Otherwise, use the word ‘Other’.

Press enter, then use the arrows in the top right corners of the Transform menus to close the menu. You should see the new column named PerceptionGroup with ‘Therapy’ or ‘Other’ written in it.

Now run the non-parametric Mann-Whitney U for self-perception of fitness between the therapists (‘Therapy’) versus the sports scientists (‘Other’).

Setup

The repeated measures t-test allows us to compare two experimental conditions where the same participants take part in both conditions. We are going to examine some sit and reach data, so open Sit and Reach.omv.

In this study, 20 participants were selected, 10 male, 10 female. Their sit and reach score was measured (in cm) at the start of the test (SnRPre) and after a 5 week intervention, their flexibility was tested again (SnRPost). A score > 0 means the participant can reach further than their toes, < 0 means they cannot reach their toes.

Repeated measures Student’s t-test

Assuming that the data satisfies our parametric assumptions we will run a paired samples t-test. Click Analyses, then T-Tests, and Paired Samples T-Test.

This new menu should look very similar to the independent measures t-test from before. The main difference is that we need to add two variables to the Paired Variables box, and they appear side-by-side to show that they are the two variables being compared.

Move both SnRPre and SnRPost into the box now. Tick Student’s, and then Mean and Effect size with their confidence intervals, Descriptives, and Normality test, similar to the independent test.

We are mainly interested in the first and third output table in our Results viewer. The Descriptives gives us the mean and standard deviation (SD) of the two groups, with the final column giving us the standard error (SE) of the mean (which shows us how far the sample mean deviates from the population mean – generally the smaller the better).

The first table then gives us our t-test result. Let’s look at it in more detail, going through each results column from left to right:

• Statistic: the actual t-statistics. This is the test statistic which Jamovi uses to calculate the significance. The t-statistic is based on the mean score (-1.97) divided by the standard error of the differences (0.481).
• Df: degrees of freedom – basically sample size minus 1.
• p: Significance of the t-test. In this case, we are saying is there is only a 1% chance that this value of t this big could happen if the null hypothesis were true (i.e. that the means are equal). Alternatively, as the p value is < 0.01 we should reject the null hypothesis at the 1% level, suggesting there is strong evidence against the null hypothesis and that we should accept the alternative hypothesis that there is a difference in the pre and post sit and reach results. From the means of the two groups we can infer that the post sit and reach value is greater than the pre sit and reach score.
• Mean difference: This is the mean difference of the two variables. As the post sit and reach score is generally greater than the pre sit and reach score, our mean difference is negative, or rather the average increase in sit and reach score is approx. 2.0cm.
• SE difference: The standard error of the mean.
• 95% confidence intervals: The range of values that the mean difference of the population is 95% likely to be within, given the statistics calculated from our sample. As we only have a sample of data, these two fields give us a 95% confidence interval for the population mean difference, so if we took 100 such samples, then the mean difference of 95 of those samples would be between 0.97 and 2.98 cm. Because this range does not cross zero, the interval here tells us that based on our sample the true value of the difference between the means is unlikely to be zero. Therefore we can conclude that the difference is greater than zero, backing up our p value result.
• Effect size and confidence intervals: Similar to Cohen’s d in the independent t-test. Here, the effect size is d = 0.92, a large effect size.

To report the paired samples t-test results, we could write this as:

On average, participants experienced greater sit and reach scores after intervention (M = 8.10, SE = 1.11) compared to before intervention (M = 6.13, SE = 0.94), t(20) = -4.106, p = 0.001, d = 0.92.

Now you try: Keep with the same data set, but only examine the sit and reach scores of the Male participants. To do this, we will need to create a Filter on our data, to only include participants where Gender is Male in any tests we run.

Click Data, then Filters (the icon looks like a funnel). This is simpler than the Compute/Transform variables - just type the filter condition straight into the function box. We can use either the numbers underlying our Gender data, or just type the label we want itself. So, enter in the function box:

Gender == 'Male'

Notice how the Filter has been applied in a new column named Filter1. There should be green ticks next to all the Male participants, and red crosses next to all the Female participants. This shows us that only the Male participants will be included in our tests.

Now run the paired-samples t-test again.

Wilcoxon signed-rank test

If the data is not normally distributed, then we need to choose a different test. Here we will use the Wilcoxon signed-rank test.

In this test, we are going to take the Sit and Reach value, but add in a new score of overall flexibility. What we would like to do is see if our intervention affected the overall flexibility of the participant, not just their sit and reach score. Don’t worry about how this is measured, except that it is on a continuous scale, so can be considered high level data.

Open the Flexibility.omv data file. This file is similar to the sit and reach data, but has the additional measures of pre and post flexibility.

Use the Compute variable we used before (for BMI, etc) and the Descriptives menu to obtain the Shapiro-Wilk results for the difference scores between Pre-Flexibility and Post-Flexibility. You should find that p = 0.046 for the difference, so we have moderate evidence against the hypothesis that our data is normally distributed. Our assumption of normality has been violated and we should run a non-parametric test.

Let’s run the test. Go into the paired samples t-test menu as before and add FlexibilityPre and FlexibilityPost to the paired variables box. Untick Student’s, and tick Wilcoxon rank. Turn on all the same other options we’ve used before.

We can read our results much the same as our previous tests, only this test uses the W statistics rather than t.

The null hypothesis for this test is that when the samples are ordered there is no difference in the median position, i.e. the two data sets are ‘the same’. As we have a p-value of < 0.01, there is strong evidence against the null hypothesis, i.e. there is evidence to suggest that the two distributions are different. We can conclude that the overall flexibility is different in the pre- and post-intervention.

Finally then, we just have to write up our results, so we could report something like:

For our sample of students, overall flexibility was significantly higher post intervention (Mdn = 8.35) compared to pre-intervention (Mdn = 8.55), W = 33.000, p = 0.008, rank biserial correlation = 0.69.

Now you try: Keep with the same data set, but only examine the difference in flexibility scores of the Female participants using a filter as we did earlier. We will assume the data is not normally distributed.