Assignment 4 Example
From Visualization Sp06
I was interested in seeing if there were significant differences in response time due to the visualization type presented. First, I grouped the response time data into three columns, separated by the visualization type. The means and standard deviations of these groups are shown in Figure 1, showing that on average bar charts outperformed pie charts which in turn outperformed tables. However, the deviations show a high overlap of the distributions. To test for significance I performed a Single Factor Analysis of Variance (ANOVA). The ANOVA results show a significant difference indeed exists between the groups (F(2,4317)=8.871, MSE=2603422, p<0.05). Having found a significant difference, I then performed post-hoc paired t-tests to look for pairwise differences between the sample groups. All differences were found to be statistically significant: bar chart and pie chart (t(1439)=-2.273, p<0.05, two-tailed), bar chart and table (t(1439)=-5.678, p<0.05, two-tailed), and pie chart and table (t(1439)=-2.036, p<0.05, two-tailed).
Notes: The above statistics were computed using the Data Analysis options within Microsoft Excel. For the ANOVA, the F values and MSE (Mean-Square-Error, which is the same as the Within-Group Mean-Square (MS) estimate of variance) are provided by the software, as is a more specific bound for the p-value, but here I report only with respect to our target significance threshold of 0.05. The F value is reported with the degrees of freedom (df) for both Between Groups and Within Groups, in that order -- e.g., F(dfBG, dfWG).
For the t-tests, I used the results of a two-tailed test, so as not to presume any particular relationship between the means of the samples a priori. The t-statistic appears in the t(...) block and is reported by Excel as the "t Stat". The p-value used was the value labeled by Excel as "P(T<=t) two-tail". Again, the value provided by Excel provides a more specific bound, but the statistics are presented here with respect to our target significance threshold of 0.05.
