Graphic Design and Gestalt Principles
From CS160 User Interfaces Fa06
Lecture on Nov 8, 2006
Readings
- How To Interpret Experimental Results. Doing Psychology Experiments. Chap 2. Martin.
You should skim the next reading. It looks longer that it really is. You do not have to comment on this one (unless you really want to), but please do read it.
- Organization and Visual Structure. Designing Visual Interfaces. Chap 4. Mullet and Sano.
Ramy Ghabrial - Nov 06, 2006 07:22:28 pm
How to interpret experimental results: This reading is a continuation of what the professor was talking about in class today. I do not think it is not very useful for our project, since we will not be testing enough users to really employ any statistical analysis. However, it is important to HCI and for testing interfaces in real-world scenarios, where more users (and money to entice them with) are available. This is also not the first time I have seen this material, and I suspect it will not be the last, so this information is also useful in other fields and classes -- pretty much any field that deals with data, in fact.
Maksim Lirov - Nov 07, 2006 12:45:46 am
How To Interpret Experimental Results: This reading covered how to analyze collected data from a qualitative experiment and how to present these results. I thought the section on "Inferential Statistics" was very interesting and relevant to our Pilot Usability Study assignment. We will only be interviewing around 3 people regarding our interactive prototype, but it would be helpful if we could predict how a bigger population would react to our design. The concepts of mean, median, and mode could be used for statistics such as number of errors made, time needed to complete each task, etc.... Ultimately, I agree with Ramy's comment above that these statistical techniques will not be very useful for the purposes of this class but undoubtedly are useful in studies involving a bigger testing population.
Qingyun Tang - Nov 07, 2006 12:04:53 pm
How To Interpret Experimental Results: Again this article gives a brief statistical review. It talks about a lot of statistic concepts such as mode, mean, average, standard deviation, scatterplot, etc. I suspect we will not be able to use much of it for our project or the coming assignments. Since we only have three test users, there is no point to calculate standard deviation and other statistical data that is supposed to be applied to a large group. However, I also found the bean blight example in inferential statistic section to be interesting. And learning these statistical analyses for the data would help if we want to work in HCI field in the future. UI designers must know these to perform their jobs in order to satisfy users’ needs.
David Hoffman - Nov 07, 2006 12:39:21 pm
How to interpret experimental results: This chapter discusses the value of statistics to analyze a data set. It doesn't actually discuss any of the specific statistical procedures to use, but instead gives some of the theory behind why a statistical analysis could prove useful. Martin strongly emphasizes that the strength of statistics is not prove that something is as we describe it, but more that it could only be as we have described it due to chance some small percentage of the time. Thus statistics is a powerful way of describing a difference as being worth appreciating, as opposed to be an artifact of an incomplete or flawed data collection procedure. The paper also includes a very cursory discussion of correlation and basic data presentation. The most valuable thing which it reminds the readers is that the computer is a stupid tool, and can give spurious results if the data is not fed into it in the exact format that it is configured to accept. This makes visually inspecting the data an important aspect of a scientific study.
Jonathan Yen - Nov 07, 2006 06:38:23 pm
How to Interpret Experimental Results: A good amount of the information in this reading seems to be review, but it is still good stuff to know. While it is true that we probably won't be dealing with large amounts of data in our experiment, I think that the concepts are important to keep in mind for the long run. Probably the most interesting thing that I got out of this reading was the section on inferential statistics and the section about misinterpreting statistical tests. In general, I think this reading gives some good practical knowledge to keep in mind for the long run.
David Eitan Poll - Nov 07, 2006 10:44:09 pm
How to interpret experimental results: For me, much of the content of this chapter was review. That's, of course, not to say that it wasn't useful. It seems the theme of some of the recent readings has been ensuring validity and accuracy in experimentation, and reinforcing good practices. I think this reading also helps provide some of the tools and vocabulary for describing the results of experimentation accurately and demonstrating both visually and verbally these findings. The inferential statistics section was also very interesting and important, encouraging a "read-between-the-lines (but do so accurately and carefully)" approach.
Andrew Tran - Nov 07, 2006 10:35:21 pm
How to interpret experimental results: This chapter was a great refresher on things learned in high school. I have completely forgotten about the shapes of graphs, such as being normal (bell curve), bimodal, skewed, or truncated. It was also a good refresher on how the standard deviation is computed. If your forgot or never learned how to interpret graphs, this chapter helped explain how to perfectly. Some things i was expecting from this chapter but was never mentioned was how to interpret results that are not pure data and numbers. It is possible to have results from experiments that cannot be statistically analyzed. I can't believe that the level of significance is that low, being less than 5 or 1 percent, and anything higher the majority of psychologist would throw out. I agree with Martin when he says experimenters need to determine the importance of the results. With the techniques mentioned in the chapter on determining important results, those results are useless if experimenters find them to be non-important.
Bowen Li - Nov 07, 2006 11:43:55 pm
Interpreting Experimental Results: This article kind of glosses over the mathematics of what a normal distribution is and what it can be used for. As someone who has taken probability and statistics in a more mathematical setting, it would be interesting for me to learn more about their application in this setting, so I'm kind of disppointed that this doesn't go into more detail. Overall it felt like the chapter was trying to cover a lot of technical detail without going into too much detail about the mathematics. I think it would be too laymen for technical people, and too technical for the casual reader. Also, this section doesn't mention percent errors at all, or how to handle them, which I think is vital when looking at data from experiments.
Kang Chen - Nov 07, 2006 11:19:57 pm
How to interpret experimental results
This chapter of the Doing Psychology Experiments covered many topics from statistic courses I've taken in the past. Although it's an extremely extended version of it, I found the chapter to be a good review. Particularly, the different types of plots and graphs might aid in presenting the survey data from our project in an easy to interpret fashion. As mentioned by many of the previous comments posted, the section on inferential statistics was the most engaging section. I personally found "misinterpreting statistical tests" to be very useful. Often, we would see a lot of numbers being provided in studies or reports to try and convince the public of the validity of the experiments. However, it's also crucial to think about the practical significance and not simply the statistical significance.
Antonis Mannaris - Nov 08, 2006 12:47:42 am
How To Interpret Experimental Results
I hate to just be repeating what my classmates said, but it is true that for our usability study we don't have enough users and results to go into statistical analysis. Nevertheless, this type of analysis has a wide spectrum of applications so it is very useful to be familiar with it. As it is mentioned in the article, this type of analysis can be largely automated and quite often one may see trends that are hard to predict or see by manually analyzing the results. Equally important to deriving conclusions however is not to derive false conclusions. Such a mistake could really send your efforts into a completely wrong direction. Finally, I am a bit concerned about the fact that performing experiments on a new design has the bias of "a new design". How can we accurately account for extended user experience (more than just repeating the task twice)?
Cheng-Lun Yang - Nov 08, 2006 01:00:42 am
Like last reading, the content of this chapter is high school level. Mean, median, mode, and some other terms were learned in stat class and is pretty common knowledge for everyone, especially the mean and standard deviation for us students who are forever haunted by grades. Most of the material presented in the chapter are useful for experiments that have larger and more randomized target groups. For our project at this phase, we will not be able to get as large and diverse group to utilize the theory mentioned in this chapter. But what mentioned in the chapter is happening in everyday life, just look at the midterm distribution on the slide a couple lectures ago.
Utsav Shah - Nov 08, 2006 01:06:28 am
How to interpret experimental results:
This reading answered most if not all of my statistics questions. I think the author did a good job explaining different kinds of graphs and forms of analysis. It always helps to have some methods to represent your data and analyze it, this reading provides just that. It's kind of difficult to use these methods for our project but all in all the reading was very enjoyable and informative. I particulary enjoyed reading Inferential statistics concept and using the computers to interpret the results section.
Julius Cheng - Nov 08, 2006 04:54:08 am
Again, this is another reading concerning useful information I already somewhat know, yet it serves a good purpose as a reminder. I haven't really done statistics since high school, but it came back to me after having read this. These past readings have definitely reminded me of the need for thorough and correct testing. I would personally enjoy a little bit of heavier math, so I can feel like I've learned some new concepts, but I won't complain that our readings aren't hard enough.
Chen Chang - Nov 08, 2006 05:28:36 am
How To Interpret Experimental Results: This reading brought back memories of a high school introductory statistics course. The material presented is very useful (mean, median, mode, variance, standard deviation, graph interpretations, etc) but at the same time it was just review and a good refresher for me. I think regardless of anyone's background (technical or not), these descriptive statistics vocabulary words are necessary under any setting and would be used over and over again. Pertaining to our class and our semester long project, I doubt we will be able to extensive conduct the desired statistical summaries as we will only be interviewing about three people regarding our interactive prototype during the pilot usability study. A bigger testing population outside the range of this class would be more feasible for the purposes of analyzing statistics.
Tabassum Khan - Nov 08, 2006 05:29:30 am
How to interpret experimental results:
This reading is a crash course on statistics 101. All the terms and concepts used in statistical analysis are clearly explained which was kind of a review for most of us. The most important idea to take from this reading is how to process the raw data measured in an experiment/testing and formulate a graph to give meaning to the data. This is done to make the interpretation of the result of the experiment easier because raw data on its own has no meaning. Graphs turn raw data into valuable information.
Scott Friedheim - Nov 08, 2006 06:05:17 am
How To Interpret Experimental Results:
I think I'm the only one who hasn't seen this material. Unlike the previous comments, I have not studied this before. I can see how the methods the author discusses can be applied to large data sets to transform one's data into something useful and relevant. However, just like the above comments, I don't think the statistical methods can really be applied to our projects and the data we collect from user testing. This would be more aimed at larger industrial strength studies where lots of data are collected. Okay, I can't help myself, his little drawings are so cute!
Heung Tai - Nov 08, 2006 07:39:21 am
I like the section the author talks about mistakes in interpreting statistical tests. It's so true in the world of statistics, there is no absolute yes or no. It's all about probability. If the result is not significant for a given significant level, we cannot really totally reject or accept the hypothesis. What we should do is to understand that it has some probability that this is true, and some probability that this is false. You may say, what's the point of statistics if it just tells me certain thing "has a chance to happen"? Well, it helps in making decision. For example, if there is 99% chance that you will lose for doing certain thing, you should not do it. There is no absolute true decision in this world anyway, so a decision that base on rational statistical analysis is a good enough decision.
Hiroki Terashima - Nov 08, 2006 09:29:14 am
Okay, so the y-axis is called the ordinate and the x-axis is the abscissa. The images of the guy with this horizontal and vertical mouth really helped. How to Interpret Experimental Results contained lots of information like this. The sections on determining the strength of a relationship and levels of significance were useful, because I think it's important to consider them and they might go unnoticed when doing data analysis. As the section on misinterpreting statistical tests says, we are prone to be different in our approaches to interpreting things, and evaluating practical significance is a matter of judgment. What happens if you are shy? Will you be able to use logical arguments to convince other researchers that the difference that you find actually makes a difference? It's hard enough being objective! Regarding the computers as "your friend" was a bit comical...I'd think that many people who do these things would know how not to be afraid of the computer. But the paper also points out an important fact: computer will only do what you tell it to do, so you have to understand its limitations as well as its capabilities.
Yimin Yao - Nov 08, 2006 09:20:18 am
I think this article would be a very good article both for students who have taken stats and students who have not learned the concepts before. It goes over the important concepts in doing statistical analysis through rather concise yet clear explainations. While most of the stuff is review for me as well, I found the "misinterpreting statistical tests" section very interesting and useful. I have read science articles that use "highly significant" in reporting experiement results, which seemed logical to me at the time. The section adquently reminds us to use proper terminology (such as statistically significant) in stating test results, and to keep in mind the practical significance in real life cases of statistical analysis. I agree with author's suggestion of plotting data with associated measures of dispersion as a good presentation as experimental results without the necessity of using inferential statistical tests.
For our project, while our test sample size is probably too small to produce statistically significant results, following through the procedure using our limited data would be a good practice of using such statisitical tools.
Andrew Hao - Nov 08, 2006 09:56:36 am
So forgive me if I don't understand this, but for something to be statistically significant there needs to be less than 5% chance that the sample could be obtained randomly. Does this have a direct correlation to the standard deviation (2 standard devs ~ 95%, 3 standard devs ~ 99%)? How do we calculate this? What does it mean for something to be tested at the 0.05 level or 0.01 level?
Eric Yoon - Nov 08, 2006 10:24:43 am
This article is an overview on how to measure and quantify relationships between variables in an experiment. I'm vaguely familiar with some of these concepts, but not in any particularly deep way, so I found the knowledge interesting and informative. You hear all the time that something is correlated with something else; it is neat to see that the way to plot this and understand it is by drawing up a scatterplot and seeing how much it resembles a slanted line with a slope of 1. That makes a lot of sense, of course, but somehow the explanation made this all a bit clearer.
Roland Carlos - Nov 08, 2006 10:34:49 am
Another reading that was something hard to get really interested into. Maybe just because it's not really about HCI but more about statisical analysis? Still, analysis is an important part of interface design as we have found out these past few weeks.
In any case, I'm with most of the students in saying that was pretty much a refresher on basic stats, albeit at a much more technical level. Again, it's good to bring some true definition to why we're finding the mode, median, mean, etc. and more importantly why. In fact, a lot of the talk about finding the "main effect" was a new section for me and somewhat interesting. The reading even includes a cautionary note about using computers for stat analysis, something that most people would probably have skipped over but is an important point nonetheless.
We obviously won't be doing this for our own projects (as has been stated many times before) but perhaps we could put in some sort of very scaled down version of it to use?
Alex Wallisch - Nov 08, 2006 11:14:54 am
I'm going to jump on board with everybody else and mention that this reading is useful, although not particularly applicable to our pilot usability experiment. If we really wanted to use these techniques, we'd have to make this experiment into a much bigger project, which we really don't have time for in this class. However, it was an interesting read nonetheless. While I've already been exposed to most of the ideas in here, this is the kind of information that everybody should see at least once in their education.
Sean Carr - Nov 08, 2006 11:11:58 am
How To Interpret Experimental Results:
This was again mostly a review of statistics. I hope this is information that all science majors have had some exposure to before coming to Cal. The later sections do get into more complex stats information that I had definitely forgot since high school though. Specifically I find interest in inferential statistics because I like applications and I see that as a very good real world application of statistics. Many times intuition is wrong and it is nice to have statistics to back it up. I do wish this chapter would have had more equation sections rather than putting them in paragraph form. The graphs and visuals were easy to understand, but when you want to use this stuff you don't want to dig through a bunch of text to find what you are looking for. I guess that is because this isn't really expected to be used as a reference though, more of an introduction.
Sung Yi - Nov 08, 2006 11:43:57 am
This article just reminded me of all the stuffs I learned in statistics in high school. It talks about all the statistics concepts like mean, standard deviations, skewness, how to extrapolate experimental data to get some useful analysis and so on. However, I strongly doubt that we will be using the concept introduced in the reading for our assignment since we have only around three user cases, and it is not appropriate to base analysis on the data extracted only from them. We need much more sample to do statistics.
Ming Huang - Nov 08, 2006 11:47:38 am
I agree with others that this chapter gives us a brief overview of statistical analysis, with particular emphasis on analyzing experimental data to find the best way to support our findings. Rather than scoffing at the relative boringness of the material perhaps we can get insight from it by relating their meaning with uses in our experiments. The general idea could be seen as that there is simply no one single best metric that we can use to draw our conclusions from. They vary in terms of usefulness for different purposes in our user testing. Some of them represent peculiarities that might indicate measurement errors or flaws on the experiment’s design and other indicate general trends useful when determining correlations. Mastering the skill of choosing the right facets of the data to support our conclusions is perhaps and much more valuable skill to take away than number crunching.
Melissa Jiang - Nov 08, 2006 11:39:01 am
This article was a good review since I have not taken a statistics course in a while. The examples were pretty clear and easy to understand. I also agree that we probably will not be able to use these techniques since we only have 3 subject. However, I do see how real companies or groups will need these techniques if they plan on designing new products. I assume 3 subjects probably would not be enough for them to adequately test their design (correct me if I am wrong).
On another, random, note, was labeling the x-axis "Abscissa" and the y-axis "Ordinate" really necessary? All the other terms, histograph, scatterplot, correlation coefficient, etc, are important and commonly used in statistics. However, I cannot recall the last time any statistics teacher called the x-axis and the y-axis "Abscissa" and "Ordinate".
Robert Held - Nov 08, 2006 11:49:52 am
How to Interpret Experimental Results
Statistics are absolutely critical whenever you start talking about presenting quantitative results from an experiment. The chapter discusses how to use p-values to establish the statistical significance of results and also mentions how today's scientific literature seems obsessed with thresholds for p-values. I enjoyed their discussion about treating a p-value in the context of the experiment, rather than always requiring it to be less than 0.05 or 0.01 to merit serious consideration. Along with the author's emphasis to use computers but avoid naively relying on them, I think the take home message is that a researcher needs to tailor his/her analysis to the experiment on hand. Otherwise, it would be easy to simple throw data into a computer algorithm and critique the outcome based upon some generic test that may not be appropriate for the nature of the study.
Rayhan Lal - Nov 08, 2006 11:59:59 am
How to Interpret Experimental Results: I work in a neuronimaging lab so these sorts of statistics are something I deal with on a day-to-day basis. I can assure you this information is absolutely vital to any research be it in HCI, psychology or any field. (Just because it’s not applicable to one assignment does not mean it’s not important.). If one at least knows the information presented in the chapter it becomes much easier to evaluate research papers. Adding what we learned about different statistical tests from lecture would have made this chapter even more useful.
Designing Visual Interfaces: I just wanted to say that this was an enjoyable piece, very artistic and somewhat refreshing. The points made about alignment intrigued me because I had not realized so much effort went into typography.
Tak Wong - Nov 08, 2006 11:47:18 am
The statistical issues discussed in the reading can be helpful in user study in the UI context. We should keep in mind that we shouldn't find the most convenient people we can find around us. These samples may be the cause of skewed distribution or false statistical significance. Since for our class project we don't have budget to get strangers as users, it's a bit hard to do the analysis for us. Also, 3 users is not big enough of a sample size to accurately interpret the results.
Aleksandr (Sasha) Ashpis - Nov 08, 2006 12:17:37 pm
This chapter is full of useful statistical jargon and approaches. A lot of the things mentioned have been derived over many years of experiments, and have just been accepter as standards, like the levels of significance should be <= .05, why? On the other hand, others believe that that is too high and say that the more appropriate level of significance should be <= .01, once again why?
When I took a lower division math class several years back, my GSI told me that statistics are like art but worse, because if one chose to, they could manipulate the numbers to demonstrate any bias they wanted, while at the same time trying to pawn it of as scientific.
Jae Chang - Nov 08, 2006 12:22:08 pm
How To Interpret Experimental Results:
This reading is about statistics and reminded me the concepts I learned in a statistic class. The reading was good for reviewing the statistical materials. However, as other people mentioned above, it is true that we do not have enough test users and experimental results to use statistical analysis introduced in this reading. Even if this reading cannot be applied on the current group project, I believe that the statistic analysis is strong method to visualize data and find relationship among variables. In real field, I am sure that collecting data and doing statistical analysis is widely used and very good way to convince other people.
Vahe Oughourlian - Nov 08, 2006 11:51:58 am
How to interpret experimental results
For the most part, this paper falls into the same category as the previous reading; dry definitions that any high-school level math or science student should be familiar with. The more usful sections came later, in the discussions about both statistical significance and meta-analysis. The statistical siginficance portion pointed out that the numbers may not always tell us what we want to hear, or that they may be telling us something we don't realize. I found it amusing that some investigators use the p (the level of statistical significance) as a strict benchmark as the utility of thier experiment. It seems like a rather childish thing to do, and it shows us the greater care that must be taken in interpreting results. The meta-analysis section was also interesting in that it took the focus farther off the specific test data and made it a more "global" analysis. Maybe a paper in greater detail about meta-analysis would clear up my concern, but, according to this presentation, I have to agree with the detractors of this method. Leaving it up to the investigator to qualify and weight the results of potentially flawed data makes it much more likely that the analysis that will come from the meta-data result be flawed as well, in my opinion. Perhaps there is some statistical viability to applying this method to all elements of a large data set, but it just seems like a stopgap solution, and I'm not really sure that the method (the mean of the group not treated subtracted from the mean of the group in treatment divided by the standard devation) would give you a particularly useful result.
Tom McClure - Nov 08, 2006 12:19:05 pm
The picture mnemonics in this text didn't bother me as much as they seem to have bugged some others. This chapter is a nice toplevel review of basic statistics concepts, and will no doubt come in handy for Monday's assignment. I got a kick out of the discussion of computers and statistics at the end of the chapter, where the author warns the computer-happy people to be wary of trusting the computer output too much, and invites the computer-fearing crowd to the computer desk with the encouraging phrase "[computers] are your friends and are getting friendlier all the time." I guess that's our job in this class, making them friendlier...
Michael Mai - Nov 08, 2006 02:09:07 am
This chapter goes over the basic concepts of a beginner statistics class using examples without the numbers. Although useful if we had a substantial amount of people to apply our projects on, I feel that the only real application I can find for all of this data would be to determine how I did on my assignments and tests in the class. Although briefly mentioned, one of the concepts should be taken into account for our results. The notion that we are testing on a sample size of a general population should make us realize that while we may please some individuals it needs broader testing for completeness.
PS. I notice that the time is off, it's really 12:27pm
Patrick Rodriguez - Nov 08, 2006 12:26:52 pm
I'm with all of the other students who have seen a good portion of this material in other classes and contexts. It's good to have a refresher every now and then, but is the topic relevant to our task at hand? It seems like a little overkill since we don't have the time or capability to gather such large datasets regarding the usability of our project. But it may prove to be useful, nonetheless.
Huangnankun - Nov 08, 2006 12:34:21 pm
The article talks about tools to do data analysis. These are tools which can be applied to raw data in order to make them a lot easier to understand and interpret. First the idea of frequency plotting was introduced along with a few different distributions which they can fall under, I think the shape of the distributaion can tell us a lot of about the data collected, for example a bimodel distribution can signify 2 main groups of people. The next set of tools introduced are graphing and correlation, although the article only introduce 2-dimensional plots, I think we can extend this analogy to multiple dimensions when there are multiple variables which we want to plot against ( for example, score against both age and income level). The topic of inferential statistics allow us to make educated guesses about the general scheme based on a selected sample. The level of significance and other statistical tools given in the article also gives us an idea of how "sure" we can be of our experimental results
Siyan Wang - Nov 08, 2006 12:37:03 pm
This article was a bit more dry and technical than his previous ones, but still contained useful information. The tools for data analysis he presents seem like simple statistics, but still seem pretty useful, especially since he presented it in such a simple and straight-forward fashion. However, I wonder how this will help us with our own pilot studies, since our subject pool is so small that we don't really have any statistically significant data. However, the part it talks about drawing conclusions from your data, such as trying interpret correlation or causation sounds like it would be very helpful.
Johnathan Hawley - Nov 08, 2006 12:43:32 pm
How to Interpret Experimental Results - This section was a nice little review of statistics. I occasionally still get mean, median, and mode confused. A lot of this I have seen in other psychology classes I've taken: interpreting positive and negative relationships, interpreting a correlation coefficient. Testing for interactions seems useful for our user interface. We could additionally test to see if there is an interaction between age and usability of our design. We could test visibility by observing if there is an interaction between our color scheme and time it takes to accomplish a task.
Siu Pang Chu - Nov 08, 2006 12:30:12 pm
This is a article that gives me a review of Statistics. It first explained some of basic statistics terms like mode ,mean , median and mean. Those are defined to describe the central tendency of some data. Then , it go to explain another important measurement dispersion, range and standard deviation. We can describe the strength of a relationship, like correlation coefficients is a number between +/- 1.0. Finally, the article tell us how interpret the result, such as discover the main effect and interactions.
Jason Shangkuan - Nov 08, 2006 12:47:08 pm
How To Interpret Experimental Results:
This article seems interesting because inferential results can be used to understand a much larger audience and user base. Most of the concepts such as statistical model of mode, mean, standard deviation are all relatively common concepts and really quantify results. However, these quantifications might not necessarily apply in our pilot study or final design. In our design cycle we focus on users and tasks, and not really a wider audience than those who will be using our system. This article would be most beneficial to someone making a general website. I think for interpreting experimental results quantitatively, tabulating responses for certain feature and characteristics would be helpful in evolving our designs.
Dexter Lau - Nov 08, 2006 12:47:59 pm
How to Interpret Experimental Results The bulk of the article is merely explaining how to use statistical analysis to make sense of the data drawn out from the experiment. This includes things like understanding the correlation between the dependent and independent variables and graphing their relationship. Other useful things include frequency distributions, which can be used to find th mean and standard deviations of the results. Something I was not aware of was meta-analysis, which can be used to combine the results of multiple tests together.
Yen Pai - Nov 08, 2006 12:51:20 pm
How to Interpret Experimental Results: A good review of statistical techniques - as others have mentioned, not quite as immediately applicable since we are not performing larger scale experiments in this class. Meta-analysis is interesting and opens up the possibility of bias depending on which studies were chosen to form broader conclusions.
Designing Visual Interfaces: A very informative reading which puts into more formal terms why a particular design or interface looks "right" while another one does not. One thing I think which is often overlooked in applications is visual design. Many times, a great deal of effort seems to have been put into making individual elements look nice (nice icons, 3D-modeled buttons, etc) but when it all comes together, overall integrity and organization is lacking.
Michael Moeng - Nov 08, 2006 12:51:30 pm
Although most of the information in the article was familiar, the section on Meta-Analysis was new to me. I think that using other people's study data would be useful-although one would have to be careful when combining the data to not produce misleading results, by weighing certain data sets incorrectly
Robert Taylor - Nov 09, 2006 02:09:58 pm
How to Interpret Experiment Results: As many have already said, I'm not sure how helpful this reading will be for our next assignment in particular, as we are only going to be doing the study on a very, very small number of users. At the same time, some of the graphs and terms will at least provide a kind of conceptual framework for when we do the Pilot Usability Study. I think this might have been better as a handout before we did the assignment rather than as an assigned reading.
Patti Bao - Nov 11, 2006 09:28:16 pm
How to Interpret Experimental Results: I think this chapter helped to demonstrate why statistics and lies are often associated with each other - it's so easy to misinterpret, not to mention misrepresent, statistical tests, and in particular statistically significant tests. Isn't Freakonomics all about drawing connections between unexpectedly related variables? How do you know when it's reliable and valid to draw those connections? That's what makes meta-analysis such is an interesting concept to me. While I don't quite see how different results can be accurately combined if they use different methodologies, the notion of combining results at all is certainly a useful one.
Patti Bao - Nov 11, 2006 10:16:02 pm
Organization and Visual Structure: I liked learning about the Gestalt principles, and I have noticed that we do tend to group things together based on things like position and color. It was really interesting to see how many different methods can be used to establish grouping, whether that means balancing window-based GUIs or adjusting alignment of elements. I was glad to see a section on the importance of white space, which I think is often underrated in design. While this reading offers a helpful series of steps to achieve each grouping method, I wonder where the place of creativity is in all of this - surely it is not just following a recipe of success.
Tony Yu Tung Lai - Nov 13, 2006 05:20:31 pm
Martin: Chap 12
It is refreshing for me to read about statistics in a Psychology text rather than from a Stats text. One small but interesting difference between how I learned stats and the stats in this article is the way variance and standard deviation was presented. In this article, the concept of standard deviation was first presented as the measurement of closeness to the mean, and then later it was also mentioned that it can be think of as the measurement of error as well. But for me, when I learn about the concept of standard deviation, I was first told to think of it as measuremen of error, and then later make the connection that it is also the measurement of closeness to the mean. Although measurement of error and closeness to mean are basically the same thing, I find that the correlation between where the concept is being learned from and the order of which side to present first to be ineteresting.
I really agree with Geoffrey Loftus' point of view regarding not needing the inferential test, but having better plots and graphs is more important. I don't mean it in such a way that we should completelt eliminating inferential testings, but I think nowadays we are concentrating too much on these tests and p-value that not many people really know about. Rather than throwing the word 'statstically significant' around, graphs and plot would be a lot more convincing to most people. Also, like Martin mentioned in a near by paragraph, statistical significance and practical significance has a big difference, where as graphs and plots are up to the reader's interpretation and wouldn't have that problem.
Yang Wang - Nov 14, 2006 01:36:16 pm
This reading is very interesting. It looked daunting but it is acutally not that hard to read, especially if you have taken statistic. It talks about how to interpret a statistically result and it is very comprehensive, almost like straight out a statistic book. Although it is generally useful, but I don't see how we can apply it to this course. Since we don't really have that much subject that we can even apply simple things like mean and mode. Only if we do have more test subject, but it will take a lot of time and energy to gather the test subject and probably even more time to interpret the result.
Yang Wang - Nov 14, 2006 01:40:12 pm
One thing that I want to comment is how our test might be deeply skewed statistically. First, for our group project, we are testing current flickr users, which we are developing the project for. However, our assumption shouldn't include only college student. But for our project, we only test it with the college students. This is a very skewed testing because such limitation of age group and culture differences (since I consider college a different culture than rest of the world) is a completely unreasonable assumption. I know that there are plenty of people using it who are in thirties and fourties. Their skill with the computer and internet varies drastically from our test subjects.
Another thing that I want to point out is that we developed this project for people who travels and take pictures. Thus it is perfect for explorers or people who are going on trips a lot. For a college student, travelling is not very feasible. Thus, as one of my test subject points out. He doesn't think it is useful unless he will be travelling. Thus this drastically changes our feedback on how useful the system is, because we are testing on people who it is not developed for (since we have no access to explorer or scientist who travels)
Simon Tan - Nov 15, 2006 03:54:51 am
Comment 1: I agree with most people that the knowledge gained from this portion of the readings/course did not really come into play during our project work. The statistical analysis done with the meager data we had felt tacked on and almost unnecessary. It was data we could analyze and discuss, but it really was so unrepresentative of a larger study that the results gained were probably heavily skewed. If we were to continue our projects and do longer and more involved studies, however, knowing how to perform these experimental routines could help us produce more professional and widely respected results.
Comment 2: Most of the knowledge contained herein I learned from AP Statistics. Getting told what mean, mode, etc. were again was unsettling at first, but I understand that it was the basis for discussion on higher-level statistics, such as significance levels and parametric/non-parametric tests (which was an interesting refresher). Meta-analysis was actually something I haven't heard of before, so that was interesting to see. I just can't get over the feeling that a majority of this technical content could be better covered in a analytics/psychology class, but I see how CS 160 may be the most logical CS-oriented course to have this content in.
Randy Hilarbo - Nov 15, 2006 09:54:58 am
The article is somewhat a refresher of those statistics terms. It's nice to read about them in the context of interpreting experiment results. It's nice to know which kind of statistical measure should be considered depending on your experimental results: how mean, median, mode, and variance place special focus on a particular characteristic of the data.
In this article, I also realized how experiment results can be understand differently through different kind of visualizations used to interpret them. Listed data is indeed harder to interpret than if they are interpreted in a graph. One way to graph data can also be better interpreted by another way of graphing the same data. In short, visualization can really help us interpreting data much more easily.
Michael Udaltsov - Nov 15, 2006 11:38:40 am
Just like the previous chapters, this one also contains the general statistics concepts about plotting results and finding relationships in them. I find the difference between statistical significance and practical significance very important. As the text mentions, with enough data it's possible to show that something is statistically significant, while it would make practically no difference (such as the speed reading example from the text). There are often similar claims of "significance" in advertising, which people tend to believe and not investigate further to determine if there really is a practical significance.
The statement about using computers - "garbage in, garbage out" is also very important. Even at a simpler level such as graphing calculators, people tend to trust the results without verifying their correctness. When dealing with statistics, it may be easy to get a result that's calculated incorrectly, and will lead to wrong conclusions, but will not always be noticed. One suggestion given in the reading is to do a quick check to see if the results make sense, but it won't necessarily prevent against skewed or biased results. Checking some of the work by hand is often troublesome, and is more prone to mistakes, so it's not always trustworthy either. I think for anyone doing statistical analysis, it's important to understand the details of the entire process, and know what to expect, so any errors get caught in advance.
CharlesLeung - Nov 15, 2006 12:25:38 pm
Like the last reading, I think that most of this chapter is review because I think I've gone over topics like this chapter's in discrete math and in a lower div stats class. I'm kind of curious though how one interprets data for a non-normal data set. I mean, even though we can't use the SD and mean like we usually do for data sets that approximately look like normal curves, there must be some way to quantitatively measure those kinds of data sets. I'm kind of curious how people treat those non-ideal sets of data.
I've always been curious why there is an actualy definition for something that is statistically significant. To me, it seems like a bunch of statisticians got together and came up with this term. I mean, to a layperson I think that they would be impressed if they heard that something was statistically significant. However, to me all it says is that the data follows some arbitrarily set criteria.
Bryce Lee - Nov 15, 2006 12:42:19 pm
I find it interesting that Martin does not put more emphasis in this section on the size of the study related to a certain set of statistics. He mentions the basic ways of analyzing the data (different averages, measures); however, all of these end results are pointless without a substantial user group. Patterns that show up from merely three users is not enough to make a strong statement about anything. At best, it infers that further study should be done to verify the pattern. Aside from this oversight, I think Martin's point about not discounting practical significance because of a lack of statistical significance is important. The large focus on absolute numbers has led to worse statistics where companies purposely coerce results into fitting into the acceptable range.
I also thought it was interesting how Martin did not go into any detail about other ways to approach statistical data, such as analyzing the different quartiles and plotting the data using a boxplot. I have found both of these representations to be more straightforward and critical of data than plots. Also, the absorption of multiple data sets seems to be too riddled with issues. Unless everyone was conducting the experiment with the same intentions and end results, such aggregation seems groundless.
Anirudh Vemprala - Dec 13, 2006 12:54:17 pm
How to Interpret Experimental Results:
The Marin reading was quite an entertaining read for a discussion on statistics. I felt that the discussion on standard deviation, main effects and statistical significance could have been more detailed. That said, the reading did explain the basic ideas like median, mode and means very well along with the kinds of distribution.
It might have been a good idea to situate this reading with something a little more mathematical to make concrete the ideas presented in this chapter.
Organization and Visual Structure: This reading was a great introduction to some of the artistic concepts that are used in information visualization. I had heard of these terms in reference to elements of modern art (integrity, figure-ground relationships, balance etc) before. I think its particularly fascinating to note that the infovis field is a blend of science and art. That said, there are great resources out there for those people interested in the fundamentals of graphic design that might probably be superior to the introduction provided by this reading. Gregg Berryman's book <a href="http://www.amazon.com/Notes-Graphic-Design-Visual-Communication/dp/1560520442">Notes on Graphic Design and Visual Communication</a> is a classic in the field. The examples w.r.t to GUI design that were presented at the end of the chapter, however, were a valuable and interesting read.
Robin Franco - Dec 15, 2006 12:18:04 pm
I remember taking a statistics course in high school. I considered it to be rather irrelevant for my studies. It was refreshing to read this article and learn how it is applicable to my field. Even though we did not use these techniques during the semester for our studies, I can see how this could be useful for our future career. Having a concrete way of determining the statistical significance of data is definitely something one would use when presenting the viability of a new technology.
