From CS294-10 Visualization Sp11

Revision as of 19:14, 5 April 2011 by Matthew Can (Talk | contribs)
Jump to: navigation, search



Group Members

  • Sally Ahn

Visualizing Aggregate Image Edits

Often times, visualization techniques focus on transforming numerical or text data into a single image that conveys the story of millions of numbers/words. However, sometimes the data are images to begin with. This is the case in my current research in computational aesthetics of photographs. One way of gaining insight into general aesthetic preferences of photo compositions is to collect images that represent individual preferences from many users. The data domain then consists of many images (50-100), and aggregating these images in a meaningful way can be challenging, especially if some of these images are unreliable and should be discarded. A good visualization tool can aid in analyzing such data by allowing the user to interactively hide or view specific subsets of images. The relevant subsets can be specified through the image itself, through image-based filtering. For example, brushing and linking techniques could enable the user to select an section of the original image that should (or should not) be present in the cropped image from the dataset. The filtered set of images could then be presented in a way that highlights visual patterns or similarities to aid in detecting aesthetic preferences, along with its statistical data. Although I am focusing on photo croppings for this project, I think this visualization tool can be extended to analyze other aspects of image editing, such as color, contrast, etc.

Initial Problem Presentation


Jvoytek - Apr 04, 2011 03:40:36 pm

This is a very interesting and difficult problem. The challenge seems to be to make an intuitive link between the crop data and the statistical measures.

Brandon Liu - Apr 04, 2011 03:42:58 pm

Q: I really like the idea of showing a density map for many user's crops. How will you handle validating input on Mechanical Turk? It seems like you run the risk of getting a lot of bad data from Mechanical Turk; one strategy is to give Turk users a "training task". What training task would you use that has a known optimal crop? Also, what about giving users a picture with more than one area of interest and then asking them to pick out individual items?

A: Use a verify step on MTurk to make sure crops are valid.

Julian Limon - Apr 04, 2011 05:48:57 pm

This looks like a very interesting challenge. Since you already plan to use Mechanical Turk, I believe you could take advantage of the task and ask people for more detailed questions. For example, you could have a question that says "Which picture is more pleasant?", another question that says "Which picture do you prefer?" or "Which picture is more interesting?". Taste is a very subjective task, so if you're able to determine a few different characteristics of people's preferences the results could be even more interesting.

I also like the idea of using pixel voting to chose the best image. However, I wonder if this could lead to non-conventional crops such as diagonal lines.

Michael Cohen - Apr 04, 2011 11:59:39 pm

In addition to the options you mentioned, it might be interesting to look at color as a way of showing the aggregated croppings. You could vary along the saturation dimension so that rarely selected pixels are grayscale and selected ones are in their usual colors. Of course, there would be a lot of crosstalk with the color information present in the image -- for instance, an image that has dull colors to begin with won't give you very much room to show intermediate degrees of selection. It might be more effective to grayscale the whole image, then add saturation of a single color to represent the level of selection; I believe photoshop does something like this for masks that support partial selection of pixels.

Siamak Faridani - Apr 05, 2011 01:18:55 am

This looks like an interesting contribution both from the visualization perspective and from the human computation perspective. I have one comment about some related work. My adviser, Ken Goldberg and his former PhD student Dez Song have worked on a number of interesting spacial voting models and these might be helpful for you work. For example see the following

Unsupervised Scoring for Internet-based Collaborative Tele-operation, Ken Goldberg, Dezhen Song, In Yong Song, Jane McGonigal, Wei Zheng, and Dana Plautz, IEEE International Conference on Robotics and Automation (ICRA), April 2004

Matthew Can - Apr 05, 2011 01:55:19 pm

You defined the problem well and seem to have a solid idea of the tools and visualizations that you plan to build. In addition to the density map, you might want to create some dynamic query tools to search through all of the cropped images. For example, if I select a portion of the original image, the system returns all cropped images that include that portion.

[add comment]
Personal tools