Gaze-Based Interaction for Semi-Automatic Photo Cropping

Anthony Santella, Maneesh Agrawala, Doug DeCarlo, David Salesin, Michael Cohen


We present an interactive method for cropping photographs given minimal information about the location of important content, provided by eye tracking. Cropping is formulated in a general optimization framework that facilitates adding new composition rules, as well as adapting the system to particular applications. Our system uses fixation data to identify important content and compute the best crop for any given aspect ratio or size, enabling applications such as automatic snapshot recomposition, adaptive documents, and thumbnailing. We validate our approach with studies in which users compare our crops to ones produced by hand and by a completely automatic approach. Experiments show that viewers prefer our gaze-based crops to uncropped images and fully automatic crops.

Original well-composed images (left), adapted to two different aspect ratios using our gaze-based approach. An ADL document (right) using our crops. If eye movements are collected passively during document construction, our approach allows adaptation of images to arbitrary aspect ratios with no explicit user effort.

Research Paper

PDF (2.2M)

Gaze-Based Interaction for Semi-Automatic Photo Cropping
ACM Human Factors in Computing Systems (CHI), 2006. pp. 771-780.