SceneSkim: Searching and Browsing Movies Using Synchronized Captions, Scripts and Plot Summaries
Amy Pavel (UC Berkeley), Dan B Goldman (Adobe Research), Björn Hartmann (UC Berkeley), Maneesh Agrawala (Stanford)
Searching for scenes in movies is a time-consuming but crucial task for film studies scholars, film professionals, and new media artists. Our formative interviews reveal that such users search for a wide variety of entities — actions, props, dialogue phrases, character performances, locations — and they return to particular scenes they have seen in the past. Today, these users find relevant clips by watching the entire movie, scrubbing the video timeline, or navigating with opaque DVD chapter menus. We introduce SceneSkim, a tool for searching and browsing movies using synchronized captions, scripts and plot summaries. Our interface integrates information from different documents to allow expressive search at several levels of granularity: Captions provide access to accurate dialogue, scripts describe shot-by-shot actions and settings, and plot summaries contain high-level event descriptions. We propose new algorithms for finding word-level caption to script alignments, parsing text scripts, and aligning plot summaries to scripts. Film studies graduate students evaluating SceneSkim expressed enthusiasm about the usability of the proposed system for their research and teaching.