Abstract

We present techniques for automatically parsing existing sewing patterns and converting them into 3D garment models. Our parser takes a sewing pattern in PDF format as input and starts by extracting the set of panels and styling elements (e.g. darts, pleats and hemlines) contained in the pattern. It then applies a combination of machine learning and integer programming to infer how the panels must be stitched together to form the garment. Our system includes an interactive garment simulator that takes the parsed result and generates the corresponding 3D model. Our fully automatic approach correctly parses 68% of the sewing patterns in our collection. Most of the remaining patterns contain only a few errors that can be quickly corrected within the garment simulator. Finally we present two applications that take advantage of our collection of parsed sewing patterns. Our garment hybrids application lets users smoothly interpolate multiple garments in the 2D space of patterns. Our sketch-based search application allows users to navigate the pattern collection by drawing the shape of panels.

Our parser automatically converted a diverse set of sewing patterns into 3D garment models for this small crowd of women.

Research Paper

PDF (11.6M)

Video

MP4 (150M)

More Results

Supplemental Materials

Abstract

Vector graphics represent images with compact, editable and scalable primitives. Skillful vector artists employ these primitives to produce vivid depictions of material appearance and lighting. However, such stylized imagery often requires building complex multi-layered combinations of colored fills and gradient meshes. We facilitate this task by introducing vector shade trees that bring to vector graphics the flexibility of modular shading representations as known in the 3D rendering community. In contrast to traditional shade trees that combine pixel and vertex shaders, our shade nodes encapsulate the creation and blending of vector primitives that vector artists routinely use. We propose a set of basic shade nodes that we design to respect the traditional guidelines on material depiction described in drawing books and tutorials. We integrate our representation as an Adobe Illustrator plug-in that allows even inexperienced users to take a line drawing, apply a few clicks and obtain a fully colored illustration. More experienced artists can easily refine the illustration, adding more details and visual features, while using all the vector drawing tools they are already familiar with. We demonstrate the power of our representation by quickly generating illustrations of complex objects and materials.

We describe Vector Shade Trees that represent stylized materials as a combination of basic shade nodes composed of vector graphics primitives (a). Combining these nodes allows the depiction of a variety of materials while preserving traditional vector drawing style and practice. We integrate our vector shade trees in a vector drawing tool that allows users to apply stylized shading effects on vector line drawings (b,c). This pdf contains png versions of vector art to avoid viewer compatibility problems; full vector versions of our results are in supplemental material. Original line drawing from koconmus, openclipart.org , colored version the authors .

Research Paper

PDF (43.2M)

Supplemental Material

ZIP (156.3M)

Video

MP4 (14.0M)

Abstract

Depictions with traditional media such as painting and drawing represent scene content in a stylized manner. It is unclear however how well stylized images depict scene properties like shape, material and lighting. In this paper, we describe the first study of material perception in stylized images (specifically painting and cartoon) and use non photorealistic rendering algorithms to evaluate how such stylization alters the perception of gloss. Our study reveals a compression of the range of representable gloss in stylized images so that shiny materials appear more diffuse in painterly rendering, while diffuse materials appear shinier in cartoon images. From our measurements we estimate the function that maps realistic gloss parameters to their perception in a stylized rendering. This mapping allows users of NPR algorithms to predict the perception of gloss in their images. The inverse of this function exaggerates gloss properties to make the contrast between materials in a stylized image more faithful. We have conducted our experiment both in a lab and on a crowdsourcing website. While crowdsourcing allows us to quickly design our pilot study, a lab experiment provides more control on how subjects perform the task. We provide a detailed comparison of the results obtained with the two approaches and discuss their advantages and drawbacks for studies like ours.

Each stylization affects gloss perception differently. In painterly rendering, opaque strokes (b) removes some highlights and semi- transparent strokes (c) blend colors, making shiny materials appear more diffuse. In contrast, cartoon rendering exaggerates shininess (d). In this paper, we evaluate how people perceive gloss in stylized images, and we derive the function that predicts for a given gloss how it will be perceived after stylization, as shown here in insets.

Research Paper

PDF (17.4M)

Supplemental Material

ZIP (46.3M)

Abstract

How-things-work visualizations use a variety of visual techniques to depict the operation of complex mechanical assemblies. We present an automated approach for generating such visualizations. Starting with a 3D CAD model of an assembly, we first infer the motions of the individual parts and the interactions across the parts based on their geometry and a few user-specified constraints. We then use this information to generate visualizations that incorporate motion arrows, frame sequences, and animation to convey the causal chain of motions and mechanical interactions across parts. We demonstrate our system on a wide variety of assemblies.

We analyze a given geometric model of a mechanical assembly to infer how the individual parts move and interact with each other and encode this information as a time-varying interaction graph. once the user indicates a driver part, we use the interaction graph to compute the motion of the assembly and generate an annotated illustration to depict how the assembly works. We also produce a corresponding causal chain sequence to help the viewer mentally animate the motion.

Research Paper

PDF (3.3M)

Technical Perspective PDF (458K)

Video

MOV (87.9M)

Abstract

Reading a visualization can involve a number of tasks such as extracting, comparing or aggregating numerical values. Yet, most of the charts that are published in newspapers, reports, books, and on the Web only support a subset of these tasks. In this paper we introduce graphical overlays—visual elements that are layered onto charts to facilitate a larger set of chart reading tasks. These overlays directly support the lower-level perceptual and cognitive processes that viewers must perform to read a chart. We identify five main types of overlays that support these processes; the overlays can provide (1) reference structures such as gridlines, (2) highlights such as outlines around important marks, (3) redundant encodings such as numerical data labels, (4) summary statistics such as the mean or max and (5) annotations such as descriptive text for context. We then present an automated system that applies user-chosen graphical overlays to existing chart bitmaps. Our approach is based on the insight that generating most of these graphical overlays only requires knowing the properties of the visual marks and axes that encode the data, but does not require access to the underlying data values. Thus, our system analyzes the chart bitmap to extract only the properties necessary to generate the desired overlay. We also discuss techniques for generating interactive overlays that provide additional controls to viewers. We demonstrate several examples of each overlay type for bar, pie and line charts.

Examples of visual overlays organized by type. These overlays were manually generated to illustrate the different overlay types.

Research Paper

PDF (11.8M)

More Results

Interactive Gallery

Abstract

We present a system for producing 3D animations using physical objects (i.e., puppets) as input. Puppeteers can load 3D models of familiar rigid objects, including toys, into our system and use them as puppets for an animation. During a performance, the puppeteer physically manipulates these puppets in front of a Kinect depth sensor. Our system uses a combination of image-feature matching and 3D shape matching to identify and track the physical puppets. It then renders the corresponding 3D models into a virtual set. Our system operates in real time so that the puppeteer can immediately see the resulting animation and make adjustments on the fly. It also provides 6D virtual camera and lighting controls, which the puppeteer can adjust before, during, or after a performance. Finally our system supports layered animations to help pup- peteers produce animations in which several characters move at the same time. We demonstrate the accessibility of our system with a variety of animations created by puppeteers with no prior animation experience.

Our system allows puppeteers to use toys and other physical props to directly perform 3D animations.

Research Paper

PDF (8.8M) | Hi Res PDF (58.8M)

Additional Materials

Source Code

Video

MP4 (43.5M) | Hi Quality MOV (340M) | YouTube

Animation Gallery

Stop, Look, and Listen MP4 (7.3M) | YouTube

The Bully Meets His Match MP4 (11.4M) | YouTube

Mean Minnow MP4 (2.3M) | YouTube

Clear the Crash (10.4M) | YouTube

Cheatin' Duck MP4 (4.4M) | YouTube

Sea Dance MP4 (6.4M) | YouTube

Penguin Mugger MP4 (2.7M) | YouTube

Abstract

Audio producers often use musical underlays to emphasize key moments in spoken content and give listeners time to reflect on what was said. Yet, creating such underlays is time-consuming as producers must carefully (1) mark an emphasis point in the speech (2) select music with the appropriate style, (3) align the music with the emphasis point, and (4) adjust dynamics to produce a harmonious composition. We present UnderScore, a set of semi-automated tools designed to facilitate the creation of such underlays. The producer simply marks an emphasis point in the speech and selects a music track. UnderScore automatically refines, aligns and adjusts the speech and music to generate a high-quality underlay. UnderScore allows producers to focus on the high-level design of the underlay; they can quickly try out a variety of music and test different points of emphasis in the story. Amateur producers, who may lack the time or skills necessary to author underlays, can quickly add music to their stories. An informal evaluation of UnderScore suggests that it can produce high-quality underlays for a variety of examples while significantly reducing the time and effort required of radio producers.

A musical underlay highlights an emphasis point in an audio story. The music track contains three segments; (1) a music pre-solo that fades in before the emphasis point, (2) a music solo that starts at the emphasis point and plays at full volume while the speech is paused, and (3) a music post-solo that fades down as the speech resumes. At the beginning of the solo, the music often changes in some significant way (e.g. a melody enters, the tempo quickens, etc.) Aligning this change point in the music with a pause in speech and a rapid increase in the music volume further draws attention to the emphasis point in the story.

Research Paper

PDF (2.7M)

Video

MP4 (67.5M)

More Results

Demo | Supplemental Materials (coming soon!)

Abstract

Proton++ is a declarative multitouch framework that allows developers to describe multitouch gestures as regular expressions of touch event symbols. It builds on the Proton framework by allowing developers to incorporate custom touch attributes directly into the gesture description. These custom attributes increase the expressivity of the gestures, while preserving the benefits of Proton: automatic gesture matching, static analysis of conflict detection, and graphical gesture creation. We demonstrate Proton++’s flexibility with several examples: a direction attribute for describing trajectory, a pinch attribute for detecting when touches move towards one another, a touch area attribute for simulating pressure, an orientation attribute for selecting menu items, and a screen location attribute for simulating hand ID. We also use screen location to simulate user ID and enable simultaneous recognition of gestures by multiple users. In addition, we show how to incorporate timing into Proton++ gestures by reporting touch events at a regular time interval. Finally, we present a user study that suggests that users are roughly four times faster at interpreting gestures written using Proton++ than those written in procedural event-handling code commonly used today.

In Proton++ the developer can specify a custom touch direction attribute. (a) The direction is computed by taking the vector formed by the last two positions of the touch and (b) binning it to one of the four cardinal directions. Combining the hit-target and direction attributes, the developer can specify a gesture to translate a star (denoted as 's') with varying degrees of specificity: (c) north only, (d) north and south only, (e) in any direction.

Research Paper

PDF (2.2M)

Video

MP4 (29.0M) | YouTube

Additional Materials

Source Code

Abstract

We present a set of tools designed to help editors place cuts and create transitions in interview video. To help place cuts, our interface links a text transcript of the video to the corresponding locations in the raw footage. It also visualizes the suitability of cut locations by analyzing the audio/visual features of the raw footage to find frames where the speaker is relatively quiet and still. With these tools editors can directly highlight segments of text, check if the endpoints are suitable cut locations and if so, simply delete the text to make the edit. For each cut our system generates visible (e.g. jump-cut, fade, etc.) and seamless, hidden transitions.We present a hierarchical, graph-based algorithm for efficiently generating hidden transitions that considers visual features specific to interview footage. We also describe a new data-driven technique for setting the timing of the hidden transition. Finally, our tools offer a one click method for seamlessly removing ’ums’ and repeated words as well as inserting natural-looking pauses to emphasize semantic content. We apply our tools to edit a variety of interviews and also show how they can be used to quickly compose multiple takes of an actor narrating a story.

Our interface. Selecting text in the transcript highlights corresponding regions of the timeline. Vertical bars in the transcript and blue bars in the timeline visualize cut suitability; places where a cut is least likely to interrupt the flow of the video.

Research Paper

PDF (5.5M)

Video

MOV (84.9M)

More Results

Supplemental Materials

Abstract

We present a semi-automated technique for selectively de-animating video to remove the large-scale motions of one or more objects so that other motions are easier to see. The user draws strokes to indicate the regions of the video that should be immobilized, and our algorithm warps the video to remove the large-scale motion of these regions while leaving finer-scale, relative motions intact. However, such warps may introduce unnatural motions in previously motionless areas, such as background regions. We therefore use a graph-cut-based optimization to composite the warped video regions with still frames from the input video; we also optionally loop the output in a seamless manner. Our technique enables a number of applications such as clearer motion visualization, simpler creation of artistic cinemagraphs (photos that include looping motions in some regions), and new ways to edit appearance and complicated motion paths in video by manipulating a de-animated representation. We demonstrate the success of our technique with a number of motion visualizations, cinemagraphs and video editing examples created from a variety of short input videos, as well as visual and numerical comparison to previous techniques.

Large-scale motions of the guitar body can make it difficult to follow the finer-scale motions of the strings and fingers. We visualize the amount of movement by averaging the frames of the input video (left) and find that the body and fretboard of the guitar, as well as the strings and fingers are blurred because they move a lot. With our selective de-animation technique, we remove the large-scale motions of the guitar to make it easier to see the finer scale motions. Averaging the frames of our de-animated result (right) shows that the body and fretboard are sharp and therefore immobilized. Note that while the strings and fingers are sharper than in the visualization of the input video, they remain blurry because their fine-scale motions are retained in our de-animated result. We encourage the reader to view the paper video, to see this comparison in video form.

Research Paper

PDF (4.8M)

Hi-Res PDF (39.3M)

Video

MP4 (104M)

Teaser MP4 (28.6M)

Abstract

In this work we increase the apparent resolution of videos when viewed on a high-refresh rate display by making use of perceptual properties of the visual system. We achieve this enhancement by exploiting the viewer’s natural tendency to track moving objects in videos which causes the screen pixels to be projected at different sub-pixel offsets onto the retina. We estimate the eye motion using optical flow and use it to compute multiple low-resolution frames for each input frame. By watching these new frames at a high frame-rate, the viewer’s eyes integrate them over time and merges them into a single perceived frame with a denser pixel layout. In this work we also advance the existing approaches for resolution enhancement in the following ways. We combine current display resolution enhancement with super-resolution methods to enhance input videos that are at the display resolution. We derive a new perceived video model that accounts for actual camera sensor and display pixel shapes in order to achieve optimal enhancement. We analyze the degeneracies that certain motion velocities introduce to super-resolution and resolution enhancement, and offer algorithmic solutions for handling these scenarios as well as other difficulties that arise when dealing with the optical flow of natural videos. A user study finds that our approach achieves a noticeable increase in the apparent resolution for videos even when viewed on regular hardware (60Hz), and further enhances resolution when viewed on higher refresh rate displays (120Hz).

A comparison between frames a viewer perceives when watching the input video (a), and the ones produced by our method (b).

Research Paper

PDF (7.9M)

Abstract

Current multitouch frameworks require application developers to write recognition code for custom gestures; this code is split across multiple event-handling callbacks. As the number of custom gestures grows it becomes increasingly difficult to 1) know if new gestures will conflict with existing gestures, and 2) know how to extend existing code to reliably recognize the complete gesture set. Proton is a novel framework that addresses both of these problems. Using Proton, the application developer declaratively specifies each gesture as a regular expression over a stream of touch events. Proton statically analyzes the set of gestures to report conflicts, and it automatically creates gesture recognizers for the entire set. To simplify the creation of complex multitouch gestures, Proton introduces gesture tablature, a graphical notation that concisely describes the sequencing of multiple interleaved touch actions over time. Proton contributes a graphical editor for authoring tablatures and automatically compiles tablatures into regular expressions. We present the architecture and implementation of Proton, along with three proof-of-concept applications. These applications demonstrate the expressiveness of the framework and show how Proton simplifies gesture definition and conflict resolution.

Proton represents a gesture as a regular expression describing a sequence of touch events. Using Proton’s gesture tablature, developers can design a multitouch gesture graphically by arranging touch sequences on horizontal tracks. Proton converts the tablature into a regular expression. When Proton matches the expression with the touch event stream, it invokes callbacks associated with the expression.

Research Paper

PDF (2.6M)

Video

MP4 (20.5M) | YouTube

Additional Materials

Source Code

Abstract

Web-based social data analysis tools that rely on public discussion to produce hypotheses or explanations of the patterns and trends in data, rarely yield high-quality results in practice. Crowdsourcing offers an alternative approach in which an analyst pays workers to generate such explanations. Yet, asking workers with varying skills, backgrounds and motivations to simply "Explain why a chart is interesting" can result in irrelevant, unclear or speculative explanations of variable quality. To address these problems, we contribute seven strategies for improving the quality and diversity of worker-generated explanations. Our experiments show that using (S1) feature-oriented prompts, providing (S2) good examples, and including (S3) reference gathering, (S4) chart reading, and (S5) annotation subtasks increases the quality of responses by 28% for US workers and 196% for non-US workers. Feature-oriented prompts improve explanation quality by 69% to 236% depending on the prompt. We also show that (S6) pre-annotating charts can focus workers' attention on relevant details, and demonstrate that (S7) generating explanations iteratively increases explanation diversity without increasing worker attrition. We used our techniques to generate 910 explanations for 16 datasets, and found that 63% were of high quality. These results demonstrate that paid crowd workers can reliably generate diverse, high-quality explanations that support the analysis of specific datasets.

In our analysis workflow an analyst first selects charts, then uses crowd workers to carry out analysis microtasks and rating microtasks to generate and rate possible explanations of outliers, trends and other features in the data. Our approach makes it possible to quickly generate large numbers of good candidate explanations for outliers and trends in data.

Research Paper

PDF (2.0M)

Video

MOV (31.8M) | YouTube

Abstract

Tutorials and sample workflows for complicated, feature- rich software packages are widely available online. As a result users must differentiate between workflows to choose the most suitable one for their task. We present Delta, an interactive workflow visualization and comparison tool that helps users identify the tradeoffs between workflows. We conducted an initial study to identify the set of attributes users attend to when comparing workflows, finding that they consider result quality, their knowledge of commands, and the efficiency of the workflow. We then designed Delta to surface these attributes at three granularities: a high-level, clustered view; an intermediate-level list view that contains workflow summaries; and a low-level detail view that allows users to compare two individual workflows. Finally, we conducted an evaluation of Delta on a small corpus of 30 workflows and found that the intermediate list view provided the best information density. We conclude with thoughts on how such a workflow comparison system could be scaled up to larger corpora in the future.

Delta’s workflow explorer. The list view (left) displays workflow summaries. The cluster view (top right) groups workflows by similarity. The detail view (bottom right) shows individual steps and union graphs of shared commands.

Research Paper

PDF (1.8M)

Video

MP4 (30.9M) | YouTube

Abstract

We present a method that makes use of the retinal integration time in the human visual system for increasing the resolution of displays. Given an input image with a resolution higher than the display resolution, we compute several images that match the display’s native resolution. We then render these low-resolution images in a sequence that repeats itself on a high refresh-rate display. The period of the sequence falls below the retinal integration time and therefore the eye integrates the images temporally and perceives them as one image. In order to achieve resolution enhancement we apply small-amplitude vibrations to the display panel and synchronize them with the screen refresh cycles. We derive the perceived image model and use it to compute the low-resolution images that are optimized to enhance the apparent resolution of the perceived image. This approach achieves resolution enhancement without having to move the displayed content across the screen and hence offers a more practical solution than existing approaches. Moreover, we use our model to establish limitations on the amount of resolution enhancement achievable by such display systems. In this analysis we draw a formal connection between our display and super-resolution techniques and find that both methods share the same limitation, yet this limitation stems from different sources. Finally, we describe in detail a simple physical realization of our display system and demonstrate its ability to match most of the spectrum displayable on a screen with twice the resolution.

Given a high resolution input image (a), our method generates four low-resolution images (b). These four images are integrated by the observer’s eye when viewed on our vibrating display and lead to a perceived image of higher resolution (c). The perceived image is an actual photograph taken of the display. Image credits: Eddy Van Leuven.

Research Paper

PDF (40.0M)

Video

MP4 (66.4M)

More Results

Supplemental Materials