A1-VirajKulkarni

Name: Viraj Kulkarni

Assignment: Texture Synthesis

Course: CS294-69, Fall 2011, UC Berkeley

Paper Implemented: Region-Filling and Object Removal by Exemplar-Based Inpainting. Antonio Criminisi, Patrick PĂ©rez, and Kentaro Toyama. CVPR 2004. (pdf)

Implementation

I have implemented the assignment in MATLAB.

The program needs the following inputs:

• Input image
• Object map: The object to be removed in the input image is painted in white and the rest of the image is painted black.

The program produces the following outputs:

• Output image with the region removed
• An image showing a plot of the confidence terms
• An image showing a plot of the data terms

I have listed a few examples where the algorithm worked well and a few where it did not work well.

Successful Examples

Below are a few examples where the algorithm (and my implementation of it) works reasonably well.

The insect has been removed from the picture:

The girl has been removed from the picture:

The frame of the swing has been removed from the picture:

Analysis

Let me take one of the above examples to break down the process for object removal.

Input Image

This is the input image.

Object Map

The region to be removed and filled in is painted in white color and the remainder of the image is painted in black. Multiple regions can be marked in white and they can be disjoint.

Patch Priorities

Region filling is done on a 'patch by patch' basis and not on a 'pixel by pixel' basis. As per the paper, this approach is not only faster but also gives better results. Every patch on the boundary of the removal region is assigned a priority which is calculated from two attributes assigned to every pixel in that patch.

• Confidence term: This represents how confident we are of the value of that pixel. As the region filling proceeds, the values of the confidence term goes on decreasing. The confidence term gives priority to the pixels that were filled early on.
• Data term: This gives priority to pixels on the continuation of image structures. For instance, if there is a strong edge leading into the removal region, the pixels inside the removal region which would lie on this edge will have higher data terms.
• Balance: The confidence term enforces a concentric fill order while the data term gives preference to filling those patches which lie on continuation of structures. These terms, in a way, balance each other.

Confidence Term Plot

This plot shows the confidence terms of the pixels in the above example:

Data Term Plot

This plot shows the confidence terms of the pixels in the above example:

Result

The filling proceeds patch by patch. The confidence values of each pixel are updated after filling the patch. This is the final result.

Unsuccessful Examples

The examples listed below are the ones for which the algorithm does not produce images which seem correct.

This is my photo!

The king is removed from the picture.

One of the people has been removed :

More Examples

More example outputs along with their confidence and data plots can be found in a zipped file File:A1-VirajKulkarni-Outputs.zip