HW3: Homography mosaics

Due June 3 in EEE DropBox at 11:00 am

In this assignment, entirely prepared by the fearless David Martin, you will align multiple photographs together using homogrpahy transformations. In the first part, you will do this by manually clicking on corresponding points. In the second part, you will automate this process using SIFT patches and correspondence estimation with RANSAC. A useful reference is this paper on using SIFT for stitching mosaics. The following directory contains the source images for alignment, along with skeleton code:

• Downloads

Part I:

The first image in each example provided is the central image. It's simplest to construct a mosaic from a central image and a set of peripheral images, since we then need to find just one homography for each peripheral image. Use the cpselect and cpcorr functions to manually select and refine at least 4 pairs of corresponding points in each overlap region between the central and peripheral images. These points should be located on high contrast corners.

Next, compute the homography using SVD for each set of correspondences to provide the mapping from points in the central images and points in the peripheral images. You'll probably want to write two functions such as:

function [H] = computeHomography(x1,y1,x2,y2)
function [x2,y2] = applyHomography(H,x1,y1)

Note that the inverse mapping is given simply by inv(H).

1. Use the central image's coordinate system for the final mosaic. You need to figure out the min and max pixel coordinates when the peripheral images are mapped into the central image's coordinate system. Simply apply your homographies to the peripheral images' corner points to find the extent of the final mosaic. Note the final mosaic will be larger than the reference image, and so has a different origin!

2. Now generate a meshgrid of points to cover the mosaic, send these points through each homography in turn, and use interp2 to extract pixel values from the source images. For each input image, this produces an image the size of the final mosaic; pixels that are outside the source image are set to NaN.

Finally, assemble the mosaic from the remapped source images. Use isnan(I) to get a mask for each one that tells you which pixels are valid and which are invalid. For Part I, you may simply take a pixel from any valid image.

Part II:

1. The first step is to compute SIFT interest points and image descriptors on each image. There are numerous matlab packages for such computations. I highly suggest using this code.

2. For each patch in the reference image, compute the best-matching patch in the other image. For each overlapping image pair, use RANSAC on the paired corner locations to find the best homography.

3. Construct the homography as in Part I using the automatically computed homography.

4. Use the above code to align each peripheral image to the central reference image.

Writeup:

This assignment must be submitted electronically to the EEE dropbox. Create a single zip/tar file with your code and final writeup.

1. Show the resulting mosaic (from manual correspondences) for one of the provided examples. Show also the remapped source images before they are blended into the final mosaic.

2. Show the mosaic constructed with RANSAC.

3. Show all the interest point patches that were in-liers in the homography estimated by RANSAC.

Matlab Tips

1. Don't forget the hold on command if you want to plot one thing on top of another.

2. Running cpselect is a pain. Don't do it more than once for any pair of images! You can use the save and load functions to save and restore variables from your matlab workspace.

3. Do not use the full size images. Resize them down so that your debugging goes fast. For your final results, 25% size is sufficient.

4. You can treat the central image as just another source image by using eye(3) as its homography.

5. Use 'linear' interpolation with interp2; it's fast and looks good.

Extra-credit