Spring 2010 - CS477/577 - An Introduction to Computer Vision

Assignment Four
Due: Saturday, March 6, 2010 (before sunrise Sunday).
Credit (U-grad): Approximately 7 points (Relative, and very rough absolute weighting)
Credit (Grad): Approximately 7 points (Relative, and very rough absolute weighting)

This assignment must be done individually

You can do this assignment in either Matlab or C/C++.

It is probably a bit easier in Matlab, but doing some of the assignments in C may prove to be a useful exercise. If you think that vision research or programming may be in your future, you might want to consider doing some of the assignments in C/C++. If you are stretched for time, and like Matlab, you may want to stick to Matlab. Your choice!

Information for those working in C/C++.

Assignment specification

This assignment has three parts, two of which are required for undergrads.

To simplify things, you should hard code the file names in the version of your program that you give to the TA. This request might be superseded by information provided by the TA on the list serve.

The big picture.

In this assignment you will reinforce your understanding of shape from shading in the simplest case of photometric stereo. The treatment for the first two parts will basically follow the notes. For the last part, grad students will study things a bit more in detail.

Below you will find images of a synthetic Lambertian surface imaged under seven different known lights that are provided. The goal is to determine the surface normals at each pixel, and then from that, a depth map for the surface.

Part A

The following seven files

    /cs/www/classes/cs477/spring10/ua_cs_only/assignments/4-1.tiff  
    /cs/www/classes/cs477/spring10/ua_cs_only/assignments/4-2.tiff 
    /cs/www/classes/cs477/spring10/ua_cs_only/assignments/4-3.tiff 
    /cs/www/classes/cs477/spring10/ua_cs_only/assignments/4-4.tiff 
    /cs/www/classes/cs477/spring10/ua_cs_only/assignments/4-5.tiff 
    /cs/www/classes/cs477/spring10/ua_cs_only/assignments/4-6.tiff 
    /cs/www/classes/cs477/spring10/ua_cs_only/assignments/4-7.tiff

(smaller versions in case the above are too slow in Matlab)

    /cs/www/classes/cs477/spring10/ua_cs_only/assignments/small/4-1.tiff  
    /cs/www/classes/cs477/spring10/ua_cs_only/assignments/small/4-2.tiff 
    /cs/www/classes/cs477/spring10/ua_cs_only/assignments/small/4-3.tiff 
    /cs/www/classes/cs477/spring10/ua_cs_only/assignments/small/4-4.tiff 
    /cs/www/classes/cs477/spring10/ua_cs_only/assignments/small/4-5.tiff 
    /cs/www/classes/cs477/spring10/ua_cs_only/assignments/small/4-6.tiff 
    /cs/www/classes/cs477/spring10/ua_cs_only/assignments/small/4-7.tiff

are seven images taken of a Lambertian surface with the light at seven different directions. Those directions are (in order):

      0.44721360   0.00000000   0.89442719
      0.27872325   0.34950790   0.89451528
     -0.09788109   0.42884510   0.89805967
     -0.37613011   0.18113471   0.90868936
     -0.35093186  -0.16899988   0.92102436
     -0.09599213  -0.42056900   0.90216807
      0.26886278  -0.33714326   0.90224566

You can assume that the projection is orthographic, with the z axis being normal to the image plane. You may recall that this means that the point (x,y,z) is simply projected to (x,z,0). If we ignore issues of rotation and units, this is like an aerial photograph where the points are far enough away that the relief does not matter.

A possible point of confusion regarding data formats is that images are typically indexed by row (increases in the direction that you normally think of as negative Y), and column, (increases in the direction that you normally think of as positive X). It is best to keep this convention for this assignment. If you use a different one, such as using X as the horizontal axis, you will get a different solution that is more difficult to interpret.

Now the meat:

Use photometric stereo to compute the surface normals at each point of the image. Notice that the surface has different albedo in the four quadrants. The normals that you compute must be independent of this. Demonstrate that you have good values for the normals by creating an image of the surface as if it had uniform albedo, and though it was illuminated by a light in the direction of the camera (i.e., in direction (0, 0, 1). You can assume that the albedo/light combination is such that a surface that is perpendicular to the camera direction (i.e., the normal is in the direction (0, 0, 1) has pixel value (250, 250, 250).

Submit code to produce this image, and the image.

Part B

Now compute a depth map of the surface, and make a 3D plot the surface z=f(x,y). Do this by integrating the partial derivatives along a path. You can specify any point you like to be the reference "sea-level" point with z=0.

For those working in C: I have provided a simple 3D plotting routine. Unfortunately, it is a bit crude, but it is sufficient for this assignment.

Submit code to produce this plot, and the plot in an image file (a screen shot is OK).

Part C (Required for grad students only).

Extend the assignment in some way, and let the TA know what you did in sufficient detail. Hand in what is appropriate. Full marks require going beyond the minimum effort. Extensions that come to mind include:

Explore the effect of noise. Add noise to the images in increasing quantities, and see how the surface reconstruction degrades. Does averaging over multiple paths improve the reconstruction? How can you measure how well you are doing?

Explore the effect of assuming a different shading model than Lambertian. For example, what if the brightness was proportional to the square root or square of the dot product.

Create some data for yourself that for some different shapes and surfaces, and try to solve for them.

Attempt to solve the problem without the light directions. This is a bit challenging, and a perfect solution is not required. However, you should explain what you have tried.

Your own creative idea!

What to Hand In

To hand in the above, use the turnin program available on lectura (turnin key is cs477_hw4).