Team Eclipse
------------

   Paul Blaer <psb15@columbia.edu>
   Lenny Volchok <lv71@columbia.edu>
   Naveed Hasan <naveed@cs.columbia.edu>
   Corey Tripp <crt17@columbia.edu>


Programs
--------

   desk_scan is the program that actually computes the point cloud
     from the calibration information and the series of images.

   merge is a small tool used to merge too already registered scans
     together into a single scan.


Usage
-----

   desk_scan <light_cal> <camera_cal> <contrast> <desk_plane_epsilon>
             <top-ref-row> <bottom-ref-row> <output> <scan_files...>

   light_cal -- the light calibration parameter file 
                (output from find_light)

   camera_cal -- the camera calibration parameter file (output from
                 ccalib)

   contrast -- the contrast used for thresholding.  We generally used 
               a contrast of 40 for our images.
   
   desk_plane_epsilon -- this is used to cull out the desk plane from 
                         the resulting point cloud.  It eliminates any
                         point thats further than this value from the
                         desk plane (as computed from the camera 
                         calibration).  If you set this value to 0, it
                         will have no effect.  To get rid of the points
                         on the plane we generally used a value of 25.

   top-ref-row -- this is the reference at the top of the image.  This
                  should be a row where at no time does the shadow pass
                  over an object, it always should project onto the desk
                  plane.  If you place your camera correctly, a value of
                  20 should usually be sufficient for this.

   top-ref-bottom -- this is the reference at the bottom of the image.
                     This should be a row where at no time does the
                     shadow pass over an object, it always should
                     project onto the desk plane.  If you place your
                     camera correctly, a value of 400 should usually
                     be sufficient for this.

   output -- this is the name of the file in which we output the
             final pts file.

   scan-files -- these are the actually images of the stick passing
                 over the object.  They should be listed here in
                 chronological order (as the stick is going from left to
                 right).


   merge <cloud1> <cloud2> <outcloud>

   cloud1 -- the first point cloud
   
   cloud2 -- the second point cloud

   output -- the output point cloud


Notes 
-----

Before we get to this stage, we assume that we already have a set of
images of the object being scanned with a stick passing over it,
casting a shadow on the object.  We assume that the stick is passing
from left to right in the images.  We also assume that we have already
completed the light and camera calibration steps of the process.

The 3D point extraction is divided into 3 distinct phases:
    1) Compute the shadow plane for each image (time step).
    2) Compute the exact time that the shadow crosses each pixel.
    3) Compute the intersection between the ray shot through a given
       pixel with the shadow plane at the exact moment that the shadow
       crosses that pixel.

   Computing the shadow plane for each image is accomplished by
finding the three points that define the shadow plane.  First we have
the 3D coordinate of the light source.  This has already been computed
during the light calibration phase and is actually command line
argument to the program.  The other two points are computed from the
"reference rows."  Near the top and the bottom of the image we pick a
pair of rows that never intersect the object that we are scanning.
Therefore those rows pass through the projection of the shadow
directly on the desk plane.  For each image, we find the exact column
that the shadow edge (we are tracking the left edge of the shadow)
crosses each of these two rows.  Since we know that these two points
are on the desk plane we can directly apply the transformation given
to us by the camera calibration to recover their 3D coordinates.
These 3 points are computed for each image (the light coordinate
always stays the same, so we are really only computing two points per
image).

   We use a thresholding technique on the intensities of each pixel
along that reference row to determine the exact column where the left
edge of the shadow intersects with the reference row.  We need to keep
track of a number of different intensities.  These are grey scale
images, so the intensities range from 0 (black) to 255 (white):

   Imax -- The maximum grey scale intensity of any pixel along the
reference row.

   Imin -- The minimum grey scale intensity of any pixel along the
reference row.

   Icontrast -- The difference between the maximum and minimum values.
If this contrast is lower than the contrast threshold that we specify
(40 in our experiments) then we simply cannot distinguish between
parts of the image in shadow and parts that are not in shadow along
this row.  Images that do not have a contrast over this threshold on
their reference rows are discarded.

   Ishadow -- The threshold value for determining whether we are
inside of a shadow or not.  This is defined to be the average between
the minimum and maximum intensities.

   Once we have computed all of these values across the reference
row, we then go back and scan across the row a second time.  We look
for the pixel that crosses the Ishadow threshold for the first time.
(Pixels at the beginning of the image should be above Ishadow, and we
are looking for the first pixel that drops below this threshold).  We
then do simple linear interpolation between the pixel immediately
before we crossed the threshold and the pixel immediately after the
threshold to find the exact pixel at which the shadow edge crossed
the reference row.  Since we are interpolating, we get a column value
at the sub-pixel level. 

   Computing the time that the shadow edge crosses each pixel is done
in a very similar manner to how we compute the reference points.
However, in this case we look at a given pixel's value over time,
holding the row and column fixed.  In the previous case we hold the
row and time fixed and varied the column value.

   We again compute the intensities: Imax, Imin, Icontrast, and
Ishadow.  We get these values for every pixel.  If a particular pixel
never has an Icontrast value greater than the threshold set, we can
say that the pixel has either never been crossed by the shadow, or was
always in shadow.  In most cases this occurs in pixels that are always
in the shadow of the object that we are scanning.  No data is computed
for that pixels always in shadow.

   After making a first pass of the images and getting the intensity
values, we make a second pass of the image and find the first frame in
which the image has crossed the Ishadow threshold.  We then
interpolate between that frame and the frame immediately before it to
find the exact time (frame number) that the shadow edge crossed that
pixel.  We get this value at the sub-pixel level.

  We now have all the information that we need to extract the 3D point
value.  

  From the camera calibration, we have the normal of the desk plane
and the perpendicular distance between the desk plane and the optical
center of the camera (the origin of our system).  From this we compute
the equation of the desk plane in the format: ax + by + cz = d.

   To find the 3D coordinate of a pixel in the plane, we look up the
shadow crossing time for that pixel (computed above for all pixels).
We then go to our list of reference points that we computed for each
image and find the reference points for the exact time frame that the
shadow edge crossed this pixel.  Since the time is at the sub-frame
level we do another linear interpolation of the reference points,
which were only computed at for each whole frame.

These reference points are in 2D image coordinates (row,column).  By
shooting a ray through the reference points on the image and
intersecting that array with the desk plane, we get the 3D coordinates
of the two reference points for the time frame of the pixel that we
are trying to compute.  These two reference points, along with the
light position (computed from the light calibration) define another
plane, the shadow plane.  We intersect the ray shot through the pixel
that we are trying to compute with the shadow plane, and this gives us
the 3D coordinate of that pixel.  We repeat this process for all pixels
in the image to get the point cloud.
