Shapes from shading
by Wojciech Chojnacki
The SFS problem, i.e., the problem of reconstructing shape from
shading, can be stated in the form of the following question:
How can the shape of a smooth surface be reconstructed from its
black-and-white photograph? The varying intensity of the picture
is called shading.
The problem dates from the 1950s. Initially it consisted in determining
the topography of the Moon from its pictures. In the mid-70s it
attracted the attention of computer vision experts, who intended
to provide robots with an ability of reconstructing a 3D event
scene from 2D pictures, a feature necessary for the robot's free
displacement around the scene, as the observed fragment of space
is termed. A variant of the SFS problem appeared again in the
mid-80s as the problem of radar picture interpretation.
The first step in the consideration of the problem is to establish
the relation between the shape of the surface and the shading.
The relation defines in turn the picture irradiancy equation.
Irradiancy is the technical term for the intensity of the picture,
while the radiance of the scene, a related term, is the intensity
of light emitted by the scene. Picture irradiancy is proportional
to the scene radiance, with a coefficient not greater than 1.
To solve the SFS problem effectively, some simplifying assumptions
must be added. The basic assumption requires that the surface
be smooth. It is also assumed that the surface does not shade
itself and that the contribution of repeated light reflections
from the surface to the picture is negligible. Moreover, the light
source and the device where the picture is generated are usually
taken to be at a large distance from the surface. This implies
that we can assume the surface image to be produced by a beam
of parallel rays hitting the surface perpendicularly. Even more,
in that case the shading due to a small patch of the surface depends
only on the orientation of the patch in space rather than on the
absolute position of the surface in space. This assumption is
satisfied in practice when, for instance, the light source is
Sun, the device used to generate the picture is a photo camera
fixed on board of a plane flying high over the ground and the
surface observed is a fragment of the Earth's surface.
Let's fix a Cartesian system of co-ordinates in space so that the axes x and y be in the picture plane. Let u(x, y) be the height of the surface above the picture plane at the point (x, y). Thus every surface point is described by a triple (x, y, u(x, y)) and the image of a point so described is (x, y). With the simplifying assumptions mentioned above the picture irradiancy equation is of the following form:
where E(x, y) is the picture irradiancy at (x, y), R is the so-called reflectivity function, l is the unit vector for the direction of the light source and the last argument is the internal surface reflectivity, i.e., the albedo at (x, y, u(x, y)). The partial derivatives involved in the equation uniquely determine the vector normal to the surface at (x, y, u(x, y)) given by the formula
This vector defines for the plane tangent to the surface at (x,
y, u(x, y)) its orientation in space.
The tangent plane, when reduced to a small environment of the
point (x, y, u(x, y)), is quite
good an approximation of the surface patch included in the environment.
In consequence we may assume that the partial derivatives determine
the orientation of a small surface patch around (x, y,
u(x, y)). Given R, l
and , the picture irradiancy
equation defines all the admissible orientations of the patch
that lead to a picture of brightness E(x, y)
at (x, y).
As a condition on u(x, y), the picture irradiancy
equation is a first-order partial differential equation with two
independent variables. As it turns out, with R, l
and fixed and for an
arbitrary (non-negative) irradiancy E, such an equation
usually has infinitely many local solutions defined on
a fragment of the image domain. There may also exist many global
solutions, defined on the entire domain of the image, and there
may be no global solution at all as well. To get a unique global
solution, some boundary conditions must be imposed. The determination
of global solutions that satisfy the boundary conditions is an
interesting mathematical problem itself, still intensely studied.