Kobus Barnard, "Practical Colour Constancy," Phd thesis, Simon Fraser University, School of Computing (1999)
Full text with out colour plates (PDF)
Full text with out colour plates (gzipped postscript)
Colour plates (8 tiff files, 36 Megs)
The image recorded by a camera depends on three factors: The physical content of the scene, the illumination incident on the scene, and the characteristics of the camera. This leads to a problem for many applications where the main interest is in the physical content of the scene. Consider, for example, a computer vision application which identifies objects by colour. If the colours of the objects in a database are specified for tungsten illumination (reddish), then object recognition can fail when the system is used under the very blue illumination of a clear sky. This is because the change in the illumination affects object colours far beyond the tolerance required for reasonable object recognition. Thus the illumination must be controlled, determined, or otherwise taken into account.
The ability of a vision system to diminish, or in the ideal case, remove, the effect of the illumination, and therefore "see" the physical scene more precisely, is called colour constancy. There is ample evidence that the human vision system exhibits some degree of colour constancy. Interest in human vision, as well as robotics and image reproduction applications, has led to much research into computational methods to achieve colour constancy. Much progress has been made, but most work has addressed the problem in the context of synthetic data and quite simple physical conditions. However, in order to serve the needs of the proposed applications, it is necessary to develop and test computational colour constancy algorithms for real image data. This practical development of computational colour constancy is the focus of this work.
In order to study and to use computational models for real image data, it is necessary to develop a model of the physical characteristics of the vision system of interest. Specifically, we want to predict the camera response based on spectral input, and as part of this work I propose a new method for doing this. In addition, the spatial variation of the optical system, and its noise characteristics are considered.
The next part of this study is the comprehensive testing of current colour constancy algorithms. I present results for a number of algorithms both on synthetic data, and on a large database of real images. The image database consists of images of 33 scenes under 11 illuminants. The illuminants were chosen to approximate a uniform coverage of the span of common natural and man made illuminants. The results from this study, as well as the testing paradigm that was developed, provide a foundation for the rest of the work, which specifically sets out to improve computational colour constancy on image data.
The first area studied to improve computational colour constancy is the sensor sharpening method of Finlayson et al. Sensor sharpening has been proposed as a method of improving colour constancy, but it has not yet been tested in conjunction with real colour constancy algorithms. In this work, I set out to test the degree to which sensor sharpening can help current colour constancy algorithms. I find that the current sharpening methods do not address the needs of this domain, and thus as part of this work, I propose a pragmatic new sharpening method.
I then propose several improvements to variants of Forsyth's CRULE algorithm. The first is a new method of choosing a solution from the feasible set. The second reduces the reliance of the method on the diagonal model. This method enables CRULE to be extended to work with fluorescent surfaces, and is the first algorithm to deal with such input. In order to consider colour constancy in the face of fluorescence, I propose a simple, but effective model for characterizing such surfaces. In addition to the CRULE extension, I also use this characterization to implement a fluorescent capable version of Finlayson et al.'s Colour by Correlation method. These algorithms are tested on generated data, the 321 images taken for the comparison work, and, most importantly, on a set of 59 images with fluorescent surfaces taken for this purpose.
A third modification to gamut mapping algorithms proposed in this work allows them to use specularities to an advantage. Specularities have long been put to use by colour constancy algorithms, but existing algorithms that use specular information are limited in that they require such information to be present. The method presented here combines the use of specular and non-specular information. Thus the algorithm can provide good performance when good specularities are present, or when there is a reasonable diversity of matte surfaces, or some combination of these conditions. Furthermore, it should be noted that the method uses the fact that specular reflection tends to be relatively bright. Most current methods which use specularities do not consider this. Finally, the method is applicable to specular reflection from coloured metallic surfaces, and is, in fact, the first colour constancy algorithm which can use such input gainfully.
Another algorithm chosen for close study is Finlayson et al.'s chromaticity based Colour by Correlation method. This method is attractive because, unlike the CRULE derivatives, it can take advantage of statistical information about the world. However, the comparison work done as part of this thesis indicates that the pixel brightness is also a very important source of information--information that cannot be used by chromaticity based methods. Therefore, I developed a three-dimensional version of Colour by Correlation. This algorithm performs better than all algorithms considered in this thesis when tested on synthetic data. Notably, it performs significantly better than the chromaticity version on both generated and image data. Unfortunately, the performance of the new algorithm on image data still lags slightly behind the best of the CRULE derivatives.
In summary, the many pragmatic difficulties encountered when computational colour constancy meets the real world demand an approach that embraces modeling the physical nature of the world, analysis of camera characteristics, and the use of real images for comprehensive testing and development of algorithms. These requirements for successfully applying colour constancy to the real world have driven the present work.
Keywords: colour, color, colour constancy, specularities, fluourescence, gamut mapping colour constancy, diagonal models, colour camera calibration, colour based object recognition, colour by correlation
Full text with out colour plates (PDF)
Full text with out colour plates (gzipped postscript)
Colour plates (8 tiff files, 36 Megs)