Color Seamlessness in Multi Projector Displays

Large are high resolution displays are essential for scientific visualization, entertainment and defense applications. A popular way to realize such displays is to tile multiple projectors together to create one large display. As opposed to a 19” diagonal monitor with 60 pixels/inch resolution, tiled multi projector displays are often 10ft x 8ft is size has 100-300 pixels/inch resolution. Thus we have in the order of 100 million pixels.

One of the most important issues is to make these displays seamless. Lot of work has been done in the past on the geometric registration methods. But there is no comprehensive model that models, defines and corrects for the color variation problem in multi-projector displays. The figure on the left explains the gravity of the color variation problem. Here, though we have a perfect geometric alignment, the color variation problem alone breaks the illusion of a single seamless display. Thus, our goal is to generate a multi-projector display that would look like a single projector display, i.e. we should not be able to tell the number of projectors making the display. Correcting this problem is often called achieving color correction or color matching. We call this achieving color seamlessness since we will show later in this document that color matching is a special case of achieving color seamlessness. Such a display (results generated from my research) is shown in the figure below.

 Display Before Correction Display After Correction

Background on Color

Color is a three dimensional quantity and can be represented in many ways. We will be using two different representations. In the perceptual representation, color is defined by its luminance (or radiance)  and chrominance. The one-dimensional luminance defines the brightness of a color while the two-dimensional chrominance defines the hue and saturation of a color.

The second representation is the one that we usually use in computer graphics where a color is defined by three primaries of red, green and blue (RGB). This is the representation in which the color is defined in the projector hardware. The range of colors that can be produced by a device can be expressed by a 3D volume which is called the color gamut of a device. Here is an example of the power point palette where we often work with both these representation.

Here the color palette consists of a two dimensional chrominance palette and a one dimensional luminance slider. We humans usually find it comfortable to work with. Once we choose a color using this representation, in the bottom we also get the corresponding RGB representation of the same color.

Main Contributions

My thesis has two main contributions.

1)     I design the emineoptic function that models the color variation in multi projector displays. The emineoptic function helps us to provide a formal definition of color seamlessness as an optimization problem.

2)     From several empirical analysis of projectors, we find that the chrominance variation is almost negligible when compared to the luminance variation across a multi-projector display, especially when the projectors are all of same model. Hence, we design a practical system that solves for the restricted problem of luminance or radiance variation and thus achieves photometrically seamless displays. We call this system PRISM for Perceptual Radiance Seamlessness in Multi Projector Displays. The term ‘perceptual’ here will become clear subsequently.

Properties of Color Variation

I categorize the color variation in multi projector displays in three different categories.

 Spatial variation in the chromaticity coordinates across a single projector. The green surface shows the luminance variation for input (1,1,1) across a single projector. The red surface shows the black offset.

Intra Projector Variation:  This is defined as the color variation within a single projector. The spatial luminance variation across a single projector shows significant fall off from center to the fringes. However, the chrominance remains almost constant spatially. These are illustrated on the left. Further, usually in any display device, the amount of light projected for input (0,0,0) should be zero. However, this is not true for projectors, and some light is always projected. This is called the black offset as is illustrated on the left.

Inter Projector Variation: This is defined as the color variation across different projectors. From extensive empirical analysis, we found that the variation in chrominance across different projectors of same model is negligible. However, the variation is luminance is significant. Even for projectors of same model, the variation in chrominance is much smaller when compared to that of luminance.

Overlap Variation: This is defined as the variation caused by the overlap between the projectors. If the chrominance of the overlapping projectors are similar, the chrominance of the overlap region varies negligibly than the non overlapping regions. However, even if exactly identical projectors are overlapped, the luminance in the overlap region is multiplied by the number of overlapping projectors.

Emineoptic Function

 Predicted Response Actual Response

To correct for the color variation, one needs to first capture the color variation in a multi projector displays. To this effect, we design the emineoptic function that models both the luminance and the chrominance variation in multi projector displays in a compact manner using a few parameters. The emineoptic function defines the color of the light reaching a viewer from a display coordinate for a particular input projected from all contributing projectors at that display coordinate. Now, to capture the color variation, instead of taking 224 images for 224 color which is both compute and storage intensive, one can reconstruct the few parameters of the emineoptic function and predict the color variation using the emineoptic function.

Thus, the emineoptic function identifies the different parameters that causes the color variation and provides a comprehensive framework to describe the intra, inter and overlap color variation. Thus, it provides an unifying framework to study all different algorithms for correcting the color variation problem across multi projector displays. It can be shown that any algorithm to correct the color variation should reconstruct the emineoptic function, modify it and then reproject the modified emineoptic function on the display. The accuracy of the method depends on the parameters of the emineoptic function reconstructed. The success of the color correction achieved depends on the method used to modify the emineoptic function. The interactiveness of the method depends on the parameters chosen for the reprojection. Further, this model is general and can also be used to model the color of cameras or any other device.

To verify the model, I reconstructed the different model parameters and predicted the response of the display to an test image displayed on it using the emineoptic function. This is called the predicted response. Then, I compared this with actual picture taken of the test image being projected on the display. This is called the actual response, as shown above.

Color Seamlessness

There has been a general mindset that to achieve truly seamless displays, one needs to match the color response at every pixel of the display. However, due to the acute spatial variation, this often means that we match the color response of all pixels to the worst possible pixel ignoring all the good pixels which are very much in majority. This has been the main assumptions behind many of the previous work in this direction like blending, gamut matching and using a common bulb for different projectors. Now, we are going to show that achieving seamlessness does not mean a strict uniformity of color responses across all display pixels.

 Luminance variation across a single projector for input (0,0,1) at all display coordinates. Luminance variation across a 2x2 array of four projectors for input (0,1,0) at all display coordinates.

Why do we see color seams?

As mentioned before, the goal is to make a multi projector display look like a single display. With that end, let us compare the luminance variation of a single projector with that of a multiple projector as seen in the left. Note that when a flat green field is displayed on a single projector, it looks flat to the human eye. But the luminance response is anything but flat. In fact, it can show as large as 80% fall off from center to fringes. However, the difference of this from the luminance response of a multi projector display is that it does not have sharp discontinuities, but has a slow gradual fall off. These sharp discontinuities in the luminance variation of multi projector displays are the cause of the color seams. This shows that perceptual uniformity does not necessarily mean strict photometric uniformity. This is also supported by many perceptual studies that show that humans cannot detect smooth color variations.

 Luminance response of the display before correction Luminance response of the display after photometric uniformity.

The next question is how to remove these discontinuities from the luminance function. A most simple way to do this is to match the luminance at every display coordinate to the minimum luminance as shown in the left. This is called photometric uniformity. However, there are two disadvantages of this. First, this leads to a severe compression in the dynamic range of the display. Second, it does not make use to available high system resources. Thus, though we achieve seamlessness, the quality of display reduces dramatically as shown below.

 A 5x3 array of fifteen projectors before correction A 5x3 array of fifteen projectors after photometric uniformity

This idea can also be generalized to 3D color from just luminance. Thus, strict color uniformity which is the goal of all color matching algorithms would yield poor display quality. This shows that strict color uniformity may not be the most desirable option. We provide formal definitions of both photometric uniformity and color uniformity using the emineoptic function.

Optimization Problem

We pose the problem of achieving color seamlessness as an optimization problem where the goal is to minimize the perceptual color variations while maximizing the display quality.  When this optimization is done only on the luminance, we achieve photometric seamlessness as opposed to color seamlessness.  These two can also be defined formally using the emineoptic function. Based on this idea we develop a system called PRISM that achieves such a photometric seamlessness.

 A 5x3 array of fifteen projectors before correction A 5x3 array of fifteen projectors after photometric seamlessness.

PRISM: Perceptual Radiance Seamlessness in Multi Projector Displays

PRISM consists of two steps.

1)     Off line Calibration Procedure:  This is repeated periodically and generates what we call the smoothing maps for each projectors.

2)     Online Image Correction:  These smoothing maps can then be used to correct any imagery on the display at interactive rates.

 The maximum luminance function of the green channel for a 5x3 array for fifteen projectors.

Calibration

The calibration has three steps.

Reconstruction: In this step a digital camera is used to reconstruct the luminance parameters of the emineoptic function of the display. This includes the maximum spatial luminance variation for each channel and the black offset, as shown in the left. Also, the non- linear luminance response of each channel of the projector is reconstructed.

Modification: Next, these luminance functions are modified using a gradient based optimization process. Thus the modified luminance function has an overall high dynamic range while the variation is controlled to be imperceptible. This is achieved by a linear constraints and objective functions and is implemented using linear programming. The amount of variation that can be tolerated also depends on the content. For example, for a high frequency movie, a large amount of variation can be tolerated, but for desktop environment that involves many flat colors, not much variation can be tolerated. Thus, smoothing parameter is used to decide the smoothness the luminance variation. Thus, the smoothest luminance function is one that is flat. This is nothing but the luminance function corresponding to the photometric uniformity. Thus, photometric uniformity is a special case of photometric seamlessness.  As we make the luminance functions smoother and smoother, the display dynamic range becomes smaller and smaller, reaching the minimum for photometric uniformity. These are illustrated in the figure below.

Reprojection (Part I):  In this step, smoothing maps (an attenuation map and an offset map) is generated from the modified and the reconstructed luminance functions. Further, a channel linearization function for each channel of a projector is generated.

Image Correction

Reprojection (Part II):  In the image correction step, the attenuation and offset map and the channel linearization function are used to correct any image to be projected on the display. These can be achieved in real time using the pixel shaders of the commodity graphics hardware.

 The modified luminance function achieved using different smoothing parameters. The smoothest is the flat luminance function. The corresponding displays. As the luminance function becomes smoother, the dynamic range of the display reduces.

Results

Here are a few results.

 A 5x3 array of fifteen projectors before color correction A 5x3 array of fifteen projectors after photometric seamlessness A 3x2 array of six projectors before color correction A 3x2 array of six projectors after photometric seamlessness

Related Publications

1. Properties of color variation in multi projector display, Aditi Majumder, Proceedings of SID Eurodisplay, 2002.
2. Color Non Uniformity in Multi Projector Displays: Analysis and Solutions, Aditi Majumder and Rick Stevens,  IEEE Transactions on Visualization and Computer Graphics, 2003.
3. LAM: Luminance Attenuation Map for Photometric Uniformity in Projection Based Displays, Aditi Majumder and Rick Stevens, ACM Virtual Reality Software and Technology, 2002.
4. Identifying and Optimizing the Emineoptic Function for Color Seamlessness in Multi Projector Displays, Aditi Majumder and Rick Stevens, Argonne National Laboratory Technical Report #260, 2003.

Related Courses

1. Large Area Displays for the Masses, Aditi Majumder and Michael Brown, ACM Siggraph, 2003.
2. Building Large Area Displays, Aditi Majumder and Michael Brown, Eurographics 2003.

Keywords

Large Area Displays, Tiled Displays, Multi Projector Displays, Color Matching, Color Balancing, Color Correction, Color Seamlessness.