Technical Evolution and Creative Constraint

One of the problems with digital SLRs is their large footprint. Not only the size and weight of the camera, but also that of the lens attached to it, especially for serious photographers interested in large apertures and high-quality glass. This is an issue that affects professionals and amateurs alike, since both groups might want to have a smaller, more unassuming device at hand for convenience, portability, or unobtrusiveness. It’s one of the reasons rangefinder cameras have remained popular for many types of photography, from journalism to weddings.

The problem is, any rangefinder or DSLR is still pretty big, or heavy, or both. There are lots and lots of compact, pocketable digital cameras on the market, but most of them produce images of less than desirable quality or make advanced photographic control difficult, or both.

Sean Reid and others have called these cameras “small sensor” cameras and remarked thoughtfully on the new affordances and constraints they bring to the medium — such as the large depth-of-field brought about by the very wide focal lengths required to create familiar fields of view on sensors far smaller than a strip of 35mm film. But because of their small size, the sensors in these cameras still often have trouble recording fine detail, especially in low light. The result is the frequently noisy images characteristic of compact digital cameras.

For several years now, Sigma has been designing a compact camera that would use a larger sensor, one roughly the same size as those used in most current DSLRs. The camera itself would remain small, with a fixed 28mm effective field of view lens and a set of sophisticated manual and automatic exposure controls. The result, dubbed the Sigma DP1, is billed as “a full spec compact camera with all the power of a DSLR.”

But the DP1’s sensor is not just larger than the average compact camera, it is also of a totally different type than the kind normally found in digital cameras of any size.

Most digital cameras use an imaging technology known as a Bayer sensor. Bayer sensors have a grid of photocells that see only shades of grey. An array sits in front of the sensors with a grid of red, green, and blue filters, one for each photocell. The filters are arranged in a specific pattern, producing a grid of shaded, colored pixels conforming to the Bayer color filter pattern. To turn this into a “normal” color photography, the camera runs an algorithm that interpolates a pixel’s color based on itself and its neighbors. To insure that the color information gets spread around a bit to avoid undesirable color moiré patterns, most Bayer sensor cameras have an antialias filter in front of the sensor, which blurs the incoming light somewhat to avoid distortion.

Sigma finally released the DP1 last month worldwide. I couldn’t resist buying one, partly because my Leica M8 has been off in Germany getting warranty service since February. This was my first encounter with the Foveon sensor, and I was pretty startled at the results. The files I got out of the camera totally defied my preconceptions about what such a small device could be capable of.

One of the first things I did with the camera was to take some comparison shots at different ISO levels. The detail and lack of noise at higher ISO was impressive, but something else bothered me. I seemed to notice the colors in my images change as I increased the ISO. First I wondered if the camera’s auto white balance was off, or if I had failed to use commensurate exposures. But when I checked on those things, neither was to blame. The colors in an image at ISO 400 and ISO 800 just looked slightly different — less saturated, it seemed — than at ISO 100, especially in the green hues.

So, I did a test shoot of the same subject at different ISO to see if my suspicions would be confirmed.

(click for a bigger version)

I’ve made the following tentative conclusions based on this quick attempt to get a recording the Foveon’s color and saturation shifts. In each case, the ISO 100 image is the baseline for comparison.

At ISO 200:

Color and saturation shifts are not noticeable.

At ISO 400:

Red shifts toward yellow by about 17 degrees.

Green shifts toward cyan slightly, and desaturates by around 20%.

At ISO 800:

Red shifts toward yellow by about 28 degrees.

Green shifts away from cyan slighly, perhaps by 5 degrees, but desaturates almost entirely.

I can’t say yet whether these shifts will always be correctable in post-processing. I was able to correct all of the above images back to something closely resembling the ISO 100 image, although I did not attempt to make any prints.

If my guesses are even remotely accurate, it would be possible for Sigma’s PhotoPro Foveon RAW conversion software to do the adjustment automatically based on the ISO reported in each image’s EXIF metadata But the Foveon and PhotoPro have been around for a few years already for use with Sigma’s line of DSLRs. I can’t imagine that they wouldn’t have attempted such a simple, even if optional, postprocessing correction if it were that straightforward. As a rule-of-thumb, it certainly would be possible to create a saved adjustment in another digital post tool like Adobe Lightroom or Apple Aperture. Even so, images vary in many ways and I doubt a single solution will apply in all cases.

The test gave me a better sense of what was happening, but not why it was happening. I did some more research and it appears that this color shift is characteristic to the Foveon sensor itself — it’s a property of the sensor.

To understand why, we have to compare the way Foveon and Bayer sensors work. We learned that in a Bayer sensor, each photocell is sensitive to only one wavelength of light — red, green, or blue. The camera’s software interpolates color based on the luminance values of a photocell and its neighbors.

But the Foveon sensor measures all wavelengths of light at each photocell. The sensor designers embedded a photosensitive material onto the silicon of the chip itself, making it possible for the sensor to record all wavelengths at once. As such, no interpolation is required — and moreover, no blur filter to insure that interpolation is possible without moiré. In theory, then, Foveon sensors offer both better color rendition and sharper images than Bayer sensors.

The color shifts come as a consequence of the Foveon sensor’s design. In a Bayer sensor, the increased light sensitivity of an ISO increase is implemented by amplifying the sensor’s signal before processing. Amplification increases both signal and noise, making both the measured luminance of each pixel and its interpolated color subject to increased error. This is why we see increased noise at higher ISO in images created on Bayer sensor digital cameras.

But in a Foveon sensor, the silicon itself is photosensitive to different wavelengths of light at different layers of the sensor. When the sensor signal is amplified for greater light sensitivity, it still uses the same basic method for detecting luminance. Color, however, is only measured when the light passes through the silicon to stimulate the photosensitive array below. I don’t yet fully understand why the green wavelengths are the most susceptible to color loss. Apparently the short wavelengths of blue are still reaching the color sensitive part of the photocell, but the green isn’t. Given the shape of the area of red sensitivity, it also makes sense that the reds absorb, but at slightly shifted values.

High ISO Foveon images also exhibit some luminance noise, but at levels far lower than equivalent Bayer filters. As such, I think it’s fair to say that color shift is the Foveon’s high ISO equivalent of Bayer image noise.

All those questions answered, the color shifts still really bothered me. Not because I expect perfect images from the DP1 under all conditions — every digital camera comes with trade-offs and its more interesting to me to consider those trade-offs as constraints or quirks instead of deficits.

No, what bothered me was how weird color shifting felt as a consequence of the photographer’s decision to boost exposure through higher light sensitivity rather than more light (aperture) or longer light (shutter speed).

I struggled for some time to figure out why this bothered me so much. Here’s a theory: the Bayer sensor’s method of amplifying light sensitivity is analogous to the film emulsion’s method of amplifying light sensitivity. But the Foveon sensor’s method of amplifying light sensitivity is not.

Higher speed films are more light sensitive because the grains of silver halide on the emulsion are larger than in slower speed film. When photons strike the crystals, they cause a chemical reaction that creates a small covering of silver on the film. The size and distribution of these coverings vary in proportion to the size of the grain.

There is an analogous relationship between film grain and image noise, especially luminance noise or chroma and luminance noise when a digital image is converted to black and white. Some digital photographers use high ISO noise as an aesthetic effect in the same way that film photographers choose a particular stock for the size of its grain. Whether the effect is pleasurable to a particular photographer or not is inconsequential. What is important is this: for photography in general, the stippling of Bayer image noise is aesthetically and materially coupled to the stippling of film grain, and both are produced when higher light sensitivity is introduced into the photographic process.

That’s grain. What about color?

Different film emulsions do produce different color effects — for example, Fuji Velvia is known for its high contrast and saturation. But those effects run more or less across the color gamut for a particular emulsion. Velvia does have a tendency to color-shift toward purple when exposed for long durations, over 10 seconds or so. But this is an edge case, not a condition brought about by increased native light sensitivity — Velvia’s measures ISO 50.

Another example of selective color effects in film comes from way back in the late 19th century, when films were orthochromatic, or sensitive only to blue or green light. Most photographic emulsions were orthochromatic from 1880 into the 1920s, when the market introduction of panchromatic film provided sensitivity to full spectrum light. Of course, the color filtration of orthochromatic images were less noticeable because the plates and the prints were all in black and white.

Interesting examples though they may be, neither the Velvia nor the orthochromatic examples describe what’s happening in the DP1. It would seem that there are no simple, photographically analogous relationships between light sensitivity and selective color shifts of the kind the Foveon exhibits.

Charles Maurer, however, has offered a fascinating optical analogy to explain what happens in the Foveon sensor. Maurer essentially argues that the Foveon simulates mesopic vision, the effect that human eyes experience in dim light when we see neither in black and white nor in color. This is perhaps a more apt analogy, although to make sense of it as a photographic guideline requires wrapping one’s brain around the idea that the dimness is relative to the sensor, not the human eye. Perhaps with practice, this sort of thing will become as second nature to the digital shooter as is grain to the film shooter.

In digital photography, ISO is an exposure tool as much or more than it is an aesthetic one. That is to say, because ISO is simulated on the sensor rather than made material on the emulsion, the photographer can change it at any time — no need to wait and load another roll. ISO’s role in exposure decisions has increased dramatically with digital photography — one might increase ISO by a stop when shooting with a lens that will open no further, for example. For this reason, the DP1’s Foveon sensor offers a new and somewhat mind-bending challenge to the photographer: such an act now means more light at the cost of color shift.

It is certainly possible to attribute the present state of affairs to a failing in the design of the Foveon itself, in Sigma’s implementation of it in the DP1, or in their interpretation of its results in the PhotoPro RAW conversion software. In the false situations like those of the test images above, it’s hard to imagine selective color shift as a virtue. But those images also do not push the effect in a particular aesthetic direction; they merely suffer its uncontrolled result.

I’m sure both Foveon and Sigma are looking for ways to improve performance in the hardware and software. And one part of me looks forward to a firmware or RAW converter update. But part of me wonders why technical evolution must always represent fault, rather than opportunity. It is not everyday that a creative constraint of a whole new order emerges in an art form. The Sigma DP1 along with its DSLR cousins offer just such a thing: a coupling between light sensitivity and selective color rendering.

(thanks to Mark Norton for suggestions that improved misleading statements in the section on Bayer sensors and moiré)