Bringing color to the color-blind

The world may be in living color, but not everyone sees it that way.

Peter Jones and longtime business partner Dennis Purcell, who met at Polaroid, over the years have developed technology for color meters for commercial photography and film, and invented an architectural model camera that Polaroid, once a leading player in the photography business, licensed and produced.

Now at Boston-based Tenebraex--where Jones is president and Purcell is senior scientist--the two have taken their color technology knowledge into both the dark night and the digital world of color-coded data.

Their refinement of the previously unsubstantiated Retinex theory of color vision, put forth by Polaroid founder Edwin Land, may both help soldiers carrying out nighttime missions and bring some relief to the 8 percent of American men and half-percent of American women who struggle daily with color blindness.

Jones, the photographer turned entrepreneur, gives his opinion on color blindness in a digital world driven by colorful data, and the stark reality of what color night vision, technology Tenebraex is pioneering, can do for the military.

Q: You've developed color night vision and software for the color-blind as well. How did you get interested in color technology?
Jones: Both Dennis and I used to work for Polaroid in years past, back when they were a big deal, and color was always of interest to us. Dennis actually invented a color meter for photographers that Polaroid licensed back in the early '80s. He came up with innovative ways for measuring the characteristics of light so you could come up with a set of filters that you could use on slide film, which was very intolerant of different kinds of light sources.

Dennis and I had invented a camera type that we had actually licensed to Polaroid and they put in production years ago. It was an architectural model camera. We've always had an interest in color, in color matters. Edwin Land (the founder of Polaroid) developed a theory of how your brain perceives color, which he called Retinex theory, that we always thought was correct.

If Edwin Land's theory wasn't right, you would not see color when you looked through our device.

What is the theory?
Jones: Retinex says that rather than measuring, what's first important to you is the name of the color, and being consistently independent of what color light is illuminating a scene. So if you're seeing a tiger at sunset or a tiger under a green forest canopy, you want it to still look orange even though the light that's hitting your eye may be a completely different color depending on what the illuminating light is like. And his Retinex theory said that your brain compared images of the three different channels--the red, green and blue senses in your retina--and from that, looking at the relative brightness, it figured out what the correct color name was and that's what you saw.

Depending on the brightness?
Jones: Well, by comparing the brightness. For instance, with your green senses in your retina, a piece of orange paper looks dark, while a piece of green paper looks bright. By comparing the two, your brain figures out what the correct color should be. That's why, for instance, if you take slide pictures under fluorescence, incandescent (light) or daylight, the pictures would be orange or green or blue or normal depending on the color of light. At the same time, to your eye, everything looks consistently the same color. Land's theory was about how colors look consistently the same pretty much irrespective of what color light is illuminating.

They stopped teaching his theory in school because he was just a businessman. We (Jones and Purcell) always thought he was correct. Four-plus years ago we figured, whatever the inspiration comes from, this conceptual framework might allow us to just figure out a way of making a practical four-color night vision system. And for our color night-vision system we figured out a way to make your brain see all of the colors while using only two channels, not three, which most people say you can't do.

You know, at conferences I'll say, "well, we're working with the Retinex theory" and, you know, people go, "Ha, ha, ha; he was just a businessman. He wasn't a real scientist." Well, he was one of the smartest people in the last century and we think his theory was right. And if his theory wasn't right, you would not see color when you looked through our device.

Can you explain that?
Jones: Most people think you need red, green and blue (RGB) or cyan, magenta and yellow (CMYK) in order to render all of the colors. But that's not true. You can do it with two dimensions. I don't know if you've ever seen a color diagram that has all the colors of the spectrum, but you can see all the colors by rendering it on a two-dimensional graph. What you need a third dimension for is so you can see, say, dark red versus light red or dark blue versus light red. The vertical access is only for brightness.

Are the ColorPath color night vision goggles and the EyePilot software for the color-blind born out of the same technology?
Jones: It's in the same researching way of thinking about the problem because then we said...You know, I don't know where this thought came from. But one way of describing a color-blind man, and most people who are color-blind are men, is that the output of their red and green channels are fused and they really only have a two-channel system. Well, I wondered if anything we learned from the ColorPath with a two-channel full-color system would allow us to help color-blind men see the full range of colors, to differentiate the full range of colors.

Are you color-blind, if you don't mind me asking?
Jones: No.

So what gave you inspiration to translate that theory and the color vision technology into developing this type of software?
Jones: You know, when I was a kid I went to school and learned from black text on white paper with an Encyclopedia Britannica. My daughter started elementary school doing homework on computers with broadband and the Web, and the way computers interact with people, color is a way that you differentiate lots of data on computer. You have pie charts, and stock charts, and weathers charts and scattered graphs.

In the real world the color-blind person will have cues. The red light on the stoplight is generally at the top, things like that. On the computer if you're color-blind, you don't know when you're looking at something if you're missing most of the information or you're getting it.

It affects 1 out of every 12 men, which means on average one kid in every class is color-blind. They're doing their research and testing and whatever on a computer, and sometimes they can get all the information and sometimes they don't have a clue. And it's a funny clientele. You know, guys tend to be taught to suck it up, don't complain. And you can't identify other color-blind people by looking at them. So, it's very hard.

You know, there are an equivalent number of men who have ADHD (attention deficit hyperactivity disorder) and they have buildings and foundations and things that study ADHD. But for the color-blind they don't, even though there are some safety and usability issues in terms of red lights and such. Say you're a really good (stock) trader, but the company is throwing you a lot of data and it's all color-coded. Maybe it takes you 20 minutes to do something someone else is doing in 6 minutes, to read these complex charts, and so you're a little bit less efficient.

So is the EyePilot software corrective? Will it allow someone with color-blindness to see the same thing I see when I look at the color wheel?
Jones: No. What we're doing basically among a lot of the tools is, it's either simplifying or putting some information into things other than colors. For instance, to a color-blind person there may be eight segments on a pie chart and they can only tell apart two or three so they don't know looking at it what anything means. We have a Flash tool. If you have a pie chart and you say OK, I want to know what this segment means, you click on it in the key and everything in the EyePilot window with that color flashes.

Color-blind people don't want to draw attention (to their condition). And so the idea with the cell phone is that people will think you're text messaging.

Or if you're looking at a color-coded subway map and you're trying to get from here to there and they all cross over each other, you're not sure which route is consistent. You can click on one route, and everything in the EyePilot window with that exact color stays the same, and everything else changes to a gray-scale image.

It isn't just for the color-blind.

You mean, it can help anyone overwhelmed by too many colors?
Jones: Yeah. Computers have gotten so capable and able to relate so many data sets. Color is what computers use, but computers now can exceed the resolution ability of everyone. The information is important, and you do want to differentiate all those different kinds, and a map is the most effective, fastest way...but you can do millions of colors, so the natural progression is, wow, let's use 20 colors or 30 colors.

So does it rearrange the color scheme so that someone with color blindness can more easily distinguish a pie chart?
Jones: One of the tools does hue. It interactively remaps all the colors in the image. We see a little rotating dial and you can turn it until you see a place where now you can separate adjacent colors clearly.

Can you set the software once for your personal type of color blindness and then leave it?
Jones: No, because a color-blind person can't differentiate all the colors in the spectrum at the same time. For each image the color-blind person needs change. A color-blind person sees a two-dimensional slice through a three-dimension color space because they only have two channels. So any one shift in the color is just rotating the orientation of that slice, but it's still only a slice. What you need to do is to simplify or separate. For instance, two colors on a map that look quite different to someone with normal color vision look almost identical to a color-blind person. Using the hue tool they can rotate and all of a sudden there'll be a place where, "Oh! These two colors, which look the same, look like there's one area of color now. I can see clearly there's two sets of information here."

So what type of color blindness does the software accommodate?
Jones: Any type. And you don't need to do any testing, and that's because it's interactive. There've been other techniques that people have tried with filtered glasses and so forth. They make some colors easier to tell apart, but at the same time (they)'re going to make some colors that they could tell apart more difficult to tell apart. There's no technology that we know of that will allow a color-blind person to differentiate all colors.

Is the EyePilot software available for both Mac and PC users?
Jones: Yes. For $35. There's a 30-day free trial at Colorhelper.com.

Can you now explain how the cell phone version of EyePilot works?
Jones: We used as a test platform a Palm Pilot with a camera so that you would look at the real world on your screen and then be able to find out, using the different EyePilot tools, what a certain target color was or use a tool to differentiate.

Based on talking with color-blind people, they don't want to draw attention (to their condition). And so the idea with the cell phone is that people will think you're text messaging. No one knows you're using an assistive aid. You're just tapping on your cell phone. Meanwhile, the cell phone is telling you when you're standing in front of the bus map that's the route you take or that tie is green. But it's quiet, it's in the background and you're not drawing attention to yourself, but you're empowering yourself. That's why we think a cell phone platform is a good way to go with it.

When would the software for the cell phone be out?
Jones: Well, our job now is to go out and convince cell phone service providers or the hardware providers that this is a good teacher. Part of it is an educational aspect first, because most people don't realize how many people are color-blind. Certainly most people don't realize how debilitating it is in certain areas. Again, statistically it's 1 out of every 12 men.

Can you briefly explain the technology behind your ColorPath color night vision goggles?
Jones: The expensive core of a night vision device is the image intensifier tube. It takes the light from the front end and magnifies it 10,000 times and then projects it on a green screen at the back. We have two very specific, very difficult-to-make filters, one in the front and one in behind the tube. But there are two channels and that's our innovation. Each filter has two channels and it alternately puts one channel in front of the tube. So we're on the simple end mechanically of the technology spectrum. The composition of the filters is what took us a couple of years to get control of and figure out how to leverage.

Because the colors that you "see" aren't actually hitting your eye. It goes back to that theory that Land had about how your brain looks at the data it's getting from your eyes and figures out how to paint the colors that you see in your brain. We came up with a way to give your brain these two channels of information, to fill in the colors.

Your eyes actually are actually are pretty crappy if you think about. It's a single lens made out of biological material and optical nerves that aren't very big.

None of those colors, or few of those colors, are actually hitting your eye, but your brain is making the scene look correct for you. The Retinex theory gave us the framework to think about how to do that. The practical advantage is that U.S. military owns tens of thousands of image intensifier tubes, the expensive part, and they don't need to get new ones. They can use an existing tube in our housing.

What does this mean for medics and military working in the field at night? What can they do now that was impossible before?
Jones: Well, let's take the medic case first. Under green night vision, blood looks the same color as water. So if you trying to set an IV with the regular green night vision you can't tell, because the fluid looks the same color as the blood. (Editors' note: Jones explained that medics need to see a little blood to know if they've hit a vein.) Obviously in combat we tend to fight at night using night vision because the bad guys by and large don't have night vision. You don't want to turn on the light if you're treating someone because that makes you a target.

There are other things where color at night is useful. Imagine you're looking for a kid who's lost in the woods and has a red sweater.

One of the guys we were showing prototypes to worked after (Hurricane) Katrina. He said you're going into a place, you've lost power, everything is ripped apart and you're looking at something trying to figure if it's an electric line or a water hose that you're about to step on.

We've been working with various military and Special Forces contacts to get their feedback. Our first production we expect to have in the market sometime this summer. And then we start the whole process of selling it into the Army, which we have done with other innovative technologies, which has surprisingly a (long, drawn-out process).

Is this only going to be sold to the military, or could medics working in civilian life be able to purchase these?
Jones: Our first concentration is on the military medic market.

Depth of field is another problem with night vision goggles. Does ColorPath deal with that issue?
Jones: It helps because your brain uses color for what's called scene segmentation. Your eyes actually are pretty crappy if you think about. It's a single lens made out of biological material and optical nerves that aren't very big. But you have a very expensive computer inside your head that takes these lousy images and does all sorts of enhancements. Your brain is using color at that first glance to help you make sense of what you're seeing in front of you.

Your brain uses color at a very primitive level for recognition. For instance, if you're looking with regular night vision out across a golf course, you have to sit there and try to figure out what's grass, what's the sand trap, what's the water hazard. With color you go, "Oh, grass, sand, water, and then there's some more grass on the other side," because the colors are giving you the answers instantly and you don't have to make sense of the different gray-scale values. So it isn't depth perception per se, but you're orienting yourself very quickly, so it does help a lot in terms of how quickly I make sense.

What is the battery usage?
Jones: You can get overnight with a pair of AA (batteries), which is the usual military goal.

I'm assuming this works in real time?
Jones: Yes, and that's another issue because there are other ways you could think about doing color, but now it has a quarter-second delay and that's not a good thing.

Can you explain the difference between the medic and the military ops version?
Jones: We can give you color down to a thin crescent moon, but below that our technology is taking too much energy out of the system for you to see well. We can move the filters out of the way (with the combat one). You just turn the knob, the filters go out of the way, you're back with what you started with (green night vision), but that isn't necessary for the medic.