I work at an advanced materials company called Nanosys that manufactures Quantum Dots for displays. In my work the display business, I’ve noticed a lot of confusion around color science and image quality.
I’ve started this blog as a way to get to the bottom of some of this stuff and share what I’ve learned. My aim is to synthesize information I’ve accumulated in order to provide a useful way for us to understand why great color is so important and exciting.
CES 2014 has come to a close and while many predicted a lackluster year, there were actually a number of interesting developments in displays. These are my top three CES 2014 display technology takeaways:
4K is here now, content isn’t the issue anymore
Analysts are still having a tough time figuring out exactly how quickly 4K will be adopted. According to data presented by the LCD TV Association at the show, last year analysts thought we’d see about 2 million 4K sets in 2014. Actual numbers turned out to be about 13 million (with 10 million predicted in China alone). 4K is clearly happening faster than most predicted but, if anyone still doubted that 4K will be mainstream in the next couple of years, this year’s CES should have made it clear that its here today.
Just about every major set maker showed off 4K sets this year in every flavor imaginable from LCD to OLED. But, hardware has never been the real barrier to 4K adoption– it’s all about the content or lack thereof. At CES 2014, the content issue was resolved a couple of different ways: Netflix is making 4K delivery a priority and upscaling is starting to look really good. With great upscaling (in one demo I saw from Technicolor it was nearly impossible to pick native 4K from upscaled 1080P) and instantly available content from Netflix, I don’t think content availability will continue to be a barrier for 4K adoption.
Wide Color Gamut and High Dynamic Range
Both Dolby and Technicolor demonstrated some very impressive high dynamic range and wide color gamut technologies that make for much more immersive viewing experiences. With it’s new Dolby Vision technology, Dolby has created essentially a new standard that uses a layer of metadata on top of today’s broadcast standard to deliver wider gamut and dynamic range with the content creator’s intentions intact. This is significant because it won’t require a new broadcast standard. Much like their surround-sound offerings (which deliver stereo audio if you have two speakers and full surround if you have six), all you’ll need is a Dolby-capable set to see the advantages, it won’t be something the viewer has to worry about.
Similarly, Technicolor is doing some on-the-fly processing to incoming content in realtime to pull out extra dynamic range and color. Again, no change in broadcast standard required for this and that’s the key. While there’s some danger that artistic intent will be altered with this approach, the demos I saw looked great. Skin tones and memory colors were kept in check while still taking advantage of the extra saturation offered by a wide color gamut display.
Quantum Dots
One of the most impressive displays at CES 2014 was Hisense’s 85″ 4K wide color gamut Quantum Dot TV. This set promises to bring OLED-like color performance at 4K resolutions to the US market this September at LCD prices (we heard a 55″, 65″ and the 85″ will all be offered). A number of other manufacturers also demonstrated Quantum Dot displays off the main show floor. We saw displays ranging in size from 5″ smartphones, to notebooks to monitors as well as TV’s. 2014 looks to be the year that Quantum Dots gain serious traction in the display market after a strong debut in 2013.
In the previous post in this series, I made the case for displays with hybrid, custom color gamuts as a great way to deliver coverage of Pointer’s gamut as well as the most important broadcast standards. We can build the hardware today to support these large color gamuts so its seems like a great solution but there is a catch: nobody is broadcasting or distributing these large color gamuts today. So, are we going to have to wait for broadcasters and content creators to slowly catchup, much like we did with HDTV?
What content delivery looks like today
Content is captured and viewed in a wide variety of gamuts across a range of different devices but only broadcast in one gamut.
Today, content creators are actually shooting in a wide variety of color spaces ranging from RAW to rec.709 to Adobe 1998. They are then forced to cram all of these different sources into the lowest common denominator rec.709 standard for broadcast or distribution. That same content is then displayed on devices with a range of different gamut capabilities from tablets that only cover about 70% of rec.709 to HDTVs that do meet the spec to OLED devices that oversaturate the content.
There’s a lot of diversity on both the capture and display sides and a clear bottleneck in the middle in the form of broadcast and distribution channels.
Adhering to broadcast standards is no longer sufficient to guarantee a good experience for consumers because there’s already too much diversity on the display side alone to rely on one standard. You just can’t be sure that consumers are actually looking at your content on a rec.709-capable device. We’re also losing a lot of the value that creators are capturing and could, in many cases, be delivered to end viewers who have the devices to show it.
How do we get around broadcast standards?
What content delivery looks like tomorrow
The first thing to note is that the internet is democratizing broadcast and distribution channels. With the web we can deliver whatever we want, whenever we want. Some players in the industry, notably Sony, are already doing this with 4K content. If there’s no content available and you believe in 4K resolution, you just deliver your own content directly to your customers.
Wide color gamut displays combined with good color management and the web as a broadcast platform will allow content to accurately be displayed in the original color gamut.
Still, this leaves us with some potential experience problems. If the right display gamut is not matched to the right content the results will be no different and that’s why color management is key. There are several companies working on color management solutions and certification programs for devices that will make it possible for wide color gamut displays to handle a variety of incoming gamuts. Using metadata, for example, a wide color gamut display can be alerted to the presence of Adobe RGB content and then remap that content on the fly to assure that it is displayed accurately on that specific panel.
With great color management, we can maximize the gamut on the display side and pull through the best possible gamut for the device we are looking at. In this way, we can deliver always accurate content that meets the designers intent, wether artistic or commercial.
Google announced an updated version of their Nexus 7 tablet this morning. Central to Google’s pitch was the improved display with both more pixels and more color. The device does feature an impressively high resolution, packing 2.3 million pixels into a 7″ form factor. But, I’m more interested in the color performance and, on this point, Google was vague offering only that the display, “has a 30% wider range of colors.”
What do they mean by that?
It depends on their frame of reference- what color space they are using and what color gamut standard they are comparing against. Since Google talked about the accuracy of HD video at their event, let’s assume that they are referring to the HDTV broadcast standard (rec.709) and using the common CIE 1976 (u’ v’) color space.
When I measured last year’s Nexus 7, I found it could only reproduce about 82%* of the colors found in the rec.709 standard. Color reproduction was not accurate and a little bit undersaturated on this device:
Color gamut of Google’s previous generation Nexus 7 versus the HDTV broadcast standard (rec.709). Plotted in CIE 1976 (u’ v’).
With just a simple calculation, increasing 82% by 30%, you’d get about 106% coverage of the HDTV broadcast standard. While that’s actually a slightly wider color gamut than the standard, it is not uncommon for device makers to use a wider color gamut in order to guarantee the color spec across all devices with some room for manufacturing tolerances. This means video and web content should be displayed accurately and it could make for a great looking display.
We’ll order and measure one as soon as they are available to verify so stay tuned…
* note: I always measure coverage of broadcast standards, not simply total area since that can be misleading. However, in this case, coverage and area are nearly the same since the Nexus 7’s gamut is smaller than rec.709.
Last week I looked at the three “P’s” of human color perception– physical, physiological and psychological– as a way to help define a color gamut for the ideal display. Based on real world examples from art and commerce, I concluded that the range of colors found in nature, as measured by Pointer, provided the best fit with our two design goals which were an accurate and exciting, immersive experience.
This week, I’d like to get a little more practical and take a look at existing color gamut standards to see what we might realistically be able to achieve today.
What fits best?
Color gamut of 4,000 surface colors found in nature as measured by Pointer in 1980 against the color gamut of the iPhone 5.
The first thing you’ll notice about Pointer’s gamut (pictured above again) is that it’s a pretty odd, squiggly shape. This means it is going to be difficult to cover efficiently with a three primary system that mixes just red, green and blue to create all the colors we see, like the LCD found in the iPhone. In order to cover Pointer’s with just those three colors, we’d need to make them extremely saturated. There are proposed standards that take this approach, such as rec.2020, but since they are not practical to implement today from a technology standpoint I’ve decided to ignore them for this discussion.
For the near future, we’ll need to rely on just three colors to get the job done, so what can we do now? Let’s look at two popular wide color gamut standards: Adobe 1998 and DCI-P3:
Current wide color gamut standards Adobe RGB 1998, commonly used by pro photographers and designers, and DCI-P3, used in digital cinema, compared to Pointer’s gamut in CIE 1976
Let’s start with Adobe 1998. Many people are familiar with this color gamut since it is found as an option on many consumer cameras and it is popular among creative professionals. It certainly covers a significantly wider range of colors than the HDTV broadcast standard with a very deep green point. The rich cyans that we talked about in the movie “The Ring” would look great in Adobe 1998. But, we’re not getting any more of those exciting reds and oranges. In fact, Adobe’s red point is identical to the HDTV broadcast standard.
What about DCI-P3 then? Designed to match the color gamut of color film and used in cinemas all over the world, DCI-P3 has a very wide gamut. The reds are particularly deep and, of course, all of the colors from the movies we looked at are covered. Still, it’s missing a lot of the deep greens found in Adobe 1998 and only just fits the green Pantone color of the year. So DCI-P3 is not quite perfect either.
What about a hybrid, custom gamut?
What if we combined the green from Adobe with the red from DCI-P3 and their shared blue point? We’d end up with pretty good, high 90’s percentage coverage of Pointer’s gamut, coverage of all of the existing HDTV broadcast content, full coverage of cinema content from Hollywood and a superior ecommerce experience with most of the colors from the natural world covered.
Hybrid color gamut standard that combines the green point from Adobe 1998 with the deep red of DCI-P3
Looks pretty great and we can make displays now that cover this color gamut with today’s technology. But how would it work on the content side? Would we need to get together and agree on this new standard and then wait for years while it is slowly adopted by content creators and display makers?
Next week
Next week we’ll look at how content delivery might evolve to support gamuts like this without the need for major changes to broadcast standards.
Last week I set out to define the ultimate consumer display experience in terms of color performance. I laid out some potential color performance design goals for an ideal display, suggesting that such a display should be both accurate and capable of creating an exciting, immersive experience that jumps off the shelf at retail.
Can we achieve both goals? To find out, let’s start by looking at how we perceive color.
Color Perception
The color of objects that our eyes see in nature is determined by three things: physical, physiological and psychological:
The color of objects that our eyes see in nature is determined by three things: physical, physiological and psychological.
The physical component of our color perception is a constant based on the laws of nature. It is a combination of the quality of the illumination or light source, in this case meaning spectrum it contains, and the reflectance of the object. In the image above, the ball appears red to the eye because it is reflecting red light, while absorbing most the other colors from the light source.
The physiological part of our vision is also a relative constant that is based on the electrochemical processes of the eye. The back of the retina contains photoreceptor nerve cells which transform incoming light into electrical impulses. These electrical impulses are sent to the optic nerve of the eye and onto the brain, which processes and creates the image we see. And that’s where the psychological component comes in.
Let’s look at how each of these components might affect display color performance, starting with the physical, which ought to be something we can measure.
Physical
Fortunately, a guy named Pointer has done this for us. For his 1980 publication, Pointer measured over 4,000 samples and was able to define a color gamut of real surface colors, of objects found in nature. The result is commonly called “Pointer’s Gamut:”
Color gamut of over 4,000 colors found in nature as measured by Pointer against the color gamut of the iPhone 5.
This already seems like a great place to start. It immediately looks like a great fit our first ultimate color experience criteria which was accuracy. If we could accurately capture and reproduce all of the colors found in the natural world it would make for a much improved, more accurate ecommerce experience, for example.
But how important are those extra colors? Looking at Pointer’s gamut mapped against the color gamut of the latest iPhone in the chart above, you have to wonder if we really come across these deep cyans and reds in everyday life. Are they just infrequent, rare colors or something worth pursuing for our display?
Turns out we do. As an example, Pantone’s color of the year for 2012 was a deep emerald green that falls outside of both the iPhone’s gamut and the HDTV broadcast standard. This is an important and popular color that appears a bit too yellowish on your computer monitor when you are shopping for the perfect tie on Amazon. So there are some really important colors outside of what the iPhone can display today.
But, what about our second criteria, the lifelike, exciting, immersive experience we want to give consumers? Is the gamut of the natural world enough?
Physiological
If we look at the second component of the visual system, the physiological component, we’ll see that we can actually perceive a much wider range of colors. The cells in the back of retina can actually detect the entire range of the CIE diagram. That’s almost double the range of colors that Pointer found in nature:
Color gamut of the average human eye vs gamut of colors found in nature as measured by Pointer
This is starting to sound like a much more immersive experience. Maybe we ought to pursue the full color capability of the human eye just like the industry has done for high, “retina” resolutions.
It sounds great but it would be a tall order. It would take quite a lot of power, brightness and extra bit depth to even begin to think about covering a color space this large. There certainly would be a high price to pay in terms of design tradeoffs to get there. So are there any truly valuable colors contained in that extra space, similar to the Pantone color in Pointer’s gamut, that would make us want to go for it?
Psychological
This is where the psychological component comes into play.
Seeing is not passive. Our brains add meaning to the light that our eyes detect based on context and experience and memory. We are continuously and actively re-visualizing the light that comes out of our retinas.
This may seem hard to believe but this fun demo created by neuroscientist Beau Lotto does a great job of showing just how much our brains actively interpret and change what we see.
The color of the chips has not changed in the video above, just our perception of the color. What’s happening here is our experience is telling us that the color chip in shadow must actually be a much brighter color than the chip under direct illumination, so our brain is just making the correction for us on the fly.
Artists absolutely play on this psychological element of our perception of color, sometimes using totally unrealistic or hyper real colors to make us feel or experience something new or help tell a story. In fact, one of the most influential art instructors of the 20th century, Josef Albers, once said that, “the purpose of art is not to represent nature but instead to re-present it.”
Monet’s The Poppy Field, near Argenteuil
So, whether it’s Monet using saturated and contrasting colors with equal luminance to trick our brains into seeing poppy flowers sway in an imaginary breeze in a 19th century painting or modern films which sometimes rely on the wider gamut capabilities of color film and digital cinema projection to create uniquely cinematic experiences for audiences.
Movies like “The Ring,” for example, which used a deep cyan cast throughout much of the film to create tension and help tell a scary story. Or Michael Bay’s “Transformers” movies, which use deeply saturated oranges, reds and teal greens to create an exciting, eye-popping palette appropriate for a summer blockbuster sci-fi movie about giant robots:
There’s certainly a place for wild, unexpected colors in art. But, as we go through some of these examples, I think we’ll actually find that there is a huge range of expression possible within the gamut of surface colors that Pointer measured. The full range of gamut detectable by the human eye, while exciting to think about, is not really necessary to deliver both accurate and pleasing (engaging) color to our visual system.
So where does that leave us?
In my next post I’ll look at existing wide color gamut standards and content delivery mechanisms to see both what we can do today and what’s next for wide color gamut displays.
In my last post, which focused on the trend towards ever higher resolutions in smartphone displays, I suggested that color performance might be a more useful area of focus for display makers. That’s because, in terms of color gamut, we are a long way from reproducing the full range of colors that our eyes can detect.
For context, let’s add the color gamut of one of the most popular smartphones on the market, Apple’s iPhone, to the chart from my last post:
Best performing smartphones in resolution vs iPhone color gamut performance since 2009.
The latest iPhone only covers about 1/3 of the range of colors our eyes can detect so we’re a long way from matching the acuity of the best displays on the market in terms of resolution. But, how much color do we really need for a great experience?
As a display technologist and color blogger that’s probably the question I’m most frequently asked. If I’m advocating for more colorful displays, how much more am I after? There’s got to be a reasonable limit right?
Perhaps unsurprisingly, the short answer is that it depends on a lot of things. Different applications, from ecommerce to the TV on your living room wall, all require differing amounts of color performance. The environment matters a lot too, if the display will be used outdoors brightness may be a factor. There are slowly evolving broadcast standards and content delivery infrastructure to consider as well. And, of course, technology limitations– what can be achieved today and at what cost?
These are all valid concerns for anyone designing and marketing a new display product but they don’t really answer the bigger question. To me, what we really ought to be asking here is: what would the ultimate consumer experience be in terms of color? To answer that, I think we first need to take a step back, put some the practical stuff aside for the moment, and define what that experience should look and feel like. Once we understand that we can start to put the technological pieces together to achieve it.
What do we want?
So, let’s first ask: what do we want? In a pie in the sky, ideal display, in terms of color performance?
Well, the engineers among us are probably thinking, first and foremost, it has to be accurate and that’s a great place to start. Our displays need to accurately reproduce colors found in nature for increasingly important ecommerce applications, photos we take of our family should look real, not over saturated and professionally created content should be reproduced so that it conveys the artist’s intent without distortion.
But, maybe the marketing folks among us have another criteria in mind and that is a bit more subjective. We want our displays to be immersive and engaging. We want them to jump off the shelf at retail and we want to deliver a unique and exciting experience to our customers.
Can we have both?
To find out, I’ll be taking a look at how we perceive color in my next post.
Last week DisplaySearch put out a new report on the current trend towards ever higher display resolutions. High resolution displays now make up most of the market for handhelds and 300+ ppi “retina-class” resolutions are coming on strong:
Smartphones and handheld devices are moving rapidly to high resolution. 200+ ppi will account for 54% of unit share in in 2013, with 24% of unit share to be 300+ ppi. Even higher resolution panels in the FHD class will emerge. 400-500 ppi FPDs are expected to hit the market with fast shipment growth in 2013. (source: DisplaySearch)
Not exactly earth shattering news. The display industry began rapidly moving towards higher resolutions the moment Apple first unveiled the retina display with it’s iPhone 4 in 2009. What is interesting here is that the trend shows no signs of abating, even as resolutions approach or surpass the acuity of the average human eye.
Best performing smartphones in terms of display resolution from 2009 to 2013. Shown as a percentage of what the average human eye can detect.
The HTC One is leading the charge this year at 468 ppi. According to Dr. Ray Soneira of DisplayMate, that’s already equivalent to Apple’s retina display for eyes with 20/20 vision at a distance of just 7.4 inches from the eye- much closer than an average user will typically hold the device.
The question is- just how noticeable are additional increases in resolution beyond 400-500 ppi going to be for consumers? In my view, resolutions above 530 ppi will be wasted on the vast majority of users. Unless you have near perfect vision and hold your phone excessively close to your eye, you just won’t be able to see the difference. Still, device makers seem intent on pushing resolution as far as they can- some manufacturers I spoke with at DisplayWeek 2013 even talked about 4K smartphones!
It’s a shame because there are many other display performance characteristics that would benefit users. They may sound like less exciting specs but color performance, sunlight readability (a combination of reflectance, brightness and color saturation), and efficiency would all improve usability much more than another 50 or 100 ppi in resolution.
Screenshot of Adobe’s Kuler app showing color extraction from a photo
Adobe recently released a new iPhone app called Kuler that let’s you extract colors from your surroundings using the phone’s camera. It’s a useful tool that allows designers to capture color inspiration wherever they find it and easily incorporate it into their work via color palettes.
The app also highlights a weakness in current display technology: no display on the market today can actually reproduce all the colors we see in the environment around us. So, even if the camera sensor can capture that color you love, you may not be seeing an accurate representation of it on your device.
The iPhone 5’s LCD display is designed to cover the sRGB/rec.709 color gamut standard used for HDTV broadcasts. And, it looks great but compared to the world we see around us, it’s just not quite as rich. If we plot the iPhone 5’s color gamut against the gamut of colors found in nature, the phone comes up short in important reds, greens and cyans:
Color gamut of the iPhone 5’s display compared to the gamut of colors found in nature. The iPhone 5 comes up short in red, green and cyan.
If DisplayWeek 2013 was any indication, color has once again become a hot topic in the display industry. Color gamuts are getting larger and it may not be long before we see a display that can match what our eye sees in nature. Over the course of the next year, we will start to see more wide color gamut-capable devices as OLED continues to expand marketshare and new technologies like quantum dot LCD begin to enter the market in volume.
Just back from a great DisplayWeek in Vancouver. Finally had a chance to recover, go through my notes and process everything I saw at the show. Most of the big story lines will be pretty familiar to anyone who followed last years show: TV’s are still getting bigger, OLED TV is still right around the corner, 4K is starting to ship and mobile displays are getting both sharper and more efficient.
DisplayWeek wasn’t all old news though. In fact, just like CES, this year everyone seemed to be talking about color performance. At the annual Display Industry Awards, honors in several categories went to wide gamut display technologies including the Best In Show and Component of the Year awards. And, on the show floor, major manufacturers like 3M, Samsung and LG dedicated significant booth space to wide color gamut or color management technologies.
3M’s Quantum Dot Enhancement Film (QDEF) demo at DisplayWeek 2013. Bottom display is using quantum dots to achieve a wider color gamut than OLED at higher brightness and lower cost.
3M demoed several wide color gamut LCDs based on the Quantum Dot Enhancement Film (QDEF) technology that they are partnering with Nanosys to manufacture. Ranging from smartphone all the way up to 55″ TVs in size, these devices were all showing a wider color gamut than OLED with an especially deep red. This seems like a lot of color but 3M says that in developing their Perceptual Quality Metric (PQM), a new analysis tool aimed at helping display makers model how different performance characteristics will affect end user experience, they found that color saturation positively affected the perception of quality.
In Samsung’s neighboring booth, I found a series of comparison demos designed to show that wide color gamut displays can be both accurate and pleasing to the eye. Each demo featured a camera feeding a live image of several colored objects to both standard and wide color gamut displays. In each case the wide gamut display was able to more accurately recreate the color of the objects in front of the camera. They also showed off the new color management capability of their flagship Galaxy S4 smartphone that allows the device to accurately display rec.709 content without oversaturation- something the previous generation S3 struggled with.
Samsung demonstrating the value of wide gamut displays by showing some common colors that fall outside the rec.709 broadcast gamut standard in a series of demos at DisplayWeek 2013
Finally, at LG’s booth, we saw a new LCD color filter design that allows them to cover the Adobe RGB color gamut used by photographers and print professionals.
With all of this buzz, it looks like we’ll start to see wide color gamut displays start to move into the mainstream in ever larger screen sizes over the next half of this year and into 2014.
Visceral refers to the gut, rather than the mind. Our brain may try to talk us out of jumping off a cliff, but as soon as we take that first step into the void, our guts take over. We respond with a rush of emotion and we can’t help but scream from terror or euphoria. It’s a purely visceral reaction. […]
So here’s my theory: I believe that introducing visceral elements into an app will take it past the point of just being awesome. It will make your app speak to the subconscious, built-in affinity that humans have for the physical properties I mentioned before.
That’s Rob Foster, co-founder of Mysterious Trousers, defining his theory about the importance of visceral elements in application design. The whole piece is well worth reading, especially if you are interested in design or have ever wondered just why Angry Birds is so unbelievably addictive.
In the quote above Rob is talking about the power of little kinetic events in applications like the bounce you get when scrolling to the bottom of a page on the iPhone or the satisfying little “pop” noise you hear when creating a new task in Clear. His point is well made, getting the details of these visceral elements right can clearly take an app from just useful to a truly engaging and even addictive experience for users.
While Rob’s piece focused on the impact of animation and sound, I wondered how color might factor into visceral application design.
Color choice is not just about beautiful graphics- it can also have a powerful physiological effect on us. We have a measurable response to aggressive colors like red, which may even cause a spike in testosterone levels. In fact, recent studies suggest that that the color of a uniform can affect the outcome of an Olympic wrestling match and onscreen colors can even influence how much you pay for something on eBay.
As mobile display technology improves, with more lifelike color and wider dynamic range, application designers may find that color becomes an even more powerful tool to elicit visceral responses from users.
