Reproducing Colors We’ve Never Seen Before – the “The Wizard of Oz” in HDR

Watching the latest superhero movie release – loaded with CGI and eye-melting explosions – is obviously a great way to test the cinematic benefits of High Dynamic Range (HDR) and Wide Color Gamut (WCG) formats on your brand-new TV. But what about the many thousands of films from the first hundred-plus years of cinema?

Well, it turns out that one of the earliest full color films ever produced contains a rich range of colors that audiences have not been able see since the original screening of the film in theaters over 80 years ago.

Barry Goch, writing for postPerspective on Warner’s recent 4K HDR restoration of 1939’s multi-Oscar-winning classic, The Wizard of Oz:

George Feltenstein, SVP of theatrical catalog marketing for Warner Bros. Home Entertainment, spoke about why the film was chosen for restoration. “The Wizard of Oz is among the crown jewels that we hold,” he said. “We wanted to embrace the new 4K HDR technology, but nobody’s ever released a film that old using this technology. HDR, or high dynamic range, has a color range that is wider than anything that’s come before it. There are colors [in The Wizard of Oz] that were never reproducible before, so what better a film to represent that color?” (emphasis added)

This is a fantastic use of HDR and WCG technologies. Many classic films have been restored multiple times in recent decades as new formats arose from laser disc to DVD to Blu Ray and so on. Each subsequent release brought improvements in image quality and fidelity but the color reproduction has never really been close to that of the original film. Until now.

Film and Digital Gamuts

Color Gamut of Print Film and modern Digital Camera Sensors compared to the DCI-P3 and BT.2020 standards in CIE1931*. Data source: Sony

Many color movies from the 1930’s, 40’s and 50’s were filmed with rich, vibrant colors. In fact, many popular film-stocks could reproduce a wider color gamut than even the best-performing HDR TVs on the market today. Technologies like Technicolor’s insane Three Strip Process enabled cinematographers to capture and reproduce a range of colors that may have been closer to BT.2020 than DCI-P3.

 

Wizard-of-Oz-4K-Munchkinland

Brilliant red fireball in The Wizard of Oz

The Wizard of Oz, presents a vibrant, fantastical world containing colors across the spectrum from the famous ruby red slippers to the Yellow Brick Road, the Emerald City and even the occasional bright red fireball. It is therefore a perfect fit for remastering in wide color gamut.

But it is by no means the only older film worthy of this treatment. There are quite a few movies, such as An American in Paris (1951), Singing in the Rain (1952) and countless early Disney animation films that relied on the Technicolor three strip process to create richly colorful worlds. I look forward to seeing more restorations of these classic films that bring back colors that haven’t seen in many decades and that many audiences (anyone under ~90) have never had the chance to see.

*    *    *

 

*: Why CIE 1931? Haven’t most people (including me!) moved on from this ancient color space in favor of the far more uniform CIE 1976 or u’v’ color space? Recent work by Dr. Kenichiro Masaoka (the guy who invented BT.2020) suggests that good old CIE 1931 may actually be more useful for making color volume comparisons. Recommend reading his recent JSID paper from 2019 “Color Gamut of Multi‐Chromatic Displays” for more detail: https://doi.org/10.1002/sdtp.13058 

Introducing Wide Gamut Wednesdays

This week I’m kicking off a new series of posts that set out to answer a simple question:

“Can an HDTV accurately reproduce these colors?”

I’m calling the new weekly feature “Wide Color Gamut Wednesday” or #WideColorWednesday in social media speak. Each week we will analyze a new wide color gamut image and post the results to our @dot_color Twitter feed.

In the process, I think we’ll find that “wide gamut” colors – colors that fall outside the BT.709 color gamut used by HDTVs – are actually fairly common beyond classic examples like Brazilian tree frogs or Coca Cola cans. In fact, in our first test, we found a simple image of spring flowers, taken in Rochester, NY, contained mostly colors that fall outside the BT.709 gamut.

Wide Color Gamut image analysis

62.5% of the colors in this springtime flowers image fall outside the BT.709 color gamut used by HDTVs #WideGamutWednesday

I thought it would be helpful to write up the first #WideColorWednesday image as a blog post with some background on the process used to create these images.

Continue reading

Is the rec.2020 UHD color broadcast spec really practical?

I’ve often advocated on this blog for Pointer’s Gamut as an important design goal for display makers but is it really practical today from a technology perspective? Pointer’s Gamut covers a huge area and it’s odd shape makes it awfully difficult to cover with just three primaries. Rec.2020, the leading Pointer’s-covering color gamut broadcast standard and de facto standard for upcoming UHD broadcasts, demonstrates this perfectly. It uses very deep red and green primaries to ensure that all those purples and cyans can get squeezed it into the triangle.

rec.2020 needs a very deep green to cover 99.9% of Pointer's Gamut

rec.2020 needs a very deep green to cover 99.9% of Pointer’s Gamut

It’s certainly tough to make a display that can reproduce primary colors that are that saturated and it is especially hard to do so efficienctly. Until now the displays that have come closest rely on an esoteric and power-hungry laser backlight system that can only cover up to about 91% of rec.2020 spec. That is impressive given how ambitious rec.2020 is but a bulky $6,000 laser display doesn’t exactly qualify as practical and it’s certainly not a technology that we are likely to find in a tablet or smartphone anytime soon given it’s low power efficiency.

That may be about to change.

My company, Nanosys, has been working on this problem and we now think it is practical to produce an LED LCD that covers over 97% of rec.2020 using Quantum Dot technology. The latest generation of our Quantum Dots emit light with a very narrow Full Width Half Max (FWHM) spec of below 30 nanometers for both red and green wavelengths. FWHM is pretty obscure spec to be sure but it means that the color is both very pure and accurate. That pin-point accuracy actually enabled us to demonstrate over 91% rec.2020 just by modifying an off-the-shelf, standard LCD TV set with a specially tuned sheet of Quantum Dot Enhancement Film (QDEF).

Nanosys demonstrates over 91% coverage of rec.2020 using Quantum Dots

Nanosys demonstrates over 91% coverage of rec.2020 using Quantum Dots and a standard LCD TV color filter

Very impressive and even a bit better than the performance of that laser TV but still not quite all the way there. What else could be optimized to improve the system and get us closer?

Looking at the spectrum after the color filters revealed a significant amount of blue leaking through the green filter. This leakage was causing the blue point to shift away from the rec.2020 primary. By optimizing the system and selecting a different blue color filter material with a sharper cutoff, Nanosys engineers showed that it is possible to build a display that covers over 97% of the rec.2020 standard– with great power efficiency.

Quantum Dot enhanced displays are in mass production today, they are used in commonly available displays on the market today. Their high power efficiency also means they can be used in all kinds of devices from smartphones to TVs. So, for the first time, it is actually becoming practical to build displays that cover the massive rec.2020 standard and since rec.2020 is part of the UHD broadcast spec this great news for the next generation of 4K and 8K devices.

Pointer’s Gamut follow-up by TFT Central

figure7_Pointer in CIE1976

Last summer I wrote a multi-part series here that looked at how much color gamut displays really need. In those articles I used the gamut of colors found in the natural world, as defined by Pointer, as a possible design goal for an ideal color display. Kid Jansen at TFT Central has followed-up on my piece with a much more detailed look at how several current color gamut standards and devices perform compared to Pointer’s gamut. He’s done some great analysis and it’s well worth reading, check it out here.

How much color do displays really need? Part 4: Content Delivery

In the previous post in this series, I made the case for displays with hybrid, custom color gamuts as a great way to deliver coverage of Pointer’s gamut as well as the most important broadcast standards. We can build the hardware today to support these large color gamuts so its seems like a great solution but there is a catch: nobody is broadcasting or distributing these large color gamuts today. So, are we going to have to wait for broadcasters and content creators to slowly catchup, much like we did with HDTV?

What content delivery looks like today

Content is captured and viewed in a wide variety of gamuts across a range of different devices but only broadcast in one gamut.

Content is captured and viewed in a wide variety of gamuts across a range of different devices but only broadcast in one gamut.

Today, content creators are actually shooting in a wide variety of color spaces ranging from RAW to rec.709 to Adobe 1998. They are then forced to cram all of these different sources into the lowest common denominator rec.709 standard for broadcast or distribution. That same content is then displayed on devices with a range of different gamut capabilities from tablets that only cover about 70% of rec.709 to HDTVs that do meet the spec to OLED devices that oversaturate the content.

There’s a lot of diversity on both the capture and display sides and a clear bottleneck in the middle in the form of broadcast and distribution channels.

Adhering to broadcast standards is no longer sufficient to guarantee a good experience for consumers because there’s already too much diversity on the display side alone to rely on one standard. You just can’t be sure that consumers are actually looking at your content on a rec.709-capable device. We’re also losing a lot of the value that creators are capturing and could, in many cases, be delivered to end viewers who have the devices to show it.

How do we get around broadcast standards?

What content delivery looks like tomorrow

The first thing to note is that the internet is democratizing broadcast and distribution channels. With the web we can deliver whatever we want, whenever we want. Some players in the industry, notably Sony, are already doing this with 4K content. If there’s no content available and you believe in 4K resolution, you just deliver your own content directly to your customers.

Wide color gamut displays combined with good quality color management and the web as a broadcast platform will allow content to accurately be displayed in the correct color gamut.

Wide color gamut displays combined with good color management and the web as a broadcast platform will allow content to accurately be displayed in the original color gamut.

Still, this leaves us with some potential experience problems. If the right display gamut is not matched to the right content the results will be no different and that’s why color management is key. There are several companies working on color management solutions and certification programs for devices that will make it possible for wide color gamut displays to handle a variety of incoming gamuts. Using metadata, for example, a wide color gamut display can be alerted to the presence of Adobe RGB content and then remap that content on the fly to assure that it is displayed accurately on that specific panel.

With great color management, we can maximize the gamut on the display side and pull through the best possible gamut for the device we are looking at. In this way, we can deliver always accurate content that meets the designers intent, wether artistic or commercial.

How much color gamut do displays really need? Part 3: Existing color gamut standards

Last week I looked at the three “P’s” of human color perception– physical, physiological and psychological– as a way to help define a color gamut for the ideal display. Based on real world examples from art and commerce, I concluded that the range of colors found in nature, as measured by Pointer, provided the best fit with our two design goals which were an accurate and exciting, immersive experience.

This week, I’d like to get a little more practical and take a look at existing color gamut standards to see what we might realistically be able to achieve today.

What fits best?

Color gamut of 4,000 surface colors found in nature as measured by Pointer in 1980 against the color gamut of the iPhone 5.

Color gamut of 4,000 surface colors found in nature as measured by Pointer in 1980 against the color gamut of the iPhone 5.

The first thing you’ll notice about Pointer’s gamut (pictured above again) is that it’s a pretty odd, squiggly shape. This means it is going to be difficult to cover efficiently with a three primary system that mixes just red, green and blue to create all the colors we see, like the LCD found in the iPhone. In order to cover Pointer’s with just those three colors, we’d need to make them extremely saturated. There are proposed standards that take this  approach, such as rec.2020, but since they are not practical to implement today from a technology standpoint I’ve decided to ignore them for this discussion.

For the near future, we’ll need to rely on just three colors to get the job done, so what can we do now? Let’s look at two popular wide color gamut standards: Adobe 1998 and DCI-P3:

Current wide color gamut standards Adobe RGB 1998, commonly used by pro photographers and designers, and DCI-P3, used in digital cinema, compared to Pointer's gamut in CIE 1976

Current wide color gamut standards Adobe RGB 1998, commonly used by pro photographers and designers, and DCI-P3, used in digital cinema, compared to Pointer’s gamut in CIE 1976

Let’s start with Adobe 1998. Many people are familiar with this color gamut since it is found as an option on many consumer cameras and it is popular among creative professionals. It certainly covers a significantly wider range of colors than the HDTV broadcast standard with a very deep green point. The rich cyans that we talked about in the movie “The Ring” would look great in Adobe 1998. But, we’re not getting any more of those exciting reds and oranges. In fact, Adobe’s red point is identical to the HDTV broadcast standard.

What about DCI-P3 then? Designed to match the color gamut of color film and used in cinemas all over the world, DCI-P3 has a very wide gamut. The reds are particularly deep and, of course, all of the colors from the movies we looked at are covered. Still, it’s missing a lot of the deep greens found in Adobe 1998 and only just fits the green Pantone color of the year. So DCI-P3 is not quite perfect either.

What about a hybrid, custom gamut? 

What if we combined the green from Adobe with the red from DCI-P3 and their shared blue point? We’d end up with pretty good, high 90’s percentage coverage of Pointer’s gamut, coverage of all of the existing HDTV broadcast content, full coverage of cinema content from Hollywood and a superior ecommerce experience with most of the colors from the natural world covered.

Hybrid color gamut standard that combines the green point from Adobe 1998 with the deep red of DCI-P3

Hybrid color gamut standard that combines the green point from Adobe 1998 with the deep red of DCI-P3

Looks pretty great and we can make displays now that cover this color gamut with today’s technology. But how would it work on the content side? Would we need to get together and agree on this new standard and then wait for years while it is slowly adopted by content creators and display makers?

Next week

Next week we’ll look at how content delivery might evolve to support gamuts like this without the need for major changes to broadcast standards.

How much color gamut do displays really need? Part 2: How we perceive color

Last week I set out to define the ultimate consumer display experience in terms of color performance. I laid out some potential color performance design goals for an ideal display, suggesting that such a display should be both accurate and capable of creating an exciting, immersive experience that jumps off the shelf at retail.

Can we achieve both goals? To find out, let’s start by looking at how we perceive color.

Color Perception

The color of objects that our eyes see in nature is determined by three things: physical, physiological and psychological:

The color of objects that our eyes see in nature is determined by three things: physical, physiological and psychological.

The color of objects that our eyes see in nature is determined by three things: physical, physiological and psychological.

The physical component of our color perception is a constant based on the laws of nature. It is a combination of the quality of the illumination or light source, in this case meaning spectrum it contains, and the reflectance of the object. In the image above, the ball appears red to the eye because it is reflecting red light, while absorbing most the other colors from the light source.

The physiological part of our vision is also a relative constant that is based on the electrochemical processes of the eye. The back of the retina contains photoreceptor nerve cells which transform incoming light into electrical impulses. These electrical impulses are sent to the optic nerve of the eye and onto the brain, which processes and creates the image we see. And that’s where the psychological component comes in.

Let’s look at how each of these components might affect display color performance, starting with the physical, which ought to be something we can measure.

Physical

Fortunately, a guy named Pointer has done this for us. For his 1980 publication, Pointer measured over 4,000 samples and was able to define a color gamut of real surface colors, of objects found in nature. The result is commonly called “Pointer’s Gamut:”

Color gamut of 4,000 surface colors found in nature as measured by Pointer in 1980 against the color gamut of the iPhone 5.

Color gamut of over 4,000 colors found in nature as measured by Pointer against the color gamut of the iPhone 5.

This already seems like a great place to start. It immediately looks like a great fit our first ultimate color experience criteria which was accuracy. If we could accurately capture and reproduce all of the colors found in the natural world it would make for a much improved, more accurate ecommerce experience, for example.

But how important are those extra colors? Looking at Pointer’s gamut mapped against the color gamut of the latest iPhone in the chart above, you have to wonder if we really come across these deep cyans and reds in everyday life. Are they just infrequent, rare colors or something worth pursuing for our display?

Turns out we do. As an example, Pantone’s color of the year for 2012 was a deep emerald green that falls outside of both the iPhone’s gamut and the HDTV broadcast standard. This is an important and popular color that appears a bit too yellowish on your computer monitor when you are shopping for the perfect tie on Amazon. So there are some really important colors outside of what the iPhone can display today.

But, what about our second criteria, the lifelike, exciting, immersive experience we want to give consumers? Is the gamut of the natural world enough?

Physiological

If we look at the second component of the visual system, the physiological component, we’ll see that we can actually perceive a much wider range of colors. The cells in the back of retina can actually detect the entire range of the CIE diagram. That’s almost double the range of colors that Pointer found in nature:

Color gamut of the average human eye vs gamut of colors found in nature as measured by Pointer

Color gamut of the average human eye vs gamut of colors found in nature as measured by Pointer

This is starting to sound like a much more immersive experience. Maybe we ought to pursue the full color capability of the human eye just like the industry has done for high, “retina” resolutions.

It sounds great but it would be a tall order. It would take quite a lot of power, brightness and extra bit depth to even begin to think about covering a color space this large. There certainly would be a high price to pay in terms of design tradeoffs to get there. So are there any truly valuable colors contained in that extra space, similar to the Pantone color in Pointer’s gamut, that would make us want to go for it?

Psychological

This is where the psychological component comes into play.

Seeing is not passive. Our brains add meaning to the light that our eyes detect based on context and experience and memory. We are continuously and actively re-visualizing the light that comes out of our retinas.

This may seem hard to believe but this fun demo created by neuroscientist Beau Lotto does a great job of showing just how much our brains actively interpret and change what we see.

The color of the chips has not changed in the video above, just our perception of the color. What’s happening here is our experience is telling us that the color chip in shadow must actually be a much brighter color than the chip under direct illumination, so our brain is just making the correction for us on the fly.

Artists absolutely play on this psychological element of our perception of color, sometimes using totally unrealistic or hyper real colors to make us feel or experience something new or help tell a story. In fact, one of the most influential art instructors of the 20th century, Josef Albers, once said that, “the purpose of art is not to represent nature but instead to re-present it.”

Monet's The Poppy Field, near Argenteuil

Monet’s The Poppy Field, near Argenteuil

So, whether it’s Monet using saturated and contrasting colors with equal luminance to trick our brains into seeing poppy flowers sway in an imaginary breeze in a 19th century painting or modern films which sometimes rely on the wider gamut capabilities of color film and digital cinema projection to create uniquely cinematic experiences for audiences.

Movies like “The Ring,” for example, which used a deep cyan cast throughout much of the film to create tension and help tell a scary story. Or Michael Bay’s “Transformers” movies, which use deeply saturated oranges, reds and teal greens to create an exciting, eye-popping palette appropriate for a summer blockbuster sci-fi movie about giant robots:

Wide color gamut in movies

There’s certainly a place for wild, unexpected colors in art. But, as we go through some of these examples, I think we’ll actually find that there is a huge range of expression possible within the gamut of surface colors that Pointer measured. The full range of gamut detectable by the human eye, while exciting to think about, is not really necessary to deliver both accurate and pleasing (engaging) color to our visual system.

So where does that leave us?

In my next post I’ll look at existing wide color gamut standards and content delivery mechanisms to see both what we can do today and what’s next for wide color gamut displays.

DisplayWeek 2013: Color is back

Just back from a great DisplayWeek in Vancouver. Finally had a chance to recover, go through my notes and process everything I saw at the show. Most of the big story lines will be pretty familiar to anyone who followed last years show: TV’s are still getting bigger, OLED TV is still right around the corner, 4K is starting to ship and mobile displays are getting both sharper and more efficient.

DisplayWeek wasn’t all old news though. In fact, just like CES, this year everyone seemed to be talking about color performance. At the annual Display Industry Awards, honors in several categories went to wide gamut display technologies including the Best In Show and Component of the Year awards. And, on the show floor, major manufacturers like 3M, Samsung and LG dedicated significant booth space to wide color gamut or color management technologies.

3M's Quantum Dot Enhancement Film demo at DisplayWeek 2013. Bottom display is using quantum dots to achieve a wide color gamut.

3M’s Quantum Dot Enhancement Film (QDEF) demo at DisplayWeek 2013. Bottom display is using quantum dots to achieve a wider color gamut than OLED at higher brightness and lower cost.

3M demoed several wide color gamut LCDs  based on the Quantum Dot Enhancement Film (QDEF) technology that they are partnering with Nanosys to manufacture. Ranging from smartphone all the way up to 55″ TVs in size, these devices were all showing a wider color gamut than OLED with an especially deep red. This seems like a lot of color but 3M says that in developing their Perceptual Quality Metric (PQM), a new analysis tool aimed at helping display makers model how different performance characteristics will affect end user experience, they found that color saturation positively affected the perception of quality.

In Samsung’s neighboring booth, I found a series of comparison demos designed to show that wide color gamut displays can be both accurate and pleasing to the eye. Each demo featured a camera feeding a live image of several colored objects to both standard and wide color gamut displays. In each case the wide gamut display was able to more accurately recreate the color of the objects in front of the camera. They also showed off the new color management capability of their flagship Galaxy S4 smartphone that allows the device to accurately display rec.709 content without oversaturation- something the previous generation S3 struggled with.

Samsung demonstrating the value of wide gamut displays by showing some common colors that fall outside the rec.709 broadcast gamut standard in a series of demos at DisplayWeek 2013

Samsung demonstrating the value of wide gamut displays by showing some common colors that fall outside the rec.709 broadcast gamut standard in a series of demos at DisplayWeek 2013

Finally, at LG’s booth, we saw a new LCD color filter design that allows them to cover the Adobe RGB color gamut used by photographers and print professionals.

With all of this buzz, it looks like we’ll start to see wide color gamut displays start to move into the mainstream in ever larger screen sizes over the next half of this year and into 2014.

Apple CEO Tim Cook talks color quality at Goldman Sachs conference

Apple CEO Tim Cook

Apple CEO Tim Cook spoke at Goldman Sachs’ Technology and Internet Conference yesterday. He touched on a wide range of topics from what Apple plans to do with its cash horde to the state of its retail operation. When it came to a question about making lower cost products, Tim used display quality to help make a point about creating great user experiences:

The truth is, customers want a great experience and they want quality and they want that a-ha moment each time that they use the product, and that’s rarely a function of any of those things.

If you look at displays, some people are focused on size. There’s a few other things about the display that are important. Some people use displays, like OLED displays, the color saturation is awful. And so if you ever buy anything online and you want to really know what the color is as many people do, you should really think twice before you depend on the color of the OLED display. The Retina display is twice as bright as an OLED display. I only bring these points up to say there are many attributes to the display, and what Apple does is sweat every detail.

He makes some fair points here. If a display is not bright enough to view in all conditions, not efficient enough to get you through a whole day or accurate enough to display your favorite content, the experience of the whole device suffers. Choosing the right display technology is certainly a critical part of the design process.

OLED technology’s power consumption and saturation issues have been well established already. What I find most interesting in Tim’s comments is the idea that high color saturation is intrinsically a bad experience. It certainly has been that way so far but the difference between a great color experience and the gaudy oversaturation of today’s OLEDs is in exactly the kind of implementation details he’s describing above.

OLED and emerging LCD technologies, like quantum dot displays, can actually show a much wider range of colors than today’s devices– over 40% more of the color that our eyes can detect. This means that, when paired with the right content, high saturation displays can more accurately reflect the world we see around us resulting in a more lifelike, immersive experience.

But how do we get wide color gamut content into consumers hands?

It’s a lot like the chicken and egg/content and technology dilemma facing 4K TV makers with two key differences- wide color gamut can be delivered with no change in file size and there’s plenty content out there already. As an example, movies have been shot for decades on media, both film and digital, that has a much wider color gamut than your TV does today. Much in the same way that 4K TV’s can upscale HD video, it’s also relatively easy to manage the color on a device to make it backwards compatible with today’s content.

OLED implementers have thus far been content to take advantage of the extra pop that added color saturation provides when comparing devices on a store shelf. They’ve left a tremendous amount of overall ecosystem value on the table. It’s possible to deliver video in cinema-level color quality to mobile devices, to offer developers the tools to take full advantage of a wider color palette and to implement accurate color management for existing content. Wide color gamut is ready now, it’s just waiting for the right device maker to come along and put all these pieces together to perfect the experience. 

ITU agrees on HEVC h.265 codec

h-265-logo,Q-D-349573-3The ITU announced today that it’s members have agreed upon a new high efficiency video codec. Dubbed HEVC H.265, the new format is designed to improve on and ultimately replace the current king of all codecs, H.264/MPEG-4 AVC which covers 80% of internet video today.

So far, a lot of attention has been given to the codec’s ability to deliver the same quality video as 264 with only half the bandwidth. That kind of efficiency improvement is a big deal– it could reduce strain on networks and bring high-resolution 4K content delivery over the internet closer to reality.

h265 vs h264 quality comparisonThere are also some important changes for color in the new spec. Recent drafts by the ITU’s Joint Collaborative Team on Video Coding (JCT-VC) have added support for wider color gamuts like Adobe RGB 1998 and 12-bit video. This paves the way for fantastic looking color as wide gamut-capable hardware starts to become more widely available.