PDA

View Full Version : Tone-Mapping images in Shake from REDCODE output



Wade McDonald
02-14-2007, 01:01 PM
Hey guys --

I realize that there's already an HDR thread, but ultimately I'm interested in discussing something a bit different, so rather than hijack the thread, I figured I'd start a new one.

In the other thread people were talking about wanting to use HDR techniques with REDCODE output. Generally I tend to shy away from using the term "HDR" because it brings to mind the strange looking things from Flickr that we see so often -- oversaturated colors, strange black ghosting and a complete lack of shadow. This is all fine and good and great for the occasional music video (and I'd love to see if anyone manages to get that kind of an image in-motion), but what I wanted was something that I could use when the normal range of tools (gamma, exposure, brightness, contrast, etc) just won't capture the full range of what i'm looking for in an image... I want to boost the shadow values or save the highlights without messing with any of the intermediate values... sure there are ways of masking out and replacing bits of the image, but that's really not practical with motion pictures (unless you've got a lot of time on your hands).. So I went on a quest last night to find some means of salvaging really rangey images that looked better than what I could accomplish with the normal set of tools.

Also, a lot of HDR depends on multiple exposures, etc... but for my purposes I'm more interested in mapping the luminance of the whole 11-stop range into an image displayable on a computer screen / HD / DV, while retaining the charectoristics of the original image.

disclaimer: I'm a DP and a photographer, not a programmer. I'm also not a video engineer, so I can't claim to know everything about gamma, etc... so i won't take it personally one or more of you can demonstrate a better means of doing this sort of thing -- especially those of you who work in Post and are really good at it... in fact, I encourage it... my goal in participating on this board is to share & learn.

Ok, so I found this little algorythm online:
http://graphics.cs.ucf.edu/papers/cic2005.pdf

in essence, it brigtens the luminance of really dark pixels (compared to their immediate neighbors and the overall brightness of the frame) and darkens the luminance of really bright pixels (same), so it really only affects the outliers.

I opened up Shake and Xcode and got to making. My code is messy and not optimized, so I'm not going to post it (pvt msg me if you want it anyway), but I made a Shake node and it did exactly as I had hoped.

Here's an uncompressed TIFF of part of my Shake window (all the gfx are at actual size):

http://www.pointedfilms.com/ShakeHDR.tif

As labeled, the images are (left to right):
1) the image in shake,
2) gamma corrected image + contrast modification to the way I wanted it artistically,
3) the result of the Spacial Tone Mapping node I made, which included some tweaking within the algorythm (c and lamdba values) to make it look like I wanted it to.

Overall, I think this works pretty well, and I'm planning on using it as a stopgap measure when I have out of control highlights or shadows; but what I'd like is for you all is to tell me if there's something i'm missing (besides using curves manually or using mattes to isolate parts of the image) tool-wise or technique-wise. I'm relatively new (1 yr or so) to the compositing world (Shake).

BTW, here's the original image, and the equations as used in the Shake macro:

Original Image (memorial.exr) (http://scanline.ca/exrtools/memorial.exr)

WARNING: MATH
-=-=-=-=-=-=-=-

c and u are the controls. c is the multiplying factor of the avg lum of the whole image (YA) and u controls the display color on the monitor (for our purposes it's saturation control)

Y = black and white luminance image
YA = solid greyish image of the avg of Y
GC = Global contrast factor
YL = local luminance (pixels around them)

GC = c*YA
CL = YL*[log(.0001 + YL/Y)] + GC

R = [(R/Y)^u] * YD
G = [(G/Y)^u] * YD
B = [(G/Y)^u] * YD

The c slider controls the overall light/dark, with .15 being the "middleish" range best I can tell.

I used .4 for the u value, as they did in the paper.

I cheated in the following ways:
1) I used the LumToAlpha node from the ShakeSDK for Y, when I probably should have used their standardized equation.
2) I used a hasty method of calculating YL, and in small resolutions, may create artifacting because of a lack of smoothness, so I introduced a blur on YL that's optional (didnt need to use it though), but for this image, I used 4x4 blocks of avgs, rather than the suggested 10x10 to prevent the artifacting.

I'm happy with how it rendered the dome, the shadow values in the roof, and the stained glass, but I'm not happy about the blooming (but I can live with it) or the fact that it perceptually saturated the colors (because of the way I did the Y channel and math at the end) but I think messing with the u value could have fixed it.

----
Apologies for not being terribly clear. I'm a stream of consciousness poster.

:)

-W

Corrado Silveri
02-14-2007, 08:10 PM
Really interesting...
I'll love to test your script. Need a PM?

Thanks, Elcurado.

Chris Forbes
02-14-2007, 09:53 PM
A messy work around to this process in FCP is by stacking two clips (one exposed for the highlights and one for the shadows) one on top of the other. Then apply a luma matte to the top one so that after the cutoff point the bottom clip shows through.

Brook Willard
02-14-2007, 10:32 PM
I'd love to see the script as well if you have it...

Cail Young
02-14-2007, 10:53 PM
for my purposes I'm more interested in mapping the luminance of the whole 11-stop range into an image displayable on a computer screen / HD / DV, while retaining the charectoristics of the original image.

If you brought in the images at 32-bit float and used a compress node, would that not acheive the same or similar effect? I don't really know how it would affect the midranges but Compress is designed to take wide dynamic ranges and compress them into smaller ones...

Wade McDonald
02-15-2007, 10:35 AM
ok, didn't expect so many requests for the node. ;)

here's the link:
http://www.pointedfilms.com/SpatialTM.zip

Be sure to read the documentation.

One of the operations I use is only avalable if you have the ShakeSDK installed and compiled. I can't distribute it according to the ADC agreement.... but it's free from Apple's site.

Please let me know if you make changes, so I can change the original as well.
W

Wade McDonald
02-15-2007, 10:44 AM
Need a PM?


Sorry, What's a pm? If it involves help, yes...


A messy work around to this process in FCP is by stacking two clips (one exposed for the highlights and one for the shadows) one on top of the other. Then apply a luma matte to the top one so that after the cutoff point the bottom clip shows through.

not a bad idea Chris. That would probably work for this image's highlight problems pretty well.


If you brought in the images at 32-bit float and used a compress node, would that not acheive the same or similar effect? I don't really know how it would affect the midranges but Compress is designed to take wide dynamic ranges and compress them into smaller ones...

That's what I thought. I couldn't get the compress node to produce anything like this, because it remaps the other values as well, best I can tell. The nice thing about this tonemap operation is that it only affects pixels that are Much brighter or Much darker than its neighbors, based on deviation from the avg lum of the whole... but then again, maybe I wasn't using the compress node properly... can you do it with the compress node and share your settings/script?

-W

Brook Willard
02-15-2007, 11:20 AM
I believe he was asking if you needed a private message sent to you in order to send him the script or code.

I'd love to look at the node if possible.

Wade McDonald
02-15-2007, 02:50 PM
Ah, yes. oops. ;)

The node is posted above, as a zip file that contains both the .h and the ui.h for the node.

Cail Young
02-16-2007, 03:49 AM
Well, I can't get Compress to behave like I imagined it did, so it looks like your math is the best solution... any way to make it work without the SDK? I can get it but in the long run what's your options?

Wade McDonald
02-16-2007, 11:13 AM
I used a little ditty (and a layerX operation) to create the luminance channel.

I weighted the R, G, and B channels like so:

lum = 0.299r + 0.587g + 0.114b

It gives almost exactly the same result. (I may tweak in a bit)

Here it is, the node, documentation, etc.... no ShakeSDK needed!!

Enjoy!

-W

Trevor Meier
02-16-2007, 03:16 PM
Thanks for posting this, Wade.

Cail Young
02-16-2007, 06:31 PM
Very nice! What is the advantage of LumToAlpha vs a Colorspace-Reorder ?

Wade McDonald
02-16-2007, 06:37 PM
Very nice! What is the advantage of LumToAlpha vs a Colorspace-Reorder ?

I wish I understood the difference, but the Apple .cpp code for that LumToAlpha stuff was at least 10x what my code for the whole Tonemap node was....haha.

Best I can tell, one is voodoo, one is hoodoo. :D

However, I think the weighted-colorspace approach is fine, because it's what was used by the guys who wrote the algorythm in the first place. :cool:

W

Gavin Greenwalt
02-17-2007, 09:14 PM
What's the difference between this and just adjusting Highlight/Midtone/Shadow gamma independently?

GlennChan
02-18-2007, 01:30 AM
It's spatial or area-based. Vaguely like automating crazy power windows, if you're familiar with power windows.

Or to explain it differently, it works on blurred areas, as opposed to a single pixel.

Graeme Nattress
02-18-2007, 07:37 AM
Similar to "highlight / shadow" in Photoshop??

Bruce Allen
02-18-2007, 03:31 PM
Or "highlight / shadow" in After Effects... look that one up, it even has temporal smoothing.

Also, the old wide-radius Unsharp Mask trick works for me...

Bruce Allen
director / designer
www.boacinema.com

Columbus
02-21-2007, 07:22 AM
Photoshop CS3 has Highlight Recovery and Shadow Fill function that work non destructively in Camera Raw maybe this will also be a function in Redcine which would be pretty amazing.

Gavin Greenwalt
02-21-2007, 10:22 AM
I would be very afraid of flicker from a plugin like that.