Dynamic local exposure

Hi, Habr! I present to you the translation of the article "Dynamic Local Exposure" by John Chapman.

In this article I will introduce a couple of ideas about dynamic local exposure in HDR rendering. Bart Vronsky already has a great article on this topic and I highly recommend reading it right now if you have not done so already; the ideas here are more based on his article. At the end I included several other great links.

Low / High Dynamic Range

In the good old days (1990s), games were rendered directly in the displayed LDR (narrow dynamic range) format (gamma space, 8 bits). It was simple and cheap, but, on the other hand, significantly hampered the creation of a truly photo-realistic picture.

Currently, especially with the advent of PBR (physically-based rendering), games are rendered with a huge dynamic range in linear space with higher accuracy. With this movement towards photorealism comes the real problem: how can we display the HDR image in the LDR ?

Global Auto Exposure

The standard approach to automatic exposure control is to measure the average (or average logarithmic) brightness of the scene, optionally with a weight function, preferring values close to the center of the image. This can be done very effectively by using parallel reduction or by multiple downsampling in the mipmap of the luminance buffer (brightness buffer). The latter approach has some advantages, which I will discuss in the next section.

The average brightness is subsequently converted to an exposure value, for example, by calculating the reciprocal of the maximum permissible scene brightness:

float Lavg = exp(textureLod(txLuminance, uv, 99.0).x); float ev100 = log2(Lavg * 100.0 / 12.5); ev100 -= uExposureCompensation; // optional manual bias float exposure = 1.0 / (1.2 * exp2(ev100));

Obtained from the ISO standard for calculating speed based on saturation, for full explanation see (3)

Since the potentially average brightness is unstable under dynamic conditions, it is usually smoothed over time using the exponential hysteresis function (2) :

 Lavg = Lavg + (Lnew - Lavg) * (1.0 - exp(uDeltaTime * -uRate));

Translator comment

This function should be used in the shader downsampling texture brightness and only during the calculation of the last mip level (1x1). Further it will be written about, but in my opinion it is easy to lose sight of.

Due to its global nature, this method suffers from strong shadowing or glare of image areas that have a deviation from the average brightness:

Although this corresponds to the ability of the eye to adapt to changes in light levels, the overall effect is rather far from what we really perceive in the real world.

Local auto exposure

If we generate medium brightness using downsampling , we have access to lower mip levels for luminance buffer (4) to get local average brightness.

 float Lavg = exp(textureLod(txLuminance, uv, uLuminanceLod).x;

Please note that in order for this to work, the hysteresis should be used only in the last step (when recording 1x1 mip level), otherwise there will be artifacts.

In theory, this is a great idea: each area of the image can have a good exposure, while being in contrast with the neighboring areas. However, in practice, a disgusting result is obtained:

The most unpleasant are block “halos”, which are found in areas with high contrast:

However, they can be smoothed either by pre-filtering the luminance buffer , or simply using bicubic sampling:

Still looks disgusting, but already better.

Sampling different levels of mipmap in luminance changes the radius of the halo. This parameter is useful for controlling the overall “appearance” of the result, as well as for minimizing the halo effect, although at the expense of a general decrease in contrast (it becomes a border filter) or a loss of locality of exposure control:

Still, smoothing halos is not enough. The result is generally not natural; It looks like an extreme “HDR photo” style, unlike what a person sees. However, by mixing global and local values, we can get the best of both worlds:

 float Llocal = exp(textureLod(txLuminance, uv, uLuminanceLod).x; float Lglobal = exp(textureLod(txLuminance, uv, 99.0).x; float L = mix(Lglobal, Llocal, uLocalExposureRatio); // .. use L to compute the final exposure scale as before

By changing the blending factor, you can adjust the local exposure so that as a result, you can minimize artifacts and maximize perceived realism:

Automatic mixing ratio

Adjusting the blending ratio manually is normal in situations where we have absolute control over camera position, lighting, etc. However, in many cases (for example, outdoor games with a dynamic change of day and night) this level of control is simply impossible. In this case, it would be nice to generate the mix ratio automatically.

In the image below, we have a wide dynamic range; mostly medium-low brightness values and several areas with high intensity (the sky in the windows):

Without local exposure, the color of the sky is lost. In this case, I would like a large mixing ratio :

Now consider the image below, which has a small dynamic range, mainly with a high brightness value:

In this case, the use of local exposure reduces the brightness of “bright” areas too much:

Observation data hints at a simple method of mixing local and global values: if the difference between the average and maximum brightness of the scene is larger, then the mixing ratio of the local exposure should be greater. Generating the maximum brightness of the scene can be done trivially during the calculation of brightness, using hysteresis to smooth the result in the same way as for the average value. Therefore, we can extend the previous code snippet as follows:

 float Llocal = exp(textureLod(txLuminance, uv, uLuminanceLod).x; float Lglobal = exp(textureLod(txLuminance, uv, 99.0).x; // average in x float Lmax = exp(textureLod(txLuminance, uv, 99.0).y; // max in y float Lratio = min(saturate(abs(Lmax - Lglobal) / Lmax), uLocalExposureMax); float L = mix(Lglobal, Llocal, Lratio); // .. use L to compute the final exposure scale as before

Please note that we have uLocalExposureMax at the entrance to control the absolute maximum degree of influence of local exposure. I have a good result gave uLocalExposureMax <0.3 .

Final code

 float Llocal = exp(textureLod(txLuminance, uv, uLuminanceLod).x; float Lglobal = exp(textureLod(txLuminance, uv, 99.0).x; // average in x float Lmax = exp(textureLod(txLuminance, uv, 99.0).y; // max in y float Lratio = min(saturate(abs(Lmax - Lglobal) / Lmax), uLocalExposureMax); float L = mix(Lglobal, Llocal, Lratio); float ev100 = log2(L * 100.0 / 12.5); ev100 -= uExposureCompensation; // optional manual bias float exposure = 1.0 / (1.2 * exp2(ev100)); vec3 result = hdrColor * exposure; result += bloom; //etc outColor.rgb = result;

Conclusion

The approach outlined above imposes some restrictions on when to measure the brightness of the scene. Usually, the measurement is performed immediately after the passage of the lighting in order to avoid adaptation of particle effects, bloom , etc. However, when local brightness is used, it is important that the real value that is involved in the exposure is represented in the luminance map . This means that the brightness measurement must be done immediately before applying the exposure. If this is unacceptable, then the solution will be to generate local brightness separately from the average and maximum values.

Although I think that using local and global scene brightness together is the “right” approach to creating a balanced, natural looking image, the quality of the result is obviously subjective. Whether a similar method is suitable for a particular game depends entirely on the content and desired visual style. I would be interested to hear other ideas on this.