ISP HDR Tuning Guide

Version 0.2

1. HDR TUNING GUIDE¶

High Dynamic Range (HDR) combines images from two different exposure conditions to produce a higher dynamic range than traditional single exposure (linear) mode.

Figure 1-1: HDR On / HDR Off

The adjustment of HDR is mainly focused on the distribution of screen brightness, i.e. how to squeeze as much most-wanted information as possible into the 8-bit range of 0~255. So first we will start with the AE settings, then cut into the HDR/WDR adjustments related to brightness distribution, and finally to the NR/Edge collocation and trouble-shooting of some frequently asked questions.

1.1. AE Settings¶

HDR mode uses a combination of long and short exposures to synthesize an area that is easily overexposed with short exposure, and to thereby obtain the original information on the long exposure of the overexposed area. Therefore, the exposure ratio (HDR Ratio) of the long and short exposure frames is directly related to the dynamic range of HDR.

However, the HDR Ratio is not the larger the better. The short-exposure image must be multiplied by the HDR ratio to match the long-exposure image linearly. Therefore, the larger the HDR Ratio, the more gain is multiplied and the more serious the noise problem is with the image.

There are three main parameters in AE settings, as follows:

1. Max Short Shutter

2. HDR Ratio

3. AE Target

Max Short Shutter: This parameter is related to the frames-per-second capability and the frequency of sensor operation. The maximum limit for this can be derived from the sensor spec. and our recommendation is to set this parameter to the maximum value available.

HDR Ratio: This parameter needs to be determined based on the dynamic range requirement of the scene involved. Note that the HDR ratio decides the “maximum” dynamic range of the HDR effect; the actual dynamic range of the screen depends on the settings of the Global Dark Tone Enhance feature in the subsequent WDR adjustment. Hence, the HDR ratio does not necessarily have to be very high. On the other hand, the HDR ratio, if too low, will restrict the exposure time of long exposure, causing the long exposure frame to be multiplied by the gain before the expected timing and producing a noise problem as a result. The value normally suggested is 10x (=10240). 16x (= 16384) is also supported.

AE Target: This parameter controls the global brightness of the screen, and is normally set to the same value as that used in single-frame linear mode. If the noise in dark area is not under good control with the final HDR effect after the WDR adjustment, you can adjust the target to allow AE to brighten the screen. The noise issue will be smaller than that with WDR.

1. Calculate the longest and shortest exposure time for short exposure based on sensor spec and fps, and fill out the calculated results in the short exposure table separately. Then input the intended HDR ratio in the HDR ratio field. Different HDR ratio by total gain (represented by the horizontal axis) can be set.

   

   

2. According to the HDR Ratio, multiply the Shutter in the short exposure table by the HDR Ratio and fill it into the long exposure table (for example, short exposure 2100 μs, long exposure 21000 μs).

   

1.2. HDR Settings¶

The HDR API contains basic HDR parameter settings, wherein SensorExpRatio needs to be set according to the HDR ratio discussed in the preceeding AE section. The rest of the parameters are described below:

NrEn	Extra 2DNR for short exposure frames Enabling this parameter can reduce the noise of short-exposure images; note, however, if there occurs a blurry image problem.
YwtTh1 & YwtTh2	Upper and lower threshold of long and short exposure in HDR synthesis Long-exposure image data will be selected for brightness below Th1, short exposure will be used for Th2 and above, and blending of long and short exposures will be applied to brightness in between th1 and th2. To adjust, first set th2 to 0, then change th1 to the highest possible extent without any spot of color deviation caused by long exposure saturation in the image (See Note 1). Once Th1 is decided, set th2 to a value between th1 + (50~100). If th2 is large enough, the noise performance of the bright area will be improved, but it also means that the details will be lost (due to blending of over-exposed and detailless long exposure).
NoiseLevel	Noise level, for determination of motion The horizontal axis represents brightness. The larger the value, the more likely an area will be judged as static. Default value is recommended. Fine-tune the default value if needed.
MotionTh	Motion threshold If the difference between long and short exposures after alignment, minus NoiseLevel, is greater than MotionTh, the area is judged as a dynamic area, and the long and short exposures will be blended according to the degree of motion. If smaller than MotionTh, the area is judged as a static area, and the long and short exposures will be blended according to brightness.
MotionAdjLut	Horizontal axis of dynamic area long exposure table The horizontal axis represents the degree of motion from static to dynamic. The vertical axis represents the blended long exposure ratio. The larger the value, the more long exposure is applied; the smaller the value, the more short exposure is applied. Applying more short exposure will render the image noisier, but the ghost image problem will be reduced. In general, more long exposure is used for relatively more static area, while more short exposure used for relatively more dynamic area. Hence, the curve will be a decreasing curve.
SensorExpRatio	Sensor exposure ratio of Long and short exposure frames In normal cases, the value should be identical with the one provided for the HDR Ratio referred to in AE Settings section.
DynRatioEn	Dynamic long and short exposure brightness compensation If the sensor’s actual difference in brightness between long and short exposures deviates from the set SensorExpRatio value, enabling this function can remedy the difference to a certain degree (for example, when the calculated long and short exposure shutter value cannot exactly be a specified multiple due to precision issue, you can enable this function to eliminate the difference).
DynRatioSrcSel	Reference frame selection for dynamic long and short exposure brightness compensation 0: Long exposure, 1: Short exposure
YSrcSel	Static area brightness source The long and short exposure ratio for static area will be decided based on current brightness (Yin) and YwtTh1, YwtTh2 settings. The Yin value can be obtained by 4 methods: 0: Long exposure surrounding average, 1: Long exposure surrounding maximum value, 2: Short exposure surrounding average, 3: Short exposure surrounding maximum value. The default setting is 1; adjustment is not recommended.
NoiseLevelSrcSel	Horizontal axis of NosieLevel, representing the Y source Noise level will be determined by Y. The Y value can be obtained by 4 methods: 0: Long exposure, 1: Short exposure, 2: Smaller value of long and short exposure, 3: Greater value of long and short exposure. The default setting is 3; adjustment is not recommended.
MotionMaxVal	Maximum value of dynamic area The difference between long and short exposures after alignment, minus NoiseLevel, will be futher checked by this maximum value. The smaller the value, the smaller the maximum value. That is, a small difference will be regarded as a big motion; as a result, the place where noise occurs will more easily be judged as a dynamic area, causing more short exposure to be applied and the image noisier. The default setting is 3; adjustment is not recommended.
MotionLumaTh	Dynamic area full short exposure Y threshold If judged as a dynamic area and the Y value is greater than this threshold, full short exposure will be used. The default setting is 1000; adjustment is not recommended.
MotionAdjSft	Horizontal axis of dynamic area long exposure ratio The horizontal axis represents the difference between long and short exposures, minus NoiseLevel, namely, from static to dynamic. Adjustment is not recommended.

Note 1: Since AWB gain is applied before HDR synthesis, areas where long-exposure image saturation occur are often left with various abnormal colors but not necessarily white. There is one thing in common, however -- details will be lost in all these areas. Therefore, you need to adjust th1 downward to prevent over-exposed long exposure section from appearing in the final image.

1.3. WDR Adjustment¶

After the long and short exposure images are synthesized in 16-bit space by HDR, the WDR Global Dark Tone Enhance will be used to re-allocate the brightness and reduce it to 12-bit. Generally, we need to use Global Dark Tone Enhance to make the long-exposure dark area data appear brighter, or have it restored to the brightness of the normal single frame. Only medium/high brightness area or overexposed area is integrated into the short exposure frame information through the curve.

Global Dark Tone Enhance is pre-configured with 16 curves, where 0 is a curve that maintains linearity without brightness enhancement. As the number rises, the degree of brightness enhancement will increase gradually, but the amptitude of the 8^th after the 7^th curve will first decrease and then rise again. Curves 0~7 and curves 8~15 can be regarded as two different types of curves to achieve the brightness improvement. You can employ both to choose the one yielding more desirable effect. If there is still a dark area after the adjustment by Global Dark Tone Enhance function, you can try adjusting the main Gamma to brighten the dark area.

After Global Dark Tone Enhance brightens the global screen brightness, there usually exists a problem of poor contrast and insufficient permeability. In this case, you can increase WDR strength a little to add some LTM (Local Tone Mapping) effect to the curve, thereby enhancing the sense of hierarchy. The list of the other adjustment parameters is as follows:

2. FREQUENTLY ASKED QUESTIONS¶

1. Flicker

   In HDR mode, since short-exposure shutters are often unable to be set to the shortest physical limit of de-flicker, flicker problems are often inevitable in short-exposure areas under cathode ray tube light sources. In this case, we recommend adjusting fps to a multiple of the power frequency to fix the banding problem without scrolling (e.g. 60Hz/30fps, 50Hz/25fps). Note that it is not yet possible to eradicate this problem.

2. Broken-like flashing in long and short exposure blending area

   

   This problem is often caused by the mismatch between long and short exposure brightness. It is recommended to check whether the HDR Ratio setting in the AE is consistent with the SensorExpRatio setting in the HDR API, or disable the dynamic compensation mechanism of the Dynamic Ratio function to allow the long-short exposure synthesis to refer only to the setting of YwtTh1 & YwtTh2. Then observe whether there is a clear boundary at the junction of the long and short exposures, so as to clarify if there exists a problem of brightness mismatch.

3. Image ghosting

   

   

   If this happens, making short-exposure image the main reference can eliminate the ghost image problem. Some useful measures include:

   1. Lowing MotionTh to loosen the criteria for judgement of dynamic area. But if this parameter is set too low, flat area can be judged also as a dynamic area, thus causing short exposure to be used and the image to be noisier.

   2. Lowering MotionAdjLut, to increase use of short exposure in dynamic area, but then the image will appear noisier too.

   3. Fine-tuning NosieLevel, if adjustment by Y is needed.

4. Local color shift and detail loss in areas with medium to high brightness

   

   To deal with this issue, try lower the YwtTh1 value of HDR API.

5. Indoor/outdoor/backlit face too dark

   

   If this happens, try the following adjustment in the order indicated:

   1. Set PreEnhance to a value in the range 11~15. The greater the value, the brighter the dark area and the blurrier the overall image.

   2. Adjust WDRcruve to brighten up the dark area.

   3. Increase the AE target, but then the detail in outdoor over-exposed area can be lost.

   4. Set the number of BoxNum to a medium value, avoiding values overly large. Although a greater BoxNum can achieve a better contrast, face at a relatively dark area will appear darker.

   5. For dark area, set WDRStrByY to a medium value, avoiding values overly small. If dark area balancing is to be limited, adjust the DarkLimit parameter.