AE/AWB/AF Interface

Version 1.9

1. System Architecture¶

According to the ISP architecture, apart from the main ISP core, additional 3A algorithm modules can be plugged in to the ISP block. Customers can develop AWB/AE/AF algorithm libraries of their own and connect them to ISP by registering the AWB/AE/AF Interfaces.

The CUS3A feeds hardware statistics to the AWB/AE/AF algorithm via AWB/AE/AF interface during ISP operation, and the AWB/AE/AF algorithm reports the corresponding results back to the CUS3A Host. The CUS3A Host then writes the algorithm results to the ISP middleware via MI_ISP, and the algorithm results are finally written to hardware through the corresponding ISP driver / sensor driver.

Based on this architecture, custom algorithm can focus solely on the calculation of input and output, without the need to handle any system integration related issues.

2. AE/AWB Operation Flow¶

Before proceeding with MI_SYS_Init(), customer application must register the AE/AWB callback function via CUS3A_RegInterface() first. In the subsequent ISP initialization stage, the ISP core notifies the algorithm via init() to do initialization. Once the front-end sensor starts to preview frames, the ISP core will call run() at frame interrupt and bring the hardware statistics into the algorithm. When the ISP is closed, release() will be called to request the algorithm to release all resources.

In the flowchart above, functions indicated in red represent callback functions required on the part of the algorithm.

3. AE/AWB Statistic Format¶

The format of AE/AWB statistics in code is as defined below. Each frame will produce 128*90 entries of sampling data.

In Macaron and Pudding, each frame will produce only 32*32 entries of sampling data on AE statistics.

#define isp_3A_ROW  (128)   /**< number of 3A statistic blocks in a row */
#define isp_3A_COL  (90)    /**< number of 3A statistic blocks in a column */ @brief Single AE HW statistics data*/
typedef struct
{
    U8  r;  /**< block pixel R average , 0~255*/
    U8  g;  /**< block pixel G average , 0~255, g=(Gr+Gb+1)>>1*/
    U8  b;  /**< block pixel B average , 0~255*/
    U8  y;  /**< block pixel Y average , 0~255, /
            y=(Ravg*9 +Gavg*19 +Bavg*4 + 16)>>5 */
}__attribute__((packed, aligned(1))) ISP_AE_SAMPLE;

/*! @brief Single AWB HW statistics data*/
typedef struct
{
    U8  r;  /**<center pixel R value , 0~255    */
    U8  g;  /**<center pixel G value , 0~255    */
    U8  b;  /**<center pixel B value , 0~255    */
}__attribute__((packed, aligned(1))) ISP_AWB_SAMPLE;

ISP will transmit AE/AWB statistics to the algorithm at each frame end. The data arrangement is shown in the following table. As can be seen from the table, ISP divides images into isp_3A_ROW x isp_3A_COL blocks and calculates the statistics to be filled into each block.

ISP_AWB_INFO::data[isp_3A_ROW*isp_3A_COL]

ISP_AE_INFO::data[isp_3A_ROW*isp_3A_COL]

If RGB-IR sensor is used, hardware will further extract the histogram relating to the IR information for reference by AE.

#define _3A_IR_HIST_BIN  (256)   /**< histogram type2 resolution*/

typedef struct
{
    u16 u2IRHist[_3A_IR_HIST_BIN];
} ____attribute__((packed, aligned(1))) ISP_IR_HISTX;

4. AF Statistic Format¶

By configuring AF statistic window (ISP_AF_RECT), where 16*n (n = 1~16) windows are available for configuration of (af_stats_win[16]), each frame will output the statistical results using the output structure defined by ISP_AF_STATS. For each window, the output statistics will contain 6 types of filter statistics.

The following table shows the number of filter bits supported:

Filter Bit & Chip Type	Twinkie	Pretzel	Macaron/Pudding
IIR / Sobel	34	35	35
Luma	34	32	32
YSat	24	22	22

In the case where AF ROI Mode = AF_ROI_MODE_NORMAL, refer to CusAFStats_t.stParaAPI[0].

In the case where AF ROI Mode = AF_ROI_MODE_MATRIX, refer to CusAFStats_t.stParaAPI[0~15].

typedef struct {
    MI_U32 u32StartX;   /*range : 0~1023*/
    MI_U32 u32StartY;   /*range : 0~1023*/
    MI_U32 u32EndX;     /*range : 0~1023*/
    MI_U32 u32EndY;     /*range : 0~1023*/
} AF_WINDOW_PARAM_t;

typedef struct {
    MI_U8 u8WindowIndex;
    AF_WINDOW_PARAM_t stParaAPI;
} CusAFWin_t;

//for Pretzel, Macaron, Pudding              //for Twinkie
typedef struct{                         typedef struct{
    MI_U8 high_iir[5*16];                       MI_U8 high_iir[6*16];
    MI_U8 low_iir[5*16];                        MI_U8 low_iir[6*16];
    MI_U8 luma[4*16];                           MI_U8 luma[6*16];
    MI_U8 sobel_v[5*16];                        MI_U8 sobel_v[6*16];
    MI_U8 sobel_h[5*16];                        MI_U8 sobel_h[6*16];
    MI_U8 ysat[3*16];                           MI_U8 ysat[3*16];
} AF_STATS_PARAM_t ;                        } AF_STATS_PARAM_t ;

typedef struct{                         typedef struct{
    AF_STATS_PARAM_t stParaAPI[16];             AF_STATS_PARAM_t stParaAPI;
} CusAFStats_t;                         } CusAFStats_t;

4.1. AF Filter Description¶

Parameter Name	Description	Input/Output
IIR_1	AF horizontal direction IIR filter statistics as calculated by ROI	Output
IIR_2	AF horizontal direction IIR filter statistics as calculated by ROI	Output
Luma	AF luminance statistics as calculated by ROI	Output
FIR_H	AF horizontal direction FIR filter statistics as calculated by ROI	Output
FIR_V	AF vertical direction FIR filter statistics as calculated by ROI	Output
YSat	Number of statistical values greater than YThd in AF FIR filter statistics as calculated by ROI	Output

Data Example:

General scene far focus:

General scene near focus:

Low brightness scene far focus:

Low brightness scene far focus with light source:

[PUDDING]

High brightness scene with IIR:

Low brightness scene with IIR:

5. AE Result¶

After the ISP calls run() to run AE algorithm, the AE algorithm must provide the corresponding results back to the ISP, in the format specified below:

/*! @brief ISP ae algorithm result*/
typedef struct
{
    U32 Size;       /**< struct size*/
    U32 Change;    /**< if true, apply this result to hw register*/
    U32 Shutter;    /**< Shutter in us */
    U32 SensorGain; /**< Sensor gain, 1X = 1024 */
    U32 IspGain;    /**< ISP gain, 1X = 1024 */
    U32 ShutterHdrShort;    /**< Shutter in us */
    U32 SensorGainHdrShort; /**< Sensor gain, 1X = 1024 */
    U32 IspGainHdrShort;    /**< ISP gain, 1X = 1024 */
    U32 u4BVx16384; /**< Bv * 16384 in APEX system, EV = Av + Tv = Sv + Bv */
    U32 AvgY;       /**< frame brightness */
    U32 HdrRatio;   /**< hdr ratio, 1X = 1024 */

/*CUS3A v1.1*/
    U32 FNx10;           /**< F number * 10 */
    U32 DebandFPS;      /**< Target fps when running auto debanding*/
U32 WeightY;        /**< frame brightness with ROI weight*/

/*CUS3A v1.3*/
    U16 GMBlendRatio;  /**< Adaptive Gamma Blending Ratio from AE**/
U16 IRLightStr;    /**< Setting IR light source strength , range 0~100**/
}ISP_AE_RESULT;

Size: Size of ISP_AE_RESULT, used to deal with version backward compatibility issue.

Change: If set to 1, ISP will write the value to hardware; if 0, ISP will ignore the current result.

Shutter: Sets the sensor shutter time.

SensorGain: Sets the sensor gain. Digital/Analog gain distribution rate will be decided by sensor driver.

IspGain: Sets the ISP digital gain.

ShutterHdrShort: Sets the short-exposure sensor time.

SensorGainHdrShort: Sets the short-exposure sensor gain. Digital/Analog gain distribution rate will be decided by sensor driver.

IspGainHdrShort: Sets the short-exposure ISP digital gain.

u4BVx16384: Brightness value of current scene based on APEX calculation formula. This value will be referenced by IQ algorithm.

AvgY: Average grayscale brightness of current frame. This value will be referenced by IQ algorithm.

HdrRatio: HDR long/short exposure ratio of current frame. This value will be referenced by IQ algorithm.

Fnx10: Sets the FN Number.

DebandFPS: Sets the FPS.

WeightY: The average grayscale brightness after ROI weighting of current frame. This value will be referenced by IQ algorithm.

GMBlendRatio: It only works when both AdaptiveGamma and RGBGamma of IQ are enabled. IQ will mix AdaptiveGamma and RGBGamma according to this ratio. Take the following figure as an example. The value range is 1~1025, 513 means that RGBGamma is used directly without mixing.

6. AWB Result¶

After the ISP calls run() to run AWB algorithm, the AWB algorithm must provide the corresponding results back to the ISP, in the format specified below:

/*! @brief AWB algorithm result*/
typedef struct
{
    U32 Size;       /**< struct size*/
    U32 Change;    /**< if true, apply this result to hw register*/
    U32 R_gain;     /**< AWB gain for R channel*/
    U32 G_gain;     /**< AWB gain for G channel*/
    U32 B_gain;     /**< AWB gain for B channel*/
    U32 ColorTmp;   /**< Return color temperature*/
}ISP_AWB_RESULT;

Size: Size of ISP_AWB_RESULT, used to deal with version backward compatibility issue.

Change: If set to 1, ISP will write the value to hardware; if 0, ISP will ignore the current result.

R_gain: R channel gain.

G_gain: G channel gain.

B_gain: B channel gain.

ColorTmp: Current scene color temperature.

7. AF Result¶

After the ISP calls run() to run AF algorithm, the AF algorithm must provide the corresponding results back to the ISP, in the format specified below:

/*! @brief AF algorithm result*/
typedef struct
{
    U32 Change;    /**< if true, apply this result to hw*/
    U32 Focal_pos; /**< Focal position*/
}__attribute__((packed, aligned(1))) ISP_AF_RESULT;

Change: If set to 1, AF will be notified of the calculated focal position (Focal_pos).

Focal_pos: Focal position result calculated by AF algorithm.

8. AE, AWB, AF Library Interface¶

int CUS3A_RegInterfaceEX(IspCh_e eIspCh, AlgoAdaptor eAdaptor, IspAlgoType eAlgoType, void* pAlgo)

Description

Custom AE, AWB and AF library register interfaces.

Parameter

Parameter Name	Description	Input/Output
eIspCh	ISP channel corresponding to specified algorithm	Input
eAdaptor	AlgoAdaptor corresponding to specified algorithm Presently only E_ALGO_ADAPTOR_1 can be used by customer.	Input
eAlgoType	Specify algorithm type: E_ALGO_TYPE_AE , E_ALGO_TYPE_AWB, E_ALGO_TYPE_AF	Input
pAlgo	Pointer to the algorithm library provided for implementation. If implementation algorithm is not required, fill in NULL. pAlgo type corresponds to eAlgoType as follows: E_ALGO_TYPE_AE : pAlgo = ISP_AE_INTERFACE* E_ALGO_TYPE_AWB : pAlgo = ISP_AWB_INTERFACE* E_ALGO_TYPE_AF : pAlgo = ISP_AF_INTERFACE*	Input

Return Value

Return Value	Description
0	Successful
\<0	Failed

int CUS3A_SetAlgoAdaptor(IspCh_e eIspCh, AlgoAdaptor eAdaptor,  IspAlgoType eAlgoType)

Description

Switch between registered algorithms.

Parameter

Parameter Name	Description	Input/Output
eIspCh	Specify the ISP channel for switching algorithm	Input
eAdaptor	Specify the AlgoAdaptor for switching algorithm: E_ALGO_ADAPTOR_NATIVE: SStar 3A E_ALGO_ADAPTOR_1: Custom 3A	Input
eAlgoType	Specify algorithm type: E_ALGO_TYPE_AE , E_ALGO_TYPE_AWB, E_ALGO_TYPE_AF	Input

Return Value

Return Value Description

0 Successful

\<0 Failed

Example

/*ISP0 AE -> SStarAE*/
CUS3A_SetAlgoAdaptor(eIspCh0, eAdapNative, eAlgoAE) 
/*ISP0 AWB -> Customer AWB*/
CUS3A_SetAlgoAdaptor(eIspCh0, eAdap1, eAlgoAWB)
/*ISP0 AF -> Customer AF */
CUS3A_SetAlgoAdaptor(eIspCh0, eAdap1, eAlgoAF)

int CUS3A_RegInterface(U32 nCh, ISP_AE_INTERFACE *pAe, ISP_AWB_INTERFACE *pAwb, ISP_AF_INTERFACE *pAf)

int CUS3A_AERegInterface(u32 nCh,ISP_AE_INTERFACE *pAE);
int CUS3A_AWBRegInterface(u32 nCh,ISP_AWB_INTERFACE *pAWB);
int CUS3A_AFRegInterface(u32 nCh,ISP_AF_INTERFACE *pAF);

CUS3A_RegInterface , CUS3A_AERegInterface, CUS3A_AWBRegInterface, CUS3A_AFRegInterface 不建议使用，请以CUS3A_RegInterfaceEX取代。

Description

The ISP provides AE, AWB and AF library register interfaces.

You can use CUS3A_RegInterface to register AE, AWB, and AF simultaneously, or use CUS3A_AERegInterface, CUS3A_AWBRegInterface, and CUS3A_AFRegInterface to register AE, AWB, and AF respectively.

Parameter

Parameter Name	Description	Input/Output
nCh	nCh = 0	Input
pAe	Pointer to the AE library algorithm implementation side. If you do not need to register AE, fill in NULL.	Input
pAwb	Pointer to the AWB library algorithm implementation side. If you do not need to register AWB, fill in NULL.	Input
pAf	Pointer to the AF library algorithm implementation side. If you do not need to register AF, fill in NULL.	Input

Return Value

Return Value Description

0 Successful

\<0 Failed

9. AE Callback Interface¶

int (*init)(void* pdata,ISP_AE_INIT_PARAM *init_state)

Description

The ISP notifies AE library to start initialization through this interface.
Parameter

Parameter Name Description Input/Output

pdata Algorithm private data pointer Input

init_state AE initial value Input
Return Value

Return Value Description

0 Successful

-1 Failed
```
void (*release)(void* pdata)
```
Description

The ISP notifies AE library to start release through this interface.
Parameter

Parameter Name Description Input/Output

pdata Algorithm private data pointer Input
```
void (*run)(void* pdata,const ISP_AE_INFO *info,ISP_AE_RESULT *result)
```
Description

The ISP notifies AE library to start calculation through this interface.

Parameter

Parameter Name	Description	Input/Output
pdata	Algorithm private data pointer	Input
info	ISP AE statistical data structure pointer	Input
result	ISP AE algorithm result structure pointer	Output

int (*ctrl)(void* pdata,ISP_AE_CTRL_CMD cmd,void* param)

Description

The ISP controls AE parameters through this interface.
Parameter

Parameter Name Description Input/Output

pdata Algorithm private data pointer Input

cmd AE control command Input

param AE control command parameter Input

10. AE Data Structure¶

10.1. ISP_AE_INTERFACE¶

Description

AE library registration structure. The algorithm needs to implement and fill in initialization, release, run, and control callback pointers to enable the ISP to call this structure.

Declaration

/**@brief ISP AE interface*/
typedef struct
{
    void *pdata; /**< Private data for AE algorithm.*/
    int (*init)(void* pdata, ISP_AE_INIT_PARAM *init_state);
    void (*release)(void* pdata);
    void (*run)(void* pdata, const ISP_AE_INFO *info, ISP_AE_RESULT *result);
    int (*ctrl)(void* pdata, ISP_AE_CTRL_CMD cmd, void* param);
} ISP_AE_INTERFACE;

Member

Name	Description
pdata	AE library private data pointer
init	AE library initialization callback pointer
release	AE library release callback pointer
run	AE library calculation callback pointer. ISP will request AE algorithm to start calculation when receiving the statistics of a frame.
ctrl	AE library control callback pointer

10.2. ISP_AE_INIT_PARAM¶

Description

The ISP notifies AE algorithm of the related hardware initialization status through this structure.

Declaration

typedef struct
{
    U32 Size;                   /**< struct size*/
    char sensor_id[32];             /**< sensor module id*/
    U32 shutter;                /**< shutter Shutter in us*/
    U32 shutter_step;           /**< shutter Shutter step ns*/
    U32 shutter_min;            /**< shutter Shutter min us*/
    U32 shutter_max;            /**< shutter Shutter max us*/
    U32 sensor_gain;            /**< sensor_gain Sensor gain, 1X = 1024*/
    U32 sensor_gain_min;        /**< sensor_gain_min Minimum Sensor gain, 1X = 1024*/
    U32 sensor_gain_max;        /**< sensor_gain_max Maximum Sensor gain, 1X = 1024*/
    U32 isp_gain;               /**< isp_gain Isp digital gain , 1X = 1024 */
    U32 isp_gain_max;           /**< isp_gain Maximum Isp digital gain , 1X = 1024 */
    U32 FNx10;                  /**< F number * 10*/
    U32 fps;                    /**< initial frame per second*/
    U32 shutterHDRShort_step;           /**< shutter Shutter step ns*/
    U32 shutterHDRShort_min;            /**< shutter Shutter min us*/
    U32 shutterHDRShort_max;            /**< shutter Shutter max us*/
    U32 sensor_gainHDRShort_min;        /**< sensor_gain_min Minimum Sensor gain, 1X = 1024*/
    U32 sensor_gainHDRShort_max;        /**< sensor_gain_max Maximum Sensor gain, 1X = 1024*/

    /*CUS3A v1.1*/
    U32 AvgBlkX;                  /**< HW statistics average block number*/
    U32 AvgBlkY;                  /**< HW statistics average block number*/
}ISP_AE_INIT_PARAM;

Member

Parameter Name	Description	Input/Output
Size	Size of ISP_AE_INIT_PARAM structure	Input
sensor_id[32]	Image sensor name	Input
shutter	Shutter time initial value in us	Input
shutter_step	Shutter time step in ns	Input
shutter_min	Minimum shutter time in us	Input
shutter_max	Maximum shutter time in us	Input
sensor_gain	Sensor gain initial value	Input
sensor_gain_min	Sensor gain minimum value	Input
sensor_gain_max	Sensor gain maximum value	Input
isp_gain	ISP gain initial value	Input
isp_gain_max	ISP gain maximum value	Input
FNx10	Aperture value *10	Input
fps	Number of frames per second	Input
shutterHDRShort_step	Shutter time step in ns for HDR short exposure	Input
shutterHDRShort_min	Minimum shutter time in us for HDR short exposure	Input
shutterHDRShort_max	Maximum shutter time in us for HDR short exposure	Input
sensor_gainHDRShort_min	Sensor gain minimum value for HDR short exposure	Input
sensor_gainHDRShort_max	Sensor gain maximum value for HDR short exposure	Input
AvgBlkX	Number of blocks in x-axis	Input
AvgBlkY	Number of blocks in y-axis	Input

10.3. ISP_AE_INFO¶

Description

ISP AE hardware statistics

Declaration

/*! @brief ISP report to AE, hardware statistic */
typedef struct
{
    U32 Size;       /**< struct size*/
    ISP_HIST  *hist1;   /**< HW statistic histogram 1*/
    ISP_HIST  *hist2;   /**< HW statistic histogram 2*/
    U32 AvgBlkX;  /**< HW statistics average block number*/
    U32 AvgBlkY;  /**< HW statistics average block number*/
    ISP_AE_SAMPLE *avgs; /**< HW statistics average block data*/
    U32 Shutter;                    /**< Current shutter in us*/
    U32 SensorGain;                 /**< Current Sensor gain, 1X = 1024 */
    U32 IspGain;                    /**< Current ISP gain, 1X = 1024*/
    U32 ShutterHDRShort;           /**< Current shutter in us*/
    U32 SensorGainHDRShort;        /**< Current Sensor gain, 1X = 1024 */
    U32 IspGainHDRShort;           /**< Current ISP gain, 1X = 1024*/

    /*CUS3A v1.1*/
    U32 PreAvgY;           /**< Previous frame brightness */
    U32 HDRCtlMode;        /**< 0 = HDR off;*/
  /**< 1 = Separate shutter & Separate sensor gain settings*/
  /**< 2 = Separate shutter & Share sensor gain settings*/
  /**< 3 = Share shutter & Separate sensor gain settings*/
    U32 FNx10;             /**< Aperture in FNx10 */
    U32 CurFPS;            /**< Current sensor FPS */
    U32 PreWeightY;        /**< Previous frame brightness with ROI weight */

    /*CUS3A v1.2*/
    ISP_IR_HISTX *histIR;           /**< HW statistic histogram IR*/
/*CUS3A v1.3*/
U16 IRLightStr;    /**< Setting IR light source strength , range 0~100**/
} ISP_AE_INFO;

Member

Parameter Name	Description	Input/Output
Size	Size of ISP_AE_INFO structure	Input
hist1	Histogram 1	Input
hist2	Histogram 2 (not supported by Macaron/Pudding)	Input
AvgBlkX	Number of blocks in x-axis	Input
AvgBlkY	Number of blocks in y-axis	Input
avgs	Block brightness statistics	Input
Shutter	Shutter time in us when the statistics are generated	Input
SensorGain	Sensor Gain 1X=1024 when the statistics are generated	Input
IspGain	ISP Gain 1X=1024 when the statistics are generated	Input
ShutterHDRShort	Shutter time in us when the statistics are generated (HDR short)	Input
SensorGainHDRShort	Sensor Gain 1X=1024 when the statistics are generated (HDR short)	Input
IspGainHDRShort	ISP Gain 1X=1024 when the statistics are generated (HDR short)	Input
PreAvgY	Average brightness 8bit*10 of previous AE result	Input
HDRCtlMode	Supported shutter/gain format as returned by sensor when in HDR mode	Input
FNx10	FN value applied when the statistics are generated	Input
CurFPS	Current FPS	Input
PreWeightY	Average brightness 8bit*10 of previous AE result after ROI weighting	Input
histIR	Histogram relating to IR information when RGB-IR sensor is used	Input
IRLightStr	Current IR light strength, ranging from 0 (IR closed) to 100 (max. strength)	Input

10.4. ISP_AE_RESULT¶

Description

ISP AE algorithm result

Declaration

/*! @brief ISP ae algorithm result*/
typedef struct
{
    U32 Size;           /**< struct size*/
    U32 Change;       /**< if true, apply this result to hw register*/
    U32 Shutter;         /**< Shutter in us */
    U32 SensorGain;      /**< Sensor gain, 1X = 1024 */
    U32 IspGain;         /**< ISP gain, 1X = 1024 */
    U32 ShutterHdrShort;     /**< Shutter in us */
    U32 SensorGainHdrShort;  /**< Sensor gain, 1X = 1024 */
    U32 IspGainHdrShort;     /**< ISP gain, 1X = 1024 */
    U32 u4BVx16384;      /**< Bv * 16384 in APEX system, EV = Av + Tv = Sv + Bv */
    U32 AvgY;       /**< frame brightness */
    U32 HdrRatio;   /**< hdr ratio, 1X = 1024 */

    /*CUS3A v1.1*/
    U32 FNx10;           /**< F number * 10 */
    U32 DebandFPS;      /**< Target fps when running auto debanding*/
U32 WeightY;        /**< frame brightness with ROI weight*/

/*CUS3A v1.3*/
    U16 GMBlendRatio;   /**< Adaptive Gamma Blending Ratio from AE**/
U16 IRLightStr;    /**< Setting IR light source strength , range 0~100**/
}ISP_AE_RESULT;

Member

Parameter Name	Description	Input/Output
Size	Size of ISP_AE_RESULT structure	Input
Change	Notify ISP whether the current algorithm result is to be applied to hardware	Output
Shutter	Shutter time in us as reported by AE algorithm	Output
SensorGain	Sensor gain 1X=1024 as reported by AE algorithm	Output
IspGain	ISP gain 1X=1024 as reported by AE algorithm	Output
ShutterHdrShort	Shutter time in us as reported by AE algorithm (HDR Short)	Output
SensorGainHdrShort	Sensor gain 1X=1024 as reported by AE algorithm (HDR Short)	Output
IspGainHdrShort	ISP gain 1X=1024 as reported by AE algorithm (HDR Short)	Output
u4Bvx16384	Current scene brightness estimate as reported by AE algorithm	Output
AvgY	Current frame average brightness as reported by AE algorithm	Output
HdrRatio	Current frame HDR long/short exposure ratio 1x=1024 as reported by AE algorithm	Output
FNx10	Current FN number as reported by AE algorithm	Output
DebandFPS	Current FPS as reported by AE algorithm	Output
WeightY	Current average brightness after ROI weighting as reported by AE algorithm	Output
GMBlendRatio	Blending ratio of IQ AdaptiveGamma to RGBGamma, controlled by AE algorithm. Range: 1~1025. Value 513 represents using RGBGamma directly without any blending.	Output
IRLightStr	IR light strength determined by AE algorithm, ranging from 0 (IR closed) to 100 (max. strength)	Output

10.5. ISP_AE_CTRL_CMD¶

Description

AE control command. Through the callback interface,
```
int (*ctrl)(void* pdata,ISP_AE_CTRL_CMD cmd,void* param)
```
ISP can set the AE parameter. The type of the parameter param will vary with the control command used.

Declaration

typedef enum
{
    ISP_AE_FPS_SET, /**< ISP notify AE sensor FPS has changed*/
}ISP_AE_CTRL_CMD;

Member

Name Description Parameter Type

ISP_AE_FPS_SET ISP notifies AE library the current FPS has changed U32

11. AWB Callback Interface¶

int (*init)(void* pdata)

Description

The ISP notifies AWB library to start initialization through this interface.
Parameter

Parameter Name Description Input/Output

pdata Algorithm private data pointer Input
Return Value

Return Value Description

0 Successful

-1 Failed
```
void (*release)(void* pdata)
```
Description

The ISP notifies AWB library to start release through this interface.

Parameter

Parameter Name	Description	Input/Output
pdata	Algorithm private data pointer	Input

void (*run)(void* pdata,const ISP_AWB_INFO *awb_info,const ISP_AE_INFO *ae_info,ISP_AWB_RESULT *result)

Description

The ISP notifies AWB library to start calculation through this interface.

Parameter

Parameter Name	Description	Input/Output
pdata	Algorithm private data pointer	Input
awb_info	ISP AWB statistical data structure pointer	Input
ae_info	ISP AE statistical data structure pointer	Input
result	ISP AWB algorithm result structure pointer	Output

int (*ctrl)(void* pdata,ISP_AWB_CTRL_CMD cmd,void* param)

Description

The ISP controls AWB parameters through this interface.
Return Value

Return Value Description

0 Successful

\<0 Failed
Parameter

Parameter Name	Description	Input/Output
pdata	Algorithm private data pointer	Input
cmd	AWB control command	Input
param	AWB control command parameter	Input

12. AWB Data Structure¶

12.1. ISP_AWB_INTERFACE¶

Description

AWB library registration structure. The algorithm needs to implement and fill in initialization, release, run, and control callback pointers to enable the ISP to call this structure.

Declaration

typedef struct
{
    void *pdata; /**< Private data for AE algorithm.*/
    int (*init)(void *pdata);
    void (*release)(void *pdata);
    void (*run)(void *pdata, const ISP_AWB_INFO *awb_info, const ISP_AE_INFO *ae_info, ISP_AWB_RESULT *result);
    int (*ctrl)(void *pdata, ISP_AWB_CTRL_CMD cmd, void* param);
} ISP_AWB_INTERFACE

Member

Name	Description
pdata	AWB library private data pointer
init	AWB library initialization callback pointer
release	AWB library release callback pointer
run	AWB library calculation callback pointer. ISP will request AWB algorithm to start calculation when receiving the statistics of a frame.
ctrl	AWB library control callback pointer

12.2. ISP_AWB_INFO¶

Description

ISP AWB hardware statistics

Declaration

/*! @brief AWB HW statistics data*/
typedef struct
{
    U32 AvgBlkX;
    U32 AvgBlkY;
    U32 CurRGain;
    U32 CurGGain;
    U32 CurBGain;
    void *avgs;

    /*CUS3A v1.1*/
    U8 HDRMode;       /**< Noramal or HDR mode */
    void* pAwbStatisShort;      /**< Short Shutter AWB statistic data*/
    U32 u4BVx16384;    /**< From AE output, Bv * 16384 in APEX system, EV = Av + Tv = Sv + Bv  */
    S32 WeightY;       /**< frame brightness with ROI weight0 */
}__attribute__((packed, aligned(1))) ISP_AWB_INFO;

Member

Parameter Name	Description	Input/Output
AvgBlkX	Number of blocks in x-axis	Input
AvgBlkY	Number of blocks in y-axis	Input
CurRGain	AWB R Gain of current frame	Input
CurGGain	AWB G Gain of current frame	Input
CurBGain	AWB B Gain of current frame	Input
avgs	ISP AWB block RGB statistics	Input
HDRMode	Indicates whether the current frame is in HDR mode	Input
pAwbStatisShort	ISP AWB block RGB statistics (short exposure information under HDR mode)	Input
U4BVx16384	BV information returned by AE	Input
WeightY	Average brightness after ROI weighting as returned by AE	Input

12.3. ISP_AWB_RESULT¶

Description

ISP AWB algorithm result

Declaration

/*! @brief AWB algorithm result*/
typedef struct
{
U32 Size;       /**< struct size*/
U32 Change;    /**< if true, apply this result to hw register*/
    U32 R_gain;     /**< AWB gain for R channel*/
    U32 G_gain;     /**< AWB gain for G channel*/
    U32 B_gain;     /**< AWB gain for B channel*/
    U32 ColorTmp;   /**< Return color temperature*/
} ISP_AWB_RESULT;

Member

Parameter Name	Description	Input/Output
Size	Size of ISP_AWB_RESULT structure	Input
Change	Notify ISP whether the current algorithm result is to be applied to hardware	Output
R_gain	AWB R gain 1X = 1024 as reported by AWB algorithm	Output
G_gain	AWB G gain 1X = 1024 as reported by AWB algorithm	Output
B_gain	AWB B gain 1X = 1024 as reported by AWB algorithm	Output
ColorTmp	Current scene color temperature K as reported by AWB algorithm	Output

12.4. ISP_AWB_CTRL_CMD¶

Description

AWB control command. Through the callback interface,
```
int (*ctrl)(void* pdata,ISP_AWB_CTRL_CMD cmd,void* param)
```
ISP can set the AWB parameter. The type of the parameter param will vary with the control command used.

Declaration

typedef enum
{
    ISP_AWB_MODE_SET,
}ISP_AWB_CTRL_CMD;

Member

Name Description Parameter Type

ISP_AWB_MODE_SET ISP notifies AWB library the current AWB mode has changed U32

13. AF Callback Interface¶

int (*init)(void* pdata, ISP_AF_INIT_PARAM *param)

Description

The ISP notifies AF library to start initialization through this interface.
Parameter

Parameter Name Description Input/Output

pdata Algorithm private data pointer Input

param AF initial setting Input/Output
Return Value

Return Value Description

0 Successful

-1 Failed
```
void (*release)(void* pdata)
```
Description

The ISP notifies AF library to start release through this interface.
Parameter

Parameter Name Description Input/Output

pdata Algorithm private data pointer Input
```
void (*run)(void* pdata,const ISP_AF_INFO *af_info ,ISP_AF_RESULT *result)
```
Description

The ISP notifies AF library to start calculation through this interface.

Parameter

Parameter Name	Description	Input/Output
pdata	Algorithm private data pointer	Input
af_info	ISP AF statistical data structure pointer	Input
result	ISP AF algorithm result structure pointer	Output

int (*ctrl)(void* pdata,ISP_AF_CTRL_CMD cmd,void* param) (Reserve)

Description

The ISP controls AF parameters through this interface.
Return Value

Return Value Description

0 Successful

\<0 Failed
Parameter

Parameter Name Description Input/Output

pdata Algorithm private data pointer (Reserved) Input

cmd AF control command (Reserved) Input

param AF control command parameter (Reserved) Input

14. AF Data Structure¶

14.1. ISP_AF_INTERFACE¶

Description

AF library registration structure. The algorithm needs to implement and fill in initialization, release, run, and control callback pointers to enable the ISP to call this structure.

Declaration

typedef struct isp_af_interface
{
    void *pdata; /**< Private data for AF algorithm.*/
    int (*init)(void *pdata, ISP_AF_INIT_PARAM *param);   
void (*release)(void *pdata);
    void (*run)(void *pdata,const ISP_AF_INFO *af_info, ISP_AF_RESULT *result);
    int (*ctrl)(void *pdata,ISP_AF_CTRL_CMD cmd,void* param);
}ISP_AF_INTERFACE;

Member

Name	Description
pdata	AF library private data pointer
init	AF library initialization callback pointer
release	AF library release callback pointer
run	AF library calculation callback pointer. ISP will request AF algorithm to start calculation when receiving the statistics of a frame.
ctrl	AF library control callback pointer

14.2. ISP_AF_INIT_PARAM¶

Description

ISP uses this structure to indicate the AF algorithm related hardware initialization status.

Declaration

typedef struct _isp_af_init_param
{
    u32 Size; /**< struct size*/
    ISP_AF_RECT af_stats_win[16];
/**< CUS3A v1.3*/
    u32 CurPos; //motor current position
    u32 MinPos; //motor down position limit
    u32 MaxPos; //motor up position limit
    u32 MinStep;//motor minimum step
    u32 MaxStep;//motor maximum step
} ISP_AF_INIT_PARAM;

Member

Name	Description	輸出/輸入
Size	Size of ISP_AF_INIT_PARAM structure	Input
Af_stats_win[16]	Current AF windows set value	Input
CurPos	Current AF motor position	Input
MinPos	AF motor position lower limit	Input
MaxPos	AF motor position upper limit	Input
MinStep	AF motor minimum step	Input
MaxStep	AF motor maximum step

14.3. ISP_AF_INFO¶

Description

ISP AF hardware statistics

Declaration

/*! @brief AF HW statistics data*/
typedef struct
{
    U32 Size;       /**< struct size*/
    ISP_AF_STATS pStats; /**< AF statistic*/
    u32 CurPos;     /**<motor current position*/
    u32 MinPos;     /**<motor down position limit*/
u32 MaxPos;     /**<motor up position limit*/
}ISP_AF_INFO;

Member

Parameter Name	Description	Input/Output
Size	Size of ISP_AF_INFO data structure	Input
pStats	ISP AF statistics	Output
CurPos	Current AF motor position	Input
MinPos	AF motor position lower limit	Input
MaxPos	AF motor position upper limit	Input

14.4. ISP_AF_STATS¶

Description

ISP AF hardware statistics

Declaration

//for Pretzel, Macaron, Pudding              //for Twinkie
typedef struct{                         typedef struct{
    MI_U8 high_iir[5*16];                       MI_U8 high_iir[6*16];
    MI_U8 low_iir[5*16];                        MI_U8 low_iir[6*16];
    MI_U8 luma[4*16];                           MI_U8 luma[6*16];
    MI_U8 sobel_v[5*16];                        MI_U8 sobel_v[6*16];
    MI_U8 sobel_h[5*16];                        MI_U8 sobel_h[6*16];
    MI_U8 ysat[3*16];                           MI_U8 ysat[3*16];
} AF_STATS_PARAM_t ;                        } AF_STATS_PARAM_t ;

typedef struct{                         typedef struct{
    AF_STATS_PARAM_t stParaAPI[16];             AF_STATS_PARAM_t stParaAPI;
} CusAFStats_t;

Member

Parameter Name	Description	Input/Output
IIR_1	AF horizontal direction IIR filter statistics as calculated by ROI	Output
IIR_2	AF horizontal direction IIR filter statistics as calculated by ROI	Output
Luma	AF luminance statistics as calculated by ROI	Output
FIR_h	AF horizontal direction FIR filter statistics as calculated by ROI	Output
FIR_v	AF vertical direction FIR filter statistics as calculated by ROI	Output
YSat	Number of statistical values greater than YThd in AF FIR filter statistics as calculated by ROI	Output

* By default, IIR_1 is IIR Low, and IIR_2, IIR High.
* Each filter is divided into 16 blocks. The first 9 blocks correspond to the 9 cells of the 3x3 magic square by default. The 16^th block is user-definable, and is normally used for touch AF.
* Higher IIR FIR values represent greater image clarity. Luma means brightness.

Usage Recommendation

Block weighting weight: We suggest using ”center weight.”

Statistics: We suggest using IIR_1 (IIR Low) in all cases.

IIR_2 (IIR High) will change significantly when the image is clear, but it will hardly change when the image is blurred; for this reason, it is suitable for determination of clearest position.

FIR is biased in nature and is only suitable for strength change references.

14.5. ISP_AF_RESULT¶

Description

ISP AF algorithm result

Declaration

/*! @brief AF algorithm result*/
typedef struct
{
    u32 Size;           /**< struct size*/
    u32 Change; /**< if true, apply this result to hw*/
    u32 NextPos; /**< Next position*/
/**< CUS3A v1.3*/
    u32                   ChangeParam;      /** if true, apply the following 4 results to hw register **/
    CusAFRoiMode_t        ROIMode;          /** roi mode configuration**/
    CusAFWin_t            Window[16];       /** AF statistics window position **/
    CusAFFilter_t         Filter;           /** AF filter paramater**/
    CusAFFilterSq_t       FilterSq;         /** AF filter square parameter**/
} ISP_AF_RESULT;

Member

Parameter Name	Description	Input/Output
Change	Notify ISP whether the current algorithm result is to be applied to hardware	Output
NextPos	AF motor target position	Output
ChangeParam	Notify ISP whether ROIMode, Window, Filter, and FilterSq have been updated	Output
ROIMode	AF window ROI mode	Output
Window	AF window position	Output
Filter	AF filter parameter	Output
FilterSq	AF filter square	Output

15. Sample Code¶

#include <mi_isp.h>

//// AE INTERFACE TEST ////

int ae_init(void* pdata, ISP_AE_INIT_PARAM *init_state)
{
    printf("****** ae_init ,shutter=%d,shutter_step=%d,sensor_gain_min=%d,sensor_gain_max=%d *******\n",
        (int)init_state->shutter,
        (int)init_state->shutter_step,
        (int)init_state->sensor_gain,
        (int)init_state->sensor_gain_max
        );
    return 0;
}

void ae_run(void* pdata, const ISP_AE_INFO *info, ISP_AE_RESULT *result)
{
#define log_info 1

    // Only one can be chosen (the following three define)
#define shutter_test 0
#define gain_test 0
#define AE_sample 1

#if (shutter_test) || (gain_test)
    static int AE_period = 4;
#endif
    static unsigned int fcount = 0;
    unsigned int max = info->AvgBlkY * info->AvgBlkX;
    unsigned int avg = 0;
    unsigned int n;
#if gain_test
    static int tmp = 0;
    static int tmp1 = 0;
#endif

    result->Change              = 0;
    result->u4BVx16384          = 16384;
    result->HdrRatio            = 10; //infinity5 //TBD //10 * 1024;   //user define hdr exposure ratio
    result->IspGain             = 1024;
    result->SensorGain          = 4096;
    result->Shutter             = 20000;
    result->IspGainHdrShort     = 1024;
    result->SensorGainHdrShort  = 1024;
    result->ShutterHdrShort     = 1000;
    //result->Size         = sizeof(CusAEResult_t);

    for(n = 0; n < max; ++n)
    {
        avg += info->avgs[n].y;
    }
    avg /= max;

    result->AvgY         = avg;

#if shutter_test // shutter test under constant sensor gain
    int Shutter_Step = 100; //per frame
    int Shutter_Max = 33333;
    int Shutter_Min = 150;
    int Gain_Constant = 10240;

    result->SensorGain = Gain_Constant;
    result->Shutter = info->Shutter;

    if(++fcount % AE_period == 0)
    {
        if (tmp == 0)
        {
            result->Shutter = info->Shutter + Shutter_Step * AE_period;
            //printf("[shutter-up] result->Shutter = %d \n", result->SensorGain);
        }
        else
        {
            result->Shutter = info->Shutter - Shutter_Step * AE_period;
            //printf("[shutter-down] result->Shutter = %d \n", result->SensorGain);
        }
        if (result->Shutter >= Shutter_Max)
        {
            result->Shutter = Shutter_Max;
            tmp = 1;
        }
        if (result->Shutter <= Shutter_Min)
        {
            result->Shutter = Shutter_Min;
            tmp = 0;
        }
    }
#if log_info
    printf("fcount = %d, Image avg = 0x%X \n", fcount, avg);
    printf("tmp = %d, Shutter: %d -> %d \n", tmp, info->Shutter, result->Shutter);
#endif
#endif

#if gain_test // gain test under constant shutter
    int Gain_Step = 1024; //per frame
    int Gain_Max = 1024 * 100;
    int Gain_Min = 1024 * 2;
    int Shutter_Constant = 20000;

    result->SensorGain = info->SensorGain;
    result->Shutter = Shutter_Constant;

    if(++fcount % AE_period == 0)
    {
        if (tmp1 == 0)
        {
            result->SensorGain = info->SensorGain + Gain_Step * AE_period;
            //printf("[gain-up] result->SensorGain = %d \n", result->SensorGain);
        }
        else
        {
            result->SensorGain = info->SensorGain - Gain_Step * AE_period;
            //printf("[gain-down] result->SensorGain = %d \n", result->SensorGain);
        }
        if (result->SensorGain >= Gain_Max)
        {
            result->SensorGain = Gain_Max;
            tmp1 = 1;
        }
        if (result->SensorGain <= Gain_Min)
        {
            result->SensorGain = Gain_Min;
            tmp1 = 0;
        }
    }
#if log_info
    printf("fcount = %d, Image avg = 0x%X \n", fcount, avg);
    printf("tmp = %d, SensorGain: %d -> %d \n", tmp, info->SensorGain, result->SensorGain);
#endif
#endif

#if AE_sample
    int y_lower = 0x28;
    int y_upper = 0x38;
    int change_ratio = 10; // percentage
    int Gain_Min = 1024 * 2;
    int Gain_Max = 1024 * 1000;
    int Shutter_Min = 150;
    int Shutter_Max = 33333;

    result->SensorGain = info->SensorGain;
    result->Shutter = info->Shutter;

    if(avg < y_lower)
    {
        if (info->Shutter < Shutter_Max)
        {
            result->Shutter = info->Shutter + (info->Shutter * change_ratio / 100);
            if (result->Shutter > Shutter_Max) result->Shutter = Shutter_Max;
        }
        else
        {
            result->SensorGain = info->SensorGain + (info->SensorGain * change_ratio / 100);
            if (result->SensorGain > Gain_Max) result->SensorGain = Gain_Max;
        }
        result->Change = 1;
    }
    else if(avg > y_upper)
    {
        if (info->SensorGain > Gain_Min)
        {
            result->SensorGain = info->SensorGain - (info->SensorGain * change_ratio / 100);
            if (result->SensorGain < Gain_Min) result->SensorGain = Gain_Min;
        }
        else
        {
            result->Shutter = info->Shutter - (info->Shutter * change_ratio / 100);
            if (result->Shutter < Shutter_Min) result->Shutter = Shutter_Min;
        }
        result->Change = 1;
    }

#if 0 //infinity5 //TBD
    //hdr demo code
    result->SensorGainHdrShort = result->SensorGain;
    result->ShutterHdrShort = result->Shutter * 1024 / result->HdrRatio;
#endif

#if log_info
    printf("fcount = %d, Image avg = 0x%X \n", fcount, avg);
    printf("SensorGain: %d -> %d \n", (int)info->SensorGain, (int)result->SensorGain);
    printf("Shutter: %d -> %d \n", (int)info->Shutter, (int)result->Shutter);
#endif

#endif
}

void ae_release(void* pdata)
{
    printf("************* ae_release *************\n");
}

//// AWB INTERFACE TEST ////

int awb_init(void *pdata)
{
    printf("************ awb_init **********\n");
    return 0;
}
 
void awb_run(void* pdata, const ISP_AWB_INFO *info, ISP_AWB_RESULT *result)
{
#define log_info 1

    static u32 count = 0;
    int avg_r = 0;
    int avg_g = 0;
    int avg_b = 0;
    int tar_rgain = 1024;
    int tar_bgain = 1024;
    int x = 0;
    int y = 0;

    result->R_gain = info->CurRGain;
    result->G_gain = info->CurGGain;
    result->B_gain = info->CurBGain;
    result->Change = 0;
    result->ColorTmp = 6000;

    if (++count % 4 == 0)
    {
        //center area YR/G/B avg
        for (y = 30; y < 60; ++y)
        {
            for (x = 32; x < 96; ++x)
            {
                avg_r += info->avgs[info->AvgBlkX * y + x].r;
                avg_g += info->avgs[info->AvgBlkX * y + x].g;
                avg_b += info->avgs[info->AvgBlkX * y + x].b;
            }
        }
        avg_r /= 30 * 64;
        avg_g /= 30 * 64;
        avg_b /= 30 * 64;

        if (avg_r < 1)
            avg_r = 1;
        if (avg_g < 1)
            avg_g = 1;
        if (avg_b < 1)
            avg_b = 1;

#if log_info
        printf("AVG R / G / B = %d, %d, %d \n", avg_r, avg_g, avg_b);
#endif

        // calculate Rgain, Bgain
        tar_rgain = avg_g * 1024 / avg_r;
        tar_bgain = avg_g * 1024 / avg_b;

        if (tar_rgain > info->CurRGain)
        {
            if (tar_rgain - info->CurRGain < 384)
                result->R_gain = tar_rgain;
            else
                result->R_gain = info->CurRGain + (tar_rgain - info->CurRGain) / 10;
        }
        else
        {
            if (info->CurRGain - tar_rgain < 384)
                result->R_gain = tar_rgain;
            else
                result->R_gain = info->CurRGain - (info->CurRGain - tar_rgain) / 10;
        }

        if (tar_bgain > info->CurBGain)
        {
            if (tar_bgain - info->CurBGain < 384)
                result->B_gain = tar_bgain;
            else
                result->B_gain = info->CurBGain + (tar_bgain - info->CurBGain) / 10;
        }
        else
        {
            if (info->CurBGain - tar_bgain < 384)
                result->B_gain = tar_bgain;
            else
                result->B_gain = info->CurBGain - (info->CurBGain - tar_bgain) / 10;
        }

        result->Change = 1;
        result->G_gain = 1024;

#if log_info
        printf("[current] r=%d, g=%d, b=%d \n", (int)info->CurRGain, (int)info->CurGGain, (int)info->CurBGain);
        printf("[result] r=%d, g=%d, b=%d \n", (int)result->R_gain, (int)result->G_gain, (int)result->B_gain);
#endif
    }
}

void awb_release(void *pdata)
{
    printf("************ awb_release **********\n");
}

//// AF INTERFACE TEST ////

int af_init(void *pdata, ISP_AF_INIT_PARAM *param)
{
    MI_U32 u32ch = 0;
    MI_U8 u8win_idx = 16;
    CusAFRoiMode_t taf_roimode;
    CusAFWin_t taf_win;

    printf("************ af_init **********\n");
#if 1
    //Init Normal mode setting
    static CusAFWin_t afwin[16] =
    {
        { 0, {   0,    0,  255,  255}},
        { 1, { 256,    0,  511,  255}},
        { 2, { 512,    0,  767,  255}},
        { 3, { 768,    0, 1023,  255}},
        { 4, {   0,  256,  255,  511}},
        { 5, { 256,  256,  511,  511}},
        { 6, { 512,  256,  767,  511}},
        { 7, { 768,  256, 1023,  511}},
        { 8, {   0,  512,  255,  767}},
        { 9, { 256,  512,  511,  767}},
        {10, { 512,  512,  767,  767}},
        {11, { 768,  512, 1023,  767}},
        {12, {   0,  768,  255, 1023}},
        {13, { 256,  768,  511, 1023}},
        {14, { 512,  768,  767, 1023}},
        {15, { 768,  768, 1023, 1023}}
    };
    for(u8win_idx = 0; u8win_idx < 16; ++u8win_idx){
        MI_ISP_CUS3A_SetAFWindow(u32ch, &afwin[u8win_idx]);
    }

taf_roimode.mode = AF_ROI_MODE_NORMAL;
    taf_roimode.u32_vertical_block_number = 1;
    MI_ISP_CUS3A_SetAFRoiMode(u32ch, &taf_roimode);

#else
    //Init Matrix mode setting
    static CusAFWin_t afwin[16] =
    {
#if 1
        //full image, equal divide to 16x16
        //window setting need to multiple of two
        {0, {16, 0, 62, 62}},
        {1, {64, 64, 126, 126}},
        {2, {128, 128, 190, 190}},
        {3, {192, 192, 254, 254}},
        {4, {256, 256, 318, 318}},
        {5, {320, 320, 382, 382}},
        {6, {384, 384, 446, 446}},
        {7, {448, 448, 510, 510}},
        {8, {512, 512, 574, 574}},
        {9, {576, 576, 638, 638}},
        {10, {640, 640, 702, 702}},
        {11, {704, 704, 766, 766}},
        {12, {768, 768, 830, 830}},
        {13, {832, 832, 894, 894}},
        {14, {896, 896, 958, 958}},
        {15, {960, 960, 1022, 1022}}
#else
        //use two row only => 16x2 win
        {0, {16, 0, 62, 62}},
        {1, {64, 64, 126, 126}},
        {2, {128, 0, 190, 2}},      //win2 v_str, v_end doesn't use, set to (0, 2)
        {3, {192, 0, 254, 2}},
        {4, {256, 0, 318, 2}},
        {5, {320, 0, 382, 2}},
        {6, {384, 0, 446, 2}},
        {7, {448, 0, 510, 2}},
        {8, {512, 0, 574, 2}},
        {9, {576, 0, 638, 2}},
        {10, {640, 0, 702, 2}},
        {11, {704, 0, 766, 2}},
        {12, {768, 0, 830, 2}},
        {13, {832, 0, 894, 2}},
        {14, {896, 0, 958, 2}},
        {15, {960, 0, 1022, 2}}        
#endif
    };

    for(u8win_idx = 0; u8win_idx < 16; ++u8win_idx){
        MI_ISP_CUS3A_SetAFWindow(u32ch, &afwin[u8win_idx]);
    }

taf_roimode.mode = AF_ROI_MODE_MATRIX;
    taf_roimode.u32_vertical_block_number = 16; //16xN, N=16
    MI_ISP_CUS3A_SetAFRoiMode(u32ch, &taf_roimode);

#endif

    static CusAFFilter_t affilter =
    {
#if Twinkie || Pretzel || Macaron
        //filter setting with sign value
        //{63, -126, 63, -109, 48, 0, 320, 0, 1023},
        //{63, -126, 63, 65, 55, 0, 320, 0, 1023}

        //convert to hw format (sign bit with msb)
        63, 126+1024, 63, 109+128, 48, 0, 320, 0, 1023,
        63, 126+1024, 63, 65, 55, 0, 320, 0, 1023,
#elseif Pudding
        //filter setting with sign value
        //{s9, s10, s9, s7, s7}
        //high: 37, 0, -37, 83, 40; 37, 0, -37, -54, 34; 32, 0, -32, 14, 0
        //low:  15, 0, -15, -79, 44; 15, 0, -15, -115, 55; 14, 0, -14, -91, 37

        //convert to hw format (sign bit with msb)
        37, 0, 37+512, 83, 40, 0, 1023, 0, 1023,    //high
        15, 0, 15+512, 79+128, 44, 0, 1023, 0, 1023,    //low
        1, 37, 0, 37+512,  54+128, 34, 1, 32, 0, 32+512, 14, 0, //high-e1, e2
        1, 15, 0, 15+512, 115+128, 55, 1, 14, 0, 14+512, 91+128, 37,  //low-e1, e2
#endif
    };

    MI_ISP_CUS3A_SetAFFilter(0, &affilter);
    return 0;
}

void af_run(void *pdata, const ISP_AF_INFO *af_info, ISP_AF_RESULT *result)
{
    #define af_log_info 1

#if af_log_info
    int i=0,x=0;

    printf("\n\n");

    //print row0 16wins
    x=0;
    for (i=0; i<16; i++){
        printf("[AF]win%d-%d iir0: 0x%02x%02x%02x%02x%02x, iir1:0x%02x%02x%02x%02x%02x, luma:0x%02x%02x%02x%02x, sobelh:0x%02x%02x%02x%02x%02x, sobelv:0x%02x%02x%02x%02x%02x ysat:0x%02x%02x%02x\n",
            x, i,
            af_info->af_stats.stParaAPI[x].high_iir[4+i*5],af_info->af_stats.stParaAPI[x].high_iir[3+i*5],af_info->af_stats.stParaAPI[x].high_iir[2+i*5],af_info->af_stats.stParaAPI[x].high_iir[1+i*5],af_info->af_stats.stParaAPI[x].high_iir[0+i*5],
            af_info->af_stats.stParaAPI[x].low_iir[4+i*5],af_info->af_stats.stParaAPI[x].low_iir[3+i*5],af_info->af_stats.stParaAPI[x].low_iir[2+i*5],af_info->af_stats.stParaAPI[x].low_iir[1+i*5],af_info->af_stats.stParaAPI[x].low_iir[0+i*5],
            af_info->af_stats.stParaAPI[x].luma[3+i*4],af_info->af_stats.stParaAPI[x].luma[2+i*4],af_info->af_stats.stParaAPI[x].luma[1+i*4],af_info->af_stats.stParaAPI[x].luma[0+i*4],
            af_info->af_stats.stParaAPI[x].sobel_h[4+i*5],af_info->af_stats.stParaAPI[x].sobel_h[3+i*5],af_info->af_stats.stParaAPI[x].sobel_h[2+i*5],af_info->af_stats.stParaAPI[x].sobel_h[1+i*5],af_info->af_stats.stParaAPI[x].sobel_h[0+i*5],
            af_info->af_stats.stParaAPI[x].sobel_v[4+i*5],af_info->af_stats.stParaAPI[x].sobel_v[3+i*5],af_info->af_stats.stParaAPI[x].sobel_v[2+i*5],af_info->af_stats.stParaAPI[x].sobel_v[1+i*5],af_info->af_stats.stParaAPI[x].sobel_v[0+i*5],
            af_info->af_stats.stParaAPI[x].ysat[2+i*3],af_info->af_stats.stParaAPI[x].ysat[1+i*3],af_info->af_stats.stParaAPI[x].ysat[0+i*3]
            );
    }

    //print row15 16wins
    x=15;
    for (i=0; i<16; i++){
        printf("[AF]win%d-%d iir0: 0x%02x%02x%02x%02x%02x, iir1:0x%02x%02x%02x%02x%02x, luma:0x%02x%02x%02x%02x, sobelh:0x%02x%02x%02x%02x%02x, sobelv:0x%02x%02x%02x%02x%02x ysat:0x%02x%02x%02x\n",
            x, i,
            af_info->af_stats.stParaAPI[x].high_iir[4+i*5],af_info->af_stats.stParaAPI[x].high_iir[3+i*5],af_info->af_stats.stParaAPI[x].high_iir[2+i*5],af_info->af_stats.stParaAPI[x].high_iir[1+i*5],af_info->af_stats.stParaAPI[x].high_iir[0+i*5],
            af_info->af_stats.stParaAPI[x].low_iir[4+i*5],af_info->af_stats.stParaAPI[x].low_iir[3+i*5],af_info->af_stats.stParaAPI[x].low_iir[2+i*5],af_info->af_stats.stParaAPI[x].low_iir[1+i*5],af_info->af_stats.stParaAPI[x].low_iir[0+i*5],
            af_info->af_stats.stParaAPI[x].luma[3+i*4],af_info->af_stats.stParaAPI[x].luma[2+i*4],af_info->af_stats.stParaAPI[x].luma[1+i*4],af_info->af_stats.stParaAPI[x].luma[0+i*4],
            af_info->af_stats.stParaAPI[x].sobel_h[4+i*5],af_info->af_stats.stParaAPI[x].sobel_h[3+i*5],af_info->af_stats.stParaAPI[x].sobel_h[2+i*5],af_info->af_stats.stParaAPI[x].sobel_h[1+i*5],af_info->af_stats.stParaAPI[x].sobel_h[0+i*5],
            af_info->af_stats.stParaAPI[x].sobel_v[4+i*5],af_info->af_stats.stParaAPI[x].sobel_v[3+i*5],af_info->af_stats.stParaAPI[x].sobel_v[2+i*5],af_info->af_stats.stParaAPI[x].sobel_v[1+i*5],af_info->af_stats.stParaAPI[x].sobel_v[0+i*5],
            af_info->af_stats.stParaAPI[x].ysat[2+i*3],af_info->af_stats.stParaAPI[x].ysat[1+i*3],af_info->af_stats.stParaAPI[x].ysat[0+i*3]
            );
    }
#endif}

void af_release(void *pdata)
{
    printf("************ af_release **********\n");
}

int main(int argc,char** argv)
{
    //TO DO: Register 3rd party AE/AWB library
    ISP_AE_INTERFACE tAeIf;
    ISP_AWB_INTERFACE tAwbIf;

    CUS3A_Init();

    /*AE*/
    tAeIf.ctrl = NULL;
    tAeIf.pdata = NULL;
    tAeIf.init = ae_init;
    tAeIf.release = ae_release;
    tAeIf.run = ae_run;

    /*AWB*/
    tAwbIf.ctrl = NULL;
    tAwbIf.pdata = NULL;
    tAwbIf.init = awb_init;
    tAwbIf.release = awb_release;
    tAwbIf.run = awb_run;

CUS3A_RegInterface(0,&tAeIf,&tAwbIf,NULL);
// or use below reginterface
//CUS3A_AERegInterface(0,&tAeIf);
//CUS3A_AWBRegInterface(0,&tAwbIf);

//TO DO: Open MI
MI_SYS_Init();
while(1)
{
usleep(5000);
}
}

15.1. CUS3A_AeAvgDownSample¶

A function for converting AE statistics from 32x32 down to 16x16 is available.

Set forth below is a sample code:

void ae_run(void* pdata, const ISP_AE_INFO *info, ISP_AE_RESULT *result)
{
        ISP_AE_SAMPLE *pInBuf = info->avgs;
        ISP_AE_SAMPLE *pOutBuf = info->avgs;
        unsigned int u32InBlkNumX = info->AvgBlkX;  //value is 32 for MACARON, PUDDING
        unsigned int u32InBlkNumY = info->AvgBlkY;  //value is 32 for MACARON, PUDDING
        unsigned int u32OutBlkNumX = 16;
        unsigned int u32OutBlkNumY = 16;

        CUS3A_AeAvgDownSample(pInBuf, pOutBuf, u32InBlkNumX, u32InBlkNumY, u32OutBlkNumX, u32OutBlkNumY);
        max = u32OutBlkNumX*u32OutBlkNumY;
}

16. Setting AE Crop Range Using MI_ISP¶

16.1. MI_ISP_CUS3A _SetAECropSize¶

Purpose

Set AE statistical value range.

Syntax

MI_RET MI_ISP_CUS3A_SetAECropSize(U32 nChannel, CusAEAWBCropSize_t *data);

Description
1. Call MI_ISP_CUS3A_SetAECropSize to set the AE crop size.
2. Call MI_ISP_CUS3A_SetAEWindowBlockNumber to divide the region within the crop size into MxN blocks.
Parameter

Parameter Name Description

nChannel Video input channel number (normally 0)

data Set the AE statistical value range
Return Value

Return Value Description

MI_RET_SUCCESS Successful

MI_RET_FAIL Failed
Requirement

header file：mi_isp_twinkie.h / mi_isp_pretzel.h / mi_isp.h (for Macaron, Pudding)

.so：libmi_isp_twinkie.so / libmi_isp_pretzel.so / libmi_isp.so (for Macaron, Pudding)

16.2. CusAEAWBCropSize_t¶

Purpose

Set AE statistical value range.

Syntax

typedef struct {
    MI_U16 u2CropX;
    MI_U16 u2CropY;
    MI_U16 u2CropW;
    MI_U16 u2CropH;
} CusAEAWBCropSize_t;

Parameter

Parameter Name Description

U2CropX AE/AWB crop X start (0~1023)

U2CropY AE/AWB crop Y start (0~1023)

U2CropW AE/AWB crop width (0~1023)

U2CropH AE/AWB crop height (0~1023)
CropSize Description
Note

The image width and height already correspond to 1023, so the crop information is based on 1023. FW will automatically take the current actual image size and perform the mapping action. Therefore, user can simply set the relative position without the need to know the current image size.

Assume CropX = 10, CropY = 10, CropW = 300, and CropH = 200:

If the actual image width and height = 1920x1080, the CropX, CropY, CropW and CropH corresponding to 1920x1080 will be as follows:

CropX = 1920*10/1023 = 18

CropY = 1080*10/1023 = 10

CropW = 1920*300/1023 = 563

CropH = 1080*200/1023 = 211

17. Setting AE Block Number Using MI_ISP¶

17.1. MI_ISP_CUS3A_SetAEWindowBlockNumber¶

Purpose

Set AE window block number

Syntax

MI_RET MI_ISP_CUS3A_SetAEWindowBlockNumber(U32 nChannel, MS_CUST_AE_WIN_BLOCK_NUM_TYPE_e eBlkNum);

Description

Call this interface to set AE block number
Parameter

Parameter Name Description

nChannel Video input channel number (normally 0)

eBlkNum Block number, by which an image is divided into X*Y blocks
Return Value

Return Value Description

MI_RET_SUCCESS Successful

MI_RET_FAIL Failed
Requirement

header file: mi_isp_twinkie.h / mi_isp_pretzel.h / mi_isp.h (for Macaron, Pudding)

.so: libmi_isp_twinkie.so / libmi_isp_pretzel.so / libmi_isp.so (for Macaron, Pudding)

17.2. MS_CUST_AE_WIN_BLOCK_NUM_TYPE_e¶

Description

AE block

Definition

typedef enum {
    AE_16x24 = 0,
    AE_32x24,
    AE_64x48,
    AE_64x45,
    AE_128x80,
    AE_128x90
} AeWinBlockNum_e;

Member

Parameter Name	Description
AE_16x24	Image divided into 16*24 blocks
AE_32x24	Image divided into 32*24 blocks
AE_64x48	Image divided into 64*48 blocks
AE_64x45	Image divided into 64*45 blocks
AE_128x80	Image divided into 128*80 blocks
AE_128x90	Image divided into 128*90 blocks

Note

Macaron and Pudding do not support this API. The image will be divided into 32*32 blocks automatically.

18. Setting AE Histogram Window Using MI_ISP¶

18.1. MI_ISP_CUS3A_SetAEHistogramWindow¶

Purpose

Window setting of AE brightness distribution statistics

Syntax

MI_RET MI_ISP_CUS3A_SetAEHistogramWindow(MI_U32 Channel, CusAEHistWin_t *data);

Description

Call this interface to set the window position and size of AE brightness distribution statistics.
Member

Parameter Name Description

nChannel Video input channel number (normally 0)

data Window position and size of AE brightness distribution statistics, for window number (0 or 1)
Return Value

Return Value Description

MI_RET_SUCCESS Successful

MI_RET_FAIL Failed
Requirement

header file: mi_isp_twinkie.h / mi_isp_pretzel.h / mi_isp.h (for Macaron, Pudding)

.so: libmi_isp_twinkie.so / libmi_isp_pretzel.so / libmi_isp.so (for Macaron, Pudding)
Note

Currently, only two windows (0/1) can be set at the same time. The window areas can overlap.

18.2. HistWin_t¶

Description

Window setting of AE brightness distribution statistics

Definition

typedef struct {
    MI_U16 u2Stawin_x_offset;
    MI_U16 u2Stawin_x_size;
    MI_U16 u2Stawin_y_offset;
    MI_U16 u2Stawin_y_size;
    MI_U16 u2WinIdx;
} CusAEHistWin_t;

Member

Parameter Name	Description
u2Stawin_x_offset	Window start coordinate x-axis offset
u2Stawin_x_size	X-axis direction window size
u2Stawin_y_offset	Window start coordinate y-axis offset
u2Stawin_y_size	Y-axis direction window size
u2WinIdx	Window number (0 or 1)

Parameter Range

X-axis offset: 0~127

Y-axis offset: 0~89

X-axis (offset + size) <= 128

Y-axis (offset + size) <= 90
Note

For Macaron and Pudding, the parameter range is as follows:

X-axis offset: 0~31

Y-axis offset: 0~31

X-axis (offset + size) <= 32

Y-axis (offset + size) <= 32

19. Setting AWB Crop Range Using MI_ISP¶

19.1. MI_ISP_CUS3A _SetAWBCropSize¶

Purpose

Set AWB statistical value range.

Syntax

MI_RET MI_ISP_CUS3A_SetAWBCropSize(U32 nChannel, CusAEAWBCropSize_t *data);

Description
1. Call MI_ISP_CUS3A_SetAWBCropSize to set the AWB crop size.
2. Divide the region within the crop size fixedly into 12890 blocks.
Parameter

Parameter Name Description

nChannel Video input channel number (normally 0)

data Set the AWB statistical value range
Return Value

Return Value Description

MI_RET_SUCCESS Successful

MI_RET_FAIL Failed
Requirement

header file: mi_isp_twinkie.h / mi_isp_pretzel.h / mi_isp.h (for Macaron, Pudding)

.so: libmi_isp_twinkie.so / libmi_isp_pretzel.so / libmi_isp.so (for Macaron, Pudding)

19.2. CusAWBCropSize_t¶

Purpose

Set AWB statistical value range.

Syntax

typedef struct {
    MI_U16 u2CropX;
    MI_U16 u2CropY;
    MI_U16 u2CropW;
    MI_U16 u2CropH;
} CusAEAWBCropSize_t;

Parameter

Parameter Name Description

U2CropX AE/AWB crop X start (0~1023)

U2CropY AE/AWB crop Y start (0~1023)

U2CropW AE/AWB crop width (0~1023)

U2CropH AE/AWB crop height (0~1023)
CropSize Description
Note

The image width and height already correspond to 1023, so the crop information is based on 1023. FW will automatically take the current actual image size and perform the mapping action. Therefore, user can simply set the relative position without the need to know the current image size.

Assume CropX = 10, CropY = 10, CropW = 300, and CropH = 200:

If the actual image width and height = 1920x1080, the CropX, CropY, CropW and CropH corresponding to 1920x1080 will be as follows:

CropX = 1920*10/1023 = 18

CropY = 1080*10/1023 = 10

CropW = 1920*300/1023 = 563

CropH = 1080*200/1023 = 211

20. Setting AWB Sampling Using MI_ISP¶

20.1. MI_ISP_CUS3A_SetAwbSampling¶

Purpose

Set AWB sampling size

Syntax

MI_RET MI_ISP_CUS3A_SetAwbSampling(U32 nChannel, CusAWBSample_t *data)

Description

Call this interface to set AWB sampling size
Member

Parameter Name Description

nChannel Video input channel number (normally 0)

SizeX X-axis sampling size

SizeY Y-axis sampling size

IncRatio Multiply the statistics by a ratio
Return Value

Return Value Description

MI_RET_SUCCESS Successful

MI_RET_FAIL Failed
Requirement

header file: mi_isp_twinkie.h / mi_isp_pretzel.h / mi_isp.h (for Macaron, Pudding)

.so: libmi_isp_twinkie.so / libmi_isp_pretzel.so / libmi_isp.so (for Macaron, Pudding)
Note

An image is divided into 128*90 blocks, where SizeX * SizeY pixels are taken from each block as the AWB sampling size.

SizeX: SizeX >=4 && SizeX \<= Block_size_X

SizeY: SizeY >=2 && SizeY \<= Block_size_Y

When AE brightness is very low, you can multiply the statistics by a ratio(IncRatio) to avoid statistical values of 0.

21. Setting AF Filter Using MI_ISP¶

21.1. MI_ISP_CUS3A_SetAFFilter¶

Purpose

Set AF Filter parameter

Syntax

MI_RET MI_ISP_CUS3A_SetAFFilter (MI_U32 Channel, CusAFFilter_t *data);

Description

Call this interface to set AF Filter parameter
Parameter

Parameter Name Description

nChannel Video input channel number (normally 0)

data AF Filter parameter
Return Value

Return Value Description

MI_RET_SUCCESS Successful

MI_RET_FAIL Failed

21.2. CusAFFilter_t¶

Description

ISP AF hardware statistics

Definition

typedef struct
{
    MI_U16 u16IIR1_a0;
    MI_U16 u16IIR1_a1;
    MI_U16 u16IIR1_a2;
    MI_U16 u16IIR1_b1;
    MI_U16 u16IIR1_b2;
    MI_U16 u16IIR1_1st_low_clip;
    MI_U16 u16IIR1_1st_high_clip;
    MI_U16 u16IIR1_2nd_low_clip;
    MI_U16 u16IIR1_2nd_high_clip;
    MI_U16 u16IIR2_a0;
    MI_U16 u16IIR2_a1;
    MI_U16 u16IIR2_a2;
    MI_U16 u16IIR2_b1;
    MI_U16 u16IIR2_b2;
    MI_U16 u16IIR2_1st_low_clip;
    MI_U16 u16IIR2_1st_high_clip;
    MI_U16 u16IIR2_2nd_low_clip;
    MI_U16 u16IIR2_2nd_high_clip;

    //for Pudding only
    MI_U16 u16IIR1_e1_en;
    MI_U16 u16IIR1_e1_a0;
    MI_U16 u16IIR1_e1_a1;
    MI_U16 u16IIR1_e1_a2;
    MI_U16 u16IIR1_e1_b1;
    MI_U16 u16IIR1_e1_b2;
    MI_U16 u16IIR1_e2_en;
    MI_U16 u16IIR1_e2_a0;
    MI_U16 u16IIR1_e2_a1;
    MI_U16 u16IIR1_e2_a2;
    MI_U16 u16IIR1_e2_b1;
    MI_U16 u16IIR1_e2_b2;

    MI_U16 u16IIR2_e1_en;
    MI_U16 u16IIR2_e1_a0;
    MI_U16 u16IIR2_e1_a1;
    MI_U16 u16IIR2_e1_a2;
    MI_U16 u16IIR2_e1_b1;
    MI_U16 u16IIR2_e1_b2;
    MI_U16 u16IIR2_e2_en;
    MI_U16 u16IIR2_e2_a0;
    MI_U16 u16IIR2_e2_a1;
    MI_U16 u16IIR2_e2_a2;
    MI_U16 u16IIR2_e2_b1;
    MI_U16 u16IIR2_e2_b2;
} CusAFFilter_t;

Member

The IIR parameters are as listed below:

Name	Bit Representation	Description	IIR1 Default	IIR2 Default
a0	S+9	a0 multiplier	63	63
a1	S+10	a1 multiplier	-126	-126
a2	S+9	a2 multiplier	63	63
b1	S+7	b1 multiplier	65	-109
b2	S+7	b2 multiplier	55	48
1^st_low_clip	10	X(n) input low clip	0	0
1^st_high_clip	10	X(n) input high clip	320	320
2^nd_low_clip	10	Y(n) output low clip	0	0
2^nd_high_clip	10	Y(n) output high clip	1023	1023

** By default, IIR1 is IIR High, and IIR2, IIR Low.

The following parameters are for Pudding only:

Name	Bit Representation	Description	IIR1 Default	IIR2 Default
a0	S+9	a0 multiplier	37	15
a1	S+10	a1 multiplier	0	0
a2	S+9	a2 multiplier	-37	-15
b1	S+7	b1 multiplier	83	-79
b2	S+7	b2 multiplier	40	44
1^st_low_clip	10	X(n) input low clip	0	0
1^st_high_clip	10	X(n) input high clip	1023	1023
2^nd_low_clip	10	Y(n) output low clip	0	0
2^nd_high_clip	10	Y(n) output high clip	1023	1023
e1_en	1	Extra1 enable	1	1
e1_a0	S+9	a0 multiplier	37	15
e1_a1	S+10	a1 multiplier	0	0
e1_a2	S+9	a2 multiplier	-37	-15
e1_b1	S+7	b1 multiplier	-54	-115
e1_b2	S+7	b2 multiplier	34	55
e2_en	1	Extra2 enable	1	1
e2_a0	S+9	a0 multiplier	32	14
e2_a1	S+10	a1 multiplier	0	0
e2_a2	S+9	a2 multiplier	-32	-14
e2_b1	S+7	b1 multiplier	14	-91
e2_b2	S+7	b2 multiplier	0	37

** By default, IIR1 is IIR High, and IIR2, IIR Low.

The architecture of IIR coefficients is shown in the diagram below:

Requirement

header file: mi_isp_twinkie.h / mi_isp_pretzel.h / mi_isp.h (for Macaron, Pudding)

.so: libmi_isp_twinkie.so / libmi_isp_pretzel.so / libmi_isp.so (for Macaron, Pudding)

22. Setting AF Filter Square Using MI_ISP¶

22.1. MI_ISP_CUS3A_SetAFFilterSq¶

Purpose

Set AF Filter Square parameter

Syntax

MI_RET MI_ISP_CUS3A_SetAFFilterSq (MI_U32 Channel, CusAFFilterSq_t *data);

Description

Call this interface to set AF Filter Square parameter
Parameter

Parameter Name Description

nChannel Video input channel number (normally 0)

data AF Filter Square parameter
Return Value

Return Value Description

MI_RET_SUCCESS Successful

MI_RET_FAIL Failed

22.2. CusAFFilterSq_t¶

Description

ISP AF Square hardware statistics

Definition

typedef struct
{
    MI_BOOL bFIRYSatEn;
    MI_U16 u16FIRYThd;

    MI_BOOL bIIRSquareAccEn;
    MI_BOOL bFIRSquareAccEn;

    MI_U16 u16IIR1Thd;
    MI_U16 u16IIR2Thd;
    MI_U16 u16FIRHThd;
    MI_U16 u16FIRVThd;
    MI_U8 u8AFTblX[12];
    MI_U16 u16AFTblY[13];
} CusAFFilter_t;

Member

Name	Description
bFIRYSatEn	This switch involves two actions: FIR Filter Y threshold control Y_sat statistics setting control
u16FIRYThd	If bFIRYSatEn = 1, FIR Filter Y threshold control: when the pixel brightness is less than u16FIRYThd, it will be included in the FIR Filter calculation. Y_sat statistics setting control: the number of pixels larger than u16FIRYThd will be returned and shown in Y_sat statistics. The value range is 0 ~ 1023.
bIIRSquareAccEn	IIR Filter enhancement control
bFIRSquareAccEn	FIR Filter enhancement control
u16IIR1Thd	IIR Filter Output = IIR Filter Output – IIRThd. The value range is 0 ~ 1023.
u16IIR2Thd	IIR Filter Output = IIR Filter Output – IIRThd. The value range is 0 ~ 1023.
u16FIRHThd	FIR Filter Output = FIR Filter Output – FIR Thd. The value range is 0 ~ 1023.
u16FIRVThd	FIR Filter Output = FIR Filter Output – FIR Thd. The value range is 0 ~ 1023.
u8AFTblX[12]	Non-linear mapping between IIR and FIR Filters. u8AFTblX is the horizontal axis, and the node is the accumulation of the power of two, which must cumulatively be greater than 1024. The value range is 0 ~ 15.
u16AFTblY[13]	Non-linear mapping between IIR and FIR Filters. u8AFTblY is the horizontal axis. The value range is 0 ~ 8191.

SquareACC can be used against coarse edge with middle to high contrast, for enhancement. Values lower than Thd will be taken as 0, to deal with noise under low luminance.

Reference setting (with enhanced statistics)

u8AFTblX = 6,7,7,6,6,6,7,6,6,7,6,6

u16AFTblY = 0,1,53,249,431,685,1023,1999,2661,3455,5487,6749,8191

Reference setting (with original statistics)

u8AFTblX = 6,7,7,6,6,6,7,6,6,7,6,6

u16AFTblY = 0,64,192,320,384,448,512,640,704,768,896,960,1023

Requirement

header file: mi_isp_twinkie.h / mi_isp_pretzel.h / mi_isp.h (for Macaron, Pudding)

.so: libmi_isp_twinkie.so / libmi_isp_pretzel.so / libmi_isp.so (for Macaron, Pudding)
Note

Twinkie does not support this API.

23. Setting AF ROI Mode Using MI_ISP¶

23.1. MI_ISP_CUS3A_SetAFRoiMode¶

Purpose

Set AF statistical mode

Syntax

MI_RET MI_ISP_CUS3A_SetAFRoiMode (U32 nChannel, CusAFRoiMode_t *data);

Description

Call this interface to set source of AF statistics
Parameter

Parameter Name Description

nChannel Video input channel number (normally 0)

data AF statistical mode
Return Value

Return Value Description

MI_RET_SUCCESS Successful

MI_RET_FAIL Failed

23.2. CusAFRoiMode_t¶

Description

Set AF statistical mode

Definition

typedef enum __attribute__ ((aligned (1)))
{
    AF_ROI_MODE_NORMAL,
    AF_ROI_MODE_MATRIX
} ISP_AF_ROI_MODE_e;

typedef struct
{
    ISP_AF_ROI_MODE_e mode;
    MI_U32 u32_vertical_block_number;
} CusAFRoiMode_t;

Member

Name	Description
mode	AF_ROI_MODE_NORMAL: Can be divided into 16 sets of ROI blocks. The window size and position can be set according to user preference. This is the default mode. AF_ROI_MODE_MATRIX: Can be divided into 16 * N sets of ROI windows. The window size and position are relatively more limited, but more blocks can be used (N= u32_vertical_block_number).
u32_vertical_block_number	If mode = AF_ROI_MODE_MATRIX, can divide the window into 16 * N sets of ROI blocks (N= u32_vertical_block_number, 1~16).

AF_ROI_MODE_NORMAL: AF ROI division method

AF_ROI_MODE_MATRIX: AF ROI division method

There are 16 fixed ROI blocks in the vertical axis direction.

As for the horizontal axis direction, the number (16*N) of ROI blocks will be decided by u32_vertical_block_number. In the above diagram, u32_vertical_block_number=16. That means the number of ROI blocks is 16*16.

Requirement

header file: mi_isp_twinkie.h / mi_isp_pretzel.h / mi_isp.h (for Macaron, Pudding)

.so: libmi_isp_twinkie.so / libmi_isp_pretzel.so / libmi_isp.so (for Macaron, Pudding)
Note

Twinkie does not support this API.

24. Setting AF Window Using MI_ISP¶

24.1. MI_ISP_CUS3A_SetAFWindow¶

Purpose

Set AF Window

Syntax

MI_RET MI_ISP_CUS3A_SetAFWindow (U32 nChannel, CusAFWin_t *data);

Description

AF window is set during AF_Init stage. CMOS sensor image is divided into 1024*1024 grids for AF window setting.

The coordinate and actual crop range conversion formula is:

Image width * X /1024 = actual X coordinate

Image_height * Y/1024 = actual Y coordinate

For example,

Suppose AF window is set as:

Start_X = 458

End_X = 566

Start_Y = 418

End_Y = 606

Then the AF window coordinates for 1920x1080 CMOS sensor case are:

Real_Start_X = 1920 *458/1024 = 858

Real_End_X = 1920 *566/1024 = 1061

Real_Start_Y = 1080 *418/1024 = 440

Real_End_Y = 1080 *606/1024 = 639
Parameter

Parameter Name Description

nChannel Video input channel number (normally 0)

data AF Window setting
Return Value

Return Value Description

MI_RET_SUCCESS Successful

MI_RET_FAIL Failed

24.2. CusAFWin_t¶

Syntax

typedef struct AF_WINDOW_PARAM_s { MI_U32 u32StartX; /range : 0~1023/ MI_U32 u32StartY; /range : 0~1023/ MI_U32 u32EndX; /range : 0~1023/ MI_U32 u32EndY; /range : 0~1023/ } AF_WINDOW_PARAM_t;

typedef struct { MI_U8 u8WindowIndex; AF_WINDOW_PARAM_t stParaAPI; } CusAFWin_t;
Parameter

Parameter Name Description

u8WindowIndex AF ROI Index (0~15), for NORMAL & MARIX mode

stParaAPI AF ROI position
Description

When AF ROI Mode = AF_ROI_MODE_NORMAL, StrX, StrY, EndX, and EndY represent the position of the Window Index.

When AF ROI Mode = AF_ROI_MODE_MATRIX, StrX and EndX represent the crop position of horizontal-axis window, and StrY and EndY the crop position of vertical-axis window.

Limitation:

If AF ROI Mode = AF_ROI_MODE_NORMAL,
- The respective window width and height can overlap.
- h/v end > h/v start
If AF ROI Mode = AF_ROI_MODE_MATRIX,
- The respective window width can overlap, but not in the case of window height.
- h/v end > h/v start
- win0_v_start \< win0_v_end \< win1_v_start \< win1_v_end \< win2…
- win(0,0), win(1,0), …, win(15,0) share same h_start, h_end, and setting by af0_h_start, h_end
- win(0,1), win(1,1), …, win(15,1) share same h_start, h_end, and setting by af1_h_start, h_end, and so on for other windows
- win(0,0), win(0,1), …, win(0,15) share same v_start, v_end, and setting by af0_v_start, v_end
- win(1,0), win(1,1), …, win(1,15) share same v_start, v_end, and setting by af1_v_start, v_end, and so on for other windows
When setting to Matrix mode, you need to finish setting AF Window before switching to Matrix mode.
Requirement

header file: mi_isp_twinkie.h / mi_isp_pretzel.h / mi_isp.h (for Macaron, Pudding)

.so: libmi_isp_twinkie.so / libmi_isp_pretzel.so / libmi_isp.so (for Macaron, Pudding)

25. Setting AF YParam Using MI_ISP¶

25.1. MI_ISP_CUS3A_SetAFYParam¶

Purpose

Set AF Y Param

Syntax

MI_RET MI_ISP_CUS3A_SetAFYParam (U32 nChannel, CusAFYParam_t *data);

Description

When AF calculates statistical values, the algorithm will convert Bayer data to Y data. This function provides control over the respective conversion ratio of R/G/B.

Default ratios for the conversion of BayerRGB to Y are (76, 75, 29), respectively. When the value is set to 128, the ratio would be 1x (i.e. remains unchanged). This function will be useful in backlight scenario or under darker environment, where less brightness results in lower AF statistics, which, in turn, impede the determination of peak value. In such cases, you might raise the RGB ratios to — for example — (255, 252, 97), thereby enhancing the Y signal and making the peak value more obvious. The values cannot be set greater than 255.
Parameter

Parameter Name Description

nChannel Video input channel number (normally 0)

data AF Window setting
Return Value

Return Value Description

MI_RET_SUCCESS Successful

MI_RET_FAIL Failed

25.2. CusAFYParam_t¶

Syntax

typedef struct { MI_U8 r; //0~255 MI_U8 g; //0~255 MI_U8 b; //0~255 } CusAFYParam_t;

Parameter

Parameter Name	Description
r	Bayer to Y conversion ratio of r channel. Range: 0 ~ 255.
g	Bayer to Y conversion ratio of g channel. Range: 0 ~ 255.
b	Bayer to Y conversion ratio of b channel. Range: 0 ~ 255.

Requirement

header file: mi_isp_twinkie.h / mi_isp_pretzel.h / mi_isp.h (for Macaron, Pudding)

.so: libmi_isp_twinkie.so / libmi_isp_pretzel.so / libmi_isp.so (for Macaron, Pudding)

26. Setting AF Source Using MI_ISP¶

26.1. MI_ISP_CUS3A_SetAFSource¶

Purpose

Select the source of AF statistical value.

Syntax

MI_RET MI_ISP_CUS3A_SetAFSource (U32 nChannel, CusAFSource_e *data);

Description

Select the source of AF statistical value.
Parameter

Parameter Name Description

nChannel Video input channel number (normally 0)

data AF Window setting
Return Value

Return Value Description

MI_RET_SUCCESS Successful

MI_RET_FAIL Failed

26.2. CusAFSource_e¶

Syntax

typedef enum { AF_SOURCE_FROM_SE_OBC_BF_HDR = 2, AF_SOURCE_FROM_SE_WBG_BF_HDR = 3, AF_SOURCE_FROM_ALSC_AF_HDR = 4, AF_SOURCE_FROM_WBG_AF_HDR = 5, AF_SOURCE_FROM_LE_OBC_BF_HDR = 6, AF_SOURCE_FROM_LE_WBG_BF_HDR = 7, } CusAFSource_e;

Parameter

HDR Mode

Parameter Name	Description
AF_SOURCE_FROM_SE_OBC_BF_HDR	Selects short exposure result before HDR (after optical black correction, or OBC).
AF_SOURCE_FROM_SE_WBG_BF_HDR	Selects short exposure result before HDR (after white balance gain, or WBGain).
AF_SOURCE_FROM_ALSC_AF_HDR	Selects post-HDR result after Shading.
AF_SOURCE_FROM_WBG_AF_HDR	Selects post-HDR result after WBGain.
AF_SOURCE_FROM_LE_OBC_BF_HDR	Selects long exposure result before HDR (after OBC).
AF_SOURCE_FROM_LE_WBG_BF_HDR	Selects long exposure result before HDR (after WBGain).

Linear Mode

Parameter Name	Description
AF_SOURCE_FROM_SE_OBC_BF_HDR	Selects post-OBC result.
AF_SOURCE_FROM_SE_WBG_BF_HDR	Selects post-WBGain result
AF_SOURCE_FROM_ALSC_AF_HDR	Selects the result before color interpolation (CI) and after Shading.
AF_SOURCE_FROM_WBG_AF_HDR	Selects the result before CI and after WBGain.
AF_SOURCE_FROM_LE_OBC_BF_HDR	Not supported.
AF_SOURCE_FROM_LE_WBG_BF_HDR	Not supported.

Requirement

header file: mi_isp_twinkie.h / mi_isp_pretzel.h / mi_isp.h (for Pudding)

.so: libmi_isp_twinkie.so / libmi_isp_pretzel.so / libmi_isp.so (for Pudding)
Note

Macaron is not supported.

27. Getting CUST3A VERSION¶

Please refer to isp_cus3a_if.h. The version definition is as follows:

#define CUS3A_VER_STR "CUS3A_V1.1"
#define CUS3A_VER_MAJOR 1
#define CUS3A_VER_MINOR 1

unsigned int CUS3A_GetVersion(char* pVerStr);

To get the version number, you can:

Check the header file, or
Check CUS3A_GetVersion.

28. Twinkie/Pretzel/Macaron/Pudding Difference Table¶

The following table lists the difference in functions supported among Twinkie, Pretzel, Macaron and Pudding:

Parameter Name	Twinkie	Pretzel	Macaron	Pudding
AE Statistics	128*90	128*90	32*32	32*32
AE Hist2	V	V	X	X
AF Statistics	IIR(34), FIR(34), Luma(34), YSat(24)	IIR(35), FIR(35), Luma(32), YSat(22)	IIR(35), FIR(35), Luma(32), YSat(22)	IIR(35), FIR(35), Luma(32), YSat(22)
MI_ISP_CUS3A_SetAEWindowBlockNumber	V	V	X	X
MI_ISP_CUS3A_SetAEHistogramWindow	offset + size limit: Max. 128*90	offset + size limit: Max. 128*90	offset + size limit: Max. 32*32	offset + size limit: Max. 32*32
MI_ISP_CUS3A_SetAFFilter	V	V	V	V (with new extra parameter)
MI_ISP_CUS3A_SetAFFilterSq	X	V	V	V
MI_ISP_CUS3A_SetAFRoiMode	X	V	V	V
MI_ISP_CUS3A_SetAFSource	V	V	X	V

Parameter Name	Description	Input/Output
pdata	Algorithm private data pointer (Reserved)	Input
cmd	AF control command (Reserved)	Input
param	AF control command parameter (Reserved)	Input

Parameter Name	Description
nChannel	Video input channel number (normally 0)
data	Set the AE statistical value range

Parameter Name	Description
U2CropX	AE/AWB crop X start (0~1023)
U2CropY	AE/AWB crop Y start (0~1023)
U2CropW	AE/AWB crop width (0~1023)
U2CropH	AE/AWB crop height (0~1023)

Parameter Name	Description
nChannel	Video input channel number (normally 0)
eBlkNum	Block number, by which an image is divided into X*Y blocks

Parameter Name	Description
nChannel	Video input channel number (normally 0)
SizeX	X-axis sampling size
SizeY	Y-axis sampling size
IncRatio	Multiply the statistics by a ratio

Parameter Name	Description
u8WindowIndex	AF ROI Index (0~15), for NORMAL & MARIX mode
stParaAPI	AF ROI position