SDK Architecture Introduction

1. Hardware Diagram¶

2. Keywords¶

3. Basic Concepts¶

• Data Stream

  Each mi module can be regarded as a data processing unit, and the data flow is uniformly scheduled by MI SYS. The input data stream represents the data received by the unit, and the output data stream represents the data processed by the unit.

• Control Stream

  APP controls the parameters of each mi module data processing process, such as setting MI_VENC encoding parameters, starting and stopping MI_VPE channel, setting MI_VPE channel output port resolution and format, etc.

• Data Stream

  1. A continuous data stream that an instance of the mi module needs to process or output, such as the original video frame sequence output by VIF, the raw ES data sequence input to VDEC, and the decoded frame sequence output by VDEC.

  2. There is no context correlation between graphics frames that need to be processed by MI_GFX.

• Channel

  1. For an mi module that needs to process or output a stream, a channel represents the time-multiplexed context and related control flow settings that it processes or outputs a stream.

  2. The time-multiplexed module can support multiple channels, such as MI_VDEC, MI_VPE, MI_DIVP, MI_VENC, MI_VDF, virtual DISP.

  3. Modules that are not time-multiplexed can only support one channel, such as MI_DISP (0/1).

  4. Except for VIF, it can be considered to have 32 channels.

• Port

  1. The input port is the location of the channel's input data stream, and the output port is the location of the channel's output data stream.

  2. A channel can have multiple input ports and output ports. For example, one MI_VENC channel has one input port and one output port, one MI_VPE channel has one input port and four output ports, and one MI_DISP (0/1) has 16 input port but no output port

  

• MI module classification

  	Data stream that needed to be handle	Which has no data stream concept
  Algorithm module (exists in user mode)	MI_VDF, MI_IAE	MI_IVE，MI_MD,MI_OD
  Driver module (exists in kernel mode)	MI_VIF, MI_VDEC, MI_VPE, MI_DIVP,MI_VENC, MI_DISP, MI_AI, MI_AO	MI_GFX, MI_IVE

  • The MI module without the concept of data stream is relatively simple to process the data model. The input and output data are both the user module directly calling its private interface to input data and read back. It does not have a unified model.

  • For modules with the concept of data stream, the data input and output are all dispatched through the MI_SYS.

• Concept of MI module channel/port

  MI module	Channel number and concept	Input port number and concept	Output port number and concept
  MI_VIF	16 channels, each channel represents a BT656/1120 input image signal	None	1 or 2 output ports, one original size output and one H/V half scaled output, the ipc chip can only support one output port
  MI_VDEC	16 channels (TBD), each channel represents one channel of video decoding case	1 input port, decode the input RAW ES data stream	1 output port, decode the input frame sequence
  MI_VPE	2 channels (TBD), each channel represents a VPE video processing case	1 input port, input processing image frame sequence	3~4 output ports, after VPE processing, output frame sequences of different resolutions, the output size, format and frame rate of each port are independently controlled
  MI_DIVP	6 channels (TBD), each channel represents a DIVP video processing case	1 input port, input processed image frame sequence	1 output port, DIVP image enhancement processing and output frame sequence after scaling
  MI_VENC	16 channels, each channel represents a video encoding case	1 input port, input the video frame sequence to be encoded	1 output port, compressed ES RAW data data stream
  MI_DISP(0/1)	1 channel, representing a physical output port (HDMI, VGA, etc)	16 input ports, 16-channel video frame sequence that can be displayed at once time	None
  MI_DISP(virtual DISP)	Max 4 channels, each channel represents a SW splicing image case	16 input ports, 16-channel video frame input sequence of the image to be spliced	1 output port, output 1 video frame sequence after spliced (need cpu resources)
  MI_AI	16 channel, 16 independent audio inputs	None	1 output port, output audio PCM stream
  MI_AO	1 channel	1 input port, output audio PCM stream	None
  MI_VDF	16 channels (TBD), each channel represents a case for intelligent identification of video streams	1 input port, input video stream	1 output port, recognition result
  MI_IAE	16 channels (TBD), each channel represents an audio intelligent identification case	1 input port, input audio stream	1 output port, recognition result

4. MI_SYS¶

The following describes the responsibilities of MI_SYS.

• Manage the channel, input port, and output port of each MI module

  • Each MI module registers the number of channel/input port/output port with the MI_SYS module when it is started, and implements the related callbacks in mi_sys_DevPassOpsInfo_t

    

• Provide different MI Module input port and output port bind functions

• BIND can be called the SW PINPON function, that is, the data output by the serially connected output port, MI_SYS is automatically pushed to the corresponding input port

  

• MI_SYS supports the user APP to push data to the input port of the channel

  

• MI_SYS supports users to directly obtain the output data of the channel output port

  

• MI_SYS supports automatic calculation of the output frequency of the channel output port for sw FRC

  

• MI_SYS provides utility function to connect the data stream between kernel mode and user mode

  

5. Typical Application Scenario¶

5.1. DVR¶

5.2. NVR¶

5.3. IPC¶