README.md
1# Video Processor Engine (VPE) High Level Documentation
2
3## Overview
4
5VPE is a hardware pipeline that does baseline video processing from DRAM to DRAM.
6
7The goal of the VPE is to reduce the dependency on graphics engine and therefore saving the graphic engine power. The main functionality of VPE is to process video DRAM in/out.
8
9It is capable of doing pixel re-gamma, scaling, CSC, gamut remap, tone mapping, programmable gamma. Future versions may include more support like blending, mirror and rotation, etc.
10
11There are three implementations of VPE:
12
131 - Single pipeline (VPE 1.0)
14
15
16
172 - Dual independent pipeline (VPE 1.1)
18
19
20
21The VPE hardware is controlled by the VPE driver encapsulated in the VPELIB library.
22
23## VPELIB API
24
25The VPE hardware pipeline is controlled by a dedicated firmware. This firmware expects a series of commands to execute. Commands can be present in Ring buffer, command buffer and embedded buffer.
26
27Ring Buffer at the current implementation stage is maintained by Kernel mode driver (KMD), and ring buffer commands are constructed by KMD. In the future, ring buffer could be owned and ring buffer command will be possible to be owned by the user mode driver.
28
29User Mode Driver (UMD)'s responsibility at the current stage is to construct the command buffer. With VPE engine, the command buffer itself will store a series of VPE descriptor commands, which essentially contain address pointers to other descriptors or other GPU memory (like 3DLUT). The other descriptors, like plane descriptor, config descriptors etc., are located in the embedded buffer.
30
31Once constructed, the vpelib's caller will submit the command buffer to the Engine's queue, KMD will construct the ring buffer command called "Indirect Buffer Command" to point to this command buffer.
32
33Once a Hardware queue is enabled, the VPE Firmware will fetch the commands from the ring buffer one by one and then eventually when it sees the "Indirect command" in the ring buffer, which points to the UMD-constructed command buffer, it will fetch the command buffer content to execute.
34
35The mechanism is presented below:
36
37
38
39The main functions of the VpeLib API are:
40
41### vpe_create
42
43```
44/** @brief Create the VPE lib instance.
45 *
46 * Caller provides the current asic info,
47 * logging and system memory APIs.
48 * It initializes all the necessary resources for the asic
49 * and returns the general capabilities of the engines.
50 *
51 * For capabilities based on conditions,
52 * shall be done by vpe->cap_funcs.*
53 *
54 *
55 * @param[in] params provide the asic version, APIs for logging and memory
56 * @return vpe instance if supported. NULL otherwise
57 */
58struct vpe *vpe_create(const struct vpe_init_data *params);
59```
60
61### vpe_destroy
62
63```
64/** @brief Destroy the VPE lib instance and resources
65 *
66 * @param[in] vpe the vpe instance created by vpe_create
67 */
68void vpe_destroy(struct vpe **vpe);
69```
70
71### vpe_check_support
72```
73/**
74* @brief Check if the VPE operation is supported.
75*
76* Caller must call this to check if the VPE supports
77* the requested operation before calling vpe_build_commands().
78* If operation is supported, it returns the memory requirement.
79*
80* The caller has to prepare those required memories
81* and pass them to the vpe_build_commands().
82*
83* @param[in] vpe vpe instance returned by vpe_initialize()
84* @param[in] param build params
85* @param[out] req memory required for the command buffer and
86 embedded data if return VPE_OK.
87 caller has to alloc them and provide it to build_vpbilts API.
88* @return VPE_OK if supported
89 */
90enum vpe_status vpe_check_support(struct vpe *vpe, const struct vpe_build_param *param, struct vpe_bufs_req *req);
91```
92
93### vpe_build_commands
94
95 ```
96/**
97 * @brief Build the command descriptors for the given param.
98 * caller must call vpe_check_support() before this function,
99 * unexpected result otherwise.
100 *
101 * @param[in] vpe vpe instance created by vpe_create()
102 * @param[in] param vpe build params
103 * @param[in,out] bufs [in] memory allocated for the command buffer, embedded buffer and 3dlut.
104 * If size is 0, it reports the required size for this checked
105 * operation. [out] the next write address and the filled sizes.
106 * @return status
107 */
108enum vpe_status vpe_build_commands(
109 struct vpe *vpe, const struct vpe_build_param *param, struct vpe_build_bufs *bufs)
110```
111
112This function can be used to query the size requirement of command buffer and embedded buffer by
113setting the buffer size to 0.
114
115With the proper buffer size, this API will start to construct the command buffer and embedded buffer content.
116
117### vpe_build_noops
118
119```
120/**
121 * @brief Build the command descriptors for No-Op operation
122 * @param[in] vpe vpe instance created by vpe_create()
123 * @param[in] num_dwords number of noops
124 * @param[in,out] ppcmd_space in: dword aligned command buffer start address
125 * out: dword aligned next good write address
126 * @return status
127 */
128enum vpe_status vpe_build_noops(struct vpe *vpe, uint32_t num_dwords, uint32_t **ppcmd_space);
129```
130
131
132# VPE Tutorial - bypass RGB TO RGB
133
134In order to process an input image bypass to output, one has to do the followings:
135
136- establish the input image array
137- set the source image dimensions
138- set the color space of the input image (RGB)
139- create an input texture with the above parameters
140- set the source destination dimensions , same like the input size
141- set the destination color format (RGB, same like input)
142- Set the linearity of the output the same like the input
143- Add alpha channel if that is the case
144- trigger the VPBlit which will use the VPE hardware
145
146In order to accomplish the aboves, the caller needs to create an instance of VPE first.
147
148```
149struct vpe *vpe_create(const struct vpe_init_data *params);
150```
151
152Then check if the created VPE instance supports the given parametrs.
153
154```
155enum vpe_status vpe_check_support(
156 struct vpe *vpe, const struct vpe_build_param *param, struct vpe_bufs_req *req)
157```
158
159The input is of type vpe_build_params which states the parameters for input and output.
160
161```
162struct vpe_build_param {
163 uint32_t num_streams; /**< Number of source streams */
164 struct vpe_stream *streams; /**< List of input streams */
165 struct vpe_surface_info dst_surface; /**< Destination/Output surface */
166 struct vpe_rect target_rect; /**< rectangle in target surface to be blt'd.
167 Ranges out of target_rect won't be touched */
168 struct vpe_color bg_color; /**< Background Color */
169 enum vpe_alpha_mode alpha_mode; /**< Alpha Mode. Output alpha in the output
170 surface */
171 struct vpe_hdr_metadata hdr_metadata; /**< HDR Metadata */
172 struct vpe_reserved_param dst_reserved_param; // data flags
173 struct {
174 uint32_t hdr_metadata : 1;
175 uint32_t reserved : 31;
176 } flags;
177```
178
179For this example, set num_streams to a one and provide a single stream for the streams parameter.
180
181The destination surface must be populated with these parameters. The most important are the destination address, plane_size, color space and format.
182
183For this example, destination format and color space should be the same as the input format and color space.
184
185```
186struct vpe_surface_info {
187 struct vpe_plane_address address; /**< Address */
188 enum vpe_swizzle_mode_values swizzle; /**< Swizzle mode */
189 struct vpe_plane_size plane_size; /**< Pitch */
190 struct vpe_plane_dcc_param dcc;
191 enum vpe_surface_pixel_format format; /**< Surface pixel format */
192 struct vpe_color_space cs; /**< Surface color space */
193};
194```
195
196Plane size should be populated accordingly. As this test is RGB to RGB, ignore the YUV fields.
197
198```
199struct vpe_plane_size {
200 struct vpe_rect surface_size; /**< Plane rectangle */
201 struct vpe_rect chroma_size; /**< Chroma plane rectangle for semi-planar YUV formats */
202 uint32_t surface_pitch; /**< Horizintal pitch alignment of the plane in pixels */
203 uint32_t chroma_pitch; /**< Horizintal pitch alignment of the chroma plane for
204 semi-planar YUV formats in pixels */
205 uint32_t surface_aligned_height; /**< Vertical alignment of the plane in pixels */
206 uint32_t chrome_aligned_height; /**< Vertical alignment of the chroma plane for semi-planar
207 YUV formats in pixels */
208};
209```
210
211For this example, `struct vpe_rect target_rect` should be identical with the input/output size.
212
213After calling the support call, the caller needs to call
214``` enum vpe_status vpe_build_commands(
215 struct vpe *vpe, const struct vpe_build_param *param, struct vpe_build_bufs *bufs);
216```
217with the same parameters provided to the support call to build the command descriptor.
218
219After the job is done and no more jobs for the VPE, call `vpe_destroy(struct vpe **vpe)` to free the allocated memory to the vpe instance.
220# Generate API Doxygen Documentation
221
222By Compiling and building VPE with MSVS, the Doxygen documentation of the public API will automatically get generated in VPE_API_manual directory.
223
224To manually generate the Doxygen documentation without building VPE, create a Doxygen config file similar to the current config file stored as [VPEDoxy.in](./docs/VPEDoxy.in) so that the INPUT field points to the [inc](./inc/) directory of VPE, and OUTPUT_DIRECTORY points to a directory to store the generated output. Then run: ` doxygen.exe <path_to_config_file>`
225
