1<section id="selection-api"> 2 3 <title>Experimental API for cropping, composing and scaling</title> 4 5 <note> 6 <title>Experimental</title> 7 8 <para>This is an <link linkend="experimental">experimental</link> 9interface and may change in the future.</para> 10 </note> 11 12 <section> 13 <title>Introduction</title> 14 15<para>Some video capture devices can sample a subsection of a picture and 16shrink or enlarge it to an image of arbitrary size. Next, the devices can 17insert the image into larger one. Some video output devices can crop part of an 18input image, scale it up or down and insert it at an arbitrary scan line and 19horizontal offset into a video signal. We call these abilities cropping, 20scaling and composing.</para> 21 22<para>On a video <emphasis>capture</emphasis> device the source is a video 23signal, and the cropping target determine the area actually sampled. The sink 24is an image stored in a memory buffer. The composing area specifies which part 25of the buffer is actually written to by the hardware. </para> 26 27<para>On a video <emphasis>output</emphasis> device the source is an image in a 28memory buffer, and the cropping target is a part of an image to be shown on a 29display. The sink is the display or the graphics screen. The application may 30select the part of display where the image should be displayed. The size and 31position of such a window is controlled by the compose target.</para> 32 33<para>Rectangles for all cropping and composing targets are defined even if the 34device does supports neither cropping nor composing. Their size and position 35will be fixed in such a case. If the device does not support scaling then the 36cropping and composing rectangles have the same size.</para> 37 38 </section> 39 40 <section> 41 <title>Selection targets</title> 42 43 <para> 44 <figure id="sel-targets-capture"> 45 <title>Cropping and composing targets</title> 46 <mediaobject> 47 <imageobject> 48 <imagedata fileref="selection.png" format="PNG" /> 49 </imageobject> 50 <textobject> 51 <phrase>Targets used by a cropping, composing and scaling 52 process</phrase> 53 </textobject> 54 </mediaobject> 55 </figure> 56 </para> 57 58 <para>See <xref linkend="v4l2-selection-targets" /> for more 59 information.</para> 60 </section> 61 62 <section> 63 64 <title>Configuration</title> 65 66<para>Applications can use the <link linkend="vidioc-g-selection">selection 67API</link> to select an area in a video signal or a buffer, and to query for 68default settings and hardware limits.</para> 69 70<para>Video hardware can have various cropping, composing and scaling 71limitations. It may only scale up or down, support only discrete scaling 72factors, or have different scaling abilities in the horizontal and vertical 73directions. Also it may not support scaling at all. At the same time the 74cropping/composing rectangles may have to be aligned, and both the source and 75the sink may have arbitrary upper and lower size limits. Therefore, as usual, 76drivers are expected to adjust the requested parameters and return the actual 77values selected. An application can control the rounding behaviour using <link 78linkend="v4l2-selection-flags"> constraint flags </link>.</para> 79 80 <section> 81 82 <title>Configuration of video capture</title> 83 84<para>See figure <xref linkend="sel-targets-capture" /> for examples of the 85selection targets available for a video capture device. It is recommended to 86configure the cropping targets before to the composing targets.</para> 87 88<para>The range of coordinates of the top left corner, width and height of 89areas that can be sampled is given by the <constant> V4L2_SEL_TGT_CROP_BOUNDS 90</constant> target. It is recommended for the driver developers to put the 91top/left corner at position <constant> (0,0) </constant>. The rectangle's 92coordinates are expressed in pixels.</para> 93 94<para>The top left corner, width and height of the source rectangle, that is 95the area actually sampled, is given by the <constant> V4L2_SEL_TGT_CROP 96</constant> target. It uses the same coordinate system as <constant> 97V4L2_SEL_TGT_CROP_BOUNDS </constant>. The active cropping area must lie 98completely inside the capture boundaries. The driver may further adjust the 99requested size and/or position according to hardware limitations.</para> 100 101<para>Each capture device has a default source rectangle, given by the 102<constant> V4L2_SEL_TGT_CROP_DEFAULT </constant> target. This rectangle shall 103over what the driver writer considers the complete picture. Drivers shall set 104the active crop rectangle to the default when the driver is first loaded, but 105not later.</para> 106 107<para>The composing targets refer to a memory buffer. The limits of composing 108coordinates are obtained using <constant> V4L2_SEL_TGT_COMPOSE_BOUNDS 109</constant>. All coordinates are expressed in pixels. The rectangle's top/left 110corner must be located at position <constant> (0,0) </constant>. The width and 111height are equal to the image size set by <constant> VIDIOC_S_FMT </constant>. 112</para> 113 114<para>The part of a buffer into which the image is inserted by the hardware is 115controlled by the <constant> V4L2_SEL_TGT_COMPOSE </constant> target. 116The rectangle's coordinates are also expressed in the same coordinate system as 117the bounds rectangle. The composing rectangle must lie completely inside bounds 118rectangle. The driver must adjust the composing rectangle to fit to the 119bounding limits. Moreover, the driver can perform other adjustments according 120to hardware limitations. The application can control rounding behaviour using 121<link linkend="v4l2-selection-flags"> constraint flags </link>.</para> 122 123<para>For capture devices the default composing rectangle is queried using 124<constant> V4L2_SEL_TGT_COMPOSE_DEFAULT </constant>. It is usually equal to the 125bounding rectangle.</para> 126 127<para>The part of a buffer that is modified by the hardware is given by 128<constant> V4L2_SEL_TGT_COMPOSE_PADDED </constant>. It contains all pixels 129defined using <constant> V4L2_SEL_TGT_COMPOSE </constant> plus all 130padding data modified by hardware during insertion process. All pixels outside 131this rectangle <emphasis>must not</emphasis> be changed by the hardware. The 132content of pixels that lie inside the padded area but outside active area is 133undefined. The application can use the padded and active rectangles to detect 134where the rubbish pixels are located and remove them if needed.</para> 135 136 </section> 137 138 <section> 139 140 <title>Configuration of video output</title> 141 142<para>For output devices targets and ioctls are used similarly to the video 143capture case. The <emphasis> composing </emphasis> rectangle refers to the 144insertion of an image into a video signal. The cropping rectangles refer to a 145memory buffer. It is recommended to configure the composing targets before to 146the cropping targets.</para> 147 148<para>The cropping targets refer to the memory buffer that contains an image to 149be inserted into a video signal or graphical screen. The limits of cropping 150coordinates are obtained using <constant> V4L2_SEL_TGT_CROP_BOUNDS </constant>. 151All coordinates are expressed in pixels. The top/left corner is always point 152<constant> (0,0) </constant>. The width and height is equal to the image size 153specified using <constant> VIDIOC_S_FMT </constant> ioctl.</para> 154 155<para>The top left corner, width and height of the source rectangle, that is 156the area from which image date are processed by the hardware, is given by the 157<constant> V4L2_SEL_TGT_CROP </constant>. Its coordinates are expressed 158in in the same coordinate system as the bounds rectangle. The active cropping 159area must lie completely inside the crop boundaries and the driver may further 160adjust the requested size and/or position according to hardware 161limitations.</para> 162 163<para>For output devices the default cropping rectangle is queried using 164<constant> V4L2_SEL_TGT_CROP_DEFAULT </constant>. It is usually equal to the 165bounding rectangle.</para> 166 167<para>The part of a video signal or graphics display where the image is 168inserted by the hardware is controlled by <constant> 169V4L2_SEL_TGT_COMPOSE </constant> target. The rectangle's coordinates 170are expressed in pixels. The composing rectangle must lie completely inside the 171bounds rectangle. The driver must adjust the area to fit to the bounding 172limits. Moreover, the driver can perform other adjustments according to 173hardware limitations. </para> 174 175<para>The device has a default composing rectangle, given by the <constant> 176V4L2_SEL_TGT_COMPOSE_DEFAULT </constant> target. This rectangle shall cover what 177the driver writer considers the complete picture. It is recommended for the 178driver developers to put the top/left corner at position <constant> (0,0) 179</constant>. Drivers shall set the active composing rectangle to the default 180one when the driver is first loaded.</para> 181 182<para>The devices may introduce additional content to video signal other than 183an image from memory buffers. It includes borders around an image. However, 184such a padded area is driver-dependent feature not covered by this document. 185Driver developers are encouraged to keep padded rectangle equal to active one. 186The padded target is accessed by the <constant> V4L2_SEL_TGT_COMPOSE_PADDED 187</constant> identifier. It must contain all pixels from the <constant> 188V4L2_SEL_TGT_COMPOSE </constant> target.</para> 189 190 </section> 191 192 <section> 193 194 <title>Scaling control</title> 195 196<para>An application can detect if scaling is performed by comparing the width 197and the height of rectangles obtained using <constant> V4L2_SEL_TGT_CROP 198</constant> and <constant> V4L2_SEL_TGT_COMPOSE </constant> targets. If 199these are not equal then the scaling is applied. The application can compute 200the scaling ratios using these values.</para> 201 202 </section> 203 204 </section> 205 206 <section> 207 208 <title>Comparison with old cropping API</title> 209 210<para>The selection API was introduced to cope with deficiencies of previous 211<link linkend="crop"> API </link>, that was designed to control simple capture 212devices. Later the cropping API was adopted by video output drivers. The ioctls 213are used to select a part of the display were the video signal is inserted. It 214should be considered as an API abuse because the described operation is 215actually the composing. The selection API makes a clear distinction between 216composing and cropping operations by setting the appropriate targets. The V4L2 217API lacks any support for composing to and cropping from an image inside a 218memory buffer. The application could configure a capture device to fill only a 219part of an image by abusing V4L2 API. Cropping a smaller image from a larger 220one is achieved by setting the field 221&v4l2-pix-format;<structfield>::bytesperline</structfield>. Introducing an image offsets 222could be done by modifying field &v4l2-buffer;<structfield>::m_userptr</structfield> 223before calling <constant> VIDIOC_QBUF </constant>. Those 224operations should be avoided because they are not portable (endianness), and do 225not work for macroblock and Bayer formats and mmap buffers. The selection API 226deals with configuration of buffer cropping/composing in a clear, intuitive and 227portable way. Next, with the selection API the concepts of the padded target 228and constraints flags are introduced. Finally, &v4l2-crop; and &v4l2-cropcap; 229have no reserved fields. Therefore there is no way to extend their functionality. 230The new &v4l2-selection; provides a lot of place for future 231extensions. Driver developers are encouraged to implement only selection API. 232The former cropping API would be simulated using the new one. </para> 233 234 </section> 235 236 <section> 237 <title>Examples</title> 238 <example> 239 <title>Resetting the cropping parameters</title> 240 241 <para>(A video capture device is assumed; change <constant> 242V4L2_BUF_TYPE_VIDEO_CAPTURE </constant> for other devices; change target to 243<constant> V4L2_SEL_TGT_COMPOSE_* </constant> family to configure composing 244area)</para> 245 246 <programlisting> 247 248 &v4l2-selection; sel = { 249 .type = V4L2_BUF_TYPE_VIDEO_CAPTURE, 250 .target = V4L2_SEL_TGT_CROP_DEFAULT, 251 }; 252 ret = ioctl(fd, &VIDIOC-G-SELECTION;, &sel); 253 if (ret) 254 exit(-1); 255 sel.target = V4L2_SEL_TGT_CROP; 256 ret = ioctl(fd, &VIDIOC-S-SELECTION;, &sel); 257 if (ret) 258 exit(-1); 259 260 </programlisting> 261 </example> 262 263 <example> 264 <title>Simple downscaling</title> 265 <para>Setting a composing area on output of size of <emphasis> at most 266</emphasis> half of limit placed at a center of a display.</para> 267 <programlisting> 268 269 &v4l2-selection; sel = { 270 .type = V4L2_BUF_TYPE_VIDEO_OUTPUT, 271 .target = V4L2_SEL_TGT_COMPOSE_BOUNDS, 272 }; 273 struct v4l2_rect r; 274 275 ret = ioctl(fd, &VIDIOC-G-SELECTION;, &sel); 276 if (ret) 277 exit(-1); 278 /* setting smaller compose rectangle */ 279 r.width = sel.r.width / 2; 280 r.height = sel.r.height / 2; 281 r.left = sel.r.width / 4; 282 r.top = sel.r.height / 4; 283 sel.r = r; 284 sel.target = V4L2_SEL_TGT_COMPOSE; 285 sel.flags = V4L2_SEL_FLAG_LE; 286 ret = ioctl(fd, &VIDIOC-S-SELECTION;, &sel); 287 if (ret) 288 exit(-1); 289 290 </programlisting> 291 </example> 292 293 <example> 294 <title>Querying for scaling factors</title> 295 <para>A video output device is assumed; change <constant> 296V4L2_BUF_TYPE_VIDEO_OUTPUT </constant> for other devices</para> 297 <programlisting> 298 299 &v4l2-selection; compose = { 300 .type = V4L2_BUF_TYPE_VIDEO_OUTPUT, 301 .target = V4L2_SEL_TGT_COMPOSE, 302 }; 303 &v4l2-selection; crop = { 304 .type = V4L2_BUF_TYPE_VIDEO_OUTPUT, 305 .target = V4L2_SEL_TGT_CROP, 306 }; 307 double hscale, vscale; 308 309 ret = ioctl(fd, &VIDIOC-G-SELECTION;, &compose); 310 if (ret) 311 exit(-1); 312 ret = ioctl(fd, &VIDIOC-G-SELECTION;, &crop); 313 if (ret) 314 exit(-1); 315 316 /* computing scaling factors */ 317 hscale = (double)compose.r.width / crop.r.width; 318 vscale = (double)compose.r.height / crop.r.height; 319 320 </programlisting> 321 </example> 322 323 </section> 324 325</section> 326
