Lines Matching +full:on +full:- +full:device
1 .. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
6 Sub-device Interface
13 components as software blocks called sub-devices.
15 V4L2 sub-devices are usually kernel-only objects. If the V4L2 driver
16 implements the media device API, they will automatically inherit from
17 media entities. Applications will be able to enumerate the sub-devices
21 In addition to make sub-devices discoverable, drivers can also choose to
23 sub-device driver and the V4L2 device driver support this, sub-devices
24 will feature a character device node on which ioctls can be called to
26 - query, read and write sub-devices controls
28 - subscribe and unsubscribe to events and retrieve them
30 - negotiate image formats on individual pads
32 - inspect and modify internal data routing between pads of the same entity
34 Sub-device character device nodes, conventionally named
35 ``/dev/v4l-subdev*``, use major number 81.
37 Drivers may opt to limit the sub-device character devices to only expose
38 operations that do not modify the device state. In such a case the sub-devices
39 are referred to as ``read-only`` in the rest of this documentation, and the
46 Most V4L2 controls are implemented by sub-device hardware. Drivers
47 usually merge all controls and expose them through video device nodes.
48 Applications can control all sub-devices through a single interface.
55 single device, all but one of the identical controls are hidden.
57 Applications can access those hidden controls through the sub-device
59 behave identically as when issued on V4L2 device nodes, with the
61 sub-device.
63 Depending on the driver, those controls might also be exposed through
64 one (or several) V4L2 device nodes.
70 V4L2 sub-devices can notify applications of events as described in
71 :ref:`event`. The API behaves identically as when used on V4L2 device
73 the sub-device. Depending on the driver, those events might also be
74 reported on one (or several) V4L2 device nodes.
77 .. _pad-level-formats:
79 Pad-level Formats
84 Pad-level formats are only applicable to very complex devices that
85 need to expose low-level format configuration to user space. Generic
94 Image formats are typically negotiated on video capture and output
102 hardware configurations. One such example is shown on
103 :ref:`pipeline-scaling`, where image scaling can be performed on both
107 .. _pipeline-scaling:
109 .. kernel-figure:: pipeline.dot
113 Image Format Negotiation on Pipelines
120 scaling on the sensor is required to achieve higher frame rates.
121 Depending on the use case (quality vs. speed), the pipeline must be
125 Drivers that implement the :ref:`media API <media-controller-intro>`
126 can expose pad-level image format configuration to applications. When
130 negotiate formats on a per-pad basis.
132 Applications are responsible for configuring coherent parameters on the
138 Pad-level image format configuration support can be tested by calling
139 the :ref:`VIDIOC_SUBDEV_G_FMT` ioctl on pad
140 0. If the driver returns an ``EINVAL`` error code pad-level format
141 configuration is not supported by the sub-device.
145 ------------------
147 Acceptable formats on pads can (and usually do) depend on a number of
148 external parameters, such as formats on other pads, active links, or
149 even controls. Finding a combination of formats on all pads in a video
150 pipeline, acceptable to both application and driver, can't rely on
157 :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls operate on
159 configuration. Modifying those 'try' formats leaves the device state
161 and the hardware state stored in the device itself).
163 While not kept as part of the device state, try formats are stored in
164 the sub-device file handles. A
166 the last try format set *on the same sub-device file handle*. Several
167 applications querying the same sub-device at the same time will thus not
170 To find out whether a particular format is supported by the device,
173 verify and, if needed, change the requested ``format`` based on device
179 guaranteed to be supported by the device. In particular, drivers
181 to an :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` call as-is
182 (as long as external parameters, such as formats on other pads or links'
185 Drivers automatically propagate formats inside sub-devices. When a try
186 or active format is set on a pad, corresponding formats on other pads of
187 the same sub-device can be modified by the driver. Drivers are free to
188 modify formats as required by the device. However, they should comply
191 - Formats should be propagated from sink pads to source pads. Modifying
192 a format on a source pad should not modify the format on any sink
195 - Sub-devices that scale frames using variable scaling factors should
201 propagating them from one sub-device file handle to another.
203 explicitly with compatible formats. Identical formats on the two ends of
205 different formats matching device requirements as being compatible.
207 :ref:`sample-pipeline-config` shows a sample configuration sequence
208 for the pipeline described in :ref:`pipeline-scaling` (table columns
220 .. _sample-pipeline-config:
222 .. flat-table:: Sample Pipeline Configuration
223 :header-rows: 1
224 :stub-columns: 0
227 * -
228 - Sensor/0
231 - Frontend/0
234 - Frontend/1
237 - Scaler/0
240 - Scaler/0
243 - Scaler/1
246 * - Initial state
247 - 2048x1536
250 - (default)
251 - (default)
252 - (default)
253 - (default)
254 - (default)
255 * - Configure frontend sink format
256 - 2048x1536
259 - *2048x1536*
262 - *2046x1534*
265 - (default)
266 - (default)
267 - (default)
268 * - Configure scaler sink format
269 - 2048x1536
272 - 2048x1536
275 - 2046x1534
278 - *2046x1534*
281 - *0,0/2046x1534*
282 - *2046x1534*
285 * - Configure scaler sink compose selection
286 - 2048x1536
289 - 2048x1536
292 - 2046x1534
295 - 2046x1534
298 - *0,0/1280x960*
299 - *1280x960*
308 size and V4L2_MBUS_FMT_SGRBG8_1X8 media bus code. Formats on the
310 values, as well as the compose rectangle on the scaler's sink pad.
319 on the sink pad is set to V4L2_MBUS_FMT_SGRBG_1X8. The driver
320 propagates the size to the compose selection rectangle on the
334 be applied as-is by the driver without being modified.
337 .. _v4l2-subdev-selections:
340 ---------------------------------------------
342 Many sub-devices support cropping frames on their input or output pads
343 (or possible even on both). Cropping is used to select the area of
344 interest in an image, typically on an image sensor or a video decoder.
354 selection targets :ref:`v4l2-selections-common`.
356 On sink pads, cropping is applied relative to the current pad format.
357 The pad format represents the image size as received by the sub-device
359 represents the sub-image that will be transmitted further inside the
360 sub-device for processing.
368 rectangle on the subdev's sink pad is scaled to the size configured
371 using ``V4L2_SEL_TGT_COMPOSE`` selection target on the same pad. If the
375 On source pads, cropping is similar to sink pads, with the exception
377 COMPOSE rectangle on the sink pad. In both sink and source pads, the
382 requests on all selection targets, unless specifically told otherwise.
384 the image size either up or down. :ref:`v4l2-selection-flags`
388 --------------------------
405 pixel array is not rectangular but cross-shaped or round. The maximum
409 .. _format-propagation:
412 ---------------------------------------------
421 by the driver, depending on the properties of the underlying hardware.
425 rectangle, which refers to the sink compose bounds rectangle --- if it
441 4. Source pad actual crop selection. Crop on the source pad defines crop
456 .. _subdev-image-processing-crop:
458 .. kernel-figure:: subdev-image-processing-crop.svg
459 :alt: subdev-image-processing-crop.svg
464 In the above example, the subdev supports cropping on its sink pad. To
465 configure it, the user sets the media bus format on the subdev's sink
466 pad. Now the actual crop rectangle can be set on the sink pad --- the
473 .. _subdev-image-processing-scaling-multi-source:
475 .. kernel-figure:: subdev-image-processing-scaling-multi-source.svg
476 :alt: subdev-image-processing-scaling-multi-source.svg
489 .. _subdev-image-processing-full:
491 .. kernel-figure:: subdev-image-processing-full.svg
492 :alt: subdev-image-processing-full.svg
499 further composed on the composition bounds rectangle. From that, two
507 subdev-formats
509 .. _subdev-routing:
512 ----------------------------------------------------
514 Simple V4L2 sub-devices do not support multiple, unrelated video streams,
521 Some hardware, e.g. MIPI CSI-2, support multiplexed streams, that is, multiple
522 data streams are transmitted on the same bus, which is represented by a media
523 link connecting a transmitter source pad with a sink pad on the receiver. For
525 metadata stream, which are transmitted on the multiplexed data bus, represented
527 sink pad. The stream-aware receiver will de-multiplex the streams received on
532 non-multiplexed subdev drivers. However, if the driver at the sink end of a link
534 There may be additional limitations specific to the sink device.
542 streams from one end of the link to the other, and sub-devices have routing
546 A stream ID is a media pad-local identifier for a stream. Streams IDs of
547 the same stream must be equal on both ends of a link. In other words,
548 a particular stream ID must exist on both sides of a media
550 of the sub-device.
553 sub-device and a (pad, stream) pair. For sub-devices that do not support
559 The addition of streams to the V4L2 sub-device interface moves the sub-device
563 the same as without streams (see :ref:`format-propagation`).
565 Instead of the sub-device wide merging of streams from all sink pads
567 other. Any number of routes from streams on sink pads towards streams on
569 stream on a source pad, however, only a single route is allowed.
572 are independent of similar configurations on other streams. This is
575 Device types and routing setup
578 Different kinds of sub-devices have differing behaviour for route activation,
579 depending on the hardware. In all cases, however, only routes that have the
591 respect to routing. Typically any route between the sub-device's sink and source
594 and user-created routes are fully replaced when ``VIDIOC_SUBDEV_S_ROUTING`` is
595 called on the sub-device. Such newly created routes have the device's default
601 The configuration of the streams is done individually for each sub-device and
602 the validity of the streams between sub-devices is validated when the pipeline
607 1. Set up links. Connect the pads between sub-devices using the
611 routing table for the sub-device using :ref:`VIDIOC_SUBDEV_S_ROUTING
613 reset formats and selections in the sub-device to default values.
616 configured separately as documented for plain sub-devices in
617 :ref:`format-propagation`. The stream ID is set to the same stream ID
626 - Two identical sensors (Sensor A and Sensor B). Each sensor has a single source
629 - Multiplexer bridge (Bridge). The bridge has two sink pads, connected to the
632 - Receiver in the SoC (Receiver). The receiver has a single sink pad (pad 0),
633 connected to the bridge, and two source pads (pads 1-2), going to the DMA
636 - DMA Engines in the SoC (DMA Engine), one for each stream. Each DMA engine is
639 The sensors, the bridge and the receiver are modeled as V4L2 sub-devices,
640 exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are
647 not differ from normal non-multiplexed media controller setup.
651 .. flat-table:: Bridge routing table
652 :header-rows: 1
654 * - Sink Pad/Stream
655 - Source Pad/Stream
656 - Routing Flags
657 - Comments
658 * - 0/0
659 - 2/0
660 - V4L2_SUBDEV_ROUTE_FL_ACTIVE
661 - Pixel data stream from Sensor A
662 * - 1/0
663 - 2/1
664 - V4L2_SUBDEV_ROUTE_FL_ACTIVE
665 - Pixel data stream from Sensor B
667 .. flat-table:: Receiver routing table
668 :header-rows: 1
670 * - Sink Pad/Stream
671 - Source Pad/Stream
672 - Routing Flags
673 - Comments
674 * - 0/0
675 - 1/0
676 - V4L2_SUBDEV_ROUTE_FL_ACTIVE
677 - Pixel data stream from Sensor A
678 * - 0/1
679 - 2/0
680 - V4L2_SUBDEV_ROUTE_FL_ACTIVE
681 - Pixel data stream from Sensor B
693 stream endpoint in each sub-device.