Lines Matching +full:1 +full:- +full:to +full:- +full:1
1 .. SPDX-License-Identifier: GPL-2.0
7 ------------
10 handles the flow of image frames to and from capture devices and
15 - Image DMA Controller (IDMAC)
16 - Camera Serial Interface (CSI)
17 - Image Converter (IC)
18 - Sensor Multi-FIFO Controller (SMFC)
19 - Image Rotator (IRT)
20 - Video De-Interlacing or Combining Block (VDIC)
22 The IDMAC is the DMA controller for transfer of image frames to and from
26 re-ordering (for example UYVY to YUYV) within the same colorspace, and
27 packed <--> planar conversion. The IDMAC can also perform a simple
28 de-interlacing by interweaving even and odd lines during transfer
32 camera sensors over Parallel, BT.656/1120, and MIPI CSI-2 buses.
34 The IC handles color-space conversion, resizing (downscaling and
38 conversions concurrently: pre-process encoding, pre-process viewfinder,
39 and post-processing. Within each task, conversions are split into three
43 The IPU time-shares the IC task operations. The time-slice granularity
48 captured frames from sensors directly to memory concurrently via four
56 The VDIC handles the conversion of interlaced video to progressive, with
58 motion). The deinterlaced output frames from the VDIC can be sent to the
59 IC pre-process viewfinder task for further conversions. The VDIC also
63 In addition to the IPU internal subunits, there are also two units
66 - MIPI CSI-2 Receiver for camera sensors with the MIPI CSI-2 bus
68 - Two video multiplexers for selecting among multiple sensor inputs
69 to send to a CSI.
71 For more info, refer to the latest versions of the i.MX5/6 reference
76 --------
80 - Many different pipelines can be configured via media controller API,
81 that correspond to the hardware video capture pipelines supported in
84 - Supports parallel, BT.565, and MIPI CSI-2 interfaces.
86 - Concurrent independent streams, by configuring pipelines to multiple
89 - Scaling, color-space conversion, horizontal and vertical flip, and
92 - Many pixel formats supported (RGB, packed and planar YUV, partial
95 - The VDIC subdev supports motion compensated de-interlacing, with three
97 defined that allow sending frames to the VDIC subdev directly from the
98 CSI. There is also support in the future for sending frames to the
101 - Includes a Frame Interval Monitor (FIM) that can correct vertical sync
106 --------
109 i.MX6Q SabreAuto. Refer to these diagrams in the entity descriptions
114 is common to all i.MX5/6 platforms. For example, the SabreSD, with the
115 MIPI CSI-2 OV5640 sensor, requires the i.MX6 MIPI CSI-2 receiver. But
117 therefore does not require the MIPI CSI-2 receiver, so it is missing in
122 .. kernel-figure:: imx6q-sabresd.dot
128 .. kernel-figure:: imx6q-sabreauto.dot
135 --------
137 imx6-mipi-csi2
138 --------------
140 This is the MIPI CSI-2 receiver entity. It has one sink pad to receive
141 the MIPI CSI-2 stream (usually from a MIPI CSI-2 camera sensor). It has
142 four source pads, corresponding to the four MIPI CSI-2 demuxed virtual
143 channel outputs. Multiple source pads can be enabled to independently
146 This entity actually consists of two sub-blocks. One is the MIPI CSI-2
147 core. This is a Synopsys Designware MIPI CSI-2 core. The other sub-block
148 is a "CSI-2 to IPU gasket". The gasket acts as a demultiplexer of the
150 containing each virtual channel that are routed to CSIs or video
153 On i.MX6 solo/dual-lite, all four virtual channel buses are routed to
157 On i.MX6 Quad, virtual channel 0 is routed to IPU1-CSI0 (after selected
158 by a video mux), virtual channels 1 and 2 are hard-wired to IPU1-CSI1
159 and IPU2-CSI0, respectively, and virtual channel 3 is routed to
160 IPU2-CSI1 (again selected by a video mux).
163 -------------
165 These are the video multiplexers. They have two or more sink pads to
167 MIPI CSI-2 virtual channels from imx6-mipi-csi2 entity. They have a
168 single source pad that routes to a CSI (ipuX_csiY entities).
170 On i.MX6 solo/dual-lite, there are two video mux entities. One sits
171 in front of IPU1-CSI0 to select between a parallel sensor and any of
172 the four MIPI CSI-2 virtual channels (a total of five sink pads). The
173 other mux sits in front of IPU1-CSI1, and again has five sink pads to
174 select between a parallel sensor and any of the four MIPI CSI-2 virtual
178 IPU1-CSI0 to select between a parallel sensor and MIPI CSI-2 virtual
179 channel 0 (two sink pads). The other mux sits in front of IPU2-CSI1 to
180 select between a parallel sensor and MIPI CSI-2 virtual channel 3 (two
184 ---------
187 either a video mux or from a MIPI CSI-2 virtual channel as described
191 to the ipuX_vdic entity or the ipuX_ic_prp entity, using hardware links
194 When the direct source pad is routed to the ipuX_ic_prp entity, frames
195 from the CSI can be processed by one or both of the IC pre-processing
198 When the direct source pad is routed to the ipuX_vdic entity, the VDIC
199 will carry out motion-compensated de-interlace using "high motion" mode
202 The second source pad sends video frames directly to memory buffers
203 via the SMFC and an IDMAC channel, bypassing IC pre-processing. This
204 source pad is routed to a capture device node, with a node name of the
210 order, the capture device linked to the IDMAC source pad can capture
217 pad's field type is sequential top-bottom or bottom-top, and the
218 requested capture interface field type is set to interlaced (t-b, b-t,
220 field order as the source pad field order (interlaced-bt if source pad
221 is seq-bt, interlaced-tb if source pad is seq-tb).
226 ---------------------
232 The CSI also supports fixed divide-by-two downscaling independently in
240 rectangle is equal to incoming rectangle).
242 To give an example of crop and /2 downscale, this will crop a
243 1280x960 input frame to 640x480, and then /2 downscale in both
244 dimensions to 320x240 (assumes ipu1_csi0 is linked to ipu1_csi0_mux):
246 .. code-block:: none
248 media-ctl -V "'ipu1_csi0_mux':2[fmt:UYVY2X8/1280x960]"
249 media-ctl -V "'ipu1_csi0':0[crop:(0,0)/640x480]"
250 media-ctl -V "'ipu1_csi0':0[compose:(0,0)/320x240]"
253 ---------------------------
258 frame skip setting to the CSI to achieve the desired frame rate
264 .. code-block:: none
266 media-ctl -V "'ipu1_csi0':0[fmt:UYVY2X8/640x480@1/60]"
267 media-ctl -V "'ipu1_csi0':2[fmt:UYVY2X8/640x480@1/30]"
270 -----------------------------------
275 ---------
277 The VDIC carries out motion compensated de-interlacing, with three
291 The source pad routes to the IC pre-processing entity ipuX_ic_prp.
294 -----------
296 This is the IC pre-processing entity. It acts as a router, routing
297 data from its sink pad to one or both of its source pads.
302 This entity has two source pads. One source pad routes to the
303 pre-process encode task entity (ipuX_ic_prpenc), the other to the
304 pre-process viewfinder task entity (ipuX_ic_prpvf). Both source pads
306 ipuX_csiY. Only the source pad to the pre-process viewfinder task entity
308 from the VDIC can only be processed by the pre-process viewfinder task).
311 --------------
313 This is the IC pre-processing encode entity. It has a single sink
315 routed to a capture device node, with a node name of the format
318 This entity performs the IC pre-process encode task operations:
319 color-space conversion, resizing (downscaling and upscaling),
324 de-interlace without motion compensation, and pixel reordering.
327 -------------
329 This is the IC pre-processing viewfinder entity. It has a single sink
331 to a capture device node, with a node name of the format
334 This entity is identical in operation to ipuX_ic_prpenc, with the same
336 and process de-interlaced frames from the ipuX_vdic if ipuX_ic_prp is
342 ipuX_vdic), it's not possible to use interweave in ipuX_ic_prpvf,
343 since the ipuX_vdic has already carried out de-interlacing (with
348 -----------------
350 The following describe the various use-cases supported by the pipelines.
353 csi-2 receiver links. This depends on the type of sensor interface
354 (parallel or mipi csi-2). So these pipelines begin with:
356 sensor -> ipuX_csiY_mux -> ...
360 sensor -> imx6-mipi-csi2 -> (ipuX_csiY_mux) -> ...
362 for mipi csi-2 sensors. The imx6-mipi-csi2 receiver may need to route
363 to the video mux (ipuX_csiY_mux) before sending to the CSI, depending
364 on the mipi csi-2 virtual channel, hence ipuX_csiY_mux is shown in
368 --------------------------
370 Send frames directly from sensor to camera device interface node, with
373 -> ipuX_csiY:2 -> ipuX_csiY capture
376 ----------------------
378 This pipeline uses the preprocess encode entity to route frames directly
379 from the CSI to the IC, to carry out scaling up to 1024x1024 resolution,
382 -> ipuX_csiY:1 -> 0:ipuX_ic_prp:1 -> 0:ipuX_ic_prpenc:1 -> ipuX_ic_prpenc capture
384 Motion Compensated De-interlace:
385 --------------------------------
387 This pipeline routes frames from the CSI direct pad to the VDIC entity to
388 support motion-compensated de-interlacing (high motion mode only),
389 scaling up to 1024x1024, CSC, flip, and rotation:
391 -> ipuX_csiY:1 -> 0:ipuX_vdic:2 -> 0:ipuX_ic_prp:2 -> 0:ipuX_ic_prpvf:1 -> ipuX_ic_prpvf capture
395 -----------
397 To aid in configuration and for backward compatibility with V4L2
410 ---------------------------------------
413 camera interface, and the OV5640 module with a MIPI CSI-2
416 - https://boundarydevices.com/product/nit6x_5mp
417 - https://boundarydevices.com/product/nit6x_5mp_mipi
422 The OV5642 module is connected to the parallel bus input on the i.MX
423 internal video mux to IPU1 CSI0. It's i2c bus connects to i2c bus 2.
425 The MIPI CSI-2 OV5640 module is connected to the i.MX internal MIPI CSI-2
427 routed as follows: vc0 to the IPU1 CSI0 mux, vc1 directly to IPU1 CSI1,
428 vc2 directly to IPU2 CSI0, and vc3 to the IPU2 CSI1 mux. The OV5640 is
429 also connected to i2c bus 2 on the SabreLite, therefore the OV5642 and
433 pipelines for both sensors. The OV5642 is routed to ipu1_csi0, and
434 the OV5640, transmitting on MIPI CSI-2 virtual channel 1 (which is
435 imx6-mipi-csi2 pad 2), is routed to ipu1_csi1. Both sensors are
436 configured to output 640x480, and the OV5642 outputs YUYV2X8, the
439 .. code-block:: none
442 media-ctl -l "'ov5642 1-0042':0 -> 'ipu1_csi0_mux':1[1]"
443 media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
444 media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
446 media-ctl -l "'ov5640 1-0040':0 -> 'imx6-mipi-csi2':0[1]"
447 media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
448 media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
450 media-ctl -V "'ov5642 1-0042':0 [fmt:YUYV2X8/640x480 field:none]"
451 media-ctl -V "'ipu1_csi0_mux':2 [fmt:YUYV2X8/640x480 field:none]"
452 media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/640x480 field:none]"
454 media-ctl -V "'ov5640 1-0040':0 [fmt:UYVY2X8/640x480 field:none]"
455 media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/640x480 field:none]"
456 media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/640x480 field:none]"
459 "ipu1_csi0 capture" and "ipu1_csi1 capture". The v4l2-ctl tool can
460 be used to select any supported YUV pixelformat on the capture device
464 -------------------------------------
466 On the i.MX6Q SabreAuto, an on-board ADV7180 SD decoder is connected to the
467 parallel bus input on the internal video mux to IPU1 CSI0.
469 The following example configures a pipeline to capture from the ADV7180
472 must output sequential or alternating fields (field type 'seq-bt' for
475 .. code-block:: none
478 media-ctl -l "'adv7180 3-0021':0 -> 'ipu1_csi0_mux':1[1]"
479 media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
480 media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
482 media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x480 field:seq-bt]"
483 media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x480]"
484 media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720x480]"
486 v4l2-ctl -d4 --set-fmt-video=field=interlaced_bt
488 Streaming can then begin on /dev/video4. The v4l2-ctl tool can also be
489 used to select any supported YUV pixelformat on /dev/video4.
491 This example configures a pipeline to capture from the ADV7180
493 Compensated de-interlacing. The adv7180 must output sequential or
494 alternating fields (field type 'seq-tb' for PAL, or 'alternate').
496 .. code-block:: none
499 media-ctl -l "'adv7180 3-0021':0 -> 'ipu1_csi0_mux':1[1]"
500 media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
501 media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]"
502 media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]"
503 media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]"
504 media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]"
506 media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x576 field:seq-tb]"
507 media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x576]"
508 media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x576]"
509 media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x576 field:none]"
510 media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x576 field:none]"
511 media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x576 field:none]"
513 v4l2-ctl -d2 --set-fmt-video=field=none
515 Streaming can then begin on /dev/video2. The v4l2-ctl tool can also be
516 used to select any supported YUV pixelformat on /dev/video2.
518 This platform accepts Composite Video analog inputs to the ADV7180 on
522 --------------------------------------
524 On the i.MX6DL SabreAuto, an on-board ADV7180 SD decoder is connected to the
525 parallel bus input on the internal video mux to IPU1 CSI0.
527 The following example configures a pipeline to capture from the ADV7180
530 must output sequential or alternating fields (field type 'seq-bt' for
533 .. code-block:: none
536 media-ctl -l "'adv7180 4-0021':0 -> 'ipu1_csi0_mux':4[1]"
537 media-ctl -l "'ipu1_csi0_mux':5 -> 'ipu1_csi0':0[1]"
538 media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
540 media-ctl -V "'adv7180 4-0021':0 [fmt:UYVY2X8/720x480 field:seq-bt]"
541 media-ctl -V "'ipu1_csi0_mux':5 [fmt:UYVY2X8/720x480]"
542 media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720x480]"
544 v4l2-ctl -d0 --set-fmt-video=field=interlaced_bt
546 Streaming can then begin on /dev/video0. The v4l2-ctl tool can also be
547 used to select any supported YUV pixelformat on /dev/video0.
549 This example configures a pipeline to capture from the ADV7180
551 Compensated de-interlacing. The adv7180 must output sequential or
552 alternating fields (field type 'seq-tb' for PAL, or 'alternate').
554 .. code-block:: none
557 media-ctl -l "'adv7180 4-0021':0 -> 'ipu1_csi0_mux':4[1]"
558 media-ctl -l "'ipu1_csi0_mux':5 -> 'ipu1_csi0':0[1]"
559 media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]"
560 media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]"
561 media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]"
562 media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]"
564 media-ctl -V "'adv7180 4-0021':0 [fmt:UYVY2X8/720x576 field:seq-tb]"
565 media-ctl -V "'ipu1_csi0_mux':5 [fmt:UYVY2X8/720x576]"
566 media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x576]"
567 media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x576 field:none]"
568 media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x576 field:none]"
569 media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x576 field:none]"
571 v4l2-ctl -d2 --set-fmt-video=field=none
573 Streaming can then begin on /dev/video2. The v4l2-ctl tool can also be
574 used to select any supported YUV pixelformat on /dev/video2.
576 This platform accepts Composite Video analog inputs to the ADV7180 on
579 i.MX6Q SabreSD with MIPI CSI-2 OV5640
580 -------------------------------------
582 Similarly to i.MX6Q SabreLite, the i.MX6Q SabreSD supports a parallel
583 interface OV5642 module on IPU1 CSI0, and a MIPI CSI-2 OV5640
584 module. The OV5642 connects to i2c bus 1 and the OV5640 to i2c bus 2.
587 OV5642 and the MIPI CSI-2 OV5640, but as of this writing only the MIPI
588 CSI-2 OV5640 has been tested, so the OV5642 node is currently disabled.
589 The OV5640 module connects to MIPI connector J5. The NXP part number
590 for the OV5640 module that connects to the SabreSD board is H120729.
592 The following example configures unprocessed video capture pipeline to
593 capture from the OV5640, transmitting on MIPI CSI-2 virtual channel 0:
595 .. code-block:: none
598 media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
599 media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
600 media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
601 media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
603 media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
604 media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
605 media-ctl -V "'ipu1_csi0_mux':0 [fmt:UYVY2X8/640x480]"
606 media-ctl -V "'ipu1_csi0':0 [fmt:AYUV32/640x480]"
608 Streaming can then begin on "ipu1_csi0 capture" node. The v4l2-ctl
609 tool can be used to select any supported pixelformat on the capture
612 To determine what is the /dev/video node correspondent to
615 .. code-block:: none
617 media-ctl -e "ipu1_csi0 capture"
622 Starting the streaming via v4l2-ctl:
624 .. code-block:: none
626 v4l2-ctl --stream-mmap -d /dev/video0
628 Starting the streaming via Gstreamer and sending the content to the display:
630 .. code-block:: none
632 gst-launch-1.0 v4l2src device=/dev/video0 ! kmssink
634 The following example configures a direct conversion pipeline to capture
635 from the OV5640, transmitting on MIPI CSI-2 virtual channel 0. It also
638 .. code-block:: none
641 media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
642 media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
643 media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
644 media-ctl -l "'ipu1_csi0':1 -> 'ipu1_ic_prp':0[1]"
645 media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]"
646 media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]"
648 media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
649 media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
650 media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/640x480]"
651 media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/640x480]"
652 media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x480]"
653 media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"
655 v4l2-ctl -d /dev/video1 --set-fmt-video=pixelformat=RGB3
659 To determine what is the /dev/video node correspondent to
662 .. code-block:: none
664 media-ctl -e "ipu1_ic_prpenc capture"
670 Starting the streaming via v4l2-ctl:
672 .. code-block:: none
674 v4l2-ctl --stream-mmap -d /dev/video1
676 Starting the streaming via Gstreamer and sending the content to the display:
678 .. code-block:: none
680 gst-launch-1.0 v4l2src device=/dev/video1 ! kmssink
683 ------------
685 1. When using 90 or 270 degree rotation control at capture resolutions
688 no end-of-frame interrupts from the IDMAC channel. To work around
694 ---------
698 include/linux/imx-media.h
701 ----------
703 .. [#f1] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6DQRM.pdf
704 .. [#f2] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6SDLRM.pdf
708 -------
710 - Steve Longerbeam <steve_longerbeam@mentor.com>
711 - Philipp Zabel <kernel@pengutronix.de>
712 - Russell King <linux@armlinux.org.uk>
714 Copyright (C) 2012-2017 Mentor Graphics Inc.