Lines Matching +full:image +full:- +full:sensor
5 ------------
7 The Freescale i.MX5/6 contains an Image Processing Unit (IPU), which
8 handles the flow of image frames to and from capture devices and
11 For image capture, the IPU contains the following internal subunits:
13 - Image DMA Controller (IDMAC)
14 - Camera Serial Interface (CSI)
15 - Image Converter (IC)
16 - Sensor Multi-FIFO Controller (SMFC)
17 - Image Rotator (IRT)
18 - Video De-Interlacing or Combining Block (VDIC)
20 The IDMAC is the DMA controller for transfer of image frames to and from
23 image flip, 8x8 block transfer (see IRT description), pixel component
24 re-ordering (for example UYVY to YUYV) within the same colorspace, and
25 even packed <--> planar conversion. It can also perform a simple
26 de-interlacing by interleaving even and odd lines during transfer
30 camera sensors over Parallel, BT.656/1120, and MIPI CSI-2 busses.
32 The IC handles color-space conversion, resizing (downscaling and
36 conversions concurrently: pre-process encoding, pre-process viewfinder,
37 and post-processing. Within each task, conversions are split into three
41 The IPU time-shares the IC task operations. The time-slice granularity
42 is one burst of eight pixels in the downsizing section, one image line
43 in the main processing section, one image frame in the rotation section.
49 The IRT carries out 90 and 270 degree image rotation operations. The
57 IC pre-process viewfinder task for further conversions. The VDIC also
58 contains a Combiner that combines two image planes, with alpha blending
64 - MIPI CSI-2 Receiver for camera sensors with the MIPI CSI-2 bus
66 - Two video multiplexers for selecting among multiple sensor inputs
74 --------
78 - Many different pipelines can be configured via media controller API,
82 - Supports parallel, BT.565, and MIPI CSI-2 interfaces.
84 - Concurrent independent streams, by configuring pipelines to multiple
87 - Scaling, color-space conversion, horizontal and vertical flip, and
88 image rotation via IC task subdevs.
90 - Many pixel formats supported (RGB, packed and planar YUV, partial
93 - The VDIC subdev supports motion compensated de-interlacing, with three
99 - Includes a Frame Interval Monitor (FIM) that can correct vertical sync
104 --------
106 imx6-mipi-csi2
107 --------------
109 This is the MIPI CSI-2 receiver entity. It has one sink pad to receive
110 the MIPI CSI-2 stream (usually from a MIPI CSI-2 camera sensor). It has
111 four source pads, corresponding to the four MIPI CSI-2 demuxed virtual
115 This entity actually consists of two sub-blocks. One is the MIPI CSI-2
116 core. This is a Synopsys Designware MIPI CSI-2 core. The other sub-block
117 is a "CSI-2 to IPU gasket". The gasket acts as a demultiplexer of the
122 On i.MX6 solo/dual-lite, all four virtual channel buses are routed to
126 On i.MX6 Quad, virtual channel 0 is routed to IPU1-CSI0 (after selected
127 by a video mux), virtual channels 1 and 2 are hard-wired to IPU1-CSI1
128 and IPU2-CSI0, respectively, and virtual channel 3 is routed to
129 IPU2-CSI1 (again selected by a video mux).
132 -------------
136 MIPI CSI-2 virtual channels from imx6-mipi-csi2 entity. They have a
139 On i.MX6 solo/dual-lite, there are two video mux entities. One sits
140 in front of IPU1-CSI0 to select between a parallel sensor and any of
141 the four MIPI CSI-2 virtual channels (a total of five sink pads). The
142 other mux sits in front of IPU1-CSI1, and again has five sink pads to
143 select between a parallel sensor and any of the four MIPI CSI-2 virtual
147 IPU1-CSI0 to select between a parallel sensor and MIPI CSI-2 virtual
148 channel 0 (two sink pads). The other mux sits in front of IPU2-CSI1 to
149 select between a parallel sensor and MIPI CSI-2 virtual channel 3 (two
153 ---------
156 either a video mux or from a MIPI CSI-2 virtual channel as described
164 from the CSI can be processed by one or both of the IC pre-processing
168 will carry out motion-compensated de-interlace using "high motion" mode
172 via the SMFC and an IDMAC channel, bypassing IC pre-processing. This
182 It will also perform simple de-interlace without motion compensation,
189 - V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR
196 ---------------------
198 The CSI supports cropping the incoming raw sensor frames. This is
202 The CSI also supports fixed divide-by-two downscaling indepently in
216 .. code-block:: none
218 media-ctl -V "'ipu1_csi0_mux':2[fmt:UYVY2X8/1280x960]"
219 media-ctl -V "'ipu1_csi0':0[crop:(0,0)/640x480]"
220 media-ctl -V "'ipu1_csi0':0[compose:(0,0)/320x240]"
223 ---------------------------
234 .. code-block:: none
236 media-ctl -V "'ipu1_csi0':0[fmt:UYVY2X8/640x480@1/60]"
237 media-ctl -V "'ipu1_csi0':2[fmt:UYVY2X8/640x480@1/30]"
240 -----------------------------------
243 NTSC/PAL signal re-sync (too little or too many video lines). When
244 this happens, the IPU triggers a mechanism to re-establish vertical
246 from image to image, and can last a long time before a stable image is
248 permanent split image (one frame contains lines from two consecutive
251 From experiment it was found that during image rolling, the frame
264 to correct the rolling/split image.
270 - V4L2_CID_IMX_FIM_ENABLE
274 - V4L2_CID_IMX_FIM_NUM
277 the nominal frame interval reported by the sensor. This can reduce noise
280 - V4L2_CID_IMX_FIM_TOLERANCE_MIN
285 - V4L2_CID_IMX_FIM_TOLERANCE_MAX
292 - V4L2_CID_IMX_FIM_NUM_SKIP
297 - V4L2_CID_IMX_FIM_ICAP_CHANNEL
298 - V4L2_CID_IMX_FIM_ICAP_EDGE
322 ---------
324 The VDIC carries out motion compensated de-interlacing, with three
338 The source pad routes to the IC pre-processing entity ipuX_ic_prp.
341 -----------
343 This is the IC pre-processing entity. It acts as a router, routing
350 pre-process encode task entity (ipuX_ic_prpenc), the other to the
351 pre-process viewfinder task entity (ipuX_ic_prpvf). Both source pads
353 ipuX_csiY. Only the source pad to the pre-process viewfinder task entity
355 from the VDIC can only be processed by the pre-process viewfinder task).
358 --------------
360 This is the IC pre-processing encode entity. It has a single sink
365 This entity performs the IC pre-process encode task operations:
366 color-space conversion, resizing (downscaling and upscaling),
370 Like the ipuX_csiY IDMAC source, it can also perform simple de-interlace
374 -------------
376 This is the IC pre-processing viewfinder entity. It has a single sink
383 process de-interlaced frames from the ipuX_vdic if ipuX_ic_prp is
386 Like the ipuX_csiY IDMAC source, it can perform simple de-interlace
389 it's not possible to use simple de-interlace in ipuX_ic_prpvf, since
390 the ipuX_vdic has already carried out de-interlacing (with motion
395 -----------------
397 The following describe the various use-cases supported by the pipelines.
399 The links shown do not include the backend sensor, video mux, or mipi
400 csi-2 receiver links. This depends on the type of sensor interface
401 (parallel or mipi csi-2). So these pipelines begin with:
403 sensor -> ipuX_csiY_mux -> ...
407 sensor -> imx6-mipi-csi2 -> (ipuX_csiY_mux) -> ...
409 for mipi csi-2 sensors. The imx6-mipi-csi2 receiver may need to route
411 on the mipi csi-2 virtual channel, hence ipuX_csiY_mux is shown in
415 --------------------------
417 Send frames directly from sensor to camera device interface node, with
420 -> ipuX_csiY:2 -> ipuX_csiY capture
423 ----------------------
427 CSC, flipping, and image rotation:
429 -> ipuX_csiY:1 -> 0:ipuX_ic_prp:1 -> 0:ipuX_ic_prpenc:1 -> ipuX_ic_prpenc capture
431 Motion Compensated De-interlace:
432 --------------------------------
435 support motion-compensated de-interlacing (high motion mode only),
438 -> ipuX_csiY:1 -> 0:ipuX_vdic:2 -> 0:ipuX_ic_prp:2 -> 0:ipuX_ic_prpvf:1 -> ipuX_ic_prpvf capture
442 -----------
457 --------------------------------
460 camera interface, and the OV5640 module with a MIPI CSI-2
463 - https://boundarydevices.com/product/nit6x_5mp
464 - https://boundarydevices.com/product/nit6x_5mp_mipi
466 Note that if only one camera module is available, the other sensor
472 The MIPI CSI-2 OV5640 module is connected to the i.MX internal MIPI CSI-2
481 the OV5640, transmitting on MIPI CSI-2 virtual channel 1 (which is
482 imx6-mipi-csi2 pad 2), is routed to ipu1_csi1. Both sensors are
486 .. code-block:: none
489 media-ctl -l "'ov5642 1-0042':0 -> 'ipu1_csi0_mux':1[1]"
490 media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
491 media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
493 media-ctl -l "'ov5640 1-0040':0 -> 'imx6-mipi-csi2':0[1]"
494 media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
495 media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
497 media-ctl -V "'ov5642 1-0042':0 [fmt:YUYV2X8/640x480 field:none]"
498 media-ctl -V "'ipu1_csi0_mux':2 [fmt:YUYV2X8/640x480 field:none]"
499 media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/640x480 field:none]"
501 media-ctl -V "'ov5640 1-0040':0 [fmt:UYVY2X8/640x480 field:none]"
502 media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/640x480 field:none]"
503 media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/640x480 field:none]"
506 "ipu1_csi0 capture" and "ipu1_csi1 capture". The v4l2-ctl tool can
511 ------------------------------
513 On the SabreAuto, an on-board ADV7180 SD decoder is connected to the
518 Compensated de-interlacing. Pad field types assume the adv7180 outputs
522 .. code-block:: none
525 media-ctl -l "'adv7180 3-0021':0 -> 'ipu1_csi0_mux':1[1]"
526 media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
527 media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]"
528 media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]"
529 media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]"
530 media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]"
532 media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x480]"
533 media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x480 field:interlaced]"
534 media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x480 field:interlaced]"
535 media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x480 field:none]"
536 media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x480 field:none]"
537 media-ctl -V "'ipu1_ic_prpvf':1 [fmt:$outputfmt field:none]"
540 "ipu1_ic_prpvf capture". The v4l2-ctl tool can be used to select any
546 SabreSD with MIPI CSI-2 OV5640
547 ------------------------------
550 OV5642 module on IPU1 CSI0, and a MIPI CSI-2 OV5640 module. The OV5642
554 OV5642 and the MIPI CSI-2 OV5640, but as of this writing only the MIPI
555 CSI-2 OV5640 has been tested, so the OV5642 node is currently disabled.
560 from the OV5640, transmitting on MIPI CSI-2 virtual channel 1. $sensorfmt
565 .. code-block:: none
568 media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
569 media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
570 media-ctl -l "'ipu1_csi1':1 -> 'ipu1_ic_prp':0[1]"
571 media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]"
572 media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]"
574 media-ctl -V "'ov5640 1-003c':0 [fmt:$sensorfmt field:none]"
575 media-ctl -V "'imx6-mipi-csi2':2 [fmt:$sensorfmt field:none]"
576 media-ctl -V "'ipu1_csi1':1 [fmt:AYUV32/$sensordim field:none]"
577 media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/$sensordim field:none]"
578 media-ctl -V "'ipu1_ic_prpenc':1 [fmt:$outputfmt field:none]"
580 Streaming can then begin on "ipu1_ic_prpenc capture" node. The v4l2-ctl
586 ------------
591 no end-of-frame interrupts from the IDMAC channel. To work around
597 ---------
601 include/linux/imx-media.h
604 ----------
606 .. [#f1] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6DQRM.pdf
607 .. [#f2] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6SDLRM.pdf
611 -------
613 - Steve Longerbeam <steve_longerbeam@mentor.com>
614 - Philipp Zabel <kernel@pengutronix.de>
615 - Russell King <linux@armlinux.org.uk>
617 Copyright (C) 2012-2017 Mentor Graphics Inc.