A buffer contains data exchanged by application and
driver using one of the Streaming I/O methods. Only pointers to
buffers are exchanged, the data itself is not copied. These pointers,
together with meta-information like timestamps or field parity, are
stored in a struct v4l2_buffer
, argument to
the VIDIOC_QUERYBUF
, VIDIOC_QBUF
and VIDIOC_DQBUF
ioctl.
Nominally timestamps refer to the first data byte transmitted. In practice however the wide range of hardware covered by the V4L2 API limits timestamp accuracy. Often an interrupt routine will sample the system clock shortly after the field or frame was stored completely in memory. So applications must expect a constant difference up to one field or frame period plus a small (few scan lines) random error. The delay and error can be much larger due to compression or transmission over an external bus when the frames are not properly stamped by the sender. This is frequently the case with USB cameras. Here timestamps refer to the instant the field or frame was received by the driver, not the capture time. These devices identify by not enumerating any video standards, see Section 1.7.
Similar limitations apply to output timestamps. Typically the video hardware locks to a clock controlling the video timing, the horizontal and vertical synchronization pulses. At some point in the line sequence, possibly the vertical blanking, an interrupt routine samples the system clock, compares against the timestamp and programs the hardware to repeat the previous field or frame, or to display the buffer contents.
Apart of limitations of the video device and natural inaccuracies of all clocks, it should be noted system time itself is not perfectly stable. It can be affected by power saving cycles, warped to insert leap seconds, or even turned back or forth by the system administrator affecting long term measurements. [1]
Table 3-1. struct v4l2_buffer
__u32 | index | Number of the buffer, set by the application. This
field is only used for memory mapping I/O
and can range from zero to the number of buffers allocated
with the VIDIOC_REQBUFS ioctl (struct v4l2_requestbuffers count ) minus one. | |
enum v4l2_buf_type | type | Type of the buffer, same as struct v4l2_format
type or struct v4l2_requestbuffers
type , set by the application. | |
__u32 | bytesused | The number of bytes occupied by the data in the
buffer. It depends on the negotiated data format and may change with
each buffer for compressed variable size data like JPEG images.
Drivers must set this field when type
refers to an input stream, applications when an output stream. | |
__u32 | flags | Flags set by the application or driver, see Table 3-3. | |
enum v4l2_field | field | Indicates the field order of the image in the
buffer, see Table 3-8. This field is not used when
the buffer contains VBI data. Drivers must set it when
type refers to an input stream,
applications when an output stream. | |
struct timeval | timestamp | For input streams this is the
system time (as returned by the | |
struct v4l2_timecode | timecode | When type is
V4L2_BUF_TYPE_VIDEO_CAPTURE and the
V4L2_BUF_FLAG_TIMECODE flag is set in
flags , this structure contains a frame
timecode. In V4L2_FIELD_ALTERNATE
mode the top and bottom field contain the same timecode.
Timecodes are intended to help video editing and are typically recorded on
video tapes, but also embedded in compressed formats like MPEG. This
field is independent of the timestamp and
sequence fields. | |
__u32 | sequence | Set by the driver, counting the frames in the sequence. | |
In V4L2_FIELD_ALTERNATE mode the top and bottom field have the same sequence number. The count starts at zero and includes dropped or repeated frames. A dropped frame was received by an input device but could not be stored due to lack of free buffer space. A repeated frame was displayed again by an output device because the application did not pass new data in time. Note this may count the frames received e.g. over USB, without taking into account the frames dropped by the remote hardware due to limited compression throughput or bus bandwidth. These devices identify by not enumerating any video standards, see Section 1.7. | |||
enum v4l2_memory | memory | This field must be set by applications and/or drivers in accordance with the selected I/O method. | |
union | m | ||
__u32 | offset | When memory is
V4L2_MEMORY_MMAP this is the offset of the buffer
from the start of the device memory. The value is returned by the
driver and apart of serving as parameter to the mmap() function
not useful for applications. See Section 3.2 for details. | |
unsigned long | userptr | When memory is
V4L2_MEMORY_USERPTR this is a pointer to the
buffer (casted to unsigned long type) in virtual memory, set by the
application. See Section 3.3 for details. | |
__u32 | length | Size of the buffer (not the payload) in bytes. | |
__u32 | input | Some video capture drivers support rapid and
synchronous video input changes, a function useful for example in
video surveillance applications. For this purpose applications set the
V4L2_BUF_FLAG_INPUT flag, and this field to the
number of a video input as in struct v4l2_input field
index . | |
__u32 | reserved | A place holder for future extensions and custom
(driver defined) buffer types
V4L2_BUF_TYPE_PRIVATE and higher. |
Table 3-2. enum v4l2_buf_type
V4L2_BUF_TYPE_VIDEO_CAPTURE | 1 | Buffer of a video capture stream, see Section 4.1. |
V4L2_BUF_TYPE_VIDEO_OUTPUT | 2 | Buffer of a video output stream, see Section 4.3. |
V4L2_BUF_TYPE_VIDEO_OVERLAY | 3 | Buffer for video overlay, see Section 4.2. |
V4L2_BUF_TYPE_VBI_CAPTURE | 4 | Buffer of a raw VBI capture stream, see Section 4.7. |
V4L2_BUF_TYPE_VBI_OUTPUT | 5 | Buffer of a raw VBI output stream, see Section 4.7. |
V4L2_BUF_TYPE_SLICED_VBI_CAPTURE | 6 | Buffer of a sliced VBI capture stream, see Section 4.8. |
V4L2_BUF_TYPE_SLICED_VBI_OUTPUT | 7 | Buffer of a sliced VBI output stream, see Section 4.8. |
V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY | 8 | Buffer for video output overlay (OSD), see Section 4.4. Status: Experimental. |
V4L2_BUF_TYPE_PRIVATE | 0x80 | This and higher values are reserved for custom (driver defined) buffer types. |
Table 3-3. Buffer Flags
V4L2_BUF_FLAG_MAPPED | 0x0001 | The buffer resides in device memory and has been mapped into the application's address space, see Section 3.2 for details. Drivers set or clear this flag when the VIDIOC_QUERYBUF, VIDIOC_QBUF or VIDIOC_DQBUF ioctl is called. Set by the driver. |
V4L2_BUF_FLAG_QUEUED | 0x0002 | Internally drivers maintain two buffer queues, an
incoming and outgoing queue. When this flag is set, the buffer is
currently on the incoming queue. It automatically moves to the
outgoing queue after the buffer has been filled (capture devices) or
displayed (output devices). Drivers set or clear this flag when the
VIDIOC_QUERYBUF ioctl is called. After
(successful) calling the VIDIOC_QBUF ioctl it is
always set and after VIDIOC_DQBUF always
cleared. |
V4L2_BUF_FLAG_DONE | 0x0004 | When this flag is set, the buffer is currently on
the outgoing queue, ready to be dequeued from the driver. Drivers set
or clear this flag when the VIDIOC_QUERYBUF ioctl
is called. After calling the VIDIOC_QBUF or
VIDIOC_DQBUF it is always cleared. Of course a
buffer cannot be on both queues at the same time, the
V4L2_BUF_FLAG_QUEUED and
V4L2_BUF_FLAG_DONE flag are mutually exclusive.
They can be both cleared however, then the buffer is in "dequeued"
state, in the application domain to say so. |
V4L2_BUF_FLAG_KEYFRAME | 0x0008 | Drivers set or clear this flag when calling the
VIDIOC_DQBUF ioctl. It may be set by video
capture devices when the buffer contains a compressed image which is a
key frame (or field), i. e. can be decompressed on its own. |
V4L2_BUF_FLAG_PFRAME | 0x0010 | Similar to V4L2_BUF_FLAG_KEYFRAME
this flags predicted frames or fields which contain only differences to a
previous key frame. |
V4L2_BUF_FLAG_BFRAME | 0x0020 | Similar to V4L2_BUF_FLAG_PFRAME
this is a bidirectional predicted frame or field. [ooc tbd] |
V4L2_BUF_FLAG_TIMECODE | 0x0100 | The timecode field is valid.
Drivers set or clear this flag when the VIDIOC_DQBUF
ioctl is called. |
V4L2_BUF_FLAG_INPUT | 0x0200 | The input field is valid.
Applications set or clear this flag before calling the
VIDIOC_QBUF ioctl. |
Table 3-4. enum v4l2_memory
V4L2_MEMORY_MMAP | 1 | The buffer is used for memory mapping I/O. |
V4L2_MEMORY_USERPTR | 2 | The buffer is used for user pointer I/O. |
V4L2_MEMORY_OVERLAY | 3 | [to do] |
The v4l2_timecode
structure is
designed to hold a SMPTE 12M or similar timecode.
(struct timeval
timestamps are stored in
struct v4l2_buffer field timestamp
.)
Table 3-5. struct v4l2_timecode
__u32 | type | Frame rate the timecodes are based on, see Table 3-6. |
__u32 | flags | Timecode flags, see Table 3-7. |
__u8 | frames | Frame count, 0 ... 23/24/29/49/59, depending on the type of timecode. |
__u8 | seconds | Seconds count, 0 ... 59. This is a binary, not BCD number. |
__u8 | minutes | Minutes count, 0 ... 59. This is a binary, not BCD number. |
__u8 | hours | Hours count, 0 ... 29. This is a binary, not BCD number. |
__u8 | userbits [4] | The "user group" bits from the timecode. |
Table 3-6. Timecode Types
V4L2_TC_TYPE_24FPS | 1 | 24 frames per second, i. e. film. |
V4L2_TC_TYPE_25FPS | 2 | 25 frames per second, i. e. PAL or SECAM video. |
V4L2_TC_TYPE_30FPS | 3 | 30 frames per second, i. e. NTSC video. |
V4L2_TC_TYPE_50FPS | 4 | |
V4L2_TC_TYPE_60FPS | 5 |
Table 3-7. Timecode Flags
V4L2_TC_FLAG_DROPFRAME | 0x0001 | Indicates "drop frame" semantics for counting frames in 29.97 fps material. When set, frame numbers 0 and 1 at the start of each minute, except minutes 0, 10, 20, 30, 40, 50 are omitted from the count. |
V4L2_TC_FLAG_COLORFRAME | 0x0002 | The "color frame" flag. |
V4L2_TC_USERBITS_field | 0x000C | Field mask for the "binary group flags". |
V4L2_TC_USERBITS_USERDEFINED | 0x0000 | Unspecified format. |
V4L2_TC_USERBITS_8BITCHARS | 0x0008 | 8-bit ISO characters. |
[1] | Since no other Linux multimedia API supports unadjusted time it would be foolish to introduce here. We must use a universally supported clock to synchronize different media, hence time of day. |