Lines Matching +full:video +full:- +full:codecs
2 <?% config.freshness.reviewed = '2021-04-15' %?>
4 # Video coding in WebRTC
6 ## Introduction to layered video coding
8 [Video coding][video-coding-wiki] is the process of encoding a stream of
9 uncompressed video frames into a compressed bitstream, whose bitrate is lower
12 ### Block-based hybrid video coding
14 All video codecs in WebRTC are based on the block-based hybrid video coding
15 paradigm, which entails prediction of the original video frame using either
16 [information from previously encoded frames][motion-compensation-wiki] or
18 of the prediction from the original video, and
19 [transform][transform-coding-wiki] and [quantization][quantization-wiki] of the
21 transform coefficients, is losslessly [entropy coded][entropy-coding-wiki] along
25 prediction. Finally, in-loop filtering is applied and the resulting
32 more inter-frame dependencies), the prior frames must be available at the
35 required. Such a frame is called a "key frame". For real-time-communications
39 ### Single-layer coding
41 In 1:1 calls, the encoded bitstream has a single recipient. Using end-to-end
45 using "single-layer coding", where each delta frame only depends on the frame
48 ### Scalable video coding
53 [scalable video coding][svc-wiki] can be used. The idea is to introduce
56 allows for a [selective forwarding unit][sfu-webrtc-glossary] to discard upper
75 transmission in a multiway call), spatial scalability with inter-layer
79 layers. The `K-SVC` concept, where spatial inter-layer dependencies are only
80 used to encode key frames, for which inter-layer prediction is typically
86 Given the general introduction to video coding above, we now describe some
87 specifics of the [`modules/video_coding`][modules-video-coding] folder in WebRTC.
89 ### Built-in software codecs in [`modules/video_coding/codecs`][modules-video-coding-codecs]
91 This folder contains WebRTC-specific classes that wrap software codec
92 implementations for different video coding standards:
94 * [libaom][libaom-src] for [AV1][av1-spec]
95 * [libvpx][libvpx-src] for [VP8][vp8-spec] and [VP9][vp9-spec]
96 * [OpenH264][openh264-src] for [H.264 constrained baseline profile][h264-spec]
98 Users of the library can also inject their own codecs, using the
99 [VideoEncoderFactory][video-encoder-factory-interface] and
100 [VideoDecoderFactory][video-decoder-factory-interface] interfaces. This is how
101 platform-supported codecs, such as hardware backed codecs, are implemented.
103 ### Video codec test framework in [`modules/video_coding/codecs/test`][modules-video-coding-codecs-…
105 This folder contains a test framework that can be used to evaluate video quality
106 performance of different video codec implementations.
108 ### SVC helper classes in [`modules/video_coding/svc`][modules-video-coding-svc]
110 * [`ScalabilityStructure*`][scalabilitystructure] - different
111 [standardized scalability structures][scalability-structure-spec]
112 * [`ScalableVideoController`][scalablevideocontroller] - provides instructions to the video encod…
114 * [`SvcRateAllocator`][svcrateallocator] - bitrate allocation to different spatial and temporal
117 ### Utility classes in [`modules/video_coding/utility`][modules-video-coding-utility]
119 * [`FrameDropper`][framedropper] - drops incoming frames when encoder systematically
121 * [`FramerateController`][frameratecontroller] - drops incoming frames to achieve a target framer…
122 * [`QpParser`][qpparser] - parses the quantization parameter from a bitstream
123 * [`QualityScaler`][qualityscaler] - signals when an encoder generates encoded frames whose
125 * [`SimulcastRateAllocator`][simulcastrateallocator] - bitrate allocation to simulcast layers
127 ### General helper classes in [`modules/video_coding`][modules-video-coding]
129 * [`FecControllerDefault`][feccontrollerdefault] - provides a default implementation for rate
130 allocation to [forward error correction][fec-wiki]
131 * [`VideoCodecInitializer`][videocodecinitializer] - converts between different encoder configura…
134 ### Receiver buffer classes in [`modules/video_coding`][modules-video-coding]
136 * [`PacketBuffer`][packetbuffer] - (re-)combines RTP packets into frames
137 * [`RtpFrameReferenceFinder`][rtpframereferencefinder] - determines dependencies between frames b…
138 * [`FrameBuffer`][framebuffer] - order frames based on their dependencies to be fed to the decoder
140 [video-coding-wiki]: https://en.wikipedia.org/wiki/Video_coding_format
141 [motion-compensation-wiki]: https://en.wikipedia.org/wiki/Motion_compensation
142 [transform-coding-wiki]: https://en.wikipedia.org/wiki/Transform_coding
143 [motion-vector-wiki]: https://en.wikipedia.org/wiki/Motion_vector
144 [mpeg-wiki]: https://en.wikipedia.org/wiki/Moving_Picture_Experts_Group
145 [svc-wiki]: https://en.wikipedia.org/wiki/Scalable_Video_Coding
146 [sfu-webrtc-glossary]: https://webrtcglossary.com/sfu/
147 [libvpx-src]: https://chromium.googlesource.com/webm/libvpx/
148 [libaom-src]: https://aomedia.googlesource.com/aom/
149 [openh264-src]: https://github.com/cisco/openh264
150 [vp8-spec]: https://tools.ietf.org/html/rfc6386
151 [vp9-spec]: https://storage.googleapis.com/downloads.webmproject.org/docs/vp9/vp9-bitstream-specifi…
152 [av1-spec]: https://aomediacodec.github.io/av1-spec/
153 [h264-spec]: https://www.itu.int/rec/T-REC-H.264-201906-I/en
154 [video-encoder-factory-interface]: https://source.chromium.org/chromium/chromium/src/+/main:third_p…
155 [video-decoder-factory-interface]: https://source.chromium.org/chromium/chromium/src/+/main:third_p…
156 [scalability-structure-spec]: https://w3c.github.io/webrtc-svc/#scalabilitymodes*
157 [fec-wiki]: https://en.wikipedia.org/wiki/Error_correction_code#Forward_error_correction
158 [entropy-coding-wiki]: https://en.wikipedia.org/wiki/Entropy_encoding
159 [modules-video-coding]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc…
160 [modules-video-coding-codecs]: https://source.chromium.org/chromium/chromium/src/+/main:third_party…
161 [modules-video-coding-codecs-test]: https://source.chromium.org/chromium/chromium/src/+/main:third_…
162 [modules-video-coding-svc]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/we…
163 [modules-video-coding-utility]: https://source.chromium.org/chromium/chromium/src/+/main:third_part…
177 [quantization-wiki]: https://en.wikipedia.org/wiki/Quantization_(signal_processing)