1tsdemux/tsparse TODO 2-------------------- 3 4* Performance 5 * Bufferlist : Creating/Destroying very small buffers is too 6 costly. Switch to pre-/re-allocating outgoing buffers in which we 7 copy the data. 8 * Adapter : Use gst_adapter_peek()/_flush() instead of constantly 9 creating buffers. 10 11* Latency 12 * Calculate the actual latency instead of returning a fixed 13 value. The latency (for live streams) is the difference between the 14 currently inputted buffer timestamp (can be stored in the 15 packetizer) and the buffer we're pushing out. 16 This value should be reported/updated (leave a bit of extra margin 17 in addition to the calculated value). 18 19* mpegtsparser 20 * SERIOUS room for improvement performance-wise (see callgrind), 21 mostly related to performance issues mentioned above. 22 23* Random-access seeking 24 * Do minimal parsing of video headers to detect keyframes and use 25 that to compute the keyframe intervals. Use that interval to offset 26 the seek position in order to maximize the chance of pushing out the 27 requested frames. 28 29 30Synchronization, Scheduling and Timestamping 31-------------------------------------------- 32 33 A mpeg-ts demuxer can be used in a variety of situations: 34 * lives streaming over DVB, UDP, RTP,.. 35 * play-as-you-download like HTTP Live Streaming or UPNP/DLNA 36 * random-access local playback, file, Bluray, ... 37 38 Those use-cases can be categorized in 3 different categories: 39 * Push-based scheduling with live sources [0] 40 * Push-based scheduling with non-live sources 41 * Pull-based scheduling with fast random-access 42 43 Due to the nature of timing within the mpeg-ts format, we need to 44pay extra attention to the outgoing NEWSEGMENT event and buffer 45timestamps in order to guarantee proper playback and synchronization 46of the stream. 47 48 In the following, 'timestamps' correspond to GStreamer 49 buffer/segment values. The mpeg-ts PCR/DTS/PTS values are indicated 50 with their actual name. 51 52 1) Live push-based scheduling 53 54 The NEWSEGMENT event will be in time format and is forwarded as is, 55 and the values are cached locally. 56 57 Since the clock is running when the upstream buffers are captured, 58 the outgoing buffer timestamps need to correspond to the incoming 59 buffer timestamp values. 60 61 => mpegtspacketizer keeps track of PCR and input timestamp and 62 extrapolates a clock skew using the EPTLA algorithm. 63 64 => The outgoing buffers will be timestamped with their PTS values 65 (overflow corrected) corrected by that calculated clock skew. 66 67 A latency is introduced between the time the buffer containing the 68 first bit of a Access Unit is received in the demuxer and the moment 69 the demuxer pushed out the buffer corresponding to that Access Unit. 70 71 => That latency needs to be reported. 72 73 According to the ISO/IEC 13818-1:2007 specifications, D.0.1 Timing 74 mode, the "coded audio and video that represent sound and pictures 75 that are to be presented simultaneously may be separated in time 76 within the coded bit stream by ==>as much as one second<==" 77 78 => The algorithm to calculate the latency should take that into 79 account. 80 81 82 2) Non-live push-based scheduling 83 84 If the upstream NEWSEGMENT is in time format, the NEWSEGMENT event 85 is forwarded as is, and the values are cached locally. 86 87 If upstream does provide a NEWSEGMENT in another format, we need to 88 compute one by taking the default values: 89 start : 0 90 stop : GST_CLOCK_TIME_NONE 91 time : 0 92 93 Since no prerolling is happening downstream and the incoming buffers 94 do not have capture timestamps, we need to ensure the first buffer 95 we push out corresponds to the base segment start running time. 96 97 => The packetizer keeps track of PCR locations and offsets in 98 addition to the clock skew (in the case of upstream buffers 99 being timestamped, which is the case for HLS). 100 101 => The demuxer indicates to the packetizer when he sees the 102 'beginning' of the program (i.e. the first valid PAT/PMT 103 combination). The packetizer will then use that location as 104 "timestamp 0", or "reference position/PCR". 105 106 => The lowest DTS is passed to the packetizer to be converted to 107 timestamp. That value is computed in the same way as live 108 streams if upstream buffers have timestamps, or will be 109 subtracted from the reference PCR. 110 111 => The outgoing buffers will be timestamped with their PTS values 112 (overflow corrected) adjusted by the packetizer. 113 114 Latency is reported just as with the live use-case. 115 116 117 3) Random access pull-mode 118 119 We do not get a NEWSEGMENT event from upstream, we therefore need to 120 compute the outgoing values. 121 122 => The outgoing values for the newsegment are calculated like for 123 the non-live push-based mode when upstream doesn't provide 124 timestamp'ed buffers. 125 126 => The outgoing buffer timestamps are timestamped with their PTS 127 values (overflow corrected) adjusted by the packetizer. 128 129 130 131[0] When talking about live sources, we mean this in the GStreamer 132definition of live sources, which is to say sources where if we miss 133the capture, we will miss the data to be captured. Sources which do 134internal buffering (like TCP connections or file descriptors) are 135*NOT* live sources. 136