1# Media Data Muxing 2 3You can call the native APIs provided by the AVMuxer module to mux audio and video streams, that is, to store encoded audio and video data to a file in a certain format. 4 5Currently, the following muxer capabilities are supported: 6 7| Muxing Format| Video Codec Type | Audio Codec Type | Cover Type | 8| -------- | --------------------- | ---------------- | -------------- | 9| mp4 | AVC (H.264) <!--RP1--><!--RP1End--> | AAC, MPEG (MP3)| jpeg, png, bmp| 10| m4a | - | AAC | jpeg, png, bmp| 11| mp3 | - | MPEG (MP3) | - | 12| amr | - | AMR (AMR-NB and AMR-WB)| - | 13| wav | - | G.711Mu (pcm-mulaw)| - | 14 15> **NOTE** 16> 17> - When the container format is MP4 and the audio codec type is MPEG (MP3), the sampling rate must be greater than or equal to 16000 Hz. 18> - When the container format is MP4 or M4A and the audio codec type is AAC, the number of audio channels ranges from 1 to 7. 19 20<!--RP2--><!--RP2End--> 21 22**Usage Scenario** 23 24- Video and audio recording 25 26 After you encode audio and video streams, mux them into files. 27 28- Audio and video editing 29 30 After you edit audio and video, mux them into files. 31 32- Audio and video transcoding 33 34 After you transcode audio and video, mux them into files. 35 36## How to Develop 37 38Read [AVMuxer](../../reference/apis-avcodec-kit/_a_v_muxer.md) for the API reference. 39 40> **NOTE** 41> 42> To call the AVMuxer module to write a local file, request the **ohos.permission.READ_MEDIA** and **ohos.permission.WRITE_MEDIA** permissions by following the instructions provided in [Requesting User Authorization](../../security/AccessToken/request-user-authorization.md). 43 44### Linking the Dynamic Library in the CMake Script 45 46``` cmake 47target_link_libraries(sample PUBLIC libnative_media_avmuxer.so) 48target_link_libraries(sample PUBLIC libnative_media_core.so) 49``` 50 51### How to Develop 52 53The following walks you through how to implement the entire process of audio and video muxing. It uses the MP4 format as an example. 54 551. Add the header files. 56 57 ```c++ 58 #include <multimedia/player_framework/native_avmuxer.h> 59 #include <multimedia/player_framework/native_avcodec_base.h> 60 #include <multimedia/player_framework/native_avformat.h> 61 #include <multimedia/player_framework/native_avbuffer.h> 62 #include <fcntl.h> 63 ``` 64 652. Call **OH_AVMuxer_Create()** to create an **OH_AVMuxer** instance. 66 67 ```c++ 68 // Set the muxing format to MP4. 69 OH_AVOutputFormat format = AV_OUTPUT_FORMAT_MPEG_4; 70 // Create a File Descriptor (FD) in read/write mode. 71 int32_t fd = open("test.mp4", O_CREAT | O_RDWR | O_TRUNC, S_IRUSR | S_IWUSR); 72 OH_AVMuxer *muxer = OH_AVMuxer_Create(fd, format); 73 ``` 74 753. (Optional) Call **OH_AVMuxer_SetRotation()** to set the rotation angle. 76 77 ```c++ 78 // Set the rotation angle when a video image needs to be rotated. 79 OH_AVMuxer_SetRotation(muxer, 0); 80 ``` 81 824. Add an audio track. 83 84 **Method 1: Use OH_AVFormat_Create to create the format.** 85 86 ```c++ 87 int audioTrackId = -1; 88 uint8_t *buffer = ...; // Encoding configuration data. If there is no configuration data, leave the parameter unspecified. 89 size_t size =...; // Length of the encoding configuration data. Set this parameter based on project requirements. 90 OH_AVFormat *formatAudio = OH_AVFormat_Create (); // Call OH_AVFormat_Create to create a format. The following showcases how to mux an AAC-LC audio with the sampling rate of 44100 Hz and two audio channels. 91 OH_AVFormat_SetStringValue(formatAudio, OH_MD_KEY_CODEC_MIME, OH_AVCODEC_MIMETYPE_AUDIO_AAC); // Mandatory. 92 OH_AVFormat_SetIntValue(formatAudio, OH_MD_KEY_AUD_SAMPLE_RATE, 44100); // Mandatory. 93 OH_AVFormat_SetIntValue(formatAudio, OH_MD_KEY_AUD_CHANNEL_COUNT, 2); // Mandatory. 94 OH_AVFormat_SetIntValue(formatAudio, OH_MD_KEY_PROFILE, AAC_PROFILE_LC); // Optional. 95 OH_AVFormat_SetBuffer(formatAudio, OH_MD_KEY_CODEC_CONFIG, buffer, size); // Optional. 96 97 int ret = OH_AVMuxer_AddTrack(muxer, &audioTrackId, formatAudio); 98 if (ret != AV_ERR_OK || audioTrackId < 0) { 99 // Failure to add the audio track. 100 } 101 OH_AVFormat_Destroy (formatAudio); // Destroy the format. 102 ``` 103 104 **Method 2: Use OH_AVFormat_CreateAudioFormat to create the format.** 105 106 ```c++ 107 int audioTrackId = -1; 108 uint8_t *buffer = ...; // Encoding configuration data. If there is no configuration data, leave the parameter unspecified. 109 size_t size =...; // Length of the encoding configuration data. Set this parameter based on project requirements. 110 OH_AVFormat *formatAudio = OH_AVFormat_CreateAudioFormat(OH_AVCODEC_MIMETYPE_AUDIO_AAC, 44100, 2); 111 OH_AVFormat_SetIntValue(formatAudio, OH_MD_KEY_PROFILE, AAC_PROFILE_LC); // Optional. 112 OH_AVFormat_SetBuffer(formatAudio, OH_MD_KEY_CODEC_CONFIG, buffer, size); // Optional. 113 114 int ret = OH_AVMuxer_AddTrack(muxer, &audioTrackId, formatAudio); 115 if (ret != AV_ERR_OK || audioTrackId < 0) { 116 // Failure to add the audio track. 117 } 118 OH_AVFormat_Destroy (formatAudio); // Destroy the format. 119 ``` 120 1215. Add a video track. 122 123 **Method 1: Use OH_AVFormat_Create to create the format.** 124 125 ```c++ 126 int videoTrackId = -1; 127 uint8_t *buffer = ...; // Encoding configuration data. If there is no configuration data, leave the parameter unspecified. 128 size_t size =...; // Length of the encoding configuration data. Set this parameter based on project requirements. 129 OH_AVFormat *formatVideo = OH_AVFormat_Create(); 130 OH_AVFormat_SetStringValue(formatVideo, OH_MD_KEY_CODEC_MIME, OH_AVCODEC_MIMETYPE_VIDEO_AVC); // Mandatory. 131 OH_AVFormat_SetIntValue(formatVideo, OH_MD_KEY_WIDTH, 1280); // Mandatory. 132 OH_AVFormat_SetIntValue(formatVideo, OH_MD_KEY_HEIGHT, 720); // Mandatory. 133 OH_AVFormat_SetBuffer(formatVideo, OH_MD_KEY_CODEC_CONFIG, buffer, size); // Optional 134 135 int ret = OH_AVMuxer_AddTrack(muxer, &videoTrackId, formatVideo); 136 if (ret != AV_ERR_OK || videoTrackId < 0) { 137 // Failure to add the video track. 138 } 139 OH_AVFormat_Destroy(formatVideo); // Destroy the format. 140 ``` 141 142 **Method 2: Use OH_AVFormat_CreateVideoFormat to create the format.** 143 144 ```c++ 145 int videoTrackId = -1; 146 uint8_t *buffer = ...; // Encoding configuration data. If there is no configuration data, leave the parameter unspecified. 147 size_t size =...; // Length of the encoding configuration data. Set this parameter based on project requirements. 148 OH_AVFormat *formatVideo = OH_AVFormat_CreateVideoFormat(OH_AVCODEC_MIMETYPE_VIDEO_AVC, 1280, 720); 149 OH_AVFormat_SetBuffer(formatVideo, OH_MD_KEY_CODEC_CONFIG, buffer, size); // Optional 150 151 int ret = OH_AVMuxer_AddTrack(muxer, &videoTrackId, formatVideo); 152 if (ret != AV_ERR_OK || videoTrackId < 0) { 153 // Failure to add the video track. 154 } 155 OH_AVFormat_Destroy(formatVideo); // Destroy the format. 156 ``` 157 1586. Add a cover track. 159 160 **Method 1: Use OH_AVFormat_Create to create the format.** 161 162 ```c++ 163 int coverTrackId = -1; 164 OH_AVFormat *formatCover = OH_AVFormat_Create(); 165 OH_AVFormat_SetStringValue(formatCover, OH_MD_KEY_CODEC_MIME, OH_AVCODEC_MIMETYPE_IMAGE_JPG); 166 OH_AVFormat_SetIntValue(formatCover, OH_MD_KEY_WIDTH, 1280); 167 OH_AVFormat_SetIntValue(formatCover, OH_MD_KEY_HEIGHT, 720); 168 169 int ret = OH_AVMuxer_AddTrack(muxer, &coverTrackId, formatCover); 170 if (ret != AV_ERR_OK || coverTrackId < 0) { 171 // Failure to add the cover track. 172 } 173 OH_AVFormat_Destroy(formatCover); // Destroy the format. 174 ``` 175 176 **Method 2: Use OH_AVFormat_CreateVideoFormat to create the format.** 177 178 ```c++ 179 int coverTrackId = -1; 180 OH_AVFormat *formatCover = OH_AVFormat_CreateVideoFormat(OH_AVCODEC_MIMETYPE_IMAGE_JPG, 1280, 720); 181 182 int ret = OH_AVMuxer_AddTrack(muxer, &coverTrackId, formatCover); 183 if (ret != AV_ERR_OK || coverTrackId < 0) { 184 // Failure to add the cover track. 185 } 186 OH_AVFormat_Destroy(formatCover); // Destroy the format. 187 ``` 188 1897. Call **OH_AVMuxer_Start()** to start muxing. 190 191 ```c++ 192 // Call Start() to write the file header. After this API is called, you cannot set media parameters or add tracks. 193 if (OH_AVMuxer_Start(muxer) != AV_ERR_OK) { 194 // Exception handling. 195 } 196 ``` 197 1988. Call **OH_AVMuxer_WriteSampleBuffer()** to write data. The encapsulated data includes video, audio, and cover data. 199 200 ```c++ 201 // Data can be written only after Start() is called. 202 int size = ...; 203 OH_AVBuffer *sample = OH_AVBuffer_Create(size); // Create an AVBuffer instance. 204 // Write data to the sample buffer by using OH_AVBuffer_GetAddr(sample). For details, see the usage of OH_AVBuffer. 205 // Mux the cover. One image must be written at a time. 206 207 // Set buffer information. 208 OH_AVCodecBufferAttr info = {0}; 209 info.pts =...; // Playback start time of the current data, in microseconds. The relative time is used. 210 info.size = size; // Length of the current data. 211 info.offset = 0; // Offset. Generally, the value is 0. 212 info.flags |= AVCODEC_BUFFER_FLAGS_SYNC_FRAME; // Flag of the current data. For details, see OH_AVCodecBufferFlags. 213 info.flags |= AVCODEC_BUFFER_FLAGS_CODEC_DATA; // The AVC in annex-b format contains the codec configuration flag. 214 OH_AVBuffer_SetBufferAttr(sample, &info); // Set the buffer attribute. 215 int trackId = audioTrackId; // Select the audio or video track to be written. 216 217 int ret = OH_AVMuxer_WriteSampleBuffer(muxer, trackId, sample); 218 if (ret != AV_ERR_OK) { 219 // Exception handling. 220 } 221 ``` 222 2239. Call **OH_AVMuxer_Stop()** to stop muxing. 224 225 ```c++ 226 // Call Stop() to write the file trailer. After this API is called, you cannot write media data. 227 if (OH_AVMuxer_Stop(muxer) != AV_ERR_OK) { 228 // Exception handling. 229 } 230 ``` 231 23210. Call **OH_AVMuxer_Destroy()** to release the instance. Do not repeatedly destroy the instance. Otherwise, the program may crash. 233 234 ```c++ 235 if (OH_AVMuxer_Destroy(muxer) != AV_ERR_OK) { 236 // Exception handling. 237 } 238 muxer = NULL; 239 close(fd); // Close the FD. 240 ``` 241