1# Audio and Video Muxing (C/C++) 2 3You can call the native APIs provided by the **AVMuxer** module to mux audio and video streams, that is, to store encoded audio and video data to a file in a certain format. 4 5Currently, the following muxer capabilities are supported: 6 7| Muxing Format| Video Codec Type | Audio Codec Type | Cover Type | 8| -------- | --------------------- | ---------------- | -------------- | 9| mp4 | AVC (H.264) | AAC, MPEG (MP3)| jpeg, png, bmp| 10| m4a | | AAC | jpeg, png, bmp| 11 12**Usage Scenario** 13 14- Video and audio recording 15 16 After you encode audio and video streams, mux them into files. 17 18- Audio and video editing 19 20 After you edit audio and video, mux them into files. 21 22- Audio and video transcoding 23 24 After you transcode audio and video, mux them into files. 25 26## How to Develop 27 28Read [AVMuxer](../reference/apis-avcodec-kit/_a_v_muxer.md) for the API reference. 29 30> **NOTE** 31> 32> To call the muxer APIs to write a local file, request the **ohos.permission.READ_MEDIA** and **ohos.permission.WRITE_MEDIA** permissions by following the instructions provided in [Requesting User Authorization](../security/AccessToken/request-user-authorization.md). 33 34### Linking the Dynamic Library in the CMake Script 35 36``` cmake 37target_link_libraries(sample PUBLIC libnative_media_avmuxer.so) 38target_link_libraries(sample PUBLIC libnative_media_core.so) 39``` 40 41### How to Develop 42 43The following walks you through how to implement the entire process of audio and video muxing. It uses the MP4 format as an example. 44 451. Add the header files. 46 47 ```c++ 48 #include <multimedia/player_framework/native_avmuxer.h> 49 #include <multimedia/player_framework/native_avcodec_base.h> 50 #include <multimedia/player_framework/native_avformat.h> 51 #include <multimedia/player_framework/native_avbuffer.h> 52 ``` 53 542. Call **OH_AVMuxer_Create()** to create an **OH_AVMuxer** instance. 55 56 ```c++ 57 // Set the muxing format to MP4. 58 OH_AVOutputFormat format = AV_OUTPUT_FORMAT_MPEG_4; 59 // Create a File Descriptor (FD) in read/write mode. 60 int32_t fd = open("test.mp4", O_CREAT | O_RDWR | O_TRUNC, S_IRUSR | S_IWUSR); 61 OH_AVMuxer *muxer = OH_AVMuxer_Create(fd, format); 62 ``` 63 643. (Optional) Call **OH_AVMuxer_SetRotation()** to set the rotation angle. 65 66 ```c++ 67 // Set the rotation angle when a video image needs to be rotated. 68 OH_AVMuxer_SetRotation(muxer, 0); 69 ``` 70 714. Add an audio track. 72 73 **Method 1: Use OH_AVFormat_Create to create the format.** 74 75 ```c++ 76 int audioTrackId = -1; 77 uint8_t *buffer = ...; // Encoding configuration data. If there is no configuration data, leave the parameter unspecified. 78 size_t size =...; // Length of the encoding configuration data. Set this parameter based on project requirements. 79 OH_AVFormat *formatAudio = OH_AVFormat_Create(); 80 OH_AVFormat_SetStringValue(formatAudio, OH_MD_KEY_CODEC_MIME, OH_AVCODEC_MIMETYPE_AUDIO_AAC); // Mandatory. 81 OH_AVFormat_SetIntValue(formatAudio, OH_MD_KEY_AUD_SAMPLE_RATE, 44100); // Mandatory. 82 OH_AVFormat_SetIntValue(formatAudio, OH_MD_KEY_AUD_CHANNEL_COUNT, 2); // Mandatory. 83 OH_AVFormat_SetIntValue(formatAudio, OH_MD_KEY_PROFILE, AAC_PROFILE_LC); // Optional. 84 OH_AVFormat_SetBuffer(formatAudio, OH_MD_KEY_CODEC_CONFIG, buffer, size); // Optional. 85 86 int ret = OH_AVMuxer_AddTrack(muxer, &audioTrackId, formatAudio); 87 if (ret != AV_ERR_OK || audioTrackId < 0) { 88 // Failure to add the audio track. 89 } 90 OH_AVFormat_Destroy (formatAudio); // Destroy the format. 91 ``` 92 93 **Method 2: Use OH_AVFormat_CreateAudioFormat to create the format.** 94 95 ```c++ 96 int audioTrackId = -1; 97 uint8_t *buffer = ...; // Encoding configuration data. If there is no configuration data, leave the parameter unspecified. 98 size_t size =...; // Length of the encoding configuration data. Set this parameter based on project requirements. 99 OH_AVFormat *formatAudio = OH_AVFormat_CreateAudioFormat(OH_AVCODEC_MIMETYPE_AUDIO_AAC, 44100, 2); 100 OH_AVFormat_SetIntValue(formatAudio, OH_MD_KEY_PROFILE, AAC_PROFILE_LC); // Optional. 101 OH_AVFormat_SetBuffer(formatAudio, OH_MD_KEY_CODEC_CONFIG, buffer, size); // Optional. 102 103 int ret = OH_AVMuxer_AddTrack(muxer, &audioTrackId, formatAudio); 104 if (ret != AV_ERR_OK || audioTrackId < 0) { 105 // Failure to add the audio track. 106 } 107 OH_AVFormat_Destroy (formatAudio); // Destroy the format. 108 ``` 109 1105. Add a video track. 111 112 **Method 1: Use OH_AVFormat_Create to create the format.** 113 114 ```c++ 115 int videoTrackId = -1; 116 uint8_t *buffer = ...; // Encoding configuration data. If there is no configuration data, leave the parameter unspecified. 117 size_t size =...; // Length of the encoding configuration data. Set this parameter based on project requirements. 118 OH_AVFormat *formatVideo = OH_AVFormat_Create(); 119 OH_AVFormat_SetStringValue(formatVideo, OH_MD_KEY_CODEC_MIME, OH_AVCODEC_MIMETYPE_VIDEO_AVC); // Mandatory. 120 OH_AVFormat_SetIntValue(formatVideo, OH_MD_KEY_WIDTH, 1280); // Mandatory. 121 OH_AVFormat_SetIntValue(formatVideo, OH_MD_KEY_HEIGHT, 720); // Mandatory. 122 OH_AVFormat_SetBuffer(formatVideo, OH_MD_KEY_CODEC_CONFIG, buffer, size); // Optional 123 124 int ret = OH_AVMuxer_AddTrack(muxer, &videoTrackId, formatVideo); 125 if (ret != AV_ERR_OK || videoTrackId < 0) { 126 // Failure to add the video track. 127 } 128 OH_AVFormat_Destroy(formatVideo); // Destroy the format. 129 ``` 130 131 **Method 2: Use OH_AVFormat_CreateVideoFormat to create the format.** 132 133 ```c++ 134 int videoTrackId = -1; 135 uint8_t *buffer = ...; // Encoding configuration data. If there is no configuration data, leave the parameter unspecified. 136 size_t size =...; // Length of the encoding configuration data. Set this parameter based on project requirements. 137 OH_AVFormat *formatVideo = OH_AVFormat_CreateVideoFormat(OH_AVCODEC_MIMETYPE_VIDEO_AVC, 1280, 720); 138 OH_AVFormat_SetBuffer(formatVideo, OH_MD_KEY_CODEC_CONFIG, buffer, size); // Optional 139 140 int ret = OH_AVMuxer_AddTrack(muxer, &videoTrackId, formatVideo); 141 if (ret != AV_ERR_OK || videoTrackId < 0) { 142 // Failure to add the video track. 143 } 144 OH_AVFormat_Destroy(formatVideo); // Destroy the format. 145 ``` 146 1476. Add a cover track. 148 149 **Method 1: Use OH_AVFormat_Create to create the format.** 150 151 ```c++ 152 int coverTrackId = -1; 153 OH_AVFormat *formatCover = OH_AVFormat_Create(); 154 OH_AVFormat_SetStringValue(formatCover, OH_MD_KEY_CODEC_MIME, OH_AVCODEC_MIMETYPE_IMAGE_JPG); 155 OH_AVFormat_SetIntValue(formatCover, OH_MD_KEY_WIDTH, 1280); 156 OH_AVFormat_SetIntValue(formatCover, OH_MD_KEY_HEIGHT, 720); 157 158 int ret = OH_AVMuxer_AddTrack(muxer, &coverTrackId, formatCover); 159 if (ret != AV_ERR_OK || coverTrackId < 0) { 160 // Failure to add the cover track. 161 } 162 OH_AVFormat_Destroy(formatCover); // Destroy the format. 163 ``` 164 165 **Method 2: Use OH_AVFormat_CreateVideoFormat to create the format.** 166 167 ```c++ 168 int coverTrackId = -1; 169 OH_AVFormat *formatCover = OH_AVFormat_CreateVideoFormat(OH_AVCODEC_MIMETYPE_IMAGE_JPG, 1280, 720); 170 171 int ret = OH_AVMuxer_AddTrack(muxer, &coverTrackId, formatCover); 172 if (ret != AV_ERR_OK || coverTrackId < 0) { 173 // Failure to add the cover track. 174 } 175 OH_AVFormat_Destroy(formatCover); // Destroy the format. 176 ``` 177 1787. Call **OH_AVMuxer_Start()** to start muxing. 179 180 ```c++ 181 // Call Start() to write the file header. After this API is called, you cannot set media parameters or add tracks. 182 if (OH_AVMuxer_Start(muxer) != AV_ERR_OK) { 183 // Exception handling. 184 } 185 ``` 186 1878. Call **OH_AVMuxer_WriteSampleBuffer()** to write data. 188 189 including video, audio, and cover data. 190 191 ```c++ 192 // Data can be written only after Start() is called. 193 int size = ...; 194 OH_AVBuffer *sample = OH_AVBuffer_Create (size); // Create an AVBuffer instance. 195 // Write data to the sample buffer by using OH_AVBuffer_GetAddr(sample). For details, see the usage of OH_AVBuffer. 196 // Mux the cover. One image must be written at a time. 197 198 // Set buffer information. 199 OH_AVCodecBufferAttr info; 200 info.pts =...; // Playback start time of the current data, in microseconds. The relative time is used. 201 info.size = size; // Length of the current data. 202 info.offset = 0; // Offset. Generally, the value is 0. 203 info.flags |= AVCODEC_BUFFER_FLAGS_SYNC_FRAME; // Flag of the current data. For details, see OH_AVCodecBufferFlags. 204 info.flags |= AVCODEC_BUFFER_FLAGS_CODEC_DATA; // The AVC in annex-b format contains the codec configuration flag. 205 OH_AVBuffer_SetBufferAttr(sample, &info); // Set the buffer attribute. 206 int trackId = audioTrackId; // Select the track to be written. 207 208 int ret = OH_AVMuxer_WriteSampleBuffer(muxer, trackId, sample); 209 if (ret != AV_ERR_OK) { 210 // Exception handling. 211 } 212 ``` 213 2149. Call **OH_AVMuxer_Stop()** to stop muxing. 215 216 ```c++ 217 // Call Stop() to write the file trailer. After this API is called, you cannot write media data. 218 if (OH_AVMuxer_Stop(muxer) != AV_ERR_OK) { 219 // Exception handling. 220 } 221 ``` 222 22310. Call **OH_AVMuxer_Destroy()** to release the instance. 224 225 ```c++ 226 if (OH_AVMuxer_Destroy(muxer) != AV_ERR_OK) { 227 // Exception handling. 228 } 229 muxer = NULL; 230 close(fd); // Close the FD. 231 ``` 232