1# Audio Encoding (C/C++) 2 3You can call the native APIs provided by the audio codec module to encode audio, that is, to compress audio PCM data into a desired format. 4 5PCM data can be from any source. For example, you can use a microphone to record audio data or import edited PCM data. After audio encoding, you can output streams in the desired format and encapsulate the streams into a target file. 6 7Currently, the following encoding capabilities are supported: 8 9| Container Specification| Audio Encoding Type | 10| -------- | :--------------- | 11| mp4 | AAC, FLAC | 12| m4a | AAC | 13| flac | FLAC | 14| aac | AAC | 15| amr | AMR (AMR-NB and AMR-WB)| 16| raw | G711mu | 17 18**Usage Scenario** 19 20- Audio recording 21 22 Record and pass in PCM data, and encode the data into streams in the desired format. 23- Audio editing 24 25 Export edited PCM data, and encode the data into streams in the desired format. 26 27## How to Develop 28 29Read [Audio Codec](../reference/apis-avcodec-kit/_audio_codec.md) for the API reference. 30 31Refer to the code snippet below to complete the entire audio encoding process, including creating an encoder, setting encoding parameters (such as the sampling rate, bit rate, and number of audio channels), and starting, refreshing, resetting, and destroying the encoder. 32 33During application development, you must call the APIs in the defined sequence. Otherwise, an exception or undefined behavior may occur. 34 35The figure below shows the call relationship of audio encoding. 36 37 38 39### Linking the Dynamic Library in the CMake Script 40 41```cmake 42target_link_libraries(sample PUBLIC libnative_media_codecbase.so) 43target_link_libraries(sample PUBLIC libnative_media_core.so) 44target_link_libraries(sample PUBLIC libnative_media_acodec.so) 45``` 46 47### How to Develop 48 491. Add the header files. 50 51 ```cpp 52 #include <multimedia/player_framework/native_avcodec_audiocodec.h> 53 #include <multimedia/native_audio_channel_layout.h> 54 #include <multimedia/player_framework/native_avcapability.h> 55 #include <multimedia/player_framework/native_avcodec_base.h> 56 #include <multimedia/player_framework/native_avformat.h> 57 #include <multimedia/player_framework/native_avbuffer.h> 58 ``` 59 602. Create an encoder instance. In the code snippet below, **OH_AVCodec *** is the pointer to the encoder instance created. 61 62 You can create an encoder by name or MIME type. 63 64 ```cpp 65 // Namespace of the C++ standard library. 66 using namespace std; 67 // Create an encoder by name. 68 OH_AVCapability *capability = OH_AVCodec_GetCapability(OH_AVCODEC_MIMETYPE_AUDIO_AAC, true); 69 const char *name = OH_AVCapability_GetName(capability); 70 OH_AVCodec *audioEnc_ = OH_AudioCodec_CreateByName(name); 71 ``` 72 73 ```cpp 74 // Specify whether encoding is used. The value **true** means encoding. 75 bool isEncoder = true; 76 // Create an encoder by MIME type. 77 OH_AVCodec *audioEnc_ = OH_AudioCodec_CreateByMime(OH_AVCODEC_MIMETYPE_AUDIO_AAC, isEncoder); 78 ``` 79 80 ```cpp 81 // Initialize the queues. 82 class AEncBufferSignal { 83 public: 84 std::mutex inMutex_; 85 std::mutex outMutex_; 86 std::mutex startMutex_; 87 std::condition_variable inCond_; 88 std::condition_variable outCond_; 89 std::condition_variable startCond_; 90 std::queue<uint32_t> inQueue_; 91 std::queue<uint32_t> outQueue_; 92 std::queue<OH_AVBuffer *> inBufferQueue_; 93 std::queue<OH_AVBuffer *> outBufferQueue_; 94 }; 95 AEncBufferSignal *signal_; 96 ``` 97 983. Call **OH_AudioCodec_RegisterCallback()** to register callback functions. 99 100 Register the **OH_AVCodecCallback** struct that defines the following callback function pointers: 101 102 - **OH_AVCodecOnError**, a callback used to report a codec operation error 103 - **OH_AVCodecOnStreamChanged**, a callback used to report a codec stream change, for example, audio channel change 104 - **OH_AVCodecOnNeedInputBuffer**, a callback used to report input data required, which means that the encoder is ready for receiving PCM data 105 - **OH_AVCodecOnNewOutputBuffer**, a callback used to report output data generated, which means that encoding is complete 106 107 You need to process the callback functions to ensure that the encoder runs properly. 108 109 ```cpp 110 // Implement the OH_AVCodecOnError callback function. 111 static void OnError(OH_AVCodec *codec, int32_t errorCode, void *userData) 112 { 113 (void)codec; 114 (void)errorCode; 115 (void)userData; 116 } 117 // Implement the OH_AVCodecOnStreamChanged callback function. 118 static void OnOutputFormatChanged(OH_AVCodec *codec, OH_AVFormat *format, void *userData) 119 { 120 (void)codec; 121 (void)format; 122 (void)userData; 123 } 124 // Implement the OH_AVCodecOnNeedInputBuffer callback function. 125 static void OnInputBufferAvailable(OH_AVCodec *codec, uint32_t index, OH_AVBuffer *data, void *userData) 126 { 127 (void)codec; 128 // The input stream is sent to the InputBuffer queue. 129 AEncBufferSignal *signal = static_cast<AEncBufferSignal *>(userData); 130 unique_lock<mutex> lock(signal->inMutex_); 131 signal->inQueue_.push(index); 132 signal->inBufferQueue_.push(data); 133 signal->inCond_.notify_all(); 134 } 135 // Implement the OH_AVCodecOnNewOutputBuffer callback function. 136 static void OnOutputBufferAvailable(OH_AVCodec *codec, uint32_t index, OH_AVBuffer *data, void *userData) 137 { 138 (void)codec; 139 // The index of the output buffer is sent to OutputQueue_. 140 // The encoded data is sent to the outBuffer queue. 141 AEncBufferSignal *signal = static_cast<AEncBufferSignal *>(userData); 142 unique_lock<mutex> lock(signal->outMutex_); 143 signal->outQueue_.push(index); 144 signal->outBufferQueue_.push(data); 145 if (attr) { 146 signal->attrQueue_.push(*attr); 147 } 148 } 149 signal_ = new AEncBufferSignal(); 150 OH_AVCodecCallback cb_ = {&OnError, &OnOutputFormatChanged, &OnInputBufferAvailable, &OnOutputBufferAvailable}; 151 // Set the asynchronous callbacks. 152 int32_t ret = OH_AudioCodec_RegisterCallback(audioEnc_, cb_, signal_); 153 if (ret != AV_ERR_OK) { 154 // Exception handling. 155 } 156 ``` 157 1584. Call **OH_AudioCodec_Configure** to configure the encoder. 159 160 The following options are mandatory: sampling rate, bit rate, number of audio channels, audio channel type, and bit depth. 161 162 The maximum input length is optional. 163 164 For FLAC encoding, the compliance level and sampling precision are also mandatory. 165 166 The following provides the AAC invoking process. 167 168 ```cpp 169 int32_t ret; 170 // (Mandatory) Configure the audio sampling rate. 171 constexpr uint32_t DEFAULT_SAMPLERATE = 44100; 172 // (Mandatory) Configure the audio bit rate. 173 constexpr uint64_t DEFAULT_BITRATE = 32000; 174 // (Mandatory) Configure the number of audio channels. 175 constexpr uint32_t DEFAULT_CHANNEL_COUNT = 2; 176 // (Mandatory) Configure the audio channel type. 177 constexpr AudioChannelLayout CHANNEL_LAYOUT = AudioChannelLayout::STEREO; 178 // (Mandatory) Configure the audio bit depth. Only SAMPLE_F32P is available for AAC encoding. 179 constexpr OH_BitsPerSample SAMPLE_FORMAT = OH_BitsPerSample::SAMPLE_F32LE; 180 // Configure the audio compliance level. The default value is 0, and the value ranges from -2 to 2. 181 constexpr int32_t COMPLIANCE_LEVEL = 0; 182 // (Mandatory) Configure the audio sampling precision. SAMPLE_S16LE, SAMPLE_S24LE, and SAMPLE_S32LE are available. 183 constexpr OH_BitsPerSample BITS_PER_CODED_SAMPLE = OH_BitsPerSample::SAMPLE_S24LE; 184 // (Optional) Configure the maximum input length. 185 constexpr uint32_t DEFAULT_MAX_INPUT_SIZE = 1024 * DEFAULT_CHANNEL_COUNT * sizeof(float); // aac 186 OH_AVFormat *format = OH_AVFormat_Create(); 187 // Set the format. 188 OH_AVFormat_SetIntValue(format,OH_MD_KEY_AUD_CHANNEL_COUNT, DEFAULT_CHANNEL_COUNT); 189 OH_AVFormat_SetIntValue(format,OH_MD_KEY_AUD_SAMPLE_RATE, DEFAULT_SAMPLERATE); 190 OH_AVFormat_SetLongValue(format,OH_MD_KEY_BITRATE, DEFAULT_BITRATE); 191 OH_AVFormat_SetIntValue(format, OH_MD_KEY_AUDIO_SAMPLE_FORMAT, SAMPLE_FORMAT); 192 OH_AVFormat_SetLongValue(format,OH_MD_KEY_CHANNEL_LAYOUT, CHANNEL_LAYOUT); 193 OH_AVFormat_SetIntValue(format,OH_MD_KEY_MAX_INPUT_SIZE, DEFAULT_MAX_INPUT_SIZE); 194 // Configure the encoder. 195 ret = OH_AudioCodec_Configure(audioEnc_, format); 196 if (ret != AV_ERR_OK) { 197 // Exception handling. 198 } 199 ``` 200 201 The following provides the FLAC invoking process. 202 203 ```cpp 204 int32_t ret; 205 // (Mandatory) Configure the audio sampling rate. 206 constexpr uint32_t DEFAULT_SMAPLERATE = 44100; 207 // (Mandatory) Configure the audio bit rate. 208 constexpr uint64_t DEFAULT_BITRATE = 261000; 209 // (Mandatory) Configure the number of audio channels. 210 constexpr uint32_t DEFAULT_CHANNEL_COUNT = 2; 211 // (Mandatory) Configure the audio channel type. 212 constexpr AudioChannelLayout CHANNEL_LAYOUT = AudioChannelLayout::STEREO; 213 // (Mandatory) Configure the audio bit depth. Only SAMPLE_S16LE and SAMPLE_S32LE are available for FLAC encoding. 214 constexpr OH_BitsPerSample SAMPLE_FORMAT = OH_BitsPerSample::SAMPLE_S32LE; 215 // Configure the audio compliance level. The default value is 0, and the value ranges from -2 to 2. 216 constexpr int32_t COMPLIANCE_LEVEL = 0; 217 // (Mandatory) Configure the audio sampling precision. SAMPLE_S16LE, SAMPLE_S24LE, and SAMPLE_S32LE are available. 218 constexpr OH_BitsPerSample BITS_PER_CODED_SAMPLE = OH_BitsPerSample::SAMPLE_S24LE; 219 OH_AVFormat *format = OH_AVFormat_Create(); 220 // Set the format. 221 OH_AVFormat_SetIntValue(format, OH_MD_KEY_AUD_CHANNEL_COUNT, DEFAULT_CHANNEL_COUNT); 222 OH_AVFormat_SetIntValue(format, OH_MD_KEY_AUD_SAMPLE_RATE, DEFAULT_SMAPLERATE); 223 OH_AVFormat_SetLongValue(format, OH_MD_KEY_BITRATE, DEFAULT_BITRATE); 224 OH_AVFormat_SetIntValue(format, OH_MD_KEY_BITS_PER_CODED_SAMPLE, BITS_PER_CODED_SAMPLE); 225 OH_AVFormat_SetIntValue(format, OH_MD_KEY_AUDIO_SAMPLE_FORMAT, SAMPLE_FORMAT); 226 OH_AVFormat_SetLongValue(format, OH_MD_KEY_CHANNEL_LAYOUT, CHANNEL_LAYOUT); 227 OH_AVFormat_SetLongValue(format, OH_MD_KEY_COMPLIANCE_LEVEL, COMPLIANCE_LEVEL); 228 // Configure the encoder. 229 ret = OH_AudioCodec_Configure(audioEnc_, format); 230 if (ret != AV_ERR_OK) { 231 // Exception handling. 232 } 233 ``` 234 2355. Call **OH_AudioCodec_Prepare()** to prepare internal resources for the encoder. 236 237 ```cpp 238 ret = OH_AudioCodec_Prepare(audioDec_); 239 if (ret != AV_ERR_OK) { 240 // Exception handling. 241 } 242 ``` 243 2446. Call **OH_AudioCodec_Start()** to start the encoder. 245 246 ```c++ 247 unique_ptr<ifstream> inputFile_ = make_unique<ifstream>(); 248 unique_ptr<ofstream> outFile_ = make_unique<ofstream>(); 249 // Open the path of the binary file to be encoded. 250 inputFile_->open(inputFilePath.data(), ios::in | ios::binary); 251 // Configure the path of the output file. 252 outFile_->open(outputFilePath.data(), ios::out | ios::binary); 253 // Start encoding. 254 ret = OH_AudioCodec_Start(audioEnc_); 255 if (ret != AV_ERR_OK) { 256 // Exception handling. 257 } 258 ``` 259 2607. Call **OH_AudioCodec_PushInputData()** to write the data to encode. 261 262 To indicate the End of Stream (EOS), pass in the **AVCODEC_BUFFER_FLAGS_EOS** flag. 263 264 For AAC encoding, **FRAME_SIZE** (number of sampling points) is fixed at **1024**. 265 266 For FLAC encoding, set **FRAME_SIZE** based on the table below. 267 268 | Sampling Rate| FRAME_SIZE| 269 | :----: | :----: | 270 | 8000 | 576 | 271 | 16000 | 1152 | 272 | 22050 | 2304 | 273 | 24000 | 2304 | 274 | 32000 | 2304 | 275 | 44100 | 4608 | 276 | 48000 | 4608 | 277 | 88200 | 8192 | 278 | 96000 | 8192 | 279 280 > **NOTE** 281 > 282 > If **FRAME_SIZE** is not set to **1024** for AAC encoding, an error code is returned. In the case of FLAC encoding, if **FRAME_SIZE** is set to a value greater than the value listed in the table for a given sampling rate, an error code is returned; if **FRAME_SIZE** is set to a value less than the value listed, the encoded file may be damaged. 283 284 ```c++ 285 constexpr int32_t FRAME_SIZE = 1024; // aac 286 constexpr int32_t DEFAULT_CHANNEL_COUNT = 2; 287 constexpr int32_t INPUT_FRAME_BYTES = DEFAULT_CHANNEL_COUNT * FRAME_SIZE * sizeof(float); // aac 288 uint32_t index = signal_->inQueue_.front(); 289 auto buffer = signal_->inBufferQueue_.front(); 290 OH_AVCodecBufferAttr attr = {0}; 291 if (!inputFile_->eof()) { 292 inputFile_->read((char *)OH_AVBuffer_GetAddr(buffer), INPUT_FRAME_BYTES); 293 attr.size = INPUT_FRAME_BYTES; 294 attr.flags = AVCODEC_BUFFER_FLAGS_NONE; 295 } else { 296 attr.size = 0; 297 attr.flags = AVCODEC_BUFFER_FLAGS_EOS; 298 } 299 OH_AVBuffer_SetBufferAttr(buffer, &attr); 300 // Send the data to the input queue for encoding. The index is the subscript of the queue. 301 ret = OH_AudioCodec_PushInputBuffer(audioEnc_, index); 302 if (ret != AV_ERR_OK) { 303 // Exception handling. 304 } 305 ``` 306 3078. Call **OH_AudioCodec_FreeOutputBuffer()** to output the encoded stream. 308 309 ```c++ 310 uint32_t index = signal_->outQueue_.front(); 311 OH_AVBuffer *avBuffer = signal_->outBufferQueue_.front(); 312 // Obtain the buffer attributes. 313 OH_AVCodecBufferAttr attr = {0}; 314 ret = OH_AVBuffer_GetBufferAttr(avBuffer, &attr); 315 if (ret != AV_ERR_OK) { 316 // Exception handling. 317 } 318 // Write the encoded data (specified by data) to the output file. 319 outputFile_->write(reinterpret_cast<char *>(OH_AVBuffer_GetAddr(avBuffer)), attr.size); 320 // Release the output buffer. 321 ret = OH_AudioCodec_FreeOutputBuffer(audioEnc_, index); 322 if (ret != AV_ERR_OK) { 323 // Exception handling. 324 } 325 if (attr.flags == AVCODEC_BUFFER_FLAGS_EOS) { 326 // End 327 } 328 ``` 329 3309. (Optional) Call **OH_AudioCodec_Flush()** to refresh the encoder. 331 332 After **OH_AudioCodec_Flush()** is called, the current encoding queue is cleared. 333 334 To continue encoding, you must call **OH_AudioCodec_Start()** again. 335 336 You need to call **OH_AudioCodec_Flush()** in the following cases: 337 338 * The EOS of the file is reached. 339 * An error with **OH_AudioCodec_IsValid** set to **true** (indicating that the execution can continue) occurs. 340 341 ```c++ 342 // Refresh the encoder. 343 ret = OH_AudioCodec_Flush(audioEnc_); 344 if (ret != AV_ERR_OK) { 345 // Exception handling. 346 } 347 // Start encoding again. 348 ret = OH_AudioCodec_Start(audioEnc_); 349 if (ret != AV_ERR_OK) { 350 // Exception handling. 351 } 352 ``` 353 35410. (Optional) Call **OH_AudioCodec_Reset()** to reset the encoder. 355 356 After **OH_AudioCodec_Reset()** is called, the encoder returns to the initialized state. To continue encoding, you must call **OH_AudioCodec_Configure()** and then **OH_AudioCodec_Start()**. 357 358 ```c++ 359 // Reset the encoder. 360 ret = OH_AudioCodec_Reset(audioEnc_); 361 if (ret != AV_ERR_OK) { 362 // Exception handling. 363 } 364 // Reconfigure the encoder. 365 ret = OH_AudioCodec_Configure(audioEnc_, format); 366 if (ret != AV_ERR_OK) { 367 // Exception handling. 368 } 369 ``` 370 37111. Call **OH_AudioCodec_Stop()** to stop the encoder. 372 373 ```c++ 374 // Stop the encoder. 375 ret = OH_AudioCodec_Stop(audioEnc_); 376 if (ret != AV_ERR_OK) { 377 // Exception handling. 378 } 379 ``` 380 38112. Call **OH_AudioCodec_Destroy()** to destroy the encoder instance and release resources. 382 383 > **NOTE** 384 > 385 > You only need to call the API once. 386 387 ```c++ 388 // Call OH_AudioCodec_Destroy to destroy the encoder. 389 ret = OH_AudioCodec_Destroy(audioEnc_); 390 if (ret != AV_ERR_OK) { 391 // Exception handling. 392 } else { 393 audioEnc_ = NULL; // The encoder cannot be destroyed repeatedly. 394 } 395 ``` 396
