1# Audio Encoding 2 3You can call the native APIs provided by the **AudioEncoder** module to encode audio, that is, to compress audio PCM data into a desired format. 4 5PCM data can be from any source. For example, you can use a microphone to record audio data or import edited PCM data. After audio encoding, you can output streams in the desired format and encapsulate the streams into a target file. 6 7Currently, the following encoding capabilities are supported: 8 9| Container Specification| Audio Encoding Type| 10| -------- | :----------- | 11| mp4 | AAC, FLAC | 12| m4a | AAC | 13| flac | FLAC | 14| aac | AAC | 15 16**Usage Scenario** 17 18- Audio recording 19 20 Record and pass in PCM data, and encode the data into streams in the desired format. 21- Audio editing 22 23 Export edited PCM data, and encode the data into streams in the desired format. 24 25## How to Develop 26 27Read [AudioEncoder](../reference/native-apis/_audio_encoder.md) for the API reference. 28 29Refer to the code snippet below to complete the entire audio encoding process, including creating an encoder, setting encoding parameters (such as the sampling rate, bit rate, and number of audio channels), and starting, refreshing, resetting, and destroying the encoder. 30 31During application development, you must call the APIs in the defined sequence. Otherwise, an exception or undefined behavior may occur. 32 33For details about the complete code, see [Sample](https://gitee.com/openharmony/multimedia_av_codec/blob/master/test/nativedemo/audio_demo/avcodec_audio_aac_encoder_demo.cpp). 34 35The figure below shows the call relationship of audio encoding. 36 37 38 39### Linking the Dynamic Library in the CMake Script 40``` cmake 41target_link_libraries(sample PUBLIC libnative_media_codecbase.so) 42target_link_libraries(sample PUBLIC libnative_media_core.so) 43target_link_libraries(sample PUBLIC libnative_media_aenc.so) 44``` 45 46### How to Develop 47 481. Create an encoder instance. 49 50 You can create an encoder by name or MIME type. 51 52 ```cpp 53 // Namespace of the C++ standard library. 54 using namespace std; 55 // Create an encoder by name. 56 OH_AVCapability *capability = OH_AVCodec_GetCapability(OH_AVCODEC_MIMETYPE_AUDIO_AAC, true); 57 const char *name = OH_AVCapability_GetName(capability); 58 OH_AVCodec *audioEnc = OH_AudioEncoder_CreateByName(name); 59 ``` 60 ```cpp 61 // Create an encoder by MIME type. 62 OH_AVCodec *audioEnc = OH_AudioEncoder_CreateByMime(OH_AVCODEC_MIMETYPE_AUDIO_AAC); 63 ``` 64 ```cpp 65 // Initialize the queues. 66 class AEncSignal { 67 public: 68 mutex inMutex_; 69 mutex outMutex_; 70 mutex startMutex_; 71 condition_variable inCond_; 72 condition_variable outCond_; 73 condition_variable startCond_; 74 queue<uint32_t> inQueue_; 75 queue<uint32_t> outQueue_; 76 queue<OH_AVMemory *> inBufferQueue_; 77 queue<OH_AVMemory *> outBufferQueue_; 78 queue<OH_AVCodecBufferAttr> attrQueue_; 79 }; 80 AEncSignal *signal_ = new AEncSignal(); 81 ``` 822. Call **OH_AudioEncoder_SetCallback()** to set callback functions. 83 84 Register the **OH_AVCodecAsyncCallback** struct that defines the following callback function pointers: 85 86 - **OH_AVCodecOnError**, a callback used to report a codec operation error 87 - **OH_AVCodecOnStreamChanged**, a callback used to report a codec stream change, for example, audio channel change 88 - **OH_AVCodecOnNeedInputData**, a callback used to report input data required, which means that the encoder is ready for receiving PCM data 89 - **OH_AVCodecOnNewOutputData**, a callback used to report output data generated, which means that encoding is complete 90 91 You need to process the callback functions to ensure that the encoder runs properly. 92 93 ```cpp 94 // Implement the OH_AVCodecOnError callback function. 95 static void OnError(OH_AVCodec *codec, int32_t errorCode, void *userData) 96 { 97 (void)codec; 98 (void)errorCode; 99 (void)userData; 100 } 101 // Implement the OH_AVCodecOnStreamChanged callback function. 102 static void OnStreamChanged(OH_AVCodec *codec, OH_AVFormat *format, void *userData) 103 { 104 (void)codec; 105 (void)format; 106 (void)userData; 107 } 108 // Implement the OH_AVCodecOnNeedInputData callback function. 109 static void OnNeedInputData(OH_AVCodec *codec, uint32_t index, OH_AVMemory *data, void *userData) 110 { 111 (void)codec; 112 // The input stream is sent to the InputBuffer queue. 113 AEncSignal *signal = static_cast<AEncSignal *>(userData); 114 unique_lock<mutex> lock(signal->inMutex_); 115 signal->inQueue_.push(index); 116 signal->inBufferQueue_.push(data); 117 signal->inCond_.notify_all(); 118 } 119 // Implement the OH_AVCodecOnNewOutputData callback function. 120 static void OnNeedOutputData(OH_AVCodec *codec, uint32_t index, OH_AVMemory *data, OH_AVCodecBufferAttr *attr, 121 void *userData) 122 { 123 (void)codec; 124 // The index of the output buffer is sent to OutputQueue_. 125 // The encoded data is sent to the outBuffer queue. 126 AEncSignal *signal = static_cast<AEncSignal *>(userData); 127 unique_lock<mutex> lock(signal->outMutex_); 128 signal->outQueue_.push(index); 129 signal->outBufferQueue_.push(data); 130 if (attr) { 131 signal->attrQueue_.push(*attr); 132 } 133 } 134 signal_ = new AEncSignal(); 135 OH_AVCodecAsyncCallback cb = {&OnError, &OnStreamChanged, &OnNeedInputData, &OnNeedOutputData}; 136 // Set the asynchronous callbacks. 137 int32_t ret = OH_AudioEncoder_SetCallback(audioEnc, cb, signal_); 138 if (ret != AV_ERR_OK) { 139 // Exception handling. 140 } 141 ``` 1423. Call **OH_AudioEncoder_Configure** to configure the encoder. 143 144 The following options are mandatory: sampling rate, bit rate, number of audio channels, audio channel type, and bit depth. The maximum input length is optional. 145 146 For FLAC encoding, the compliance level and sampling precision are also mandatory. 147 148 The following provides the AAC invoking process. 149 ```cpp 150 #include "avcodec_audio_channel_layout.h" 151 152 int32_t ret; 153 // (Mandatory) Configure the audio sampling rate. 154 constexpr uint32_t DEFAULT_SAMPLERATE = 44100; 155 // (Mandatory) Configure the audio bit rate. 156 constexpr uint64_t DEFAULT_BITRATE = 32000; 157 // (Mandatory) Configure the number of audio channels. 158 constexpr uint32_t DEFAULT_CHANNEL_COUNT = 2; 159 // (Mandatory) Configure the audio channel type. 160 constexpr AudioChannelLayout CHANNEL_LAYOUT = AudioChannelLayout::STEREO; 161 // (Mandatory) Configure the audio bit depth. Only SAMPLE_F32P is available for AAC encoding. 162 constexpr OH_BitsPerSample SAMPLE_FORMAT = OH_BitsPerSample::SAMPLE_F32LE; 163 // Configure the audio compliance level. The default value is 0, and the value ranges from -2 to 2. 164 constexpr int32_t COMPLIANCE_LEVEL = 0; 165 // (Mandatory) Configure the audio sampling precision. SAMPLE_S16LE, SAMPLE_S24LE, and SAMPLE_S32LE are available. 166 constexpr uint32_t BITS_PER_CODED_SAMPLE = OH_BitsPerSample::SAMPLE_S24LE; 167 // (Optional) Configure the maximum input length. 168 constexpr uint32_t DEFAULT_MAX_INPUT_SIZE = 1024*DEFAULT_CHANNEL_COUNT *sizeof(float);//aac 169 OH_AVFormat *format = OH_AVFormat_Create(); 170 // Set the format. 171 OH_AVFormat_SetIntValue(format,OH_MD_KEY_AUD_CHANNEL_COUNT,DEFAULT_CHANNEL_COUNT); 172 OH_AVFormat_SetIntValue(format,OH_MD_KEY_AUD_SAMPLE_RATE,DEFAULT_SAMPLERATE); 173 OH_AVFormat_SetLongValue(format,OH_MD_KEY_BITRATE, DEFAULT_BITRATE); 174 OH_AVFormat_SetIntValue(format, OH_MD_KEY_AUDIO_SAMPLE_FORMAT, SAMPLE_FORMAT); 175 OH_AVFormat_SetLongValue(format,OH_MD_KEY_CHANNEL_LAYOUT,CHANNEL_LAYOUT); 176 OH_AVFormat_SetIntValue(format,OH_MD_KEY_MAX_INPUT_SIZE,DEFAULT_MAX_INPUT_SIZE); 177 // Configure the encoder. 178 ret = OH_AudioEncoder_Configure(audioEnc, format); 179 if (ret != AV_ERR_OK) { 180 // Exception handling. 181 } 182 ``` 183 The following provides the FLAC invoking process. 184 ```cpp 185 #include "avcodec_audio_channel_layout.h" 186 187 int32_t ret; 188 // (Mandatory) Configure the audio sampling rate. 189 constexpr uint32_t DEFAULT_SMAPLERATE = 44100; 190 // (Mandatory) Configure the audio bit rate. 191 constexpr uint64_t DEFAULT_BITRATE = 32000; 192 // (Mandatory) Configure the number of audio channels. 193 constexpr uint32_t DEFAULT_CHANNEL_COUNT = 2; 194 // (Mandatory) Configure the audio channel type. 195 constexpr AudioChannelLayout CHANNEL_LAYOUT = AudioChannelLayout::STEREO; 196 // (Mandatory) Configure the audio bit depth. Only SAMPLE_S16LE and SAMPLE_S32LE are available for FLAC encoding. 197 constexpr OH_BitsPerSample SAMPLE_FORMAT = OH_BitsPerSample::SAMPLE_S32LE; 198 // Configure the audio compliance level. The default value is 0, and the value ranges from -2 to 2. 199 constexpr int32_t COMPLIANCE_LEVEL = 0; 200 // (Mandatory) Configure the audio sampling precision. SAMPLE_S16LE, SAMPLE_S24LE, and SAMPLE_S32LE are available. 201 constexpr uint32_t BITS_PER_CODED_SAMPLE = OH_BitsPerSample::SAMPLE_S24LE; 202 OH_AVFormat *format = OH_AVFormat_Create(); 203 // Set the format. 204 OH_AVFormat_SetIntValue(format,OH_MD_KEY_AUD_CHANNEL_COUNT,DEFAULT_CHANNEL_COUNT); 205 OH_AVFormat_SetIntValue(format,OH_MD_KEY_AUD_SAMPLE_RATE,DEFAULT_SMAPLERATE); 206 OH_AVFormat_SetLongValue(format,OH_MD_KEY_BITRATE, DEFAULT_BITRATE); 207 OH_AVFormat_SetIntValue(format, OH_MD_KEY_BITS_PER_CODED_SAMPLE, BITS_PER_CODED_SAMPLE); 208 OH_AVFormat_SetIntValue(format, OH_MD_KEY_AUDIO_SAMPLE_FORMAT, SAMPLE_FORMAT); 209 OH_AVFormat_SetLongValue(format,OH_MD_KEY_CHANNEL_LAYOUT,CHANNEL_LAYOUT); 210 OH_AVFormat_SetLongValue(format, OH_MD_KEY_COMPLIANCE_LEVEL, COMPLIANCE_LEVEL); 211 // Configure the encoder. 212 ret = OH_AudioEncoder_Configure(audioEnc, format); 213 if (ret != AV_ERR_OK) { 214 // Exception handling. 215 } 216 ``` 2174. Call **OH_AudioEncoder_Prepare()** to prepare internal resources for the encoder. 218 219 ```c++ 220 OH_AudioEncoder_Prepare(audioEnc); 221 ``` 2225. Call **OH_AudioEncoder_Start()** to start the encoder. 223 224 ```c++ 225 unique_ptr<ifstream> inputFile_ = make_unique<ifstream>(); 226 unique_ptr<ofstream> outFile_ = make_unique<ofstream>(); 227 // Open the path of the binary file to be encoded. 228 inputFile_->open(inputFilePath.data(), ios::in |ios::binary); 229 // Configure the path of the output file. 230 outFile_->open(outputFilePath.data(), ios::out |ios::binary); 231 // Start encoding. 232 ret = OH_AudioEncoder_Start(audioEnc); 233 if (ret != AV_ERR_OK) { 234 // Exception handling. 235 } 236 ``` 2376. Call **OH_AudioEncoder_PushInputData()** to write the data to encode. 238 239 To indicate the End of Stream (EOS), pass in the **AVCODEC_BUFFER_FLAGS_EOS** flag. 240 241 For AAC encoding, **FRAME_SIZE** (number of sampling points) is fixed at **1024**. 242 243 For FLAC encoding, set **FRAME_SIZE** based on the table below. 244 245 | Sampling Rate| FRAME_SIZE| 246 | :----: | :----: | 247 | 8000 | 576 | 248 | 16000 | 1152 | 249 | 22050 | 2304 | 250 | 24000 | 2304 | 251 | 32000 | 2304 | 252 | 44100 | 4608 | 253 | 48000 | 4608 | 254 | 88200 | 8192 | 255 | 96000 | 8192 | 256 257 **NOTE**: If **FRAME_SIZE** is not set to **1024** for AAC encoding, an error code is returned. In the case of FLAC encoding, if **FRAME_SIZE** is set to a value greater than the value listed in the table for a given sampling rate, an error code is returned; if **FRAME_SIZE** is set to a value less than the value listed, the encoded file may be damaged. 258 259 260 ```c++ 261 constexpr int32_t FRAME_SIZE = 1024; // AAC encoding 262 constexpr int32_t DEFAULT_CHANNEL_COUNT = 2; 263 constexpr int32_t INPUT_FRAME_BYTES = DEFAULT_CHANNEL_COUNT * FRAME_SIZE * sizeof(float); // AAC encoding 264 // Configure the buffer information. 265 OH_AVCodecBufferAttr info; 266 // Set the size, offset, and timestamp. 267 info.size = INPUT_FRAME_BYTES; 268 info.offset = 0; 269 info.flags = AVCODEC_BUFFER_FLAGS_CODEC_DATA; 270 auto buffer = signal_->inBufferQueue_.front(); 271 if (inputFile_->eof()){ 272 info.size = 0; 273 info.flags = AVCODEC_BUFFER_FLAGS_EOS; 274 }else{ 275 inputFile_->read((char *)OH_AVMemory_GetAddr(buffer), INPUT_FRAME_BYTES); 276 } 277 uint32_t index = signal_->inQueue_.front(); 278 // Send the data to the input queue for encoding. The index is the subscript of the queue. 279 int32_t ret = OH_AudioEncoder_PushInputData(audioEnc, index,info); 280 if (ret != AV_ERR_OK) { 281 // Exception handling. 282 } 283 ``` 284 2857. Call **OH_AudioEncoder_FreeOutputData()** to output the encoded stream. 286 287 ```c++ 288 OH_AVCodecBufferAttr attr = signal_->attrQueue_.front(); 289 OH_AVMemory *data = signal_->outBufferQueue_.front(); 290 uint32_t index = signal_->outQueue_.front(); 291 // Write the encoded data (specified by data) to the output file. 292 outFile_->write(reinterpret_cast<char *>(OH_AVMemory_GetAdd(data)), attr.size); 293 // Release the output buffer. 294 ret = OH_AudioEncoder_FreeOutputData(audioEnc, index); 295 if (ret != AV_ERR_OK) { 296 // Exception handling. 297 } 298 if (attr.flags == AVCODEC_BUFFER_FLAGS_EOS) { 299 cout << "decode eos" << endl; 300 isRunning_.store(false); 301 break; 302 } 303 ``` 3048. (Optional) Call **OH_AudioEncoder_Flush()** to refresh the encoder. 305 306 After **OH_AudioEncoder_Flush()** is called, the current encoding queue is cleared. 307 308 To continue encoding, you must call **OH_AudioEncoder_Start()** again. 309 310 You need to call **OH_AudioEncoder_Flush()** in the following cases: 311 312 * The EOS of the file is reached. 313 * An error with **OH_AudioEncoder_IsValid** set to **true** (indicating that the execution can continue) occurs. 314 315 ```c++ 316 // Refresh the encoder. 317 ret = OH_AudioEncoder_Flush(audioEnc); 318 if (ret != AV_ERR_OK) { 319 // Exception handling. 320 } 321 // Start encoding again. 322 ret = OH_AudioEncoder_Start(audioEnc); 323 if (ret != AV_ERR_OK) { 324 // Exception handling. 325 } 326 ``` 3279. (Optional) Call **OH_AudioEncoder_Reset()** to reset the encoder. 328 329 After **OH_AudioEncoder_Reset()** is called, the encoder returns to the initialized state. To continue encoding, you must call **OH_AudioEncoder_Configure()** and then **OH_AudioEncoder_Start()**. 330 331 ```c++ 332 // Reset the encoder. 333 ret = OH_AudioEncoder_Reset(audioEnc); 334 if (ret != AV_ERR_OK) { 335 // Exception handling. 336 } 337 // Reconfigure the encoder. 338 ret = OH_AudioEncoder_Configure(audioEnc, format); 339 if (ret != AV_ERR_OK) { 340 // Exception handling. 341 } 342 ``` 34310. Call **OH_AudioEncoder_Stop()** to stop the encoder. 344 345 ```c++ 346 // Stop the encoder. 347 ret = OH_AudioEncoder_Stop(audioEnc); 348 if (ret != AV_ERR_OK) { 349 // Exception handling. 350 } 351 ``` 35211. Call **OH_AudioEncoder_Destroy()** to destroy the encoder instance and release resources. 353 354 **NOTE**: You only need to call this API once. 355 356 ```c++ 357 // Call OH_AudioEncoder_Destroy to destroy the encoder. 358 ret = OH_AudioEncoder_Destroy(audioEnc); 359 if (ret != AV_ERR_OK) { 360 // Exception handling. 361 } else { 362 audioEnc = NULL; // The encoder cannot be destroyed repeatedly. 363 } 364 ```
