• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1# Using OpenSL ES for Audio Recording
2
3OpenSL ES, short for Open Sound Library for Embedded Systems, is an embedded, cross-platform audio processing library that is free of charge. It provides high-performance and low-latency APIs for you to develop applications running on embedded mobile multimedia devices. OpenHarmony have implemented certain native APIs based on [OpenSL ES](https://www.khronos.org/opensles/) 1.0.1 API specifications developed by the [Khronos Group](https://www.khronos.org/). You can use these APIs through <OpenSLES.h\> and <OpenSLES_OpenHarmony.h\>.
4
5## OpenSL ES on OpenHarmony
6
7Currently, OpenHarmony implements parts of [OpenSL ES APIs](https://gitee.com/openharmony/third_party_opensles/blob/master/api/1.0.1/OpenSLES.h) to implement basic audio recording functionalities.
8
9If an API that has not been implemented on OpenHarmony is called, **SL_RESULT_FEATURE_UNSUPPORTED** is returned.
10
11The following lists the OpenSL ES APIs that have been implemented on OpenHarmony. For details, see the [OpenSL ES](https://www.khronos.org/opensles/) specifications.
12
13- **Engine APIs implemented on OpenHarmony**
14  - SLresult (\*CreateAudioPlayer) (SLEngineItf self, SLObjectItf \* pPlayer, SLDataSource \*pAudioSrc, SLDataSink \*pAudioSnk, SLuint32 numInterfaces, const SLInterfaceID \* pInterfaceIds, const SLboolean \* pInterfaceRequired)
15  - SLresult (\*CreateAudioRecorder) (SLEngineItf self, SLObjectItf \* pRecorder, SLDataSource \*pAudioSrc, SLDataSink \*pAudioSnk, SLuint32 numInterfaces, const SLInterfaceID \* pInterfaceIds, const SLboolean \* pInterfaceRequired)
16  - SLresult (\*CreateOutputMix) (SLEngineItf self, SLObjectItf \* pMix, SLuint32 numInterfaces, const SLInterfaceID \* pInterfaceIds, const SLboolean \* pInterfaceRequired)
17
18- **Object APIs implemented on OpenHarmony**
19  - SLresult (\*Realize) (SLObjectItf self, SLboolean async)
20  - SLresult (\*GetState) (SLObjectItf self, SLuint32 \* pState)
21  - SLresult (\*GetInterface) (SLObjectItf self, const SLInterfaceID iid, void \* pInterface)
22  - void (\*Destroy) (SLObjectItf self)
23
24- **Recorder APIs implemented on OpenHarmony**
25  - SLresult (\*SetRecordState) (SLRecordItf self, SLuint32 state)
26  - SLresult (\*GetRecordState) (SLRecordItf self,SLuint32 \*pState)
27
28- **BufferQueue APIs implemented on OpenHarmony**
29
30  The APIs listed below can be used only after <OpenSLES_OpenHarmony.h\> is introduced.
31
32  | API| Description|
33  | -------- | -------- |
34  | SLresult (\*Enqueue) (SLOHBufferQueueItf self, const void \*buffer, SLuint32 size) | Adds a buffer to the corresponding queue.<br>For an audio playback operation, this API adds the buffer with audio data to the **filledBufferQ_** queue. For an audio recording operation, this API adds the idle buffer after recording data storage to the **freeBufferQ_** queue.<br>The **self** parameter indicates the **BufferQueue** object that calls this API.<br>The **buffer** parameter indicates the pointer to the buffer with audio data or the pointer to the idle buffer after the recording data is stored.<br>The **size** parameter indicates the size of the buffer.|
35  | SLresult (\*Clear) (SLOHBufferQueueItf self) | Releases a **BufferQueue** object.<br>The **self** parameter indicates the **BufferQueue** object that calls this API.|
36  | SLresult (\*GetState) (SLOHBufferQueueItf self, SLOHBufferQueueState \*state) | Obtains the state of a **BufferQueue** object.<br>The **self** parameter indicates the **BufferQueue** object that calls this API.<br>The **state** parameter indicates the pointer to the state of the **BufferQueue** object.|
37  | SLresult (\*RegisterCallback) (SLOHBufferQueueItf self, SlOHBufferQueueCallback callback, void\* pContext) | Registers a callback.<br>The **self** parameter indicates the **BufferQueue** object that calls this API.<br>The **callback** parameter indicates the callback to be registered for the audio playback or recording operation.<br>The **pContext** parameter indicates the pointer to the audio file to be played for an audio playback operation or the pointer to the audio file to be recorded for an audio recording operation.|
38  | SLresult (\*GetBuffer) (SLOHBufferQueueItf self, SLuint8\*\* buffer, SLuint32\* size) | Obtains a buffer.<br>For an audio playback operation, this API obtains an idle buffer from the **freeBufferQ_** queue. For an audio recording operation, this API obtains the buffer that carries recording data from the **filledBufferQ_** queue.<br>The **self** parameter indicates the **BufferQueue** object that calls this API.<br>The **buffer** parameter indicates the double pointer to the idle buffer or the buffer carrying recording data.<br>The **size** parameter indicates the size of the buffer.|
39
40## Sample Code
41
42### Linking the Dynamic Library in the CMake Script
43
44``` cmake
45target_link_libraries(sample PUBLIC libOpenSLES.so)
46```
47
48Refer to the sample code below to record an audio file.
49
501. Add the header files.
51
52   ```c++
53   #include "SLES/OpenSLES.h"
54   #include "SLES/OpenSLES_OpenHarmony.h"
55   #include "SLES/OpenSLES_Platform.h"
56   ```
57
582. Use the **slCreateEngine** API to create and instantiate an **engine** object.
59
60   ```c++
61   SLObjectItf engineObject = nullptr;
62   slCreateEngine(&engineObject, 0, nullptr, 0, nullptr, nullptr);
63   (*engineObject)->Realize(engineObject, SL_BOOLEAN_FALSE);
64   ```
65
663. Obtain the **engineEngine** instance of the **SL_IID_ENGINE** API.
67
68   ```c++
69   SLEngineItf engineItf = nullptr;
70   (*engineObject)->GetInterface(engineObject, SL_IID_ENGINE, &engineItf);
71   ```
72
734. Configure the recorder information (including the input source **audiosource** and output source **audiosink**), and create a **pcmCapturerObject** instance.
74
75   ```c++
76   SLDataLocator_IODevice io_device = {
77       SL_DATALOCATOR_IODEVICE,
78       SL_IODEVICE_AUDIOINPUT,
79       SL_DEFAULTDEVICEID_AUDIOINPUT,
80       NULL
81   };
82   SLDataSource audioSource = {
83       &io_device,
84       NULL
85   };
86   SLDataLocator_BufferQueue buffer_queue = {
87       SL_DATALOCATOR_BUFFERQUEUE,
88       3
89   };
90   // Configure the parameters based on the audio file format.
91   SLDataFormat_PCM format_pcm = {
92       SL_DATAFORMAT_PCM,           // Input audio format.
93       1,                                              // Mono channel.
94       SL_SAMPLINGRATE_44_1,        // Sampling rate, 44100 Hz.
95       SL_PCMSAMPLEFORMAT_FIXED_16, // Audio sampling format, a signed 16-bit integer in little-endian format.
96       0,
97       0,
98       0
99   };
100   SLDataSink audioSink = {
101       &buffer_queue,
102       &format_pcm
103   };
104
105   SLObjectItf pcmCapturerObject = nullptr;
106   (*engineItf)->CreateAudioRecorder(engineItf, &pcmCapturerObject,
107       &audioSource, &audioSink, 0, nullptr, nullptr);
108   (*pcmCapturerObject)->Realize(pcmCapturerObject, SL_BOOLEAN_FALSE);
109
110   ```
111
1125. Obtain the **recordItf** instance of the **SL_IID_RECORD** API.
113
114   ```c++
115   SLRecordItf  recordItf;
116   (*pcmCapturerObject)->GetInterface(pcmCapturerObject, SL_IID_RECORD, &recordItf);
117   ```
118
1196. Obtain the **bufferQueueItf** instance of the **SL_IID_OH_BUFFERQUEUE** API.
120
121   ```c++
122   SLOHBufferQueueItf bufferQueueItf;
123   (*pcmCapturerObject)->GetInterface(pcmCapturerObject, SL_IID_OH_BUFFERQUEUE, &bufferQueueItf);
124   ```
125
1267. Register the **BufferQueueCallback** function.
127
128   ```c++
129   static void BufferQueueCallback(SLOHBufferQueueItf bufferQueueItf, void *pContext, SLuint32 size)
130   {
131       // Obtain the user information passed in during the registration from pContext.
132       SLuint8 *buffer = nullptr;
133       SLuint32 pSize = 0;
134       (*bufferQueueItf)->GetBuffer(bufferQueueItf, &buffer, &pSize);
135       if (buffer != nullptr) {
136           // The recording data can be read from the buffer for subsequent processing.
137           (*bufferQueueItf)->Enqueue(bufferQueueItf, buffer, size);
138       }
139   }
140   void *pContext; // This callback can be used to obtain the custom context information passed in.
141   (*bufferQueueItf)->RegisterCallback(bufferQueueItf, BufferQueueCallback, pContext);
142   ```
143
1448. Start audio recording.
145
146   ```c++
147   (*recordItf)->SetRecordState(recordItf, SL_RECORDSTATE_RECORDING);
148   ```
149
1509. Stop audio recording.
151
152   ```c++
153   (*recordItf)->SetRecordState(recordItf, SL_RECORDSTATE_STOPPED);
154   (*pcmCapturerObject)->Destroy(pcmCapturerObject);
155   ```
156