天天看點
傳回

NDK Android OpenSL ES 音頻采集與播放OpenSL ES 是什麼？OpenSL ES 的一些基本概念OpenSL ES 常用的 Object 和 InterfaceOpenSL ES Recorder 和 Player 功能建構代碼實作

作者：位元組流動

來源：

https://blog.csdn.net/Kennethdroid/article/details/86418725

OpenSL ES 是什麼？

OpenSL ES 是一個針對嵌入式系統的開放硬體音頻加速庫，也可以将其視為一套針對嵌入式平台的音頻标準，全稱為： Open Sound Library for Embedded Systems ，它提供了一套高性能、 低延遲的音頻功能實作方法，并且實作了軟硬體音頻性能的跨平台部署，大大降低了上層處理音頻應用的開發難度。

在 Android 開發中，Google 官方從 Android 2.3 （API 9）開始，便支援了 OpenSL ES 标準 ，并且對其進行了擴充。本文介紹的 OpenSL ES 是針對 Android NDK 開發來說。

OpenSL ES 的一些基本概念

基于 c 語言的面向對象接口

OpenSL ES 是基于 c 語言實作的，但其提供的接口是采用面向對象的方式實作，OpenSL ES 的大多數 API 是通過對象來調用的。例如，以下代碼片段，主要的邏輯是執行個體化引擎對象和擷取引擎對象接口：

SLresult result;

// realize the engine
result = (*engineObject)->Realize(engineObject, SL_BOOLEAN_FALSE);
assert(SL_RESULT_SUCCESS == result);
(void)result;

result = (*engineObject)->GetInterface(engineObject, SL_IID_ENGINE, &engineEngine);
assert(SL_RESULT_SUCCESS == result);
(void)result;

對象和接口概念

Object 和 Interface OpenSL ES 中的兩大基本概念，可以類比為 Java 中的對象和接口。在 OpenSL ES 中， 每個 Object 可以存在一系列的 Interface ，并且為每個對象都提供了一系列的基本操作，如 Realize，GetState，Destroy 等。重要的一點，隻有通過 GetInterface 方法拿到 Object 的 Interface ，才能使用 Object 提供的功能。

對象的生命周期

OpenSL ES 的 Object 一般有三種狀态，分别是： SL_OBJECT_STATE_UNREALIZED （不可用），SL_OBJECT_STATE_REALIZED（可用），SL_OBJECT_STATE_SUSPENDED（挂起）。

Object 處于 SL_OBJECT_STATE_UNREALIZED （不可用）狀态時，系統不會為其配置設定資源；調用 Realize 方法後便進入 SL_OBJECT_STATE_REALIZED（可用）狀态，此時對象的各個功能和資源可以正常通路；當系統音頻相關的硬體裝置被其他程序占用時，OpenSL ES Object 便會進入 SL_OBJECT_STATE_SUSPENDED （挂起）狀态，随後調用 Resume 方法可使對象重回 SL_OBJECT_STATE_REALIZED（可用）狀态；當 Object 使用結束後，調用 Destroy 方法釋放資源，是對象重回 SL_OBJECT_STATE_UNREALIZED （不可用）狀态。

OpenSL ES 常用的 Object 和 Interface

Audio 引擎對象和接口

Audio 引擎對象和接口，即 Engine Object 和 SLEngineItf Interface 。Engine Object 的主要功能是管理 Audio Engine 的生命周期，提供引擎對象的管理接口。引擎對象的使用方法如下：

SLresult result;
// 建立引擎對象
result = slCreateEngine(&engineObject, 0, NULL, 0, NULL, NULL);
assert(SL_RESULT_SUCCESS == result);
(void)result;

// 執行個體化
result = (*engineObject)->Realize(engineObject, SL_BOOLEAN_FALSE);
assert(SL_RESULT_SUCCESS == result);
(void)result;

// 擷取引擎對象接口
result = (*engineObject)->GetInterface(engineObject, SL_IID_ENGINE, &engineEngine);
assert(SL_RESULT_SUCCESS == result);
(void)result;

// 釋放引擎對象的資源
result = (*engineObject)->Destroy(engineObject, SL_BOOLEAN_FALSE);
assert(SL_RESULT_SUCCESS == result);
(void)result;

SLRecordItf 和 SLPlayItf

SLRecordItf 和 SLPlayItf 分别抽象多媒體功能 recorder 和 player ，通過 SLEngineItf 的 CreateAudioPlayer 和 CreateAudioRecorder 方法分别建立 player 和 recorder 對象執行個體。

// 建立 audio recorder 對象
result = (*engineEngine)->CreateAudioRecorder(engineEngine, &recorderObject , &recSource, &dataSink,
                                                  NUM_RECORDER_EXPLICIT_INTERFACES, iids, required);

// 建立 audio player 對象
SLresult result = (*engineEngine)->CreateAudioPlayer(
        engineEngine,
        &audioPlayerObject,
        &dataSource,
        &dataSink,
        1,
        interfaceIDs,
        requiredInterfaces
);

SLDataSource 和 SLDataSink

OpenSL ES 中的 SLDataSource 和 SLDataSink 結構體，主要用于建構 audio player 和 recorder 對象，其中 SLDataSource 表示音頻資料來源的資訊，SLDataSink 表示音頻資料輸出資訊。

// 資料源簡單緩沖隊列定位器
SLDataLocator_AndroidSimpleBufferQueue dataSou
        SL_DATALOCATOR_ANDROIDSIMPLEBUFFERQUEU
        1
};
// PCM 資料源格式
SLDataFormat_PCM dataSourceFormat = {
        SL_DATAFORMAT_PCM, // 格式類型
        wav_get_channels(wav), // 通道數
        wav_get_rate(wav) * 1000, //采樣率
        wav_get_bits(wav), // 位寬
        wav_get_bits(wav),
        SL_SPEAKER_FRONT_CENTER, // 通道屏蔽
        SL_BYTEORDER_LITTLEENDIAN // 位元組順序
};


// 資料源
SLDataSource dataSource = {
        &dataSourceLocator,
        &dataSourceFormat
};
// 針對資料接收器的輸出混合定位器(混音器)
SLDataLocator_OutputMix dataSinkLocator = {
        SL_DATALOCATOR_OUTPUTMIX, // 定位器類型
        outputMixObject // 輸出混合
};
// 輸出
SLDataSink dataSink = {
        &dataSinkLocator, // 定位器
        0,
};

OpenSL ES Recorder 和 Player 功能建構

Audio Recorder

NDK Android OpenSL ES 音頻采集與播放OpenSL ES 是什麼？OpenSL ES 的一些基本概念OpenSL ES 常用的 Object 和 InterfaceOpenSL ES Recorder 和 Player 功能建構代碼實作

Audio Player

NDK Android OpenSL ES 音頻采集與播放OpenSL ES 是什麼？OpenSL ES 的一些基本概念OpenSL ES 常用的 Object 和 InterfaceOpenSL ES Recorder 和 Player 功能建構代碼實作

PS： Audio Player 的 Data Source 也可以是本地存儲或緩存的音頻資料，以上圖檔來自于 Jhuster 的部落格。

代碼實作

以下代碼主要實作音頻資料的采集、儲存和播放。

//
// Created by haohao on 2018/1/12.
//

#include <jni.h>
#include <string>
#include <assert.h>
#include <SLES/OpenSLES.h>
#include <SLES/OpenSLES_Android.h>
#include <android/log.h>

#define AUDIO_SRC_PATH "/sdcard/audio.pcm"

#define LOGI(FORMAT, ...) __android_log_print(ANDROID_LOG_INFO,"haohao",FORMAT,##__VA_ARGS__);
#define LOGE(FORMAT, ...) __android_log_print(ANDROID_LOG_ERROR,"haohao",FORMAT,##__VA_ARGS__);

#define NUM_RECORDER_EXPLICIT_INTERFACES 2
#define NUM_BUFFER_QUEUE 1
#define SAMPLE_RATE 44100
#define PERIOD_TIME 20  // 20ms
#define FRAME_SIZE SAMPLE_RATE * PERIOD_TIME / 1000
#define CHANNELS 2
#define BUFFER_SIZE   (FRAME_SIZE * CHANNELS)

// engine interfaces
static SLObjectItf engineObject = NULL;
static SLEngineItf engineEngine = NULL;

// audio recorder interfaces
static SLObjectItf recorderObject = NULL;
static SLRecordItf recorderRecord = NULL;
static SLAndroidSimpleBufferQueueItf recorderBuffQueueItf = NULL;
static SLAndroidConfigurationItf configItf = NULL;

// pcm audio player interfaces
static SLObjectItf playerObject = NULL;
static SLPlayItf playerPlay = NULL;
static SLObjectItf outputMixObjext = NULL; // 混音器
static SLAndroidSimpleBufferQueueItf playerBufferQueueItf = NULL;

void createEngine(){
    SLEngineOption EngineOption[] = {
            {(SLuint32) SL_ENGINEOPTION_THREADSAFE, (SLuint32) SL_BOOLEAN_TRUE}
    };
    SLresult result;
    result = slCreateEngine(&engineObject, 1, EngineOption, 0, NULL, NULL);
    assert(SL_RESULT_SUCCESS == result);

    /* Realizing the SL Engine in synchronous mode. */
    result = (*engineObject)->Realize(engineObject, SL_BOOLEAN_FALSE);
    assert(SL_RESULT_SUCCESS == result);

    // get the engine interface, which is needed in order to create other objects
    result = (*engineObject)->GetInterface(engineObject, SL_IID_ENGINE, &engineEngine);
    assert(SL_RESULT_SUCCESS == result);
}

class AudioContext {
public:
    FILE *pfile;
    uint8_t *buffer;
    size_t bufferSize;

    AudioContext(FILE *pfile, uint8_t *buffer, size_t bufferSize){
        this->pfile = pfile;
        this->buffer = buffer;
        this->bufferSize = bufferSize;
    }
};

static AudioContext *recorderContext = NULL;

// 錄制音頻時的回調
void AudioRecorderCallback(SLAndroidSimpleBufferQueueItf bufferQueueItf, void *context){
    AudioContext *recorderContext = (AudioContext*)context;
    assert(recorderContext != NULL);
    if (recorderContext->buffer != NULL) {
        fwrite(recorderContext->buffer, recorderContext->bufferSize, 1, recorderContext->pfile);
        LOGI("save a frame audio data.");
        SLresult result;
        SLuint32 state;
        result = (*recorderRecord)->GetRecordState(recorderRecord, &state);
        assert(SL_RESULT_SUCCESS == result);
        (void) result;

        if (state == SL_RECORDSTATE_RECORDING) {
            result = (*bufferQueueItf)->Enqueue(bufferQueueItf, recorderContext->buffer, recorderContext->bufferSize);
            assert(SL_RESULT_SUCCESS == result);
            (void) result;
        }
    }

}

// 播放音頻時的回調
void AudioPlayerCallback(SLAndroidSimpleBufferQueueItf bufferQueueItf, void *context){
    AudioContext *playerContext = (AudioContext*)context;
    if (!feof(playerContext->pfile)) {
        fread(playerContext->buffer, playerContext->bufferSize, 1, playerContext->pfile);
        LOGI("read a frame audio data.");
        (*bufferQueueItf)->Enqueue(bufferQueueItf, playerContext->buffer, playerContext->bufferSize);
    } else {
        fclose(playerContext->pfile);
        delete playerContext->buffer;
    }
}

// 建立音頻播放器
void createAudioPlayer(SLEngineItf engineEngine, SLObjectItf outputMixObject, SLObjectItf &audioPlayerObject){
    SLDataLocator_AndroidSimpleBufferQueue dataSourceLocator = {
            SL_DATALOCATOR_ANDROIDSIMPLEBUFFERQUEUE,
            1
    };

    // PCM 資料源格式
    SLDataFormat_PCM dataSourceFormat = {
            SL_DATAFORMAT_PCM,
            2,
            SL_SAMPLINGRATE_44_1,
            SL_PCMSAMPLEFORMAT_FIXED_16,
            16,
            SL_SPEAKER_FRONT_LEFT| SL_SPEAKER_FRONT_RIGHT,
            SL_BYTEORDER_LITTLEENDIAN
    };

    SLDataSource dataSource = {
            &dataSourceLocator,
            &dataSourceFormat
    };

    SLDataLocator_OutputMix dataSinkLocator = {
            SL_DATALOCATOR_OUTPUTMIX, // 定位器類型
            outputMixObject // 輸出混合
    };

    SLDataSink dataSink = {
            &dataSinkLocator, // 定位器
            0,
    };

    // 需要的接口
    SLInterfaceID interfaceIDs[] = {
            SL_IID_BUFFERQUEUE
    };
    SLboolean requiredInterfaces[] = {
            SL_BOOLEAN_TRUE
    };

    // 建立音頻播放對象
    SLresult result = (*engineEngine)->CreateAudioPlayer(
            engineEngine,
            &audioPlayerObject,
            &dataSource,
            &dataSink,
            1,
            interfaceIDs,
            requiredInterfaces
    );
    assert(SL_RESULT_SUCCESS == result);
    (void) result;

}

extern "C" {

// 開始播放音頻
JNIEXPORT void JNICALL
Java_com_haohao_opensl_1es_AudioRecorder_startPlay(JNIEnv *env, jobject instance) {
    // 建立引擎
    if (engineEngine == NULL) {
        createEngine();
    }

    // 建立混音器
    SLresult result;
    result = (*engineEngine)->CreateOutputMix(engineEngine, &outputMixObjext, 0, 0, 0);
    assert(SL_RESULT_SUCCESS == result);
    (void) result;

    result = (*outputMixObjext)->Realize(outputMixObjext, SL_BOOLEAN_FALSE);
    assert(SL_RESULT_SUCCESS == result);
    (void) result;

    FILE *p_file = fopen(AUDIO_SRC_PATH, "r");

    // 建立播放器
    createAudioPlayer(engineEngine, outputMixObjext, playerObject);

    result = (*playerObject)->Realize(playerObject, SL_BOOLEAN_FALSE);
    assert(SL_RESULT_SUCCESS == result);
    (void) result;

    result = (*playerObject)->GetInterface(playerObject, SL_IID_BUFFERQUEUE,
                                                &playerBufferQueueItf);
    assert(SL_RESULT_SUCCESS == result);
    (void) result;

    uint8_t *buffer = new uint8_t[BUFFER_SIZE];
    AudioContext *playerContext = new AudioContext(p_file, buffer, BUFFER_SIZE);
    result = (*playerBufferQueueItf)->RegisterCallback(playerBufferQueueItf, AudioPlayerCallback,
                                                    playerContext);
    assert(SL_RESULT_SUCCESS == result);
    (void) result;

    result = (*playerObject)->GetInterface(playerObject, SL_IID_PLAY, &playerPlay);
    assert(SL_RESULT_SUCCESS == result);
    (void) result;

    result = (*playerPlay)->SetPlayState(playerPlay, SL_PLAYSTATE_PLAYING);
    assert(SL_RESULT_SUCCESS == result);

    AudioPlayerCallback(playerBufferQueueItf, playerContext);

}

// 停止播放音頻
JNIEXPORT void JNICALL
Java_com_haohao_opensl_1es_AudioRecorder_stopPlay(JNIEnv *env, jobject instance) {
    if (playerPlay != NULL) {
        SLresult result;
        result = (*playerPlay)->SetPlayState(playerPlay, SL_PLAYSTATE_STOPPED);
        assert(SL_RESULT_SUCCESS == result);
    }
}

// 開始采集音頻資料，并儲存到本地
JNIEXPORT void JNICALL
Java_com_haohao_opensl_1es_AudioRecorder_startRecord(JNIEnv *env, jobject instance) {

    if (engineEngine == NULL) {
        createEngine();
    }

    if (recorderObject != NULL) {
        LOGI("Audio recorder already has been created.");
        return ;
    }

    FILE *p_file = fopen(AUDIO_SRC_PATH, "w");

    if (p_file == NULL) {
        LOGI("Fail to open file.");
        return ;
    }

    SLresult result;

    /* setup the data source*/
    SLDataLocator_IODevice ioDevice = {
            SL_DATALOCATOR_IODEVICE,
            SL_IODEVICE_AUDIOINPUT,
            SL_DEFAULTDEVICEID_AUDIOINPUT,
            NULL
    };

    SLDataSource recSource = {&ioDevice, NULL};

    SLDataLocator_AndroidSimpleBufferQueue recBufferQueue = {
            SL_DATALOCATOR_ANDROIDSIMPLEBUFFERQUEUE,
            NUM_BUFFER_QUEUE
    };

    SLDataFormat_PCM pcm = {
            SL_DATAFORMAT_PCM, // pcm 格式的資料
            2,  // 2 個聲道（立體聲）
            SL_SAMPLINGRATE_44_1, // 44100hz 的采樣頻率
            SL_PCMSAMPLEFORMAT_FIXED_16,
            SL_PCMSAMPLEFORMAT_FIXED_16,
            SL_SPEAKER_FRONT_LEFT| SL_SPEAKER_FRONT_RIGHT,
            SL_BYTEORDER_LITTLEENDIAN
    };

    SLDataSink dataSink = { &recBufferQueue, &pcm };
    SLInterfaceID iids[NUM_RECORDER_EXPLICIT_INTERFACES] = {SL_IID_ANDROIDSIMPLEBUFFERQUEUE, SL_IID_ANDROIDCONFIGURATION};
    SLboolean required[NUM_RECORDER_EXPLICIT_INTERFACES] = {SL_BOOLEAN_TRUE, SL_BOOLEAN_TRUE};

    /* Create the audio recorder */
    result = (*engineEngine)->CreateAudioRecorder(engineEngine, &recorderObject , &recSource, &dataSink,
                                                  NUM_RECORDER_EXPLICIT_INTERFACES, iids, required);
    assert(SL_RESULT_SUCCESS == result);


    /* get the android configuration interface*/
    result = (*recorderObject)->GetInterface(recorderObject, SL_IID_ANDROIDCONFIGURATION, &configItf);
    assert(SL_RESULT_SUCCESS == result);

    /* Realize the recorder in synchronous mode. */
    result = (*recorderObject)->Realize(recorderObject, SL_BOOLEAN_FALSE);
    assert(SL_RESULT_SUCCESS == result);

    /* Get the buffer queue interface which was explicitly requested */
    result = (*recorderObject)->GetInterface(recorderObject, SL_IID_ANDROIDSIMPLEBUFFERQUEUE, (void*) &recorderBuffQueueItf);
    assert(SL_RESULT_SUCCESS == result);


    /* get the record interface */
    result = (*recorderObject)->GetInterface(recorderObject, SL_IID_RECORD, &recorderRecord);
    assert(SL_RESULT_SUCCESS == result);

    uint8_t *buffer = new uint8_t[BUFFER_SIZE];
    recorderContext = new AudioContext(p_file, buffer, BUFFER_SIZE);
    result = (*recorderBuffQueueItf)->RegisterCallback(recorderBuffQueueItf, AudioRecorderCallback, recorderContext);
    assert(SL_RESULT_SUCCESS == result);

    /* Enqueue buffers to map the region of memory allocated to store the recorded data */
    result = (*recorderBuffQueueItf)->Enqueue(recorderBuffQueueItf, recorderContext->buffer, BUFFER_SIZE);
    assert(SL_RESULT_SUCCESS == result);

    /* Start recording */
    // 開始錄制音頻
    result = (*recorderRecord)->SetRecordState(recorderRecord, SL_RECORDSTATE_RECORDING);
    assert(SL_RESULT_SUCCESS == result);
    LOGI("Starting recording");

}

// 停止音頻采集
JNIEXPORT void JNICALL
Java_com_haohao_opensl_1es_AudioRecorder_stopRecord(JNIEnv *env, jobject instance) {
    if (recorderRecord != NULL) {
        SLresult result = (*recorderRecord)->SetRecordState(recorderRecord, SL_RECORDSTATE_STOPPED);
        assert(SL_RESULT_SUCCESS == result);

        if (recorderContext != NULL) {
            fclose(recorderContext->pfile);
            delete recorderContext->buffer;
        }
    }
}

// 釋放資源
JNIEXPORT void JNICALL
Java_com_haohao_opensl_1es_AudioRecorder_release(JNIEnv *env, jobject instance) {
    if (recorderObject != NULL) {
        (*recorderObject)->Destroy(recorderObject);
        recorderObject = NULL;
        recorderRecord = NULL;
        recorderBuffQueueItf = NULL;
        configItf = NULL;
        recorderContext = NULL;
    }

    if (playerObject != NULL) {
        (*playerObject)->Destroy(playerObject);
        playerObject = NULL;
        playerPlay = NULL;
        playerBufferQueueItf = NULL;
        outputMixObjext = NULL;
    }

    // destroy engine object, and invalidate all associated interfaces
    if (engineObject != NULL) {
        (*engineObject)->Destroy(engineObject);
        engineObject = NULL;
        engineEngine = NULL;
    }
}
};

CMake 腳本 CMakeLists.txt 。

cmake_minimum_required(VERSION 3.4.1)

add_library( # Sets the name of the library.
             audio-recorder

             # Sets the library as a shared library.
             SHARED

             # Provides a relative path to your source file(s).
             src/main/cpp/audio-recorder.cpp)

target_link_libraries(audio-recorder
                      android
                      log
                      OpenSLES)

NDK 開發系列文章：

「視訊雲技術」你最值得關注的音視訊技術公衆号，每周推送來自阿裡雲一線的實踐技術文章，在這裡與音視訊領域一流工程師交流切磋。
NDK Android OpenSL ES 音頻采集與播放OpenSL ES 是什麼？OpenSL ES 的一些基本概念OpenSL ES 常用的 Object 和 InterfaceOpenSL ES Recorder 和 Player 功能建構代碼實作
存儲 緩存 java api android開發 c語言 c++ 記憶體技術 opengl android建立 android直播播放rtmp android阿裡雲oss android直播播放 android對話框進度條
上一篇: DataX在資料遷移中的應用
下一篇: “21天助力Java工程師進階”活動玩法詳解一、21天Java活動報名二、21天Java活動打卡三、活動獎品說明

繼續閱讀