文章目錄

前言
檢視代碼重點關注log
1 應用層
- 1.1 建立
- 1.2 擷取
- 1.3 設定VideoSource
- 1.4 設定輸出模式
- 1.5 設定檔案名
- 1.6 設定編碼方式
2 擷取video codec表——mediaprofile
3 MediaRecorder: prepare
3.1 建立MediaCodecSource（重點看MediaCodec的log）
- 3.2 Audio
4 MediaRecorder: start
5 小結

前言

從應用層到native層檢視mediacodec的建立

後續再看Acodec到omx

檢視代碼重點關注log

Codec相關：MediaCodec\MediaCodecSource\Acodec\MPEG4Writer\omx

音頻資料：AudioSource

視訊資料：GraphicBufferSource

1 應用層

以camera代碼為例，因為camera使用的是mediarecorder，而audiorecord隻能錄制pcm資料。

android\packages\apps\SnapdragonCamera\src\com\android\camera\captureModule.java

1.1 建立

mMediaRecorder = new MediaRecorder()

1.2 擷取

mMediaRecorder.setAudioSource(MediaRecorder.AudioSource.MIC);

int audioEncoder = SettingTranslation.getAudioEncoder(mSettingsManager.getValue(SettingsManager.KEY_AUDIO_ENCODER));

【mediacodec】MediaRecorder--MediaCodec前言檢視代碼重點關注log1 應用層2 擷取video codec表——mediaprofile3 MediaRecorder: prepare3.1 建立MediaCodecSource（重點看MediaCodec的log）4 MediaRecorder: start5 小結

1.3 設定VideoSource

mMediaRecorder.setVideoSource(MediaRecorder.VideoSource.SURFACE);

MediaRecorder: setVideoSource(2)

視訊資料來源為surface(螢幕顯示？)，非camera，存疑

1.4 設定輸出模式

mMediaRecorder.setOutputFormat(mProfile.fileFormat);

MediaRecorder: setOutputFormat(2)

選擇輸出模式為mpeg4，目前看起來輸出模式隻有兩種

預設為OUTPUT_FORMAT_THREE_GPP

存疑：為什麼輸出模式有兩種，如何對應encoder，而編碼方式中也有mpeg4

解答：這裡的mpeg_4和three_gpp都隻是檔案的存儲類型，目前視訊的存儲格式支援這兩種。

而在videoencoder看到的mpeg4是編碼類型，同類型的還有h263/h264/h265(hevc)

參考文章：媒體篇　第一章音頻 | 不隻是看客

MediaRecorder官方文檔導讀

1.5 設定檔案名

mMediaRecorder.setOutputFile(fileName);

1.6 設定編碼方式

mMediaRecorder.setVideoEncoder(mVideoEncoder);

MediaRecorder: setVideoEncoder(2)

選擇編碼方式為h264

編碼格式不同于輸出格式，以音頻為類比。

視訊編碼可以分為h264/mpeg4，音頻編碼為lpcm/aac

視訊輸出格式常見的是mp4/m4a(outputformat為mpeg4)，音頻輸出格式mp3/wav/amr

視訊編碼為h264（camera app預設設定）

07-26 22:23:32.636 3772 3772 V MediaRecorder: setVideoEncoder(2)

07-26 23:01:36.513 1129 3967 I OMXMaster: makeComponentInstance(OMX.qcom.video.encoder.avc) in android.hardwar process

視訊編碼為mpeg4

07-26 23:02:09.983 3772 3772 V MediaRecorder: setVideoEncoder(3)

07-20 23:30:40.325 972 1051 I OMXMaster: makeComponentInstance(OMX.qcom.video.encoder.mpeg4) in android.hardwar process

視訊編碼為h265

07-26 23:02:28.156 3772 3772 V MediaRecorder: setVideoEncoder(5)

07-26 23:02:28.158 1121 1216 V MPEG4Writer: initInternal

07-26 23:02:28.158 1121 1216 V MediaCodecList: matching ‘OMX.qcom.video.encoder.hevc’

07-26 23:02:28.158 1121 1216 V MediaCodecList: matching ‘c2.android.hevc.encoder’

07-26 23:02:28.158 1121 1216 V ACodec : Now uninitialized

07-26 23:02:28.159 1121 4879 V ACodec : onAllocateComponent

07-26 23:02:28.161 1121 4879 I OMXClient: IOmx service obtained

07-26 23:02:28.162 1129 4672 I OMXMaster: makeComponentInstance(OMX.qcom.video.encoder.hevc) in android.hardwar process

c2.android.hevc.encoder的size不符合

設定h265編碼時可選兩種編碼器，但是最終選擇了qcom的硬體編碼

1 判斷是否是軟解碼，依據編碼器名稱是否包含OMX.google.和c2.android.

ps：startsWithIgnoreCase表示不區分大小寫

2 判斷軟解碼

以soundrecorder為例，可以看到輸出格式為audio/3gpp

以camera mpeg4為例，可以看到輸出格式為video/mp4v-es，輸入格式為video/raw

以camera h265為例，可以看到輸出格式為video/hevc，輸入格式為video/raw

2 擷取video codec表——mediaprofile

java代碼

上層傳入encoder類型

根據video_encoder類型，擷取編碼資訊，碼率/幀率/寬/高的最大最小值

存疑：sProfiles哪裡來的

來源于/vendor/etc/media_profiles_vendor.xml

目前看起來不支援vp8，參見/vendor/etc/media_profiles_vendor.xml

3 MediaRecorder: prepare

3.1 建立MediaCodecSource（重點看MediaCodec的log）

status_t MediaCodecSource::initEncoder() {
	//1. 根據傳入的mimi等資訊，查找mediacodeclist中合适的codec名字存入Vector<AString> matchingCodecs中
	MediaCodecList::findMatchingCodecs(outputMIME.c_str(), true /* encoder */,mFlags,&matchingCodecs);
	
	//2. 周遊matchingCodecs，根據名字建立mediacodec
	//ps:還有一種方式MediaCodec::CreateByType建立mediacodec，或者函數有的時候會用到
	mEncoder = MediaCodec::CreateByComponentName(mCodecLooper, matchingCodecs[ix]);
	
	//3. 列印輸出格式
	ALOGV("output format is '%s'", mOutputFormat->debugString(0).c_str());

	//4. 配置mediacodec
	err = mEncoder->configure();

	//5. 啟動mediacodec
	err = mEncoder->start();
}

1 查找比對的codec名字

輸入mime，在MediaCodecList中查找合适的codec名字，存入matches中

mime --> MediaCodecList.matches.name

07-28 03:44:05.130   945  3290 D MediaCodecSource: youkai in MediaCodecSource::initEncoder
07-28 03:44:05.130   945  3290 V MediaCodecList: youkai in MediaCodecList::findMatchingCodecs
07-28 03:44:05.130   945  3290 V MediaCodecList: matching 'OMX.qcom.video.encoder.mpeg4'
07-28 03:44:05.130   945  3290 V MediaCodecList: matching 'c2.android.mpeg4.encoder'
07-28 03:44:05.130   945  3290 V MediaCodecList: matching 'OMX.google.mpeg4.encoder'

至此比對到三個encoder

07-28 03:44:05.130 945 3290 V MediaCodecList: matching ‘OMX.qcom.video.encoder.mpeg4’

Media_codecs.xml

07-28 03:44:05.130 945 3290 V MediaCodecList: matching ‘c2.android.mpeg4.encoder’

07-28 03:44:05.130 945 3290 V MediaCodecList: matching ‘OMX.google.mpeg4.encoder’

Media_codecs_sw.xml (frameworks\av\media\libstagefright\data)

Size不符合淘汰

2 根據名字建立mediacodec

3 打出輸出格式，這個參數是從app傳來的

07-28 03:44:05.161   945 15206 V ACodec  : [OMX.qcom.video.encoder.mpeg4] Now Loaded
07-28 03:44:05.163   945  3290 V MediaCodecSource: output format is 'AMessage(what = 0x00000000) = {
07-28 03:44:05.163   945  3290 V MediaCodecSource:   string mime = "video/mp4v-es"
07-28 03:44:05.163   945  3290 V MediaCodecSource:   int32_t width = 1920
07-28 03:44:05.163   945  3290 V MediaCodecSource:   int32_t height = 1080
07-28 03:44:05.163   945  3290 V MediaCodecSource:   int32_t stride = 1920
07-28 03:44:05.163   945  3290 V MediaCodecSource:   int32_t slice-height = 1080
07-28 03:44:05.163   945  3290 V MediaCodecSource:   int32_t color-format = 2130708361
07-28 03:44:05.163   945  3290 V MediaCodecSource:   int32_t bitrate = 20000000
07-28 03:44:05.163   945  3290 V MediaCodecSource:   int32_t frame-rate = 30
07-28 03:44:05.163   945  3290 V MediaCodecSource:   int32_t i-frame-interval = 1
07-28 03:44:05.163   945  3290 V MediaCodecSource:   int32_t priority = 0
07-28 03:44:05.163   945  3290 V MediaCodecSource:   int32_t feature-nal-length-bitstream = 1
07-28 03:44:05.163   945  3290 V MediaCodecSource:   int32_t nal-length-in-bytes = 4
07-28 03:44:05.163   945  3290 V MediaCodecSource:   int32_t isNativeRecorder = 1
07-28 03:44:05.163   945  3290 V MediaCodecSource:   int32_t create-input-buffers-suspended = 1
07-28 03:44:05.163   945  3290 V MediaCodecSource: }'
07-28 03:44:05.163   945 15205 I MediaCodec: MediaCodec will operate in async mode
07-28 03:44:05.163   945 15205 V MediaCodec: kWhatConfigure: Old mCrypto: 0x0 (0)
07-28 03:44:05.163   945 15205 V MediaCodec: kWhatConfigure: New mCrypto: 0x0 (0)
07-28 03:44:05.163   945 15205 V MediaCodec: Found 0 pieces of codec specific data.
07-28 03:44:05.163   945 15206 V ACodec  : onConfigureComponent

4 mediacodec的消息機制接受到kWhatComponentConfigured，列印出輸入和輸出格式

07-28 03:44:05.226   945 15205 V MediaCodec: [OMX.qcom.video.encoder.mpeg4] configured as input format: AMessage(what = 0x00000000) = {
07-28 03:44:05.226   945 15205 V MediaCodec:       string mime = "video/raw"
07-28 03:44:05.226   945 15205 V MediaCodec:       int32_t stride = 1920
07-28 03:44:05.226   945 15205 V MediaCodec:       int32_t slice-height = 1080
07-28 03:44:05.226   945 15205 V MediaCodec:       int32_t color-format = 2130708361
07-28 03:44:05.226   945 15205 V MediaCodec:       int32_t color-range = 0
07-28 03:44:05.226   945 15205 V MediaCodec:       int32_t color-standard = 0
07-28 03:44:05.226   945 15205 V MediaCodec:       int32_t color-transfer = 0
07-28 03:44:05.226   945 15205 V MediaCodec:       int32_t width = 1920
07-28 03:44:05.226   945 15205 V MediaCodec:       int32_t height = 1080
07-28 03:44:05.226   945 15205 V MediaCodec:     }, output format: AMessage(what = 0x00000000) = {
07-28 03:44:05.226   945 15205 V MediaCodec:       int32_t bitrate = 20000000
07-28 03:44:05.226   945 15205 V MediaCodec:       int32_t max-bitrate = 20000000
07-28 03:44:05.226   945 15205 V MediaCodec:       int32_t frame-rate = 30
07-28 03:44:05.226   945 15205 V MediaCodec:       string mime = "video/mp4v-es"
07-28 03:44:05.226   945 15205 V MediaCodec:       int32_t width = 1920
07-28 03:44:05.226   945 15205 V MediaCodec:       int32_t height = 1080
07-28 03:44:05.226   945 15205 V MediaCodec:     }
07-28 03:44:05.227   945 15206 V ACodec  : onCreateInputSurface
07-28 03:44:05.227   953  2087 V GraphicBufferSource: GraphicBufferSource

5 接受到GraphicBufferSource傳來的建立顯示消息kWhatInputSurfaceCreated

07-28 03:44:05.232   945 15205 V MediaCodec: [OMX.qcom.video.encoder.mpeg4] input surface created as input format: AMessage(what = 0x00000000) = {
07-28 03:44:05.232   945 15205 V MediaCodec:       string mime = "video/raw"
07-28 03:44:05.232   945 15205 V MediaCodec:       int32_t stride = 1920
07-28 03:44:05.232   945 15205 V MediaCodec:       int32_t slice-height = 1080
07-28 03:44:05.232   945 15205 V MediaCodec:       int32_t color-format = 2130708361
07-28 03:44:05.232   945 15205 V MediaCodec:       int32_t color-range = 2
07-28 03:44:05.232   945 15205 V MediaCodec:       int32_t color-standard = 1
07-28 03:44:05.232   945 15205 V MediaCodec:       int32_t color-transfer = 3
07-28 03:44:05.232   945 15205 V MediaCodec:       int32_t width = 1920
07-28 03:44:05.232   945 15205 V MediaCodec:       int32_t height = 1080
07-28 03:44:05.232   945 15205 V MediaCodec:       int32_t android._dataspace = 260
07-28 03:44:05.232   945 15205 V MediaCodec:       Buffer android._color-aspects = {
07-28 03:44:05.232   945 15205 V MediaCodec:         00000000:  02 00 00 00 01 00 00 00  03 00 00 00 01 00 00 00  ................
07-28 03:44:05.232   945 15205 V MediaCodec:       }
07-28 03:44:05.232   945 15205 V MediaCodec:     }, output format: AMessage(what = 0x00000000) = {
07-28 03:44:05.232   945 15205 V MediaCodec:       int32_t bitrate = 20000000
07-28 03:44:05.232   945 15205 V MediaCodec:       int32_t max-bitrate = 20000000
07-28 03:44:05.232   945 15205 V MediaCodec:       int32_t frame-rate = 30
07-28 03:44:05.232   945 15205 V MediaCodec:       string mime = "video/mp4v-es"
07-28 03:44:05.232   945 15205 V MediaCodec:       int32_t width = 1920
07-28 03:44:05.232   945 15205 V MediaCodec:       int32_t height = 1080
07-28 03:44:05.232   945 15205 V MediaCodec:       int32_t color-range = 2
07-28 03:44:05.232   945 15205 V MediaCodec:       int32_t color-standard = 1
07-28 03:44:05.232   945 15205 V MediaCodec:       int32_t color-transfer = 3
07-28 03:44:05.232   945 15205 V MediaCodec:     }
07-28 03:44:05.232   945  3290 V MediaCodecSource: setting dataspace 0x104, format 0x22
07-28 03:44:05.232   945 15206 V ACodec  : onStart
07-28 03:44:05.232   953  1026 V GraphicBufferSource: stop

3.2 Audio

07-28 03:44:05.303   945  3290 D MediaCodecSource: youkai in MediaCodecSource::initEncoder
07-28 03:44:05.303   945  3290 V ACodec  : Now uninitialized
07-28 03:44:05.304   945 15212 V ACodec  : onAllocateComponent
07-28 03:44:05.306   945 15212 I OMXClient: IOmx service obtained
07-28 03:44:05.306   714 15211 V AudioFlinger: acquireWakeLock_l() AudioIn_7E status 0
07-28 03:44:05.306   714 15211 D AudioFlinger: updateWakeLockUids_l AudioIn_7E uids:
07-28 03:44:05.306   953  1026 I OMXMaster: makeComponentInstance(OMX.qcom.audio.encoder.aac) in android.hardwar process
07-28 03:44:05.306   714 15211 V AudioFlinger: updateWakeLockUids_l() AudioIn_7E status 0
07-28 03:44:05.308   714 15211 V AudioFlinger: releaseWakeLock_l() AudioIn_7E
07-28 03:44:05.309   714 15211 V AudioFlinger: RecordThread: loop stopping
07-28 03:44:05.314   953  1026 E         :  component init: role = OMX.qcom.audio.encoder.aac
07-28 03:44:05.324     0     0 I aac_in_open: session id 2: NT mode encoder success
07-28 03:44:05.324     0     0 I aac_in_open: session id 2: success
07-28 03:44:05.327   953  1026 W media.codec: Failed to obtain quirks for omx component 'OMX.qcom.audio.encoder.aac' from XML files
07-28 03:44:05.328   945 15212 V ACodec  : [OMX.qcom.audio.encoder.aac] Now Loaded
07-28 03:44:05.329   945  3290 V MediaCodecSource: output format is 'AMessage(what = 0x00000000) = {
07-28 03:44:05.329   945  3290 V MediaCodecSource:   string mime = "audio/mp4a-latm"
07-28 03:44:05.329   945  3290 V MediaCodecSource:   int32_t aac-profile = 2
07-28 03:44:05.329   945  3290 V MediaCodecSource:   int32_t max-input-size = 2048
07-28 03:44:05.329   945  3290 V MediaCodecSource:   int32_t channel-count = 2
07-28 03:44:05.329   945  3290 V MediaCodecSource:   int32_t sample-rate = 48000
07-28 03:44:05.329   945  3290 V MediaCodecSource:   int32_t bitrate = 96000
07-28 03:44:05.329   945  3290 V MediaCodecSource:   int32_t priority = 0
07-28 03:44:05.329   945  3290 V MediaCodecSource: }'
07-28 03:44:05.330   945 15212 I MediaCodec: MediaCodec will operate in async mode
07-28 03:44:05.331   945 15212 V MediaCodec: kWhatConfigure: Old mCrypto: 0x0 (0)
07-28 03:44:05.332   945 15212 V MediaCodec: kWhatConfigure: New mCrypto: 0x0 (0)
07-28 03:44:05.332   945 15212 V MediaCodec: Found 0 pieces of codec specific data.
07-28 03:44:05.332   945 15212 V ACodec  : onConfigureComponent


07-28 03:44:05.334   945 15212 V MediaCodec: [OMX.qcom.audio.encoder.aac] configured as input format: AMessage(what = 0x00000000) = {
07-28 03:44:05.334   945 15212 V MediaCodec:       string mime = "audio/raw"
07-28 03:44:05.334   945 15212 V MediaCodec:       int32_t channel-count = 2
07-28 03:44:05.334   945 15212 V MediaCodec:       int32_t sample-rate = 48000
07-28 03:44:05.334   945 15212 V MediaCodec:       int32_t pcm-encoding = 2
07-28 03:44:05.334   945 15212 V MediaCodec:     }, output format: AMessage(what = 0x00000000) = {
07-28 03:44:05.334   945 15212 V MediaCodec:       int32_t bitrate = 96000
07-28 03:44:05.334   945 15212 V MediaCodec:       int32_t max-bitrate = 96000
07-28 03:44:05.334   945 15212 V MediaCodec:       string mime = "audio/mp4a-latm"
07-28 03:44:05.334   945 15212 V MediaCodec:       int32_t channel-count = 2
07-28 03:44:05.334   945 15212 V MediaCodec:       int32_t sample-rate = 48000
07-28 03:44:05.334   945 15212 V MediaCodec:     }
07-28 03:44:05.335   945  3290 V MediaCodecSource: setting dataspace 0x10c10000, format 0x22
07-28 03:44:05.335   945 15212 V ACodec  : onStart


07-28 03:44:05.367   945 15212 V MediaCodec: [OMX.qcom.audio.encoder.aac] output format changed to: AMessage(what = 0x00000000) = {
07-28 03:44:05.367   945 15212 V MediaCodec:       int32_t bitrate = 96000
07-28 03:44:05.367   945 15212 V MediaCodec:       int32_t max-bitrate = 96000
07-28 03:44:05.367   945 15212 V MediaCodec:       string mime = "audio/mp4a-latm"
07-28 03:44:05.367   945 15212 V MediaCodec:       int32_t channel-count = 2
07-28 03:44:05.367   945 15212 V MediaCodec:       int32_t sample-rate = 48000
07-28 03:44:05.367   945 15212 V MediaCodec:     }
07-28 03:44:05.367   945 15212 V MediaCodec: -- returned buffer timestamp 0 <= 0, ignore it
07-28 03:44:05.367   945 15204 E Utils   : csd0 too small
07-28 03:44:05.367   945 15204 E ExtendedUtils: csd0 too small
07-28 03:44:05.368   945 15212 V ACodec  : [OMX.qcom.audio.encoder.aac] calling fillBuffer 3

4 MediaRecorder: start

07-28 03:44:05.756 14402 14402 V MediaRecorder: start
07-28 03:44:05.756   945  1028 V StagefrightRecorder: start
07-28 03:44:05.756   945  1028 V MPEG4Writer: movie time scale: 1000
07-28 03:44:05.756   945  1028 V MPEG4Writer: muxer starting: mHasMoovBox 1, mHasFileLevelMeta 0
07-28 03:44:05.757   945  1028 I MPEG4Writer: limits: 11065696640/600000000 bytes/us, bit rate: 20096000 bps and the estimated moov size 405120 bytes
07-28 03:44:05.757   945  1028 V MPEG4Writer: startWriterThread
07-28 03:44:05.757   945  1028 V MPEG4Writer: initTrackingProgressStatus
07-28 03:44:05.757   945  1028 I MPEG4Writer: Start time offset: 100000 us
07-28 03:44:05.757   945 15239 V MPEG4Writer: ThreadWrapper: 0xeb153180
07-28 03:44:05.757   945 15239 V MPEG4Writer: threadFunc
07-28 03:44:05.757   945 15239 V MPEG4Writer: findChunkToWrite
07-28 03:44:05.757   945 15239 V MPEG4Writer: Nothing to be written after all
07-28 03:44:05.757   945 15204 I MediaCodecSource: MediaCodecSource (video) starting
07-28 03:44:05.758   945 15204 I MediaCodecSource: MediaCodecSource (video) started
07-28 03:44:05.758   945  1028 V MPEG4Writer: initTrackingProgressStatus
07-28 03:44:05.759   945  1028 I MPEG4Writer: Start time offset: 100000 us
07-28 03:44:05.759   945 15204 I MediaCodecSource: MediaCodecSource (audio) starting
07-28 03:44:05.759   945 15204 V MediaCodecSource: puller (audio) start
07-28 03:44:05.759   953 14820 D GraphicBufferSource: setStartTimeUs: skipFramesBeforeUs=81188109321
07-28 03:44:05.760   953  1026 V GraphicBufferSource: setSuspend=0 at time -1 us
07-28 03:44:05.763   945 15241 V AudioRecord: start(48): sync event 0 trigger session 0

5 小結

本章主要介紹從java的mediarecorder到mediacodec建立流程。

從app傳下來結果關鍵參數(編碼類型)
經過mediaprofile的過濾，選擇出支援的格式
将mime傳給mediacodeclist選擇合适的encoder名字
根據encoder名字建立mediacodec(這裡會建立Acodec，然後到omx和MPEG4writer，後續文章介紹)

mediarecorder的prepare階段重點建立音視訊的encoder，start階段簡單介紹

【mediacodec】MediaRecorder--MediaCodec前言檢視代碼重點關注log1 應用層2 擷取video codec表——mediaprofile3 MediaRecorder: prepare3.1 建立MediaCodecSource（重點看MediaCodec的log）4 MediaRecorder: start5 小結

文章目錄

前言

檢視代碼重點關注log

1 應用層

1.1 建立

1.2 擷取

1.3 設定VideoSource

1.4 設定輸出模式

1.5 設定檔案名

1.6 設定編碼方式

2 擷取video codec表——mediaprofile

3 MediaRecorder: prepare

3.1 建立MediaCodecSource（重點看MediaCodec的log）

3.2 Audio

4 MediaRecorder: start

5 小結

繼續閱讀

Android MediaCodec解碼aac,播放.

使用mediacodec做視訊解碼

MediaCodec寫死成H264視訊流

Android MediaCodec h264編碼無法設為high profile

Android平台使用MediaCodec壓縮攝像頭圖像時設定碼率無效或碼率偏差很大的問題解決

android 錄制螢幕帶聲音可直播方案截屏

MediaCodec解碼h264流

NDK中使用mediacodec編碼h264

【分享】性能比肩美拍秒拍的Android視訊錄制編輯特效解決方案【2】

MediaCodec編碼音頻pts設定

Android MediaCodec硬編相容性測試方案

Android MediaPlayer整體架構源碼分析 -【MediaCodec編解碼器插件子產品化注冊和建立處理流程】【Part 2-B】

Android平台采集攝像頭圖像和使用MediaCodec寫死的例子詳解

[RK3288][Android6.0] MediaCodec的bitrate_mode配置問題

Android MediaMuxer使用遇到的問題