目錄
-
- 前言：
- 運作環境：
- 運作效果：
- 一、ffmpeg視訊編碼基本流程
- 二、相關變量和函數的簡要說明
-
- 結構體：
-
- AVFormatContext：
- AVOutputFormat：
- AVCodecContext：
- AVDictionary：
- AVCodec：
- AVStream：
- AVPacket：
- AVFrame：
- SwsContext：
- 流程中的主要函數（不想看具體參數參考流程圖即可）
-
- av_register_all()：
- avformat_alloc_output_context2()：
- avio_open()：
- avcodec_find_encoder()：
- av_new_stream()：
- avcodec_open2()：
- avformat_write_header()：
- avcodec_encode_video2()：
- av_write_frame()：
- av_write_trailer()：
- 源碼
前言：
本篇介紹ffmpeg視訊編碼的基本流程，根據官方曆程muxing中的曆程提取的，做個筆記也讓以後用到有機會ctrl+c ctrl+v。除了主要基本流程中的函數外，還用到了其它ffmpeg中的一些相關函數，我都會做簡要介紹說明。
運作環境：
IDE： Qt Creator；Qt版本：Qt5.6.3 ；FFMpeg編譯時間： git-2019-10-22-0b8956b。
運作效果：
一、ffmpeg視訊編碼基本流程
二、相關變量和函數的簡要說明
結構體：
AVFormatContext：
描述了一個媒體檔案或媒體流的構成和基本資訊，位于avformat.h檔案中。
AVOutputFormat：
輸出資料的封裝格式。僅封裝用，調用者在avformat_write_header()之前設定。
AVCodecContext：
描述編解碼器上下文的資料結構，包含了衆多編解碼器需要的參數資訊，位于avcodec.h檔案中。
AVDictionary：
簡單的key/value存儲，經常使用AVDictionary設定或讀取内部參數，具體實作在libavutil子產品中的dict.c/h，
AVCodec：
存儲編解碼器資訊的結構體，位于avcodec.h檔案中。
AVStream：
是存儲每一個視訊/音頻流資訊的結構體，位于avformat.h檔案中。
AVPacket：
是FFmpeg中很重要的一個資料結構，它儲存了解複用（demuxer)之後，解碼（decode）之前的資料（仍然是壓縮後的資料）和關于這些資料的一些附加的資訊，如顯示時間戳（pts），解碼時間戳（dts）,資料時長（duration），所在流媒體的索引（stream_index）等等。
AVFrame：
存儲一幀的資料，在解碼中，AVFrame是解碼器的輸出；在編碼中，AVFrame是編碼器的輸入。
SwsContext：
主要用于圖檔格式轉換，用sws_getContext()擷取，sws_freeContext()釋放。
流程中的主要函數（不想看具體參數參考流程圖即可）
av_register_all()：
注冊FFmpeg所有協定、容器、編解碼器。
avformat_alloc_output_context2()：
初始化輸出碼流的AVFormatContext。
/**
* Allocate an AVFormatContext for an output format.
* avformat_free_context() can be used to free the context and
* everything allocated by the framework within it.
*
* @param *ctx is set to the created format context, or to NULL in
* case of failure
* @param oformat format to use for allocating the context, if NULL
* format_name and filename are used instead
* @param format_name the name of output format to use for allocating the
* context, if NULL filename is used instead
* @param filename the name of the filename to use for allocating the
* context, may be NULL
* @return >= 0 in case of success, a negative AVERROR code in case of
* failure
*/
int avformat_alloc_output_context2(AVFormatContext **ctx, ff_const59 AVOutputFormat *oformat,
const char *format_name, const char *filename);
avio_open()：
打開輸出檔案。
/**
* Create and initialize a AVIOContext for accessing the
* resource indicated by url.
* @note When the resource indicated by url has been opened in
* read+write mode, the AVIOContext can be used only for writing.
*
* @param s Used to return the pointer to the created AVIOContext.
* In case of failure the pointed to value is set to NULL.
* @param url resource to access
* @param flags flags which control how the resource indicated by url
* is to be opened
* @return >= 0 in case of success, a negative value corresponding to an
* AVERROR code in case of failure
*/
int avio_open(AVIOContext **s, const char *url, int flags);
avcodec_find_encoder()：
找到編碼器。
/**
* Find a registered encoder with a matching codec ID.
*
* @param id AVCodecID of the requested encoder
* @return An encoder if one was found, NULL otherwise.
*/
AVCodec *avcodec_find_encoder(enum AVCodecID id);
av_new_stream()：
建立輸出碼流的AVStream。
/**
* Add a new stream to a media file.
*
* When demuxing, it is called by the demuxer in read_header(). If the
* flag AVFMTCTX_NOHEADER is set in s.ctx_flags, then it may also
* be called in read_packet().
*
* When muxing, should be called by the user before avformat_write_header().
*
* User is required to call avcodec_close() and avformat_free_context() to
* clean up the allocation by avformat_new_stream().
*
* @param s media file handle
* @param c If non-NULL, the AVCodecContext corresponding to the new stream
* will be initialized to use this codec. This is needed for e.g. codec-specific
* defaults to be set, so codec should be provided if it is known.
*
* @return newly created stream or NULL on error.
*/
AVStream *avformat_new_stream(AVFormatContext *s, const AVCodec *c);
avcodec_open2()：
打開編碼器。
/**
* Initialize the AVCodecContext to use the given AVCodec. Prior to using this
* function the context has to be allocated with avcodec_alloc_context3().
*
* The functions avcodec_find_decoder_by_name(), avcodec_find_encoder_by_name(),
* avcodec_find_decoder() and avcodec_find_encoder() provide an easy way for
* retrieving a codec.
*
* @warning This function is not thread safe!
*
* @note Always call this function before using decoding routines (such as
* @ref avcodec_receive_frame()).
*
* @code
* avcodec_register_all();
* av_dict_set(&opts, "b", "2.5M", 0);
* codec = avcodec_find_decoder(AV_CODEC_ID_H264);
* if (!codec)
* exit(1);
*
* context = avcodec_alloc_context3(codec);
*
* if (avcodec_open2(context, codec, opts) < 0)
* exit(1);
* @endcode
*
* @param avctx The context to initialize.
* @param codec The codec to open this context for. If a non-NULL codec has been
* previously passed to avcodec_alloc_context3() or
* for this context, then this parameter MUST be either NULL or
* equal to the previously passed codec.
* @param options A dictionary filled with AVCodecContext and codec-private options.
* On return this object will be filled with options that were not found.
*
* @return zero on success, a negative value on error
* @see avcodec_alloc_context3(), avcodec_find_decoder(), avcodec_find_encoder(),
* av_dict_set(), av_opt_find().
*/
int avcodec_open2(AVCodecContext *avctx, const AVCodec *codec, AVDictionary **options);
avformat_write_header()：
寫入流标頭，也就是檔案格式的檔案頭（對于某些沒有檔案頭的封裝格式，不需要此函數。比如說MPEG2TS）。
/**
* Allocate the stream private data and write the stream header to
* an output media file.
*
* @param s Media file handle, must be allocated with avformat_alloc_context().
* Its oformat field must be set to the desired output format;
* Its pb field must be set to an already opened AVIOContext.
* @param options An AVDictionary filled with AVFormatContext and muxer-private options.
* On return this parameter will be destroyed and replaced with a dict containing
* options that were not found. May be NULL.
*
* @return AVSTREAM_INIT_IN_WRITE_HEADER on success if the codec had not already been fully initialized in avformat_init,
* AVSTREAM_INIT_IN_INIT_OUTPUT on success if the codec had already been fully initialized in avformat_init,
* negative AVERROR on failure.
*
* @see av_opt_find, av_dict_set, avio_open, av_oformat_next, avformat_init_output.
*/
int avformat_write_header(AVFormatContext *s, AVDictionary **options);
avcodec_encode_video2()：
編碼一幀視訊。即将AVFrame（存儲YUV像素資料）編碼為AVPacket（存儲視訊格式的碼流資料）。
/**
* Encode a frame of video.
*
* Takes input raw video data from frame and writes the next output packet, if
* available, to avpkt. The output packet does not necessarily contain data for
* the most recent frame, as encoders can delay and reorder input frames
* internally as needed.
*
* @param avctx codec context
* @param avpkt output AVPacket.
* The user can supply an output buffer by setting
* avpkt->data and avpkt->size prior to calling the
* function, but if the size of the user-provided data is not
* large enough, encoding will fail. All other AVPacket fields
* will be reset by the encoder using av_init_packet(). If
* avpkt->data is NULL, the encoder will allocate it.
* The encoder will set avpkt->size to the size of the
* output packet. The returned data (if any) belongs to the
* caller, he is responsible for freeing it.
*
* If this function fails or produces no output, avpkt will be
* freed using av_packet_unref().
* @param[in] frame AVFrame containing the raw video data to be encoded.
* May be NULL when flushing an encoder that has the
* AV_CODEC_CAP_DELAY capability set.
* @param[out] got_packet_ptr This field is set to 1 by libavcodec if the
* output packet is non-empty, and to 0 if it is
* empty. If the function returns an error, the
* packet can be assumed to be invalid, and the
* value of got_packet_ptr is undefined and should
* not be used.
* @return 0 on success, negative error code on failure
*
* @deprecated use avcodec_send_frame()/avcodec_receive_packet() instead
*/
int avcodec_encode_video2(AVCodecContext *avctx, AVPacket *avpkt,
const AVFrame *frame, int *got_packet_ptr);
av_write_frame()：
将編碼後的視訊碼流寫入檔案。
/**
* Write a packet to an output media file.
*
* This function passes the packet directly to the muxer, without any buffering
* or reordering. The caller is responsible for correctly interleaving the
* packets if the format requires it. Callers that want libavformat to handle
* the interleaving should call av_interleaved_write_frame() instead of this
* function.
*
* @param s media file handle
* @param pkt The packet containing the data to be written. Note that unlike
* av_interleaved_write_frame(), this function does not take
* ownership of the packet passed to it (though some muxers may make
* an internal reference to the input packet).
* <br>
* This parameter can be NULL (at any time, not just at the end), in
* order to immediately flush data buffered within the muxer, for
* muxers that buffer up data internally before writing it to the
* output.
* <br>
* Packet's @ref AVPacket.stream_index "stream_index" field must be
* set to the index of the corresponding stream in @ref
* AVFormatContext.streams "s->streams".
* <br>
* The timestamps (@ref AVPacket.pts "pts", @ref AVPacket.dts "dts")
* must be set to correct values in the stream's timebase (unless the
* output format is flagged with the AVFMT_NOTIMESTAMPS flag, then
* they can be set to AV_NOPTS_VALUE).
* The dts for subsequent packets passed to this function must be strictly
* increasing when compared in their respective timebases (unless the
* output format is flagged with the AVFMT_TS_NONSTRICT, then they
* merely have to be nondecreasing). @ref AVPacket.duration
* "duration") should also be set if known.
* @return < 0 on error, = 0 if OK, 1 if flushed and there is no more data to flush
*
* @see av_interleaved_write_frame()
*/
int av_write_frame(AVFormatContext *s, AVPacket *pkt);
av_write_trailer()：
寫檔案尾（對于某些沒有檔案頭的封裝格式，不需要此函數。比如說MPEG2TS）。
/**
* Write the stream trailer to an output media file and free the
* file private data.
*
* May only be called after a successful call to avformat_write_header.
*
* @param s media file handle
* @return 0 if OK, AVERROR_xxx on error
*/
int av_write_trailer(AVFormatContext *s);
源碼
void MainWindow::pngTomp4_2() //png圖集轉mp4
{
QString folder_path = QFileDialog::getExistingDirectory(this,"選擇檔案目錄","",QFileDialog::ShowDirsOnly
| QFileDialog::DontResolveSymlinks);
if(folder_path.isEmpty()) return;
QString outFilename = QFileDialog::getSaveFileName(this,"選擇儲存路徑",folder_path+"/視訊檔案","*.mp4;; *.avi");
if(outFilename.isEmpty()) return;
// QStringList filters;
// filters<<QString("*.png");
// 定義疊代器并設定過濾器
// QDirIterator dir_iterator(folder_path,filters, QDir::Files | QDir::NoSymLinks, QDirIterator::Subdirectories);
// QList<QString > pnglist;
// while(dir_iterator.hasNext()) //擷取目錄下的png檔案
// {
// dir_iterator.next();
// QFileInfo file_info = dir_iterator.fileInfo();
// pnglist.append(QString(tr("%1").arg(file_info.absoluteFilePath())));
// }
AVFormatContext *kAVFormatContext;
AVOutputFormat *kAVOutputFormat;
AVCodecContext *kAVCodecContext;
AVDictionary *kAVDictionary = NULL;
AVDictionary *opt = NULL;
AVCodec *video_codec;
AVStream *kAVStream;
AVFrame *frame;
AVFrame *tmp_frame;
AVPacket kAVPacket = { 0 };
struct SwsContext *kSwsContext;
AVRational time{1,24}; /*1s25幀*/
AVRational time_1{1,1}; /*1s25幀*/
uint8_t *intBuffer = 0; // 圖檔資料緩沖區
int ret;
int got_packer = 0;
int64_t next_pts = 0;
tmp_frame = av_frame_alloc();
av_init_packet(&kAVPacket);
av_register_all(); //1、注冊所有容易和編解碼器
/* 2、配置設定輸出媒體上下文 */
avformat_alloc_output_context2(&kAVFormatContext, NULL, NULL, outFilename.toLocal8Bit().data()); //上下文結構體、輸出結構體、檔案類型、路徑
if(!kAVFormatContext) //如果根據檔案名沒有找到對應的格式則預設mpeg格式
{
ret = avformat_alloc_output_context2(&kAVFormatContext, NULL, "mpeg", outFilename.toLocal8Bit().data()); //沒有找到檔案類型預設mpeg（MP4）
}
if(!kAVOutputFormat) { goto end;}
kAVOutputFormat = kAVFormatContext->oformat; //把輸出操作結構體指派出來
if (!(kAVOutputFormat->flags & AVFMT_NOFILE)) /* 如果需要，打開輸出檔案*/
{
ret = avio_open(&kAVFormatContext->pb, outFilename.toLocal8Bit().data(), AVIO_FLAG_WRITE); //3、打開輸出檔案
if (ret < 0) {qDebug()<<"打開輸出檔案失敗。"<<ret; }
}
/* 使用預設格式的編解碼器添加音頻和視訊流，并初始化編解碼器。*/
if (kAVOutputFormat->video_codec != AV_CODEC_ID_NONE)
{
video_codec = avcodec_find_encoder(kAVOutputFormat->video_codec); //4、找到編碼器
if(!video_codec) goto end;
kAVStream = avformat_new_stream(kAVFormatContext,NULL); //5、建立一個輸出流
if(!kAVStream) {qDebug()<<"建立流kAVStream失敗。";goto end;}
kAVCodecContext = avcodec_alloc_context3(video_codec); //初始化一個AVCodecContext
if(!kAVCodecContext) {qDebug()<<"用編碼器video_codec初始化的kAVCodecContext預設參數失敗";goto end;}
switch(video_codec->type)
{
case AVMEDIA_TYPE_VIDEO:
kAVCodecContext->codec_id = video_codec->id;
kAVCodecContext->bit_rate = 800000;
kAVCodecContext->width = 720;
kAVCodecContext->height = 404;
kAVStream->time_base = time;
kAVCodecContext->time_base = time;
kAVCodecContext->gop_size = 12; /*最多每十二幀發射一個内幀 */
kAVCodecContext->pix_fmt = AV_PIX_FMT_YUV420P;
if (kAVCodecContext->codec_id == AV_CODEC_ID_MPEG2VIDEO)
{
kAVCodecContext->max_b_frames = 2; /*為了測試，我們還添加了b幀*/
}
if (kAVCodecContext->codec_id == AV_CODEC_ID_MPEG1VIDEO)
{
/* 需要避免使用一些coeffs溢出的宏塊。這在正常的視訊中不會發生，
* 隻是在色度平面的運動與luma平面不比對時才會發生。 */
kAVCodecContext->mb_decision = 2;
}
break;
case AVMEDIA_TYPE_AUDIO:
break;
default:
break;
}
if(kAVOutputFormat->flags & AVFMT_GLOBALHEADER)/*有些格式希望流标頭是單獨的*/
{
kAVCodecContext->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
}
}
av_dict_copy(&opt,kAVDictionary, 0);
ret = avcodec_open2(kAVCodecContext,video_codec,&opt); //6、打開編碼器
if(ret<0){qDebug()<<"打開視訊編解碼器失敗"<<ret; goto end;}
av_dict_free(&opt);
frame = av_frame_alloc();
if(!frame) {qDebug()<<"配置設定幀失敗."; goto end;}
frame->format = kAVCodecContext->pix_fmt;
frame->width = kAVCodecContext->width;
frame->height = kAVCodecContext->height;
ret = av_frame_get_buffer(frame, 32); //配置設定記憶體空間frame必須要有上三條件，32也就是4位元組對齊
if(ret < 0){qDebug()<<"frame配置設定記憶體失敗"; goto end;}
ret = avcodec_parameters_from_context(kAVStream->codecpar,kAVCodecContext);/*将流參數複制到muxer */
if (ret < 0) {qDebug()<<"無法複制流參數";goto end; }
av_dump_format(kAVFormatContext, 0, outFilename.toLocal8Bit().data(), 1); //列印輸出檔案的詳細資訊
ret = avformat_write_header(kAVFormatContext,&kAVDictionary);/* 7、寫流标頭(如果有的話)*/
if(ret < 0){qDebug()<<"寫流标題失敗"; goto end;}
//8、寫入每一幀資料
for(int i = 0; i<527; i++)
{
// if(av_compare_ts(next_pts, kAVCodecContext->time_base, 10.0, time_1) > 0) /*這裡隻生成10s的視訊*/
// {
// qDebug()<<"暫時不需要生成新的幀"; break;
// }
if (av_frame_make_writable(frame) < 0) {goto end;} /*當我們傳遞一個幀給編碼器時，它可能會在内部保留一個對它的引用;確定我們沒有在這裡覆寫它*/
QImage img(tr("%1/%2.png").arg(folder_path).arg(i));
if(img.isNull()){qDebug()<<"打開圖檔失敗";break;}
img = img.convertToFormat(QImage::Format_RGB888);
kSwsContext = sws_getContext(kAVCodecContext->width,kAVCodecContext->height,
AV_PIX_FMT_RGB24,kAVCodecContext->width,kAVCodecContext->height,
AV_PIX_FMT_YUV420P,SWS_LANCZOS | SWS_ACCURATE_RND,NULL,NULL,NULL);
if(!kSwsContext){qDebug()<<"無法初始化圖檔轉換器"; goto end;}
intBuffer = (uint8_t*)malloc(sizeof(uint8_t)*img.bytesPerLine()*img.height());
memcpy(intBuffer,img.constBits(),sizeof(uint8_t)*img.bytesPerLine()*img.height());
avpicture_fill((AVPicture *)tmp_frame,intBuffer,AV_PIX_FMT_RGB24,kAVCodecContext->width,kAVCodecContext->height); //pAVFrame32的data指針指向了outBuffer
sws_scale(kSwsContext,(const uint8_t * const *)tmp_frame->data,
tmp_frame->linesize,0,kAVCodecContext->height,frame->data,frame->linesize); //圖檔資訊轉換
sws_freeContext(kSwsContext);
kSwsContext = 0;
frame->pts = next_pts++;
//frame->pts = i*(kAVStream->time_base.den)/((kAVStream->time_base.num)*24);
got_packer = 0;
ret = avcodec_encode_video2(kAVCodecContext,&kAVPacket,frame,&got_packer); //8、編碼、把一幀資料編碼成AVPacket
if(ret < 0) {qDebug()<<"錯誤編碼視訊幀"<<ret; goto end;}
if(got_packer)
{
av_packet_rescale_ts(&kAVPacket, kAVCodecContext->time_base, kAVStream->time_base);
kAVPacket.stream_index = kAVStream->index;
ret = av_write_frame(kAVFormatContext, &kAVPacket); /*将壓縮幀寫入媒體檔案。 */
av_free_packet(&kAVPacket);
}else
{ret = 0;}
free(intBuffer);intBuffer = 0;
if(ret<0){ qDebug()<<"寫入video檔案失敗"<<ret<<kAVPacket.stream_index; break; }
}
/* 選擇寫預告片，如果有的話。預告片必須在你之前寫好 當你寫标題時關閉CodecContexts打開;否則
av_write_trailer()可以嘗試使用被釋放的記憶體 av_codec_close()。要編碼的流 */
av_write_trailer(kAVFormatContext);
end:
/* 關閉每個編解碼器。 */
avcodec_free_context(&kAVCodecContext);
av_frame_free(&frame);
if (!(kAVOutputFormat->flags & AVFMT_NOFILE))
avio_closep(&kAVFormatContext->pb);/*關閉輸出檔案*/
avformat_free_context(kAVFormatContext); /*釋放上下文*/
}