天天看點

目标跟蹤-object track

大牛推薦

湊個數,目前相關濾波方向貢獻最多的是以下兩個組(有創新有代碼):

牛津大學:Joao F. Henriques和Luca Bertinetto,代表:CSK, KCF/DCF, Staple, CFNet (其他SiamFC, Learnet).

林雪平大學:Martin Danelljan,代表:CN, DSST, SRDCF, DeepSRDCF, SRDCFdecon, C-COT, ECO.

國内也有很多高校的優秀工作就不一一列舉了。

相關濾波

介紹最經典的高速相關濾波類跟蹤算法CSK, KCF/DCF, CN。很多人最早了解CF,應該和我一樣,都是被下面這張圖吸引了:

目标跟蹤-object track

這是KCF/DCF算法在OTB50上(2014年4月就挂arVix了, 那時候OTB100還沒有發表)的實驗結果,Precision和FPS碾壓了OTB50上最好的Struck,看慣了勉強實時的Struck和TLD,飙到高速的KCF/DCF突然有點讓人不敢相信,其實KCF/DCF就是在OTB上大放異彩的CSK的多通道特征改進版本。注意到那個超高速615FPS的MOSSE(嚴重超速這是您的罰單),這是目标跟蹤領域的第一篇相關濾波類方法,這其實是真正第一次顯示了相關濾波的潛力。和KCF同一時期的還有個CN,在2014'CVPR上引起劇烈反響的顔色特征方法,其實也是CSK的多通道顔色特征改進算法。從MOSSE(615)到 CSK(362) 再到 KCF(172FPS), DCF(292FPS), CN(152FPS), CN2(202FPS),速度雖然是越來越慢,但效果越來越好,而且始終保持在高速水準:

  • Bolme D S, Beveridge J R, Draper B A, et al. Visual object tracking using adaptive correlation filters [C]// CVPR, 2010.
  • Henriques J F, Caseiro R, Martins P, et al. Exploiting the circulant structure of tracking-by- detection with kernels [C]// ECCV, 2012.
  • Henriques J F, Rui C, Martins P, et al. High-Speed Tracking with Kernelized Correlation Filters [J]. IEEE TPAMI, 2015.
  • Danelljan M, Shahbaz Khan F, Felsberg M, et al. Adaptive color attributes for real-time visual tracking [C]// CVPR, 2014.
目标跟蹤-object track

CSK和KCF都是Henriques J F(牛津大學)João F. Henriques 大神先後兩篇論文,影響後來很多工作,核心部分的嶺回歸,循環移位的近似密集采樣,還給出了整個相關濾波算法的詳細推導。還有嶺回歸加kernel-trick的封閉解,多通道HOG特征。

Martin Danelljan大牛(林雪平大學)用多通道顔色特征Color Names(CN)去擴充CSK得到了不錯的效果,算法也簡稱CN Coloring Visual Tracking 。

MOSSE是單通道灰階特征的相關濾波,CSK在MOSSE的基礎上擴充了密集采樣(加padding)和kernel-trick,KCF在CSK的基礎上擴充了多通道梯度的HOG特征,CN在CSK的基礎上擴充了多通道顔色的Color Names。HOG是梯度特征,而CN是顔色特征,兩者可以互補,是以HOG+CN在近兩年的跟蹤算法中成為了hand-craft特征标配。最後,根據KCF/DCF的實驗結果,讨論兩個問題:

  • 1. 為什麼隻用單通道灰階特征的KCF和用了多通道HOG特征的KCF速度差異很小?

第一,作者用了HOG的快速算法fHOG,來自Piotr's Computer Vision Matlab Toolbox,C代碼而且做了SSE優化。如對fHOG有疑問,請參考論文Object Detection with Discriminatively Trained Part Based Models第12頁。

第二,HOG特征常用cell size是4,這就意味着,100*100的圖像,HOG特征圖的次元隻有25*25,而Raw pixels是灰階圖歸一化,次元依然是100*100,我們簡單算一下:27通道HOG特征的複雜度是27*625*log(625)=47180,單通道灰階特征的複雜度是10000*log(10000)=40000,理論上也差不多,符合表格。

看代碼會發現,作者在擴充後目标區域面積較大時,會先對提取到的圖像塊做因子2的下采樣到50*50,這樣複雜度就變成了2500*log(2500)=8495,下降了非常多。那你可能會想,如果下采樣再多一點,複雜度就更低了,但這是以犧牲跟蹤精度為代價的,再舉個例子,如果圖像塊面積為200*200,先下采樣到100*100,再提取HOG特征,分辨率降到了25*25,這就意味着響應圖的分辨率也是25*25,也就是說,響應圖每位移1個像素,原始圖像中跟蹤框要移動8個像素,這樣就降低了跟蹤精度。在精度要求不高時,完全可以稍微犧牲下精度提高幀率(但看起來真的不能再下采樣了)。

  • 2. HOG特征的KCF和DCF哪個更好?

大部分人都會認為KCF效果超過DCF,而且各屬性的準确度都在DCF之上,然而,如果換個角度來看,以DCF為基準,再來看加了kernel-trick的KCF,mean precision僅提高了0.4%,而FPS下降了41%,這麼看是不是挺驚訝的呢?除了圖像塊像素總數,KCF的複雜度還主要和kernel-trick相關。是以,下文中的CF方法如果沒有kernel-trick,就簡稱基于DCF,如果加了kernel-trick,就簡稱基于KCF(劇透基本各占一半)。當然這裡的CN也有kernel-trick,但請注意,這是Martin Danelljan大神第一次使用kernel-trick,也是最後一次。。。

這就會引發一個疑問,kernel-trick這麼強大的東西,怎麼才提高這麼點?這裡就不得不提到Winsty的另一篇大作:

  • Wang N, Shi J, Yeung D Y, et al. Understanding and diagnosing visual tracking systems[C]// ICCV, 2015.

一句話總結,别看那些五花八門的機器學習方法,那都是虛的,目标跟蹤算法中特征才是最重要的(就是因為這篇文章我粉了WIN叔哈哈),以上就是最經典的三個高速算法,CSK, KCF/DCF和CN,推薦。

總結:

MOSSE:單通道灰階特征的相關濾波

CSK:基于MOSSE增加Padding和 kernel-trick

KCF:多通道梯度HOG特征的CSK

CN:多通道顔色特征的CSK

DCF:沒有kernel-trick的KCF

CSK:foolwood/CSK Qiang Wang實作的

I just want to build a c++ preject for CSK.

##It looks like MATLAB.

##Simple gui(I will change it to the KCF version 2 via Trackbar) It's quite difficult to draw gui like MATLAB,but the Trackbar function is quite useful!(if you tried KCF MATLAB code)

KCF:joaofaro/KCFcpp (高人氣推薦) 原作者提供的

"KCFC++", command: ./KCF 原版HOG特征的KCF算法

Description: KCF on HOG features, ported to C++ OpenCV. The original Matlab tracker placed 3rd in VOT 2014.

"KCFLabC++", command: ./KCF lab 擴充Lab顔色特征的KCF算法

Description: KCF on HOG and Lab features, ported to C++ OpenCV. The Lab features are computed by quantizing CIE-Lab colors into 15 centroids, obtained from natural images by k-means.

The CSK tracker [2] is also implemented as a bonus, simply by using raw grayscale as features (the filter becomes single-channel). 調用灰階特征就是CSK

KCF:foolwood/KCF 還是Qiang Wang,完全實作MATLAB代碼,可以對照學習

It denpends on OpenCV, so you have to install OpenCV first.

I change fhog from computeHOG32D,newly opencv_contrib

Now I use fhog from Piotr's Computer Vision Matlab Toolbox and wrapper by Tomas Vojir

This algorithm is belong to the author of KCFJoão F. Henriques, Rui Caseiro, Pedro Martins, Jorge Batista

CN:mostafaizz/ColorTracker 按照MATLAB代碼實作的

This C++ code is an implementation of the visual tracking method proposed in [1]. The implementation is based on the Matlab code provided by the authors of the paper.

The implementation in C++ using openCV was done by Mostafa Izz.

DSST:liliumao/KCF-DSST 在原作者KCF的C++代碼基礎上,增加scale filter實作DSST,比DSST的MATLAB代碼多了gaussian-kernel

This package includes a C++ class with several tracking methods based on the Kernelized Correlation Filter (KCF) [1, 2] for translation changes and the Discriminative Scale Space Tracker (DSST) [3] .

DSST scaling changing part is added to the original kcftracker.cpp/hpp file. Original recttools.hpp and ffttools.hpp are also modified. Debug mode is added to the Cmakelists.txt.

fDSST:TuringKi/fDSST_cpp,沒用過

C++ re-implementation of fast Discriminative Scale Space Tracking.

SAMF:vojirt/kcf (高人氣推薦) 最後這個雖說是KCF代碼,但擴充了CN特征,還加了7步長的尺度檢測scale detection,這不SAMF是什麼

This is a C++ reimplementation of algorithm presented in "High-Speed Tracking with Kernelized Correlation Filters" paper. For more info and implementation in other languages visit the autor's webpage!.

It is extended by a scale estimation (use several 7 different scales steps) and by a RGB (channels) and Color Names [2] features. Data for Color Names features were obtained from SAMF tracker.

It is free for research use. If you find it useful or use it in your research, please acknowledge my git repository and cite the original paper [1].

The code depends on OpenCV 2.4+ library and is build via cmake toolchain.

https://www.zhihu.com/question/26493945 

繼續閱讀