圖像融合方法分類
low level (pixel) mide level (feature) high level (symbolic, image)
- transform fusion
- spatial fusion： fusion rules are directly applied to image pixels or image regions. 空域融合進一步分為：（1）block （2）segmentation based (3) gradient based
according to lishutao 2017 <information fusion>
1 multiscale decomposition
2 sparse representation based
3 other transforms like pca ida
4 combination of different transforms
圖像融合常見問題
artifact 人工産品，指融合後圖像不自然
halo artifact； halo 指光暈 halo artifact near some edges
contras decrease
reduction of sharpness
評價名額 object quality metric
參考文獻： zheng liu 等 Objective Assessment of Multiresolution Image Fusion Algorithms for Context Enhancement
in Night Vision: A Comparative Study TPAMI 2012
- information theory based ones
1 Normalized Mutual Information Q MI G. Qu, D. Zhang, and P. Yan, “Information Measure for Performance of Image Fusion,” Electronics Letters, vol. 38, no. 7, pp. 313-315, 2002. M. Hossny, S. Nahavandi, and D. Vreighton, “Comments on Information Measure for Performance of Image Fusion’,” Electronics Letters, vol. 44, no. 18, pp. 1066-1067, Aug. 2008.
2 Tsallis Entropy QTE R. Nava, G. Cristo´ bal, and B. Escalante-Ramı´rez, “Mutual Information Improves Image Fusion Quality Assessments,” SPIE
News Room, http://spie.org/documents/Newsroom/Imported/0824/0824-2007-08-30.pdf, Sept. 2007. 3 Nonlinear Correlation Information Eentropy Q NCIE Q. Wang, Y. Shen, and J. Jin, “Performance Evaluation of Image Fusion Techniques,” Image Fusion: Algorithms and Applications,ch. 19, T. Stathaki, ed., pp. 469-492. Elsevier, 2008.
- image feature based
4 Gradient based fusion performance QG [a.k.a, QAB/F]
C.S. Xydeas and V. Petrovic, “Objective Image Fusion Performance Measure,” Electronics Letters, vol. 36, no. 4, pp. 308-309, 2000.
5 based on a multiscale frame Q M P. Wang and B. Liu, “A Novel Image Fusion Metric Based on Multi-Scale Analysis,” Proc. IEEE Int’l Conf. Signal Processing,
pp. 965-968, 2008.
6 based on Spatial Frenquency QSF
Y. Zheng, E.A. Essock, B.C. Hansen, and A.M. Haun, “A New Metric Based on Extended Spatial Frequency and Its Application
to DWT Based Fusion Algorithms,” Information Fusion, vol. 8, no. 2, pp. 177-192, Apr. 2007.
7 based on Phase congruency QP
J. Zhao, R. Laganiere, and Z. Liu, “Performance Assessment of Combinative Pixel-Level Image Fusion Based on an Absolute Feature Measurement,” Int’l J. Innovative Computing, Information and Control, vol. 3, no. 6(A), pp. 1433-1447, Dec. 2007
- image structure similarity based
8 Piella's metric QP
G. Piella and H. Heijmans, “A New Quality Metric for Image Fusion,” Proc. Int’l Conf. Image Processing, 2003.
9 Cvejie's metric QC
N. Cvejic, A. Loza, D. Bul, and N. Canagarajah, “A Similarity Metric for Assessment of Image Fusion Algorithms,” Int’l J. Signal
Processing, vol. 2, no. 3, pp. 178-182, 2005.
10 Yang's metric QY
C. Yang, J. Zhang, X. Wang, and X. Liu, “A Novel Similarity BasedQuality Metric for Image Fusion,” Information Fusion, vol. 9,
pp. 156-160, 2008.
- human perception based
11 Chen-Varshney metric QCV
H. Chen and P.K. Varshney, “A Human Perception Inspired Quality Metric for Image Fusion Based on Regional Information,”
Information Fusion, vol. 8, pp. 193-207, 2007.
12 Y.Chen R.S.Blum QCB [a.k.a QHVS]
Y. Chen and R. Blum, “A new automated quality assessment algorithm for image fusion,” Image Vis. Comput., vol. 27, no. 10, pp. 1421–1432,
2009.
該綜述性評價文章發表于12年，此後還有别的方法。
13 Visual Information Fidility Fusion metric VIFF
Y. Han, Y. Cai, Y. Cao, and X. Xu, “A new image fusion performance metric based on visual information fidelity,” Inf. Fusion, vol. 14, pp. 127–135, 2013.
For most existing fusion methods, either in spatial domain or transform domain, the activity level measurement is essentially implemented by designing local filters to extract high-frequency details. On one hand, for most transform domain fusion methods, the images or image patches are represented using a set of predesigned bases such as wavelet or trained dictionary atoms. From the view of image processing, this is generally equivalent to convolving them with those bases . For example, the implementation of discrete wavelet transform is exactly based on filtering. On the other hand, for spatial domain fusion methods, the situation is even clearer that so many activity level measurements are based on high-pass spatial filtering. Furthermore, the fusion rule, which is usually interpreted as the weight assignment strategy for different source images based on the calculated activity level measures can be transformed into a filtering-based form as well (個人了解： activitity level measurement 或許等同于 image clarity measure, focus measure 常用方法有 energy of laplacian 以及 spatial frequency)
應用領域
數字成像（digital phtotgrapyh），主要包括 multi focus multi exposure
多模态融合（multi modality imaging）主要包括，medical image fusion, visible /infrared image fusion
遙感圖像（remote sensing） 例如 MS PAN , MS HS
深度模型
CNN
CSR （convolutional sparse representation）
SAE(stack autoencoder)
概念
CNN 能夠 multistage/ hierarchical representation 網絡結構 architecture
圖1 傳統模型， 基于多尺度變換的圖像融合
圖2 傳統模型 ，基于稀疏表示的圖像融合
以下圖示來自 yu liu 等發表于information fusion 的文章： <Deep learning for pixellevel image fusion: Recent advances and future prospects>