A Gabor Feature-based Quality Assessment Model for the Screen Content Images

Zhangkai Ni1, Huanqiang Zeng1, Lin Ma2, Junhui Hou3,Jing Chen1, and Kai-Kuang Ma4

1School of Information Science and Engineering, Huaqiao University, Fujian, China

2Tencent AI Lab, Shenzhen

3Department of Computer Science, City University of Hong Kong, Hong Kong

4School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore

[Paper] [GFM Code]

Introduction

In this paper, an accurate and efficient full-reference image quality assessment (IQA) model using the extracted Gabor features, called Gabor feature-based model (GFM), is proposed for conducting objective evaluation of screen content images (SCIs). It is well-known that the Gabor filters are highly consistent with the response of the human visual system (HVS), and the HVS is highly sensitive to the edge information. Based on these facts, the imaginary part of the Gabor filter that has odd symmetry and yields edge detection is exploited to the luminance of the reference and distorted SCI for extracting their Gabor features, respectively. The local similarities of the extracted Gabor features and two chrominance components, recorded in the LMN color space, are then measured independently. Finally, the Gabor-feature pooling strategy is employed to combine these measurements and generate the final evaluation score. Experimental simulation results obtained from two large SCI databases have shown that the proposed GFM model not only yields a higher consistency with the human perception on the assessment of SCIs but also requires a lower computational complexity, compared with that of classical and state-of-the-art IQA models.

Gabor feature-based model for SCI Quality Assessment


Fig. 1. The framework of our proposed GFM for SCI.


The framework of our proposed GFM for SCI is illustrated in Fig. 1. In our approach, therefore the Gabor features are first extracted from the luminance (i.e., the L component recorded in the LMN color space) of the reference and distorted SCI, separately. On this feature-extraction process, a specially-designed Gabor filtering (i.e., the imaginary part with odd symmetry) is conducted on the horizontal and the vertical directions, respectively. The obtained filtering results are combined to form the Gabor feature map. The degree of similarity measurement is then conducted on these maps for the luminance part and for the chrominance components inde- pendently, between the reference and distorted SCIs. Finally, the developed Gabor-feature pooling strategy is employed to combine these measurements and generate the final IQA evaluation score for the SCI under evaluation.

Experimental Results

Table 1. Performance Comparison of Different IQA models on the SIQAD and SCID databases.


Fig. 2. SROCC VS. running time of various IQA models on SCID dataset.


Computational Complexity Comparison:


In addition to the accuracy, the efficiency of the IQA model is another figure of merit that needs to be assessed, especially for practical applications. For that, the average running time per image incurred for each IQA model by experimenting on the SCID database (resolution of 1280℅720 for each image) is measured to evaluate its computational complexity. Although the PSNR, SSIM, GSIM, and GMSD are faster than the proposed GFM model, their accuracy measurements are much inferior to ours as shown in Table 1. The Fig. 2 shows the SROCC versus running time of our GFM compared to other IQA models. It can be observed that the proposed GFM model requires a relatively low computational complexity. Among the IQA models with top-four performances (i.e., GFM, SVQI, ESIM, and SQMS), the proposed GFM requires the least amount of computational time, while delivering fairly high accuracy.

Reference

GFM
Z. Ni, H. Zeng, L. Ma, J. Hou, J. Chen, and K.-K. Ma "A Gabor Feature-based Quality Assessment Model for the Screen Content Images," IEEE Transactions on Image Processing, vol. 27, no. 9, pp. 4516-4528, Sept. 2018. [Full Text]
SSIM
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image Quality Assessment: From Error Visibility to Structural Similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004. [Full Text]
IWSSIM
Z. Wang, Q. Li, "Information Content Weighting for Perceptual Image Quality Assessment," IEEE Transactions on Image Processing, vol. 20, no. 5, pp. 1185-1198, Apri. 2011. [Full Text]
VIF
H. R. Sheikh, A. C. Bovik, "Image Information and Visual Quality," IEEE Transactions on Image Processing, vol. 15, no. 2, pp. 430-444, Feb. 2006. [Full Text]
MAD
E. C. Larson, D. M. Chandler, "Most Apparent Distortion: Full-Reference Image Quality Assessment and the Role of Strategy," Journal of Electronic Imaging, vol. 19, no. 1, Jan. 2010. [Full Text]
FSIM
L. Zhang, L. Zhang, X. Mou, and D. Zhang, "FSIM: A Feature Similarity Index for Image Quality Assessment," IEEE Transactions on Image Processing, vol. 20, no. 8, pp. 2378-2386, Aug. 2011. [Full Text]
GSIM
A. Liu, W. Lin, and M. Narwaria, "Image Quality Assessment Based on Gradient Similarity," IEEE Transactions on Image Processing, vol. 21, no. 4, pp. 1500-1512, Apr. 2012. [Full Text]
GMSD
W. Xue, L. Zhang, X. Mou, and A. C. Bovik, "Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index," IEEE Transactions on Image Processing, vol. 23, no. 2, pp. 684-695, Feb. 2014. [Full Text]
VSI
L. Zhang, Y. Shen, and H. Li, "VSI: A Visual Saliency-Induced Index for Perceptual Image Quality Assessment," IEEE Transactions on Image Processing, vol. 23, no. 10, pp. 4270-4281, Oct. 2014. [Full Text]
SIQM
K. Gu, S. Wang, G. Zhai, S. Ma, and W. Lin, "Screen image quality assessment incorporating structural degradation measurement," IEEE International Symposium on Circuits and Systems, pp. 125每128, May 2015. [Full Text]
SPQA
H. Yang, Y. Fang, and W. Lin, "Perceptual quality assessment of screen content images," IEEE Transactions on Image Processing, vol. 24, no. 11, pp. 4408每4421, August 2015. [Full Text]
SQI
S. Wang, K. Gu, K. Zeng, Z. Wang, and W. Lin, "Objective quality assessment and perceptual compression of screen content images," IEEE Computer Graphics and Applications, vol. 38, no. 1, pp. 47每58, May 2016. [Full Text]
SQMS
K. Gu, S. Wang, H. Yang, W. Lin, G. Zhai, X. Yang, and W. Zhang, "Saliency-guided quality assessment of screen content images," IEEE Transactions on Multimedia, vol. 18, no. 6, pp. 1098每1110, March 2016. [Full Text]
ESIM
Z. Ni, L. Ma, H. Zeng, J. Chen, C. Cai, and K.-K. Ma, "ESIM: Edge similarity for screen content image quality assessment," IEEE Transactions on Image Processing, vol. 26, no. 10, pp. 4818每4831, June 2017. [Full Text]
SVQI
K. Gu, J. Qiao, X. Min, G. Yue, W. Lin, and D. Thalmann, "Evaluating quality of screen content images via structural variation analysis," IEEE Transactions on Visualization and Computer Graphics, November 2017, DOI: 10.1109/TVCG.2017.2771284. [Full Text]
SIQAD
H. Yang, Y. Fang, and W. Lin, "Perceptual Quality Assessment of Screen Content Images", IEEE Transactions on Image Processing, vol. 24, no. 11, pp. 4408-4421, Aug. 2015. [Full Text][Download]
SCID
Z. Ni, L. Ma, H. Zeng, J. Chen, C. Cai, and K.-K. Ma, "ESIM: Edge similarity for screen content image quality assessment," IEEE Transactions on Image Processing, vol. 26, no. 10, pp. 4818每4831, June 2017. [Full Text][Download]

Copyright

If you use these codes in your research, we kindly ask that you reference our paper listed below.


Zhangkai Ni, Huanqiang Zeng, Lin Ma, Junhui Hou, Jing Chen, and Kai-Kuang Ma, "A Gabor Feature-based Quality Assessment Model for the Screen Content Images," IEEE Transactions on Image Processing, vol. 27, no. 9, pp. 4516-4528, Sept. 2018.


Copyright (c) 2017 The Huaqiao University


All rights reserved. Permission is hereby granted, without written agreement and without license or royalty fees, to use, copy, and distribute this database (the images, the results, and the source files) and its documentation for non-commercial research and educational purposes only, provided that the copyright notice in its entirety appear in all copies of this database, and the original source of this database, Smart Visual Information Processing Laboratory (SmartVIPLab, http://smartviplab.org/index.html) at The Huaqiao University, is acknowledged in any publication that reports research using this database.


Contact Me

If you have any questions, please feel free to contact Dr. Zhangkai Ni (eezkni@gmail.com)


Back to top
SmartVIPLab.org © 2016-2025. All Rights Reserved.
關ICP掘16022269瘍