Gradient Direction for Screen Content Image Quality Assessment

Zhangkai Ni1, Lin Ma2, Huanqiang Zeng1, Canhui Cai1, and Kai-Kuang Ma3

1School of Information Science and Engineering, Huaqiao University, Fujian, China

2Huawei Noah's Ark Lab, Hong Kong

3School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore

[Paper][GSS Code]

Abstract

In this letter, we make the first attempt to explore the usage of the gradient direction on the perceptual quality assessment of the screen content image (SCI). Specifically, the proposed approach firstly extracts the gradient direction based on the local information of the image gradient magnitude, which not only preserves gradient direction consistency in local regions, but also demonstrates sensitivities to the distortions introduced to the SCI. A deviation based pooling strategy is subsequently utilized to generate the corresponding image quality index. Moreover, we investigate and demonstrate the complementary behaviors of the gradient direction and magnitude for SCI quality assessment. By jointly considering them together, our proposed SCI quality metric outperforms the state-of-the-art quality metrics in terms of correlation with human visual system perception.

The contributions of this work:

  • The gradient direction presents an important attribute of the edge. The proposed approach to compute the image gradient direction based on the local information of the image gradient magnitude, which not only preserves gradient direction consistency in local regions, but also demonstrates sensitivities to the distortions introduced to the SCI.
  • Incorporating gradient direction with magnitude. The gradient magnitude be jointly utilized to describe the perceptual quality degradation of SCI. Experimental results on public SCI database show that our proposed IQA metric outperforms the state-of-the-art quality metrics.

Gradient Similarity Score for SCI Quality Assessment

Fig. 2. The SCIs and the corresponding direction distributions.

Fig. 1. The framework of our proposed IQA for SCI.

The framework of our proposed IQA for SCI is illustrated in Fig. 1. Our proposed IQA firstly computes the gradient magnitude and direction of the reference and distorted SCI, respectively. The deviation pooling is then used to pool the magnitude and direction similarities together to a final quality score.

Gradient direction computation:

The gradient direction map is calculated in terms of twelve directions from 0 to π by individually convolving the SCI with different convolution kernel at different directions. After obtaining the gradient direction maps for all twelve directions, the final gradient direction map is generated by selecting the maximum value among the responses overall directions. Fig. 2 demonstrates an example of SCI with Gaussian blurring, motion blurring, contrast change. The blue bars denote the gradient direction distributions of the corresponding SCIs, while the red bars indicate the difference between the original and distorted SCIs.

 

Experimental Results

PLCC, SROCC and RMSE of IQA metrics on SIQAD database

Reference

GSS
Z. Ni, L. Ma, H. Zeng, C. Cai, and K.-K. Ma, "Gradient direction for screen content image quality assessment", IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1394�C1398, August 2016. [Full Text]
SSIM
Z. Wang, A. C. Bovil, H. R. Sheikh, and E. P. Simoncelli, "Image Quality Assessment: From Error Visibility to Structural Similarity", IEEE Transactions on Image Processing, 2004. [Full Text]
MSSIM
Z. Wang, E. P. Simoncelli, and A. C. Bovil, "Multi-Scale Structural Similarity for Image Quality Assessment", IEEE Conference on Signals, Systems and Computers, 2003. [Full Text]
IWSSIM
Z. Wang, and Q. Li, "Information Content Weighting for Perceptual Image Quality Assessment", IEEE Transactions on Image Processing, 2011. [Full Text]
VIF
H. R. Sheikh, and A. C. Bovik, "Image Information and Visual Quality", IEEE Transactions on Image Processing, 2006. [Full Text]
IFC
H. R. Sheikh, A. C. Bovik, and G. de Veciana "An Information Fidelity Creation for Image quality Assessment Using Natural Scene Statistics", IEEE Transactions on Image Processing, 2005. [Full Text]
VSNR
D. M. Chandler, and S. S. Hemami, "VSNR: A Wavelet-Based visual Signal-to-Noise Ratio for Natural Images", IEEE Transactions on Image Processing, 2007. [Full Text]
MAD
E. C. Larson, and D. M. Chandler, "Most Apparent Distortion: Full-Reference Image Quality Assessment and the Role of Strategy", Journal of Electronic Imaging, 2010. [Full Text]
FSIM
L. Zhang, L. Zhang, X. Mou, and D. Zhang, "FSIM: A Feature Similarity Index for Image Quality Assessment", IEEE Transactions on Image Processing, 2011. [Full Text]
GSIM
A. Liu, W. Lin, and M. Narwaria, "Image Quality Assessment Based on Gradient Similarity", IEEE Transactions on Image Processing, 2012. [Full Text]
GMSD
W. Xue, L. Zhang, X. Mou, and A. C. Bovik, "Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index", IEEE Transactions on Image Processing, 2014. [Full Text]
VSI
L. Zhang, and Y. Shen, and H. Li "VSI: A Visual Saliency-Induced Index for Perceptual Image Quality Assessment", IEEE Transactions on Image Processing, 2014. [Full Text]
SIQAD
H. Yang, Y. Fang, and W. Lin, "Perceptual Quality Assessment of Screen Content Images", IEEE Transactions on Image Processing, vol. 24, no. 11, pp. 4408-4421, Aug. 2015. [Full Text][Download]

Contact Me

If you have any questions, please feel free to contact Dr. Zhangkai Ni (eezkni@gmail.com).

Last update: Aug. 6, 2016