Abstract
High-Efficiency Video Coding (HEVC) is the new emerging video coding standard of the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG). The HEVC standard provides a significant improvement in compression efficiency in comparison with existing standards such as H264/AVC by means of greater complexity. In this paper we will examine several HEVC optimizations based on image analysis to reduce its huge CPU, resource and memory expensive encoding process. The proposed algorithms optimize the HEVC quad-tree partitioning procedure, intra/inter prediction and mode decision by means of H264-based methods and spatial and temporal homogeneity analysis which is directly applied to the original video. The validation process of these approaches was conducted by taking into account the human visual system (HVS). The adopted solution makes it possible to perform HEVC real time encoding for HD sequences on a low cost processor with negligible quality loss. Moreover, the frames pre-processing leverages the logic units and embedded hardware available on an Intel GPU, so the execution time of these stages are negligible for the encoding processor.
Similar content being viewed by others
References
Ahn S, Lee B, Kim M (2015) A novel fast CU encoding scheme based on spatio-temporal encoding parameters for HEVC inter coding. IEEE Trans Circuits Syst Video Technol 25(3):422–435
Alcocer E, Gutierrez R, Lopez-Granado O, Malumbres MP (2016) Design and implementation of an efficient hardware integer motion estimator for an HEVC video encoder. Journal of Real-Time Image Processing
Bjontegarrd G (2001) Calculation of average PSNR differences between RD curves. ITU-T SC16/Q6 13th VCEG meeting, Austin
Bossen F (2012) Common test conditions and software reference configurations. JCT-VC Document, JCTVC-K1100
Cho S, Kim M (2013) Fast CU splitting and pruning for suboptimal CU partitioning in HEVC intra coding. IEEE Trans Circuits Syst Video Technol 23(9):1555–1564
de Frutos-López M, Orellana-Quirós D, Pujol-Alcolado JC, de María FD (2010) An improved fast mode decision algorithm for intraprediction in H.264/AVC video coding. Signal Process Image Commun 25(10):709–716
Fernandez DG, Del Barrio AA, Botella G, Garcia C (2016) 4K-based intra and inter prediction techniques for HEVC. Proc. SPIE, Real-Time Image and Video Processing 2016, 98970B
Fernández DG, Del Barrio AA, Botella G, García C (2016) Fast CU size decision based on temporal homogeneity detection. In: Design of Circuits and Integrated Systems (DCIS), 2016 Conference on, 1–6
Fernández DG, Del Barrio AA, Botella G, García C (2018) Fast and effective CU size decision based on spatial and temporal homogeneity detection. Multimedia Tools and Applications 77(5):5907–5927
Fernández DG, Del Barrio AA, Botella G, García C, Meyer-Baese U, Meyer-Baese A (2016) HEVC optimizations for medical environments. In: Proc. SPIE 9871, Sensing and Analysis Technologies for Biomedical and Cognitive Applications 2016, 98710B
Fernández DG, Del Barrio AA, Botella G, García C, Prieto M, Hermida R (2018) Complexity reduction in the HEVC/H265 standard based on smooth region classification. Digital Signal Processing 73:24–39
D. G. Fernández, A. A. Del Barrio, Guillermo Botella, Uwe Meyer-Baese, Anke Meyer-Baese, Christos Grecos (2017) Information fusion based techniques for HEVC. Proc. SPIE 10223, Real-Time Image and Video Processing 2017, 102230M. 10.1117/12.2262604
Fraunhofer Institute for Telecommunications (2018) Perceptually optimized video coding. Retrieved from: https://www.hhi.fraunhofer.de/en/departments/vca/research-groups/image-video-coding/research-topics/perceptually-optimized-video-coding.html. Accessed 17 Dec 2018
Goswami K, Lee J-H, Jang K-S, Kim B-G, Kwon K-K (2014) Entropy difference-based early skip detection technique for high efficiency video coding. Journal of Real-Time Image Processing
He G, Zhou D, Goto S (2013) Transform-based fast mode and depth decision algorithm for HEVC intra prediction. IEEE 10th International Conference on ASIC (ASICON), pp. 1–4
Intel Corporation (2016) Introduction to advance motion extension for OpenCL. Retrieved from: https://software.intel.com/en-us/articles/intro-to-advanced-motion-estimation-extension-for-opencl. Accessed 17 Dec 2018
Intel Corporation White Paper (2012) Performance Interactions of OpenCL* Code and Intel® Quick Sync Video on Intel® HD Graphics 4000
ITU-R BT.500 (2012) Methodology for the subjective assessment of the quality of television pictures. International Telecommunication Union, Geneva
Jiang W, Ma H, Chen Y (2012) Gradient based fast mode decision algorithm for intra prediction in HEVC. In: 2nd International Conference on Consumer Electronics, Communications and Networks (CECNet), pp. 1836–1840
Jiang W, Ma H, Chen Y (2012) Gradient Based Fast Mode Decision Algorithm for Intra Prediction in HEVC. In: International Conference on Consumer Electronics, Communications and Networks (CECNet)
Khan MUK, Shafique M, Grellert M, Henkel J (2013) Hardware-software collaborative complexity reduction scheme for the emerging HEVC intra encoder. Design, Automation Test in Europe Conference Exhibition (DATE) 2013:125–128
Khronos OpenCL Working Group (2011) The OpenCL specification version 1.1. Revision 44
Khronos OpenCL Working Group (2016) Online documentation for cl_intel_advaanced_motion_estimation. Retrieved from: https://www.khronos.org/registry/cl/extensions/intel/cl_intel_advanced_motion_estimation.txt. Accessed 17 Dec 2018
Koumaras H, Kourtis M, Martakos D (2012) Benchmarking the encoding efficiency of h.265/HEVC and h.264/AVC,” in Future Network Mobile Summit (FutureNetw)
Leal da Silva T, Agostini LV, da Silva Cruz LA (2015) Fast intra prediction algorithm based on texture analysis for 3D-HEVC encoders” in Journal of Real-Time Image Processing
Lee JH, Park CS, Kim BG, Jun DS, Jung SH, Choi JS (2013) Novel fast PU decision algorithm for the HEVC video standard”, IEEE International Conference on Image Processing (ICIP)
Li C, Bovik AC (2009) Three-component weighted structural similarity index. Proc. SPIE 7242–72420, Image Quality and System Performance VI
Lim K, Lee J, Kim S, Lee S (2015) Fast PU skip and split termination algorithm for HEVC intra. IEEE Transactions on Circuits Systems for Video Technology 25(8)
Liu X, Liu Y, Wang P, Lai C, Chao H (2016) An adaptive mode Decision algorithm based on video texture characteristics for HEVC intra prediction. IEEE Transactions on Circuits Systems for Video Technology, vol. PP: 99
Mallikarachchi T, Fernando A, Arachchi H (2014) Efficient coding unit size selection based on texture analysis for HEVC intra prediction. IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6
McCann K, Rosewarne C, Bross B, Naccari M, Sharman K, Sullivan G (2014) High efficiency video coding (HEVC) encoder description 0v16 (HM16). JCT-VC High Efficiency Video Coding N14 703
Min B, Cheung R (2014) A fast cu size decision algorithm for HEVC intra encoder. IEEE Transactions on Circuits and Systems for Video Technology PP(99):1
Moorthy AK, Bovik AC (2010) Efficient motion weighted spatio-temporal video SSIM Index. Proc. SPIE 7527–75271, Human Vision and Electronic Imaging XV
MSU Graphics & Media Lab (Video Group) (2016) MSU video quality measurement tool. Retrieved from: http://www.compression.ru/video/quality_measure/video_measurement_tool.html. Accessed 17 Dec 2018
Multimedia Signal Processing Group (MMSPG) (2016) VQMT: video quality measurement tool. Retrieved from: http://mmspg.epfl.ch/vqmt. Accessed 17 Dec 2018
Na S, Lee W, Yoo K (2014) Edge-based fast mode decision algorithm for intra prediction in HEVC. IEEE International Conference on Consumer Electronics (ICCE), pp. 11–14
Öztekin A, ErÇelebi E (2015) An early split and skip algorithm for fast intra CU selection in HEVC. In: Journal of Real-Time Image Processing
Pastuszak G, Trochimiuk M (2015) Algorithm and architecture design of the motion estimation for the H.265/HEVC 4K-UHD encoder. Journal of Real-Time Image Processing
Ponomarenko N, Silvestri F, Egiazarian K, Carli M, Astola J, Lukin V (2007) On Between-Coefficient Contrast Masking of DCT Basis Functions. Third International Workshop on Video Processing and Quality Metrics (VPQM), Scottsdale
Ramezanpour M, Zargari F (2016) Fast HEVC I-frame coding based on strength of dominant direction of CUs. Journal of Real-Time Image Processing
Shang X, Wang G, Fan T, Li Y (2015) Fast CU size decision and PU mode decision algorithm in HEVC intra coding. IEEE International Conference on Image Processing (ICIP)
Sheikh HR, Bovik AC (2006) Image Information and Visual Quality. IEEE Trans Image Process 15(2):430–444
Sheikh HR, Bovik AC, de Veciana G (2005) An Information Fidelity Criterion for Image Quality Assessment Using Natural Scene Statistics. IEEE Trans Image Process 14(12):2117–2128
Shen L, Liu Z, Liu S, Zhang Z, An P (2009) Selective disparity estimation and variable size motion estimation based on motion homogeneity for multi-view coding. IEEE Transactions on Broadcasting 55(4)
Shen L, Liu Z, Yan T, Zhang Z, An P (2010) View-adaptive motion estimation and disparity estimation for low complexity multiview video coding. IEEE Transactions on Circuits and Systems for Video Technology 20(6)
Shen L, Liu Z, Zhang Z, Shi X (2008) Fast Inter Mode Decision Using Spatial Property of Motion Field. IEEE Transactions on Multimedia 10(6):1208–1214
Shen L, Liu Z, Zhang X, Zhao W, Zhang Z (2013) An effective CU size decision method for HEVC encoders. IEEE Transactions on Multimedia 15(2):465–470
Shen L, Zhang Z, An P (2013) Fast CU size decision and mode decision algorithm for HEVC intra coding. IEEE Transactions on Consumer Electronics 59(1)
Shen L, Zhang Z, Liu Z (2014) Effective CU size decision for HEVC intracoding. IEEE Trans Image Process 23(10):4232–4241
Sullivan G, Ohm J, Han W-J, Wiegand T (2012) Overview of the high efficiency video coding (hevc) standard. IEEE Trans Circuits Syst Video Technol 22(12):1649–1668
Sze V, Budagavi M, Sullivan GJ (2014) High Efficiency Video Coding (HEVC), 1st ed. Springer International Publishing. Available: https://link.springer.com/book/10.1007%2F978-3-319-06895-4. Accessed 17 Dec 2018
Tian G, Goto S (2012) Content adaptive prediction unit size decision algorithm for HEVC intra coding,” in Picture Coding Symposium (PCS), pp. 405–408
Ting Y-C, Chang T-S (2014) Gradient-based PU size selection for HEVC intra prediction. IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1929–1932
Vanne J, Viitanen M, Hämäläinen TD (2014) Efficient mode decision schemes for HEVC inter prediction. IEEE Transactions on Circuits and Systems for Video Technology 24(9):1579–1593
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Trans Image Process 13(4):600–612
Wang H-M, Lin J-K, Yang J-F (2006) Fast inter mode decision based on hierarchical homogeneous detection and cost analysis for h.264/AVC coders. In: IEEE International Conference on Multimedia and Expo, pp. 709–712
Wang Z, Simoncelli EP, Bovik AC (2003) Multi-scale structural similarity for image quality assessment. In: Proceedings of the 37th IEEE Asiloma Conference on Signal, Systems and Computers, Pacific Grove
Watson AB (1998) Toward a perceptual video quality metric. Human Vision, Visual Processing, and Digital Display VIII(3299):139–147
Wu D, Pan F, Lim K, Wu S, Li Z, Lin X, Rahardja S, Ko C (2005) Fast intermode decision in h.264/AVC video coding. IEEE Transactions on Circuits and Systems for Video Technology 15(7):953–958
x265 project (2017) http://x265.org/. GNU GPL 2 license, source code available at: https://bitbucket.org/multicoreware/x265/wiki/Home. Accessed 17 Dec 2018
Xiao F (2000) DCT-based Video Quality Evaluation---Final Project for EE392J
Xiong J, Li H, Meng F, Wu Q, Ngan KN (2015) Fast HEVC inter CU decision based on latent SAD estimation. IEEE Transactions on Multimedia 17(12):2147–2159
Xiong J, Li H, Meng F, Zhu S, Wu Q, Zeng B (2014) MRF-based fast HEVC inter CU decision with the variance of absolute differences. IEEE Trans Multimedia 16(8):2141–2153
Xiong J, Li H, Wu Q, Meng F (2014) A fast HEVC inter CU selection method based on pyramid motion divergence. IEEE Transactions on Multimedia 16(2):559–564
xiph.org (2016) Derf’s test media collection. Retrieved from: https://media.xiph.org/video/derf/. Accessed 17 Dec 2018
Ye T, Zhang D, Dai F, Zhang Y (2013) Fast mode decision algorithm for intra prediction in HEVC. in Proceedings of the Fifth International Conference on Internet Multimedia Computing and Service, pp. 300–304
Ye T, Zhang D, Dai F, Zhang Y (2013) Fast mode decision algorithm for intra prediction in HEVC. in the Fifth International Conference on Internet Multimedia Computing and Service (ICIMCS'13)
Zhang Y, Li Z, Li B (2012) Gradient-based fast decision for intra prediction in HEVC. In Visual Communications and Image Processing (VCIP), pp. 1–6
Zhang H, Ma Z (2014) Fast Intra mode decision for high efficiency video coding (HEVC). IEEE Transactions on Circuits Systems for Video Technology 24(4):660–668
Zhang H, Ma Z (2016) Fast intra mode and CU size decision for HEVC. IEEE Transactions on Circuits Systems for Video Technology PP(99):1–7
Acknowledgments
This work has been partially supported by Spanish research projects TIN 2015-65277-R and TIN-2012-32180, as well as the UCM-Banco Santander Grant PR26-16/20B-1.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Fernández, D.G., Botella, G., Del Barrio, A.A. et al. HEVC optimization based on human perception for real-time environments. Multimed Tools Appl 79, 16001–16033 (2020). https://doi.org/10.1007/s11042-018-7033-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-018-7033-y