×

Median filtering-based methods for static background extraction from surveillance video. (English) Zbl 1349.94023

Summary: We propose some computational methods for extracting static backgrounds from surveillance videos corrupted by noise, blur, or both. The new methods are constructed based on the fact that the matrix representation of a static background consists of identical columns; hence the idea of median filtering is embedded in these methods. These new methods significantly differ from existing methods originating from the robust principal component analysis (RPCA) in that no nuclear-norm term is involved; thus the computation of singular value decomposition can be completely avoided when solving these new models iteratively. This is an important feature because usually the dimensionality of a surveillance video is large and so the involved singular value decomposition (which is inevitable for RPCA-based models) is very expensive computationally. We show that these methods can be easily solved by well-developed operator splitting methods in optimization literature such as the alternating direction method of multipliers. We compare the new methods with their RPCA-based counterparts via testing some synthetic and real videos. Our numerical results show that compared with RPCA-based models, these median filtering-based variational models can extract more accurate backgrounds when the background in a surveillance video is static, and numerically, they can be solved much more efficiently.

MSC:

94A08 Image processing (compression, reconstruction, etc.) in information and communication theory
65K10 Numerical optimization and variational techniques

Software:

Matlab; PROPACK
Full Text: DOI

References:

[1] CollinsRT, LiptonAJ, KanadeT. Introduction to the special section on video surveillance. IEEE Transactions on Pattern Analysis and Machine Intelligence2000; 22:745-746.
[2] CutlerR, DavisL. View‐based detection and analysis of periodic motion. Proceedings of the International Conference on Pattern Recognition, Kerkyra, 1998; 255-261.
[3] ToyamaK, KrummJ, BrumittB, MeyersB. Wallflower: principles, and practice of background maintenance. Proceedings of the IEEE International Conference on Computer Vision1999; 1:255-261.
[4] MiglioreD, MatteucciM, NaccariM, BonariniA. A revaluation of frame difference in fast and robust motion detection. Proceedings of the 4th ACM International Workshop on Video Surveillance and Sensor Networks, Santa Barbara, 2006; 215-218.
[5] CucchiaraR, PiccardiM, PratiA. Detecting moving objects, ghosts, and shadows in video streams. IEEE Transactions on Pattern Analysis and Machine Intelligence2003; 23:1337-1342.
[6] ZhouQ, AggarwalJ. Tracking and classifying moving objects from videos. Proceedings of IEEE Workshop on Performance Evaluation of Tracking and Surveillance, Hawaii, 2001; 46-54.
[7] PiccardiM. Background subtraction techniques: a review. Proceedings of IEEE International Conference on Systems Man and Cybernetics, The Hague, Netherlands, 2004; 3099-3104.
[8] WrenC, AzarbayejaniA, DarrellT, PentlandA. Pfinder: real‐time tracking of the human body. IEEE Transactions on Pattern Analysis and Machine Intelligence1997; 19:780-785.
[9] MittalA, ParagiosN. Motion‐based background subtraction using adaptive kernel density estimation. IEEE Conference on Computer Vision and Pattern Recognition2004; 2:302-309.
[10] RidderC, MunkeltO, KirchnerH. Adaptive background estimation and foreground detection using Kalman filtering. Proceedings of the International Conference on Recent Advances in Mechatronics, Istanbul, Turkey, 1995; 193-199.
[11] FriedmanN, RussellS. Image segmentation in video sequence: a probabilistic approach. Proceedings of the Conference of Uncertainty in Aritificial Intelligence, Providence, Rhode Island, 1997; 175-181.
[12] SekiM, WadaT, FujiwareH, SurniK. Background subtraction based on cooccurence of image variations. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition2003; 2:65-72.
[13] OliverNM, RosarioB, PentlandAP. A Bayesian computer vision system for modeling human interactions. IEEE Transactions on Pattern Analysis and Machine Intelligence2000; 22:831-843.
[14] BouwmansT. Subspace learning for background modeling: a survey. Recent Patents on Computer Science2009; 2:223-234.
[15] CandèsEJ, LiX, MaY, WrightJ. Robust principal component analysis?Journal of the ACM2011; 58(3):Article No. 11. DOI: 10.1145/1970392.1970395. · Zbl 1327.62369
[16] ChandrasekaranV, SanghaviS, ParriloP, WillskyA. Rank‐sparsity incoherence for matrix decomposition. SIAM Journal on Optimization2011; 21:572-596. · Zbl 1226.90067
[17] TorreF, BlackM. A framework for robust subspace learning. International Journal of Computer Vision2003; 54:117-142. · Zbl 1076.68058
[18] LinZ, GaneshA, WrightJ, WuL, ChenM, MaY. Fast convex optimization algorithms for exact recovery of a corrupted low‐rank matrix. Proceedings of the International Workshop on Computational Advances in Multi‐Sensor Adaptive Processing, Aruba, Dutch Antilles, 2009; 213-216.
[19] OreifejO, LiX, ShahM. Simultaneous video stabilization and moving object detection in turbulence. IEEE Transactions on Pattern Analysis and Machine Intelligence2012; 35:450-463.
[20] TaoM, YuanXM. Recovering low‐rank and sparse components of matrices from incomplete and noisy observations. SIAM Journal on Optimization2011; 21:57-81. · Zbl 1218.90115
[21] YangJF, YuanXM. Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization. Mathematics of Computation2013; 82:301-329. · Zbl 1263.90062
[22] HansenPC, NagyJG, O’learyDP, Deblurring Images: Matrices, Spectra, and Filtering. SIAM: Philadelphia, 2006. · Zbl 1112.68127
[23] ReddyV, SandersonC, LovellBC. A low‐complexity algorithm for static background estimation from cluttered image sequences in surveillance contexts. Eurasip Journal on Image and Video Processing2011. DOI: 10.1155/2011/164956.
[24] ElgammalA, HarwoodD, DavisLS. Non‐parametric model for background subtraction. Proceedings of the 6th European Conference on Computer Vision2000; 2:751-767.
[25] GrimsonE, StaufferC. Adaptive background mixture models for real‐time tracking. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition1999; 2:246-252.
[26] HorprasertT, HarwoodD, DavisL. Arobust background subtraction and shadow detection. Proceedings of the Asian Conference on Computer Vision, Taipei, 2000; 983-988.
[27] ChiuCC, KuMY, LiangLW. A robust object segmentation system using a probability‐based background extraction algorithm. IEEE Transactions on Circuits and Systems for Video Technology2010; 20:518-528.
[28] ColombariA, FusielloA, MurinoV. Background initialization in cluttered sequences. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition2006:197-202.
[29] GutchessD, TrajkovicsM, SolalEC, LyonsD, JainA. A background model initialization algorithm for video surveillance. Proceedings of International Conference on Computer Vision2001; 1:733-740.
[30] LongW, YangY. Stationary background generation: an alternative to the difference of two images. Pattern Recognition1990; 23:1351-1359.
[31] CohenS. Background estimation as a labeling problem. Proceedings of IEEE International Conference on Computer Vision2005; 2:1034-1041.
[32] XuX, HuangTS. A loopy belief propagation approach for robust background estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition2008; 1:1-7.
[33] LoBPL, VelastinSA. Automatic congestion detection system for underground platforms. Proceedings of the International Symposium on Intelligent Multimedia, Video and Speech Processing, Hong Kong, 2001; 158-161.
[34] RakibeRS, PatilBD. Background subtraction algorithm based human motion detection. International Journal of Scientific and Research Publications2013; 3(5).
[35] HeJ, BalzanoL, SzlamA. Incremental gradient on the Grassmannian for online foreground and background separation in subsampled video. IEEE Conference on Computer Vision and Pattern Recognition2012:1568-1575.
[36] QiuCL, VaswaniN. Real‐time robust principal components’ pursuit. International Conference on Communication Control and Computing, Tamil Nadu, India, 2010; 591-598.
[37] GabayD, MercierB. A dual algorithm for the solution of nonlinear variational problems via finite‐element approximations. Computers Mathematics with Applications1976; 2:17-40. · Zbl 0352.65034
[38] GlowinskiR, MarroccoA. Sur l’approximation paréléments finis d’ordre un et résolution par pénalisationdualité d’une classe de problèmes non linéaires, ESAIM. Mathematical Modelling and Numerical Analysis‐Modélisation Mathématique et Analyse Numérique1975; 9(R2):41-76. · Zbl 0368.65053
[39] PengY, GaneshA, WrightJ, XuW, MaY. RASL: robust alignment by sparse and low‐rank decomposition for linearly correlated images. IEEE Transactions on Pattern Analysis and Machine Intelligence2012; 34:2233-2246.
[40] YuanXM, YangJF. Sparse and low‐rank matrix decomposition via alternating direction methods. Pacific Journal of Optimization2013; 9:167-180. · Zbl 1269.90061
[41] HestenesMR. Multiplier and gradient methods. Journal of Optimization Theory and Applications1969; 4:303-320. · Zbl 0174.20705
[42] PowellMJD. A method for nonlinear constraints in minimization problems. In Optimization, FletcherR (ed.) (ed.). Academic Press: New York, 283-298. · Zbl 0194.47701
[43] GlowinskiR. Numerical Methods for Nonlinear Variational Problems. Springer‐Verlag: Heidelberg, 1984. · Zbl 0536.65054
[44] HeBS, YuanXM. On the O(1/n) convergence rate of Douglas-Rachford alternating direction method. SIAM Journal on Numerical Analysis2012; 50:700-709. · Zbl 1245.90084
[45] ChenCH, HeBS, YeYY, YuanXM. The direct extension of ADMM for multi‐block convex minimization problems is not necessarily convergent. Optimization Online2013. Mathematical Programming; DOI: 10.1007/s10107‐014‐0826‐5; Online version. · Zbl 1332.90193
[46] HeBS, TaoM, YuanXM. A splitting method for separable convex programming. IMA Journal of Numerical Analysis2015; 35(1):394-426. · Zbl 1310.65062
[47] HeBS, TaoM, YuanXM. Alternating direction method with Gaussian back substitution for separable convex programming. SIAM Journal on Optimization2012; 22:313-340. · Zbl 1273.90152
[48] DonohoDL. De‐noising by soft thresholding. IEEE Transactions on Information Theory1995; 41:613-627. · Zbl 0820.62002
[49] GolubGH, Van LoanCF. Matrix Computations. Johns Hopkins University Press: Baltimore, 1996. · Zbl 0865.65009
[50] LarsenRM. Lanczos bidiagonalization with partial reorthogonalization, DAIMI PB‐357, Department of Computer Science, Aarhus University, September 1998.
[51] RudinL, OsherS, FatemiE. Nonlinear total variation based noise removal algorithms. Journal of Physics D1992; 60:259-268. · Zbl 0780.49028
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.