Fast and Generalisable License Plate Re-identification using Neural Embedding of Fisher Vectors
Authors: 
Abhinav Kumar, Shantanu Gupta, Vladimir Kozitsky, Sriganesh MadhvanathAuthors Info & Claims
Abhinav Kumar, Shantanu Gupta, Vladimir Kozitsky, Sriganesh MadhvanathAuthors Info & ClaimsArticle No.: 49, Pages 1 - 9
Abstract
We consider the license plate re-identification task, treated here as a one-shot image retrieval problem. Our objective is to learn a feature representation for license plate images, such that a single training image of a given license plate (referred to as a template image) is sufficient to perform nearest-neighbour retrieval with high accuracy at test time. Also, the feature representation should ideally be generalisable across datasets and should be extractable in real-time on resource-constrained embedded hardware or a moderately powerful cellphone.
We evaluate representations from person re-identification (re-id) literature, learned from a trained deep convolutional network as well with those derived from a trained Fisher vector. While the convolutional network features perform better than the Fisher vector, we obtain comparable results from a hybrid model projecting the Fisher vector into a lower-dimensional space via two fully connected layers called f2nn using the triplet loss. The proposed hybrid model f2nn generates features which outperform and generalise better than convolutional features on datasets dissimilar to the training corpus. The model can be trained in stages and takes significantly less time to extract features. Further, it uses much smaller feature dimensions for license plate images resulting in faster re-identification, and is therefore well-suited for resource-constrained platforms such as mobile devices.
References
[1]
Jon Almazan, Bojana Gajic, Naila Murray, and Diane Larlus. 2018. Re-ID done right: towards good practices for person re-identification. arXiv preprint arXiv:1801.05339 (2018).
[2]
Joakim And Ãl'n and StÃl'phane Mallat. 2014. Deep Scattering Spectrum. IEEE Transactions on Signal Processing 62, 16 (Aug. 2014), 4114--4128. https://doi.org/10.1109/TSP.2014.2326991 arXiv: 1304.6763.
[3]
Song Bai, Xiang Bai, and Qi Tian. 2017. Scalable Person Re-identification on Supervised Smoothed Manifold. IEEE, 3356--3365. https://doi.org/10.1109/CVPR.2017.358
[4]
Orhan Bulan, Vladimir Kozitsky, Palghat Ramesh, and Matthew Shreve. 2017. Segmentation-and annotation-free license plate recognition with deep localization and failure identification. IEEE Transactions on Intelligent Transportation Systems 18, 9 (2017), 2351--2363.
[5]
Sumit Chopra, Raia Hadsell, and Yann LeCun. 2005. Learning a similarity metric discriminatively, with application to face verification. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, Vol. 1. IEEE, 539--546.
[6]
R. Collobert, K. Kavukcuoglu, and C. Farabet. 2011. Torch7: A Matlab-like Environment for Machine Learning. In BigLearn, NIPS Workshop.
[7]
Douglas Gray and Hai Tao. 2008. Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features. In Computer Vision âĂŞECCV 2008, David Forsyth, Philip Torr, and Andrew Zisserman (Eds.). Vol. 5302. Springer Berlin Heidelberg, Berlin, Heidelberg, 262--275. https://doi.org/10.1007/978-3-540-88682-2_21
[8]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep Residual Learning for Image Recognition. arXiv preprint arXiv:1512.03385 (2015).
[9]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision. 1026--1034.
[10]
Alexander Hermans, Lucas Beyer, and Bastian Leibe. 2017. In Defense of the Triplet Loss for Person Re-Identification. arXiv preprint arXiv:1703.07737 (2017).
[11]
Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional Architecture for Fast Feature Embedding. arXiv preprint arXiv:1408.5093 (2014).
[12]
Srikrishna Karanam, Mengran Gou, Ziyan Wu, Angels Rates-Borras, Octavia Camps, and Richard J. Radke. 2016. A Systematic Evaluation and Benchmark for Person Re-Identification: Features, Metrics, and Datasets. arXiv:1605.09653 [cs] (May 2016). http://arxiv.org/abs/1605.09653 arXiv: 1605.09653.
[13]
Martin Köstinger, Martin Hirzer, Paul Wohlhart, Peter M. Roth, and Horst Bischof. 2012. Large Scale Metric Learning from Equivalence Constraints. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[14]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105.
[15]
Praveen Kulkarni, Joaquin Zepeda, Frederic Jurie, Patrick Perez, and Louis Chevallier. 2015. Hybrid multi-layer deep CNN/aggregator feature for image classification. In Acoustics, Speech and Signal Processing (ICASSP), 2015 IEEE International Conference on. IEEE, 1379--1383.
[16]
Vijay BG Kumar, Ben Harwood, Gustavo Carneiro, Ian Reid, and Tom Drummond. 2017. Smart Mining for Deep Metric Learning. arXiv preprint arXiv:1704.01285 (2017).
[17]
Svetlana Lazebnik, Cordelia Schmid, and Jean Ponce. 2006. Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories. In Computer Vision and Pattern Recognition (CVPR).
[18]
Wei Li, Rui Zhao, Tong Xiao, and Xiaogang Wang. 2014. DeepReID: Deep Filter Pairing Neural Network for Person Re-identification. IEEE, 152--159. https://doi.org/10.1109/CVPR.2014.27
[19]
Shengcai Liao, Yang Hu, Xiangyu Zhu, and Stan Z Li. 2015. Person re-identification by local maximal occurrence representation and metric learning. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2197--2206.
[20]
Chunxiao Liu, Shaogang Gong, Chen Change Loy, and Xinggang Lin. 2012. Person re-identification: What features are important?. In European Conference on Computer Vision. Springer, 391--401.
[21]
Hao Liu, Jiashi Feng, Meibin Qi, Jianguo Jiang, and Shuicheng Yan. 2017. End-to-End Comparative Attention Networks for Person Re-identification. IEEE Transactions on Image Processing 26, 7 (July 2017), 3492--3506. https://doi.org/10.1109/TIP.2017.2700762 arXiv: 1606.04404.
[22]
David G Lowe. 2004. Distinctive image features from scale-invariant keypoints. International journal of computer vision 60, 2 (2004), 91--110.
[23]
Bingpeng Ma, Yu Su, and Frédéric Jurie. 2012. Local Descriptors Encoded by Fisher Vectors for Person Re-identification. In Computer Vision - ECCV 2012. Workshops and Demonstrations. Vol. 7583. Springer Berlin Heidelberg, Berlin, Heidelberg, 413--422. https://doi.org/10.1007/978-3-642-33863-2_41
[24]
Bingpeng Ma, Yu Su, and Frédéric Jurie. 2014. Covariance descriptor based on bio-inspired features for person re-identification and face verification. Image and Vision Computing 32, 6-7 (June 2014), 379--390. https://doi.org/10.1016/j.imavis.2014.04.002
[25]
Tetsu Matsukawa, Takahiro Okabe, Einoshin Suzuki, and Yoichi Sato. 2016. Hierarchical gaussian descriptor for person re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1363--1372.
[26]
Florent Perronnin and Christopher Dance. 2007. Fisher kernels on visual vocabularies for image categorization. CVPR (2007).
[27]
Florent Perronnin and Diane Larlus. 2015. Fisher Vectors Meet Neural Networks: A Hybrid Classification Architecture. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[28]
Florent Perronnin, Jorge Sánchez, and Thomas Mensink. 2010. Improving the fisher kernel for large-scale image classification. ECCV (2010).
[29]
KS Raghunandan, Palaiahnakote Shivakumara, Hamid A Jalab, Rabha W Ibrahim, G Hemantha Kumar, Umapada Pal, and Tong Lu. 2018. Riesz fractional based model for enhancing license plate detection and recognition. IEEE Transactions on Circuits and Systems for Video Technology 28, 9 (2018), 2276--2288.
[30]
Tanzila Rahman. [n. d.]. Person Re-Identification by Localizing Discriminative Regions. ([n. d.]), 12.
[31]
Ali Sharif Razavian, Hossein Azizpour, Josephine Sullivan, and Stefan Carlsson. 2014. CNN Features Off-the-Shelf: An Astounding Baseline for Recognition. IEEE, 512--519. https://doi.org/10.1109/CVPRW.2014.131
[32]
Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016. You only look once: Unified, real-time object detection. CVPR (2016).
[33]
José A Rodríguez-Serrano, Harsimrat Sandhawalia, Raja Bala, Florent Perronnin, and Craig Saunders. 2012. Data-driven vehicle identification by image matching. In European Conference on Computer Vision. Springer, 536--545.
[34]
Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. FaceNet: A Unified Embedding for Face Recognition and Clustering. CoRR abs/1503.03832 (2015). http://arxiv.org/abs/1503.03832
[35]
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep fisher networks for large-scale image classification. In Advances in neural information processing systems. 163--171.
[36]
K. Simonyan and A. Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRR abs/1409.1556 (2014).
[37]
Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of machine learning research 15, 1 (2014), 1929--1958.
[38]
Chi Su, Jianing Li, Shiliang Zhang, Junliang Xing, Wen Gao, and Qi Tian. 2017. Pose-Driven Deep Convolutional Model for Person Re-identification. IEEE, 3980--3989. https://doi.org/10.1109/ICCV.2017.427
[39]
A. Vedaldi and B. Fulkerson. 2008. VLFeat: An Open and Portable Library of Computer Vision Algorithms. http://www.vlfeat.org/. (2008).
[40]
Kilian Q Weinberger, John Blitzer, and Lawrence K Saul. [n. d.]. Distance Metric Learning for Large Margin Nearest Neighbor Classification. ([n. d.]), 8.
[41]
Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. 2016. A Discriminative Feature Learning Approach for Deep Face Recognition. Springer International Publishing, Cham, 499--515. https://doi.org/10.1007/978-3-319-46478-7_31
[42]
Lin Wu, Chunhua Shen, and Anton van den Hengel. 2016. Deep Linear Discriminant Analysis on Fisher Networks: A Hybrid Architecture for Person Re-identification. arXiv.1606.01595 [cs] (June 2016). http://arxiv.org/abs/1606.01595 arXiv: 1606.01595.
[43]
Liang Zheng, Yi Yang, and Alexander G. Hauptmann. 2016. Person Re-identification: Past, Present and Future. arXiv:1610.02984 [cs] (Oct. 2016). http://arxiv.org/abs/1610.02984 arXiv: 1610.02984.
Index Terms
- Fast and Generalisable License Plate Re-identification using Neural Embedding of Fisher Vectors
Recommendations
License Plate Detection Based on Convolutional Neural Network: Support Vector Machine (CNN-SVM)
ICVIP '17: Proceedings of the International Conference on Video and Image ProcessingAutomatic License Plate Recognition (ALPR) implementation can be used in many applications, such as road traffic monitoring, automatic toll payments, and parking management. License plate detection is the first and very critical stage in the ALPR ...
Face recognition using discriminative locality preserving vectors
We proposed an effective face recognition method based on the discriminative locality preserving vectors method (DLPV). Using the analysis of eigenspectrum modeling of locality preserving projections, we selected the reliable face variation subspace of ...
Image retrieval using nonlinear manifold embedding
The huge number of images on the Web gives rise to the content-based image retrieval (CBIR) as the text-based search techniques cannot cater to the needs of precisely retrieving Web images. However, CBIR comes with a fundamental flaw: the semantic gap ...
Comments
Information & Contributors
Information
Published In
December 2018
659 pages
ISBN:9781450366151
DOI:10.1145/3293353
Copyright © 2018 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 03 May 2020
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICVGIP 2018
ICVGIP 2018: 11th Indian Conference on Computer Vision, Graphics and Image Processing
December 18 - 22, 2018
Hyderabad, India
Acceptance Rates
Overall Acceptance Rate 95 of 286 submissions, 33%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 52Total Downloads
- Downloads (Last 12 months)6
- Downloads (Last 6 weeks)1
Reflects downloads up to 23 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in