research-article

Fewer-Shots and Lower-Resolutions: Towards Ultrafast Face Recognition in the Wild

Authors:

Jia LiAuthors Info & Claims

MM '19: Proceedings of the 27th ACM International Conference on Multimedia

Pages 229 - 237

https://doi.org/10.1145/3343031.3351082

Published: 15 October 2019 Publication History

Abstract

Is it possible to train an effective face recognition model with fewer shots that works efficiently on low-resolution faces in the wild? To answer this question, this paper proposes a few-shot knowledge distillation approach to learn an ultrafast face recognizer via two steps. In the first step, we initialize a simple yet effective face recognition model on synthetic low-resolution faces by distilling knowledge from an existing complex model. By removing the redundancies in both face images and the model structure, the initial model can provide an ultrafast speed with impressive recognition accuracy. To further adapt this model into the wild scenarios with fewer faces per person, the second step refines the model via few-shot learning by incorporating a relation module that compares low-resolution query faces with faces in the support set. In this manner, the performance of the model can be further enhanced with only fewer low-resolution faces in the wild. Experimental results show that the proposed approach performs favorably against state-of-the-arts in recognizing low-resolution faces with an extremely low memory of 30KB and runs at an ultrafast speed of 1,460 faces per second on CPU or 21,598 faces per second on GPU.

References

[1]

Ankan Bansal, Anirudh Nanduri, Carlos D Castillo, Rajeev Ranjan, and Rama Chellappa. 2017. UMDFaces: An Annotated Face Dataset for Training Deep Networks. In IEEE International Joint Conference on Biometrics. 464--473.

[2]

Qiong Cao, Li Shen, Weidi Xie, Omkar M Parkhi, and Andrew Zisserman. 2018. VGGFace2: A Dataset for Recognising Faces Across Pose and Age. In IEEE International Conference on Automatic Face and Gesture Recognition. 67--74.

[3]

Bowen Cheng, Ding Liu, Zhangyang Wang, Haichao Zhang, and Thomas S Huang. 2018a. Visual Recognition in Very Low-Quality Settings: Delving into the Power of Pre-Training. In AAAI Conference on Artificial Intelligence. 8065--8066.

[4]

Zhiyi Cheng, Xiatian Zhu, and Shaogang Gong. 2018b. Low-Resolution Face Recognition. In Asian Conference on Computer Vision . 605--621.

[5]

Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. 2019. ArcFace: Additive Angular Margin Loss for Deep Face Recognition. In IEEE Conference on Computer Vision and Pattern Recognition. 4690--4699.

[6]

Li Fei-Fei, Rob Fergus, and Pietro Perona. 2006. One-Shot Learning of Object Categories. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 28, 4 (2006), 594--611.

Digital Library

[7]

Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. In International Conference on Machine Learning. 1126--1135.

Digital Library

[8]

Victor Garcia and Joan Bruna. 2018. Few-Shot Learning with Graph Neural Networks. In International Conference on Learning Representations .

[9]

Shiming Ge, Shengwei Zhao, Chenyu Li, and Jia Li. 2019. Low-Resolution Face Recognition in the Wild via Selective Knowledge Distillation. IEEE Transactions on Image Processing, Vol. 28, 4 (2019), 2051--2062.

Digital Library

[10]

Weifeng Ge and Yizhou Yu. 2017. Borrowing Treasures from the Wealthy: Deep Transfer Learning through Selective Joint Fine-Tuning. In IEEE Conference on Computer Vision and Pattern Recognition. 1086--1095.

[11]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems. 2672--2680.

[12]

Manuel Günther, Peiyun Hu, Christian Herrmann, Chi-Ho Chan, Min Jiang, Shufan Yang, Akshay Raj Dhamija, Deva Ramanan, Jürgen Beyerer, Josef Kittler, et almbox. 2017. Unconstrained Face Detection and Open-Set Face Recognition Challenge. In IEEE International Joint Conference on Biometrics. 697--706.

[13]

Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. 2016. MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition. In European Conference on Computer Vision. 87--102.

[14]

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2014. Distilling the Knowledge in a Neural Network. In Advances in Neural Information Processing Systems Workshop .

[15]

Junjun Jiang, Yi Yu, Jinhui Hu, Suhua Tang, and Jiayi Ma. 2018. Deep CNN Denoiser and Multi-Layer Neighbor Component Embedding for Face Hallucination. In International Joint Conference on Artificial Intelligence. 771--778.

[16]

Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. 2015. Siamese Neural Networks for One-Shot Image Recognition. In International Conference on Machine Learning Workshop .

[17]

Soheil Kolouri and Gustavo K Rohde. 2015. Transport-based Single Frame Super Resolution of Very Low Resolution Face Images. In IEEE Conference on Computer Vision and Pattern Recognition . 4876--4884.

[18]

Erik Learned-Miller, Gary B Huang, Aruni RoyChowdhury, Haoxiang Li, and Gang Hua. 2016. Labeled Faces in the Wild: A Survey. In Advances in Face Detection and Facial Image Analysis. 189--248.

[19]

Pei Li, Loreto Prieto, Domingo Mery, and Patrick J Flynn. 2019. On Low-Resolution Face Recognition in the Wild: Comparisons and New Techniques. IEEE Transactions on Information Forensics and Security, Vol. 14, 8 (2019), 2000--2012.

Digital Library

[20]

Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. 2017. SphereFace: Deep Hypersphere Embedding for Face Recognition. In IEEE Conference on Computer Vision and Pattern Recognition. 212--220.

[21]

David Lopez-Paz, Léon Bottou, Bernhard Schölkopf, and Vladimir Vapnik. 2016. Unifying Distillation and Privileged Information. In International Conference on Learning Representations .

[22]

Ze Lu, Xudong Jiang, and Alex Kot. 2018. Deep Coupled Resnet for Low-Resolution Face Recognition. IEEE Signal Processing Letters, Vol. 25, 4 (2018), 526--530.

[23]

Ping Luo, Zhenyao Zhu, Ziwei Liu, Xiaogang Wang, and Xiaoou Tang. 2016. Face Model Compression by Distilling Knowledge from Neurons. In AAAI Conference on Artificial Intelligence. 3560--3566.

[24]

Sivaram Prasad Mudunuri and Soma Biswas. 2016. Low Resolution Face Recognition Across Variations in Pose and Illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 38, 5 (2016), 1034--1040.

Digital Library

[25]

Sivaram Prasad Mudunuri, Soubhik Sanyal, and Soma Biswas. 2018. GenLR-Net: Deep Framework for Very Low Resolution Face and Object Recognition with Generalization to Unseen Categories. In IEEE Conference on Computer Vision and Pattern Recognition Workshop. 489--498.

[26]

Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, et almbox. 2015. Deep Face Recognition. In British Machine Vision Conference .

[27]

Ilija Radosavovic, Piotr Dollár, Ross Girshick, Georgia Gkioxari, and Kaiming He. 2018. Data Distillation: Towards Omni-Supervised Learning. In IEEE Conference on Computer Vision and Pattern Recognition. 4119--4128.

[28]

Sachin Ravi and Hugo Larochelle. 2017. Optimization as a Model for Few-Shot Learning. In International Conference on Learning Representations .

[29]

Joseph P Robinson, Ming Shao, Yue Wu, and Yun Fu. 2016. Families in the Wild (FIW): Large-Scale Kinship Image Database and Benchmarks. In ACM International Conference on Multimedia. 242--246.

Digital Library

[30]

Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. 2015. Fitnets: Hints for Thin Deep Nets. In International Conference on Learning Representations .

[31]

Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. FaceNet: A Unified Embedding for Face Recognition and Clustering. In IEEE Conference on Computer Vision and Pattern Recognition. 815--823.

[32]

Jake Snell, Kevin Swersky, and Richard Zemel. 2017. Prototypical Networks for Few-Shot Learning. In Advances in Neural Information Processing Systems. 4077--4087.

[33]

Yi Sun, Yuheng Chen, Xiaogang Wang, and Xiaoou Tang. 2014a. Deep Learning Face Representation by Joint Identification-Verification. In Advances in Neural Information Processing Systems. 1988--1996.

[34]

Yi Sun, Ding Liang, Xiaogang Wang, and Xiaoou Tang. 2015a. DeepID3: Face Recognition with Very Deep Neural Networks. arXiv preprint arXiv:1502.00873 (2015).

[35]

Yi Sun, Xiaogang Wang, and Xiaoou Tang. 2014b. Deep Learning Face Representation from Predicting 10,000 Classes. In IEEE Conference on Computer Vision and Pattern Recognition. 1891--1898.

Digital Library

[36]

Yi Sun, Xiaogang Wang, and Xiaoou Tang. 2015b. Deeply Learned Face Representations are Sparse, Selective, and Robust. In IEEE Conference on Computer Vision and Pattern Recognition. 2892--2900.

[37]

Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. 2018. Learning to Compare: Relation Network for Few-Shot Learning. In IEEE Conference on Computer Vision and Pattern Recognition. 1199--1208.

[38]

Yaniv Taigman, Ming Yang, Marc'Aurelio Ranzato, and Lior Wolf. 2014. DeepFace: Closing the Gap to Human-Level Performance in Face Verification. In IEEE Conference on Computer Vision and Pattern Recognition . 1701--1708.

Digital Library

[39]

Antti Tarvainen and Harri Valpola. 2017. Mean Teachers are Better Role Models: Weight-Averaged Consistency Targets Improve Semi-Supervised Deep Learning Results. In Advances in Neural Information Processing Systems. 1195--1204.

[40]

Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et almbox. 2016. Matching Networks for One Shot Learning. In Advances in Neural Information Processing Systems. 3630--3638.

Digital Library

[41]

Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. 2018a. CosFace: Large Margin Cosine Loss for Deep Face Recognition. In IEEE Conference on Computer Vision and Pattern Recognition. 5265--5274.

[42]

Hui Wang, Hanbin Zhao, Xi Li, and Xu Tan. 2018b. Progressive Blockwise Knowledge Distillation for Neural Network Acceleration. In International Joint Conference on Artificial Intelligence. 2769--2775.

[43]

Zhangyang Wang, Shiyu Chang, Yingzhen Yang, Ding Liu, and Thomas S Huang. 2016. Studying Very Low Resolution Recognition Using Deep Networks. In IEEE Conference on Computer Vision and Pattern Recognition. 4792--4800.

[44]

Bichen Wu, Alvin Wan, Xiangyu Yue, Peter Jin, Sicheng Zhao, Noah Golmant, Amir Gholaminejad, Joseph Gonzalez, and Kurt Keutzer. 2018. Shift: A Zero Flop, Zero Parameter Alternative to Spatial Convolutions. In IEEE Conference on Computer Vision and Pattern Recognition. 9127--9135.

[45]

Fuwei Yang, Wenming Yang, Riqiang Gao, and Qingmin Liao. 2018. Discriminative Multidimensional Scaling for Low-Resolution Face Recognition. IEEE Signal Processing Letters, Vol. 25, 3 (2018), 388--392.

[46]

Dong Yi, Zhen Lei, Shengcai Liao, and Stan Z Li. 2014. Learning Face Representation from Scratch. arXiv preprint arXiv:1411.7923 (2014).

[47]

Xin Yu, Basura Fernando, Richard Hartley, and Fatih Porikli. 2018. Super-Resolving Very Low-Resolution Face Images with Supplementary Attributes. In IEEE Conference on Computer Vision and Pattern Recognition. 908--917.

[48]

Xin Yu and Fatih Porikli. 2017. Hallucinating Very Low-Resolution Unaligned and Noisy Face Images by Transformative Discriminative Autoencoders. In IEEE Conference on Computer Vision and Pattern Recognition. 3760--3768.

[49]

Chenrui Zhang and Yuxin Peng. 2018. Better and Faster: Knowledge Transfer From Multiple Self-Supervised Learning Tasks via Graph Distillation for Video Classification. In International Joint Conference on Artificial Intelligence. 1135--1141.

[50]

Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao. 2016. Joint Face Detection and Alignment using Multitask Cascaded Convolutional Networks. IEEE Signal Processing Letters, Vol. 23, 10 (2016), 1499--1503.

Cited By

Joshi IGrimmer MRathgeb CBusch CBremond FDantcheva A(2024)Synthetic Data in Human Analysis: A SurveyIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.336282146:7(4957-4976)Online publication date: Jul-2024
https://doi.org/10.1109/TPAMI.2024.3362821
Song YWang CTang HHan SLi MTong G(2024)LLRFaceFormer: Lightweight Face Transformer for Real-World Low-Resolution RecognitionIEEE Internet of Things Journal10.1109/JIOT.2024.335606311:9(16769-16787)Online publication date: 1-May-2024
https://doi.org/10.1109/JIOT.2024.3356063
He ZZhang LWang H(2023)An initial prediction and fine-tuning model based on improving GCN for 3D human motion predictionFrontiers in Computational Neuroscience10.3389/fncom.2023.114520917Online publication date: 5-Apr-2023
https://doi.org/10.3389/fncom.2023.1145209
Show More Cited By

Index Terms

Fewer-Shots and Lower-Resolutions: Towards Ultrafast Face Recognition in the Wild
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
    2. Knowledge representation and reasoning

Recommendations

A novel approach inspired by optic nerve characteristics for few-shot occluded face recognition
Abstract
Although there has been a growing body of work for face recognition, it is still a challenging task for faces under occlusion with limited training samples. In this work, we propose a novel framework to address the problem of few-shot ...
Face recognition under varying illumination using gradientfaces

In this correspondence, we propose a novel method to extract illumination insensitive features for face recognition under varying lighting called the Gradientfaces. Theoretical analysis shows Gradientfaces is an illumination insensitive measure, and ...
Age-Invariant Face Recognition

One of the challenges in automatic face recognition is to achieve temporal invariance. In other words, the goal is to come up with a representation and matching scheme that is robust to changes due to facial aging. Facial aging is a complex process that ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '19: Proceedings of the 27th ACM International Conference on Multimedia

October 2019

2794 pages

ISBN:9781450368896

DOI:10.1145/3343031

General Chairs:
Laurent Amsaleg
CNRS-IRISA, France
,
Benoit Huet
EURECOM, France
,
Martha Larson
Radboud University and TU Delft (Netherlands)
,
Program Chairs:
Guillaume Gravier
CNRS-IRISA, France
,
Hayley Hung
Delft University of Technology Netherlands
,
Chong-Wah Ngo
City University of Hong Kong Hong Kong
,
Wei Tsang Ooi
National University of Singapore Singapore

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Basic Research Program of China (973 Program)
National Natural Science Foundation of China
Beijing Nova Program

Conference

MM '19

Sponsor:

SIGMM

MM '19: The 27th ACM International Conference on Multimedia

October 21 - 25, 2019

Nice, France

Acceptance Rates

MM '19 Paper Acceptance Rate 252 of 936 submissions, 27%;

Overall Acceptance Rate 995 of 4,171 submissions, 24%

Upcoming Conference

MM '24

Sponsor:
sigmm

The 32nd ACM International Conference on Multimedia

October 28 - November 1, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
335
Total Downloads

Downloads (Last 12 months)21
Downloads (Last 6 weeks)1

Reflects downloads up to 23 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Joshi IGrimmer MRathgeb CBusch CBremond FDantcheva A(2024)Synthetic Data in Human Analysis: A SurveyIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.336282146:7(4957-4976)Online publication date: Jul-2024
https://doi.org/10.1109/TPAMI.2024.3362821
Song YWang CTang HHan SLi MTong G(2024)LLRFaceFormer: Lightweight Face Transformer for Real-World Low-Resolution RecognitionIEEE Internet of Things Journal10.1109/JIOT.2024.335606311:9(16769-16787)Online publication date: 1-May-2024
https://doi.org/10.1109/JIOT.2024.3356063
He ZZhang LWang H(2023)An initial prediction and fine-tuning model based on improving GCN for 3D human motion predictionFrontiers in Computational Neuroscience10.3389/fncom.2023.114520917Online publication date: 5-Apr-2023
https://doi.org/10.3389/fncom.2023.1145209
Zhang YZheng LThing VZimmermann RGuo BYu Z(2023)FaceLivePlus: A Unified System for Face Liveness Detection and Face VerificationProceedings of the 2023 ACM International Conference on Multimedia Retrieval10.1145/3591106.3592289(144-152)Online publication date: 12-Jun-2023
https://doi.org/10.1145/3591106.3592289
Nagpal SSingh MSingh RVatsa MRatha N(2023)In-Group Bias in Deep Learning-Based Face Recognition Models Due to Ethnicity and AgeIEEE Transactions on Technology and Society10.1109/TTS.2023.32410104:1(54-67)Online publication date: Mar-2023
https://doi.org/10.1109/TTS.2023.3241010
Atzori AFenu GMarras M(2023)Demographic Bias in Low-Resolution Deep Face Recognition in the WildIEEE Journal of Selected Topics in Signal Processing10.1109/JSTSP.2023.324948517:3(599-611)Online publication date: May-2023
https://doi.org/10.1109/JSTSP.2023.3249485
Atzori AFenu GMarras M(2023)(Un)fair Exposure in Deep Face Rankings at a Distance2023 IEEE International Joint Conference on Biometrics (IJCB)10.1109/IJCB57857.2023.10449007(1-9)Online publication date: 25-Sep-2023
https://doi.org/10.1109/IJCB57857.2023.10449007
Wu HZhao YLi J(2023)Invariant and consistent: Unsupervised representation learning for few-shot visual recognitionNeurocomputing10.1016/j.neucom.2022.11.073520(1-14)Online publication date: Feb-2023
https://doi.org/10.1016/j.neucom.2022.11.073
Zhang YZimmermann RYu ZGuo B(2023)Towards an Enhanced and Lightweight Face Authentication SystemCutting Edge Applications of Computational Intelligence Tools and Techniques10.1007/978-3-031-44127-1_10(211-228)Online publication date: 1-Dec-2023
https://doi.org/10.1007/978-3-031-44127-1_10
Chen JChen JWang ZLiang CHan ZLin C(2023)Face Super-Resolution with Better Semantics and More Efficient GuidanceAdvances in Computer Graphics10.1007/978-3-031-23473-6_5(53-64)Online publication date: 1-Jan-2023
https://doi.org/10.1007/978-3-031-23473-6_5
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents