research-article

SweepLoc: Automatic Video-based Indoor Localization by Camera Sweeping

Authors:

S.-H. Gary Chan,

Chengying GaoAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 2, Issue 3

Article No.: 120, Pages 1 - 25

https://doi.org/10.1145/3264930

Published: 18 September 2018 Publication History

Abstract

Indoor localization based on visual landmarks has received much attention in commercial sites with rich features (e.g., shopping malls, museums) recently because landmarks are relatively stable over a long time. Prior arts often require a user to take multiple independent images around his/her location, and manually confirm shortlisted landmarks. The process is sophisticated, inconvenient, slow, unnatural and error-prone. To overcome these limitations, we propose SweepLoc, a novel, efficient and automatic video-based indoor localization system. SweepLoc mimics our natural scanning around to identify nearby landmarks in an unfamiliar site to localize.

In SweepLoc, a user simply takes a short video clip (about 6 to 8 seconds) of his/her surroundings by sweeping the camera. Using correlation and scene continuity between successive video frames, it automatically and efficiently selects key frames (where potential landmarks are centered) and subsequently reduces the decision error on landmarks. With identified landmarks, SweepLoc formulates an optimization problem to locate the user, taking compass noise and floor map constraint into account. We have implemented SweepLoc in Android platform. Our extensive experimental results in a food plaza and a premium mall demonstrate that SweepLoc is fast (less than 1 second to localize), and achieves substantially better accuracy as compared with the state-of-the-art approaches (reducing the localization error by 30%).

References

[1]

P. Bahl and V. N. Padmanabhan. 2000. RADAR: an in-building RF-based user location and tracking system. In Proc. of IEEE INFOCOM, Vol. 2. IEEE, 775--784 vol.2.

[2]

Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. 2006. SURF: speeded up robust features. In Proc. of Springer ECCV. Springer, Berlin, Heidelberg, 404--417.

Digital Library

[3]

Jiang Dong, Yu Xiao, Marius Noreikis, Zhonghong Ou, and Antti Ylä-Jääski. 2015. iMoon: using smartphones for image-based indoor navigation. In Proc. of ACM SenSys. New York, NY, USA, 85--97.

Digital Library

[4]

Jiang Dong, Yu Xiao, Zhonghong Ou, Yong Cui, and Antti Ylä-Jääski. 2016. Indoor Tracking Using Crowdsourced Maps. In Proc. of IEEE IPSN. Article 5, 6 pages.

Digital Library

[5]

G David Forney. 1973. The viterbi algorithm. Proc. IEEE 61, 3 (1973), 268--278.

[6]

R. Gao, Y. Tian, F. Ye, G. Luo, K. Bian, Y. Wang, T. Wang, and X. Li. 2016. Sextant: Towards ubiquitous indoor localization service by photo-taking of the environment. IEEE Trans. Mob. Comput. 15, 2 (Feb 2016), 460--474.

Digital Library

[7]

R. Gao, B. Zhou, F. Ye, and Y. Wang. 2017. Knitter: Fast, resilient single-user indoor floor plan construction. In Proc. IEEE INFOCOM. IEEE, 1--9.

[8]

Cole Gleason, Dragan Ahmetovic, Saiph Savage, Carlos Toxtli, Carl Posthuma, Chieko Asakawa, Kris M. Kitani, and Jeffrey P. Bigham. 2018. Crowdsourcing the Installation and Maintenance of Indoor Localization Infrastructure to Support Blind Navigation. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 1, Article 9 (March 2018), 25 pages.

Digital Library

[9]

Fei Gu, Jianwei Niu, and Lingjie Duan. 2017. WAIPO: A Fusion-Based Collaborative Indoor Localization System on Smartphones. IEEE/ACM Trans. Netw. 25, 4 (Aug 2017), 2267--2280.

Digital Library

[10]

B. Guo, Q. Han, H. Chen, L. Shangguan, Z. Zhou, and Z. Yu. 2017. The Emergence of Visual Crowdsensing: Challenges and Opportunities. Commun. Surveys Tuts. 19, 4 (Fourthquarter 2017), 2526--2543.

[11]

J. F. Henriques, R. Caseiro, P. Martins, and J. Batista. 2015. High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37, 3 (March 2015), 583--596.

Digital Library

[12]

Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: convolutional architecture for fast feature embedding. In Proc. of ACM MM. 675--678.

Digital Library

[13]

Hernisa Kacorri, Eshed Ohn-Bar, Kris M. Kitani, and Chieko Asakawa. 2018. Environmental Factors in Indoor Navigation Based on Real-World Trajectories of Blind Users. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, Article 56, 12 pages.

Digital Library

[14]

Alex Kendall, Matthew Grimes, and Roberto Cipolla. 2015. PoseNet: a convolutional network for real-time 6-DOF camera relocalization. In Proc. of IEEE ICCV. 2938--2946.

Digital Library

[15]

T. H. Kim, S. Nah, and K. M. Lee. 2017. Dynamic Video Deblurring using a Locally Adaptive Linear Blur Model. IEEE Trans. Pattern Anal. Mach. Intell. PP, 99 (Oct 2017), 1--1.

[16]

Patrick Lazik, Niranjini Rajagopal, Oliver Shih, Bruno Sinopoli, and Anthony Rowe. 2015. ALPS: A Bluetooth and Ultrasound Platform for Mapping and Localization. In Proc. ACM Sensys. ACM, 73--84.

Digital Library

[17]

Kai Liu, Hao Zhang, Joseph Kee-Yin Ng, Yusheng Xia, Liang Feng, Victor CS Lee, and Sang H Son. 2018. Toward Low-Overhead Fingerprint-Based Indoor Localization via Transfer Learning: Design, Implementation, and Evaluation. IEEE Trans. Ind. Informat. 14, 3 (2018), 898--908.

[18]

Z. Liu, L. Zhang, Q. Liu, Y. Yin, L. Cheng, and R. Zimmermann. 2017. Fusion of magnetic and visual sensors for indoor localization: infrastructure-free and more effective. IEEE Trans. Multimedia 19, 4 (April 2017), 874--888.

Digital Library

[19]

Chengwen Luo, Hande Hong, Mun Choon Chan, Jianqiang Li, Xinglin Zhang, and Zhong Ming. 2018. MPiLoc: Self-Calibrating Multi-Floor Indoor Localization Exploiting Participatory Sensing. IEEE Trans. Mob. Comput. 17, 1 (2018), 141--154.

[20]

Raul Mur-Artal and Juan D Tardós. 2017. Orb-slam2: An opensource slam system for monocular, stereo, and rgb-d cameras. IEEE Trans. Robot. 33, 5 (2017), 1255--1262.

[21]

S. Papaioannou, A. Markham, and N. Trigoni. 2017. Tracking People in Highly Dynamic Industrial Environments. IEEE Trans. Mob. Comput. 16, 8 (Aug 2017), 2351--2365.

[22]

Valter Pasku, Alessio De Angelis, Darmindra D. Arumugam, Marco Dionigi, Paolo Carbone, Antonio Moschitta, and David S. Ricketts. 2017. Magnetic Field-Based Positioning Systems. IEEE Commun. Surveys Tuts. 19, 3 (Mar 2017), 2003--2017.

[23]

Claudio Piciarelli. 2016. Visual indoor localization in known environments. IEEE Signal Process. Lett. 23, 10 (2016), 1330--1334.

[24]

Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: towards real-time object detection with region proposal networks. In Proc. of NIPS. MIT Press, 91--99.

Digital Library

[25]

Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary Bradski. 2011. ORB: an efficient alternative to SIFT or SURF. In Proc. of IEEE ICCV. 2564--2571.

Digital Library

[26]

Torsten Sattler, Bastian Leibe, and Leif Kobbelt. 2016. Exploiting Spatial and Co-visibility Relations for Image-Based Localization. Springer International Publishing, Cham, 165--187.

[27]

Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. International Conference on Learning Representations (2015).

[28]

Masato Sugasaki and Masamichi Shimosaka. 2017. Robust Indoor Localization Across Smartphone Models with Ellipsoid Features from Multiple RSSIs. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 3, Article 103 (Sept. 2017), 16 pages.

Digital Library

[29]

Xiaohua Tian, Mei Wang, Wenxin Li, Binyao Jiang, Dong Xu, Xinbing Wang, and Jun Xu. 2018. Improve Accuracy of Fingerprinting Localization with Temporal Correlation of the RSS. IEEE Trans. Mob. Comput. 17, 1 (2018), 113--126.

[30]

Matthew Wall. 1996. GAlib: A C++ library of genetic algorithm components. Mechanical Engineering Department, Massachusetts Institute of Technology 87 (1996), 54.

[31]

S. Wang, S. Fidler, and R. Urtasun. 2015. Lost Shopping! Monocular Localization in Large Indoor Spaces. In Proc. IEEE ICCV. IEEE, 2695--2703.

Digital Library

[32]

Yu-Lin Wei, Chang-Jung Huang, Hsin-Mu Tsai, and Kate Ching-Ju Lin. 2017. CELLI: Indoor Positioning Using Polarized Sweeping Light Beams. In Proc. of ACM MobiSys. ACM, 136--147.

Digital Library

[33]

Hongkai Wen, Sen Wang, Ronnie Clark, Savvas Papaioannou, and Niki Trigoni. 2016. Poster: Efficient Visual Positioning with Adaptive Parameter Learning. In Proc. of IEEE IPSN. IEEE Press, Piscataway, NJ, USA, Article 34, 2 pages.

Digital Library

[34]

M. Werner, M. Kessel, and C. Marouane. 2011. Indoor positioning using smartphone camera. In Proc. of IEEE IPIN. 1--6.

[35]

Chenshu Wu, Zheng Yang, and Chaowei Xiao. 2018. Automatic Radio Map Adaptation for Indoor Localization Using Smartphones. IEEE Trans. Mob. Comput. 17, 3 (2018), 517--528.

[36]

K. Wu, Jiang Xiao, Youwen Yi, Min Gao, and L. M. Ni. 2012. FILA: Fine-grained indoor localization. In Proc. IEEE INFOCOM. IEEE, 2210--2218.

[37]

Han Xu, Zheng Yang, Zimu Zhou, and Chunyi Peng Ke Yi. 2017. TUM: Towards Ubiquitous Multi-Device Localization for Cross-Device Interaction. In Proc. IEEE INFOCOM.

[38]

Han Xu, Zheng Yang, Zimu Zhou, Longfei Shangguan, Ke Yi, and Yunhao Liu. 2015. Enhancing wifi-based localization with visual clues. In Proc. of ACM UbiComp. ACM, New York, New York, USA, 963--974.

Digital Library

[39]

Han Xu, Zheng Yang, Zimu Zhou, Longfei Shangguan, Ke Yi, and Yunhao Liu. 2016. Indoor Localization via Multimodal Sensing on Smartphones. In Proc. ACM UbiComp. ACM, 208--219.

Digital Library

[40]

Yi Yao, Besma Abidi, Narjes Doggaz, and Mongi Abidi. 2006. Evaluation of sharpness measures and search algorithms for the auto focusing of high-magnification images. In Visual Information Processing XV, Vol. 6246. International Society for Optics and Photonics, 62460G.

[41]

A. Yassin, Y. Nasser, M. Awad, A. Al-Dubai, R. Liu, C. Yuen, R. Raulefs, and E. Aboutanios. 2017. Recent Advances in Indoor Localization: A Survey on Theoretical Approaches and Applications. Commun. Surveys Tuts. 19, 2 (Secondquarter 2017), 1327--1346.

[42]

Xuehan Ye, Yongcai Wang, Yuhe Guo, Wei Hu, and Deying Li. 2018. Accurate and Efficient Indoor Location by Dynamic Warping in Sequence-Type Radio-Map. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 1 (2018), 50.

Digital Library

[43]

Moustafa Youssef and Ashok Agrawala. 2005. The horus WLAN location determination system. In Proc. of ACM MobiSys. 205--218.

Digital Library

[44]

Matthew D. Zeiler and Rob Fergus. 2014. Visualizing and Understanding Convolutional Networks. In Proc. of Springer ECCV. Springer, Cham, 818--833.

[45]

Hongwei Zhang, Barry Lennox, Peter R Goulding, and Andrew YT Leung. 2000. A float-encoded genetic algorithm technique for integrated optimization of piezoelectric actuator and sensor placement and feedback gains. Smart Materials and Structures 9, 4 (2000), 552.

[46]

X. Zhang, A. K. S. Wong, C. T. Lea, and R. S. K. Cheng. 2018. Unambiguous Association of Crowd-Sourced Radio Maps to Floor Plans for Indoor Localization. IEEE Trans. Mob. Comput. 17, 2 (Feb 2018), 488--502.

[47]

Yongtuo Zhang, Wen Hu, Weitao Xu, Hongkai Wen, and Chun Tung Chou. 2016. NaviGlass: indoor localisation using smart glasses. In Proc. of ACM EWSN. 205--216.

Digital Library

[48]

Y. Zheng, G. Shen, L. Li, C. Zhao, M. Li, and F. Zhao. 2017. Travi-Navi: Self-Deployable Indoor Navigation System. IEEE/ACM Trans. Netw. 25, 5 (Oct 2017), 2655--2669.

Digital Library

[49]

Shilin Zhu and Xinyu Zhang. 2017. Enabling High-Precision Visible Light Localization in Today's Buildings. In Proc. of ACM MobiSys. ACM, 96--108.

Digital Library

[50]

Jinbo Zuo, Shuo Liu, Hao Xia, and Yanyou Qiao. 2018. Multi-Phase Fingerprint Map Based on Interpolation for Indoor Localization Using iBeacons. IEEE Sensors J. (2018), 3351--3359.

Cited By

Liu MLiao XZhang YGao Z(2024)A Multiscale Spatial-Temporal Features Fusion Framework for Indoor LocalizationIEEE Sensors Journal10.1109/JSEN.2024.339577224:14(23098-23107)Online publication date: 15-Jul-2024
https://doi.org/10.1109/JSEN.2024.3395772
Li GDeng XXu JLiu YZhang JXiong SGao F(2023)An Efficient and Accurate Convolution-Based Similarity Measure for Uncertain TrajectoriesISPRS International Journal of Geo-Information10.3390/ijgi1210043212:10(432)Online publication date: 22-Oct-2023
https://doi.org/10.3390/ijgi12100432
Ding YYang YJiang WLiu YHe TZhang D(2023)Nationwide Deployment and Operation of a Virtual Arrival Detection System in the WildIEEE/ACM Transactions on Networking10.1109/TNET.2022.319680631:2(574-589)Online publication date: Apr-2023
https://doi.org/10.1109/TNET.2022.3196806
Show More Cited By

Index Terms

SweepLoc: Automatic Video-based Indoor Localization by Camera Sweeping
1. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing systems and tools
2. Networks
  1. Network services
    1. Location based services

Recommendations

Human appearance modeling for matching across video sequences
Abstract
We present an appearance model for establishing correspondence between tracks of people which may be taken at different places, at different times or across different cameras. The appearance model is constructed by kernel density estimation. To ...
A Polygonal Method for Ranging-Based Localization in an Indoor Wireless Sensor Network

In this paper, we propose an indoor localization method in a wireless sensor network based on IEEE 802.15.4 specification. The proposed method follows a ranging-based approach using not only the measurements of received signal strength (RSS) but also ...
LiFS: low human-effort, device-free localization with fine-grained subcarrier information
MobiCom '16: Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking

Device-free localization of people and objects indoors not equipped with radios is playing a critical role in many emerging applications. This paper presents an accurate model-based device-free localization system LiFS, implemented on cheap commercial ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 2, Issue 3

September 2018

1536 pages

EISSN:2474-9567

DOI:10.1145/3279953

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 September 2018

Accepted: 01 September 2018

Revised: 01 May 2018

Received: 01 February 2018

Published in IMWUT Volume 2, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
467
Total Downloads

Downloads (Last 12 months)34
Downloads (Last 6 weeks)6

Reflects downloads up to 23 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu MLiao XZhang YGao Z(2024)A Multiscale Spatial-Temporal Features Fusion Framework for Indoor LocalizationIEEE Sensors Journal10.1109/JSEN.2024.339577224:14(23098-23107)Online publication date: 15-Jul-2024
https://doi.org/10.1109/JSEN.2024.3395772
Li GDeng XXu JLiu YZhang JXiong SGao F(2023)An Efficient and Accurate Convolution-Based Similarity Measure for Uncertain TrajectoriesISPRS International Journal of Geo-Information10.3390/ijgi1210043212:10(432)Online publication date: 22-Oct-2023
https://doi.org/10.3390/ijgi12100432
Ding YYang YJiang WLiu YHe TZhang D(2023)Nationwide Deployment and Operation of a Virtual Arrival Detection System in the WildIEEE/ACM Transactions on Networking10.1109/TNET.2022.319680631:2(574-589)Online publication date: Apr-2023
https://doi.org/10.1109/TNET.2022.3196806
Wong MLu QChan S(2023)SiFu: A Generic and Robust Multimodal Signal Fusion Platform for Pervasive LocalizationIEEE Internet of Things Journal10.1109/JIOT.2022.320836710:1(904-919)Online publication date: 1-Jan-2023
https://doi.org/10.1109/JIOT.2022.3208367
Campana MDelmastro FPagani E(2023)Transfer learning for the efficient detection of COVID-19 from smartphone audio dataPervasive and Mobile Computing10.1016/j.pmcj.2023.10175489:COnline publication date: 22-Mar-2023
https://dl.acm.org/doi/10.1016/j.pmcj.2023.101754
Kumrai TKorpela JZhang YOhara KMurakami TAbeysekera HMaekawa T(2023)Automated construction of Wi-Fi-based indoor logical location predictor using crowd-sourced photos with Wi-Fi signalsPervasive and Mobile Computing10.1016/j.pmcj.2022.10174289:COnline publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1016/j.pmcj.2022.101742
Belmonte-Fernández ÓSansano-Sansano ECaballer-Miedes AMontoliu RGarcía-Vidal RGascó-Compte A(2021)A Generative Method for Indoor Localization Using Wi-Fi FingerprintingSensors10.3390/s2107239221:7(2392)Online publication date: 30-Mar-2021
https://doi.org/10.3390/s21072392
Liu NHe THe SNiu Q(2021)Indoor Localization With Adaptive Signal Sequence RepresentationsIEEE Transactions on Vehicular Technology10.1109/TVT.2021.311333370:11(11678-11694)Online publication date: Nov-2021
https://doi.org/10.1109/TVT.2021.3113333
Niu QHe TLiu NHe SLuo XZhou F(2020)MAILProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/33973354:2(1-23)Online publication date: 15-Jun-2020
https://dl.acm.org/doi/10.1145/3397335
Niu XXie LWang JChen HLiu DChen R(2020)AtLAS: An Activity-Based Indoor Localization and Semantic Labeling Mechanism for ResidencesIEEE Internet of Things Journal10.1109/JIOT.2020.30044967:10(10606-10622)Online publication date: Oct-2020
https://doi.org/10.1109/JIOT.2020.3004496
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents