skip to main content
research-article

A Dataset of Head and Eye Movements for 360 Degree Images

Published: 20 June 2017 Publication History

Abstract

Understanding how observers watch visual stimuli like Images and Videos has helped the multimedia encoding, transmission, quality assessment and rendering communities immensely, to learn the regions important to an observer and provide to him/her an optimum quality of experience. The problem is even more paramount in case of 360 degree stimuli considering that most/a part of the content might not be seen by the observers at all, while other regions maybe extraordinarily important. Attention studies in this area has however been missing, mainly due to the lack of a dataset and guidelines to evaluate and compare visual attention/saliency in such scenarios. In this work, we present a dataset of sixty different 360 degree images, each watched by at-least 40 observers. Additionally, we also provide guidelines and tools to the community regarding the procedure to evaluate and compare saliency in omni-directional images. Some basic image/ observer agnostic viewing characteristics, like variation of exploration strategies with time and expertise, and also the effect of eye-movement within the view-port are explored. The dataset and tools are made available for free use by the community and is expected to promote Reproducible Research for all future work on computational modeling of attention in 360 scenarios.

References

[1]
J. G. Apostolopoulos, P. A. Chou, B. Culbertson, T. Kalker, M. D. Trott, and S. Wee. 2012. The Road to Immersive Communication. Proc. IEEE 100, 4 (Apr. 2012), 974--990.
[2]
C. Carlson. How I Made Wine Glasses from Sunflowers. http://blog.wolfram.com/2011/07/28/how-i-made-wine-glasses-from-sunflowers. (????). Accessed: 2015-04-02.
[3]
M-L. Champel and S. Lasserre. 2016. The Special Challenges of Offering High Quality Experience for VR Video. In SMPTE 2016 Annual Technical Conference and Exhibition, Vol. 1100. 1--10.
[4]
C. A. Curcio, K. R. Sloan, R. E. Kalina, and A. E Hendrickson. 1990. Human photoreceptor topography. Journal of Comparative Neurology 4 (1990).
[5]
N Drasdo and CW Fowler. 1974. Non-linear projection of the retinal image in a wide-angle schematic eye. The British journal of ophthalmology 58, 8 (1974), 709.
[6]
U. Engelke, W. Zhang, P. Le Callet, and H. Liu. 2016. Perceived interest versus overt visual attention in image quality assessment, In IS&T/SPIE Electronic Imaging. Proc. SPIE 9394 (2016).
[7]
M. Iwasaki and H Inomata. 1986. Relation between superficial capillaries and foveal structures in the human retina. Investigative ophthalmology & visual science 27, 12 (1986), 1698--1705.
[8]
O. Le Meur and T. Baccino. 2013. Methods for comparing scanpaths and saliency maps: strengths and weaknesses. Behavior research methods 45, 1 (2013), 251--266.
[9]
G. Marmitt and AT. Duchowski. 2002. Modeling visual attention in VR: Measuring the accuracy of predicted scanpaths. In Eurographics 2002, Short Presentations. Saarbrücken, Germany, 217--226.
[10]
Yashas Rai, Ahmed Aldahdooh, Suiyi Ling, Patrick Le Callet, and Marcus Barkowsky. 2016. Effect of content features on short-term video quality in the visual periphery. In Multimedia Signal Processing, 2016. MMSP '16. IEEE International Workshop on. 1--6.
[11]
D. D. Salvucci and J. H. Goldberg. 2000. Identifying fixations and saccades in eye-tracking protocols. In Proceedings of the 2000 symposium on Eye tracking research & applications. ACM, 71--78.
[12]
R. Szeliski. 2006. Image Alignment and Stitching: A Tutorial. Foundations and Trends in Computer Graphics and Vision 2, 1 (Jan. 2006), 1--104.
[13]
J. Thatte, J-B. Boin, H. Lakshman, G. Wetzstein, and B. Girod. 2016. Depth augmented stereo panorama for cinematic virtual reality with focus cues. In IEEE International Conference on Image Processing. 1569--1573.
[14]
I. Tosic and P. Frossard. 2009. Low bit-rate compression of omnidirectional images. In 2009 Picture Coding Symposium. 1--4.
[15]
E. Upenik, M. Rerabek, and T. Ebrahimi. 2016. A Testbed for Subjective Evaluation of Omnidirectional Visual Content. In 32nd Picture Coding Symposium.
[16]
C. Weissig, O. Schreer, P. Eisert, and P. Kauff. 2012. The ultimate immersive experience: Panoramic 3D video acquisition. Lecture Notes in Computer Science 7131 (Feb. 2012), 671--681.
[17]
M. Yu, H. Lakshman, and B. Girod. 2015. Content Adaptive Representations of Omnidirectional Videos for Cinematic Virtual Reality. In Proceedings of the 3rd International Workshop on Immersive Media Experiences. New York, New York, USA, 1--6.
[18]
M. Yu, H. Lakshman, and B. Girod. 2015. A Framework to Evaluate Omnidirectional Video Coding Schemes. In 2015 IEEE International Symposium on Mixed and Augmented Reality. 31--36.
[19]
V. Zakharchenko, K. P. Choi, and J. H. Park. 2016. Quality metric for spherical panoramic video. In Proc. of SPIE Optics and Photonics for Information Processing, Vol. 9970. 99700C.

Cited By

View all
  • (2025)Involution fused convolution for classifying eye-tracking patterns of children with Autism Spectrum DisorderEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.109475139(109475)Online publication date: Jan-2025
  • (2024)Strategies for enhancing automatic fixation detection in head-mounted eye trackingBehavior Research Methods10.3758/s13428-024-02360-056:6(6276-6298)Online publication date: 9-Apr-2024
  • (2024)Complexity matters: Normalization to prototypical viewpoint induces memory distortion along the vertical axis of scenesThe Journal of Neuroscience10.1523/JNEUROSCI.1175-23.2024(e1175232024)Online publication date: 22-May-2024
  • Show More Cited By
  1. A Dataset of Head and Eye Movements for 360 Degree Images

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MMSys'17: Proceedings of the 8th ACM on Multimedia Systems Conference
    June 2017
    407 pages
    ISBN:9781450350020
    DOI:10.1145/3083187
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    In-Cooperation

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 20 June 2017

    Permissions

    Request permissions for this article.

    Check for updates

    Badges

    Author Tags

    1. Omnidirectional images
    2. dataset
    3. eye-tracking
    4. saliency in 360 images
    5. subjective experiment

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    Conference

    MMSys'17
    Sponsor:
    MMSys'17: Multimedia Systems Conference 2017
    June 20 - 23, 2017
    Taipei, Taiwan

    Acceptance Rates

    MMSys'17 Paper Acceptance Rate 13 of 47 submissions, 28%;
    Overall Acceptance Rate 176 of 530 submissions, 33%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)163
    • Downloads (Last 6 weeks)17
    Reflects downloads up to 19 Oct 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Involution fused convolution for classifying eye-tracking patterns of children with Autism Spectrum DisorderEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.109475139(109475)Online publication date: Jan-2025
    • (2024)Strategies for enhancing automatic fixation detection in head-mounted eye trackingBehavior Research Methods10.3758/s13428-024-02360-056:6(6276-6298)Online publication date: 9-Apr-2024
    • (2024)Complexity matters: Normalization to prototypical viewpoint induces memory distortion along the vertical axis of scenesThe Journal of Neuroscience10.1523/JNEUROSCI.1175-23.2024(e1175232024)Online publication date: 22-May-2024
    • (2024)Privacy-preserving Scanpath Comparison for Pervasive Eye TrackingProceedings of the ACM on Human-Computer Interaction10.1145/36556058:ETRA(1-28)Online publication date: 28-May-2024
    • (2024)Panonut360Proceedings of the 15th ACM Multimedia Systems Conference10.1145/3625468.3652176(319-325)Online publication date: 15-Apr-2024
    • (2024)Saliency and Depth-Aware Full Reference 360-Degree Image Quality AssessmentInternational Journal of Pattern Recognition and Artificial Intelligence10.1142/S021800142351022938:01Online publication date: 9-Feb-2024
    • (2024)SISAd: Advertisement Implantation in VR Video with Saliency-constrained Instance Segmentation2024 IEEE International Conference on Consumer Electronics (ICCE)10.1109/ICCE59016.2024.10444331(1-5)Online publication date: 6-Jan-2024
    • (2024)TransUser's: A Transformer Based Salient Object Detection for Users Experience Generation in 360° Videos2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)10.1109/AIxVR59861.2024.00042(256-260)Online publication date: 17-Jan-2024
    • (2024)3D Pop-Ups: Omnidirectional image visual saliency prediction based on crowdsourced eye-tracking data in VRDisplays10.1016/j.displa.2024.10274683(102746)Online publication date: Jul-2024
    • (2024)An egocentric video and eye-tracking dataset for visual search in convenience storesComputer Vision and Image Understanding10.1016/j.cviu.2024.104129248(104129)Online publication date: Nov-2024
    • Show More Cited By

    View Options

    Get Access

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media