Skip to main content
Springer Nature Link
Log in

Crowdsourcing of labeling image objects: an online gamification application for data collection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This study aimed to improve the labeling of objects inside images in the crowdsourcing process. Images are one of the most widely used types of data on the internet. Controlling and refining images through the crowdsourcing process is time-consuming and tedious because of their quality and nature. Gamification of data collection was presented as a solution to review, categorize images, and motivate people. Because participant motivation might influence the quality of the output data, we used gamification elements to manage user interaction in this study. The proposed method has a great effect on improving the quality of output data by considering various challenges such as motivation, financial costs, and delays. The proposed algorithm calculates the average of the points specified by each user and then compares it with the average of the total correct answers. In the end, the proposed algorithm uses this comparison to decide whether to accept or reject the answer. In this research, the LabelMe, Flickr, and VOC2012 datasets were used. Implementing the proposed method in a real context showed that the proposed design improved the image labeling accuracy, which was increased by 11.3% compared to the previous methods. In this experiment, the people who interacted the most generated the most accurate data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1.
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

Similar content being viewed by others

Data availability

The dataset generated during the current study are available from the corresponding author on reasonable request.

References

  1. Adelsman RM, Whinston AB (1977) Sophisticated voting with information for two voting functions. J Econ Theory 15(1):145–159

    MathSciNet  Google Scholar 

  2. Allahverdi R, Bastanfard A, Akbarzadeh D (2012) Sasanian coins classification using discrete cosine transform. In: The 16th CSI international symposium on artificial intelligence and signal processing (AISP 2012). IEEE, pp 278–282

    Google Scholar 

  3. Amirkhani D, Bastanfard A (2019) Inpainted image quality evaluation based on saliency map features. In: 2019 5th Iranian conference on signal processing and intelligent systems (ICSPIS), pp 1–6. https://doi.org/10.1109/ICSPIS48872.2019.9066140

    Chapter  Google Scholar 

  4. Amirkhani D, Bastanfard A (2021) An objective method to evaluate exemplar-based inpainted images quality using Jaccard index. Multimed Tools Appl 80:26199–26212. https://doi.org/10.1007/s11042-021-10883-3

  5. Antin J (2012) Gamification is not a dirty word. Interactions 19(4):14–16

    Google Scholar 

  6. Aparicio AF, Vela FL, Sánchez JL, Montes JL (2012) Analysis and application of gamification. In: Proceedings of the 13th international conference on interacción persona-ordenador, pp 1–2

  7. Bagley KS (2012) Conceptual mile markers to improve time-to-value for exploratory search sessions. University of Massachusetts Lowell

    Google Scholar 

  8. Bastanfard A, Amirkhani D (2021) Improving the accuracy of the annotation algorithm in pattern-based tennis game video. In: 2021 29th Iranian conference on electrical engineering (ICEE). IEEE, pp 493–497

    Google Scholar 

  9. Bastanfard A, Aghaahmadi M, Fazel M, Moghadam M (2009) Persian viseme classification for developing visual speech training application. In: Pacific-rim conference on multimedia. Springer, Berlin, Heidelberg, pp 1080–1085

    Google Scholar 

  10. Bastanfard A, Jafari S, Amirkhani D (2019) Improving tracking soccer players in shaded playfield video. In: 2019 5th Iranian conference on signal processing and intelligent systems (ICSPIS). IEEE, pp 1–8

    Google Scholar 

  11. Bastanfard A, Amirkhani D, Mohammadi M (2022) Toward image super-resolution based on local regression and nonlocal means. Multimed Tools Appl 81:23473–23492. https://doi.org/10.1007/s11042-022-12584-x

  12. Berengueres J, Alsuwairi F, Zaki N, Ng T (2013) Gamification of a recycle bin with emoticons. In: 2013 8th ACM/IEEE international conference on human-robot interaction (HRI). IEEE, pp 83–84

  13. Bhatti UA, Huang M, Wang H, Zhang Y, Mehmood A, Di W (2018) Recommendation system for immunization coverage and monitoring. Human Vaccin Immunother 14(1):165–171

    Google Scholar 

  14. Bhatti UA, Huang M, Wu D, Zhang Y, Mehmood A, Han H (2019) Recommendation system using feature extraction and pattern recognition in clinical care systems. Enterp Inf Syst 13(3):329–351

    Google Scholar 

  15. Bhatti SS, Gao X, Chen G (2020) General framework, opportunities and challenges for crowdsourcing techniques: a comprehensive survey. J Syst Softw 167:110611

    Google Scholar 

  16. Bhatti UA, Yu Z, Chanussot J, Zeeshan Z, Yuan L, Luo W, Nawaz SA, Bhatti MA, Ain QU, Mehmood A (2021) Local similarity-based spatial–spectral fusion hyperspectral image classification with deep CNN and Gabor filtering. IEEE Trans Geosci Remote Sens 60:1–15

    Google Scholar 

  17. Bhatti UA, Zeeshan Z, Nizamani MM, Bazai S, Yu Z, Yuan L (2022) Assessing the change of ambient air quality patterns in Jiangsu Province of China pre-to post-COVID-19. Chemosphere 288:132569

    ADS  CAS  PubMed  Google Scholar 

  18. Bhatti UA, Yu Z, Hasnain A, Nawaz SA, Yuan L, Wen L, Bhatti MA (2022) Evaluating the impact of roads on the diversity pattern and density of trees to improve the conservation of species. Environ Sci Pollut Res 29(10):14780–14790

    Google Scholar 

  19. Bista SK, Nepal S, Colineau N, Paris C (2012) Using gamification in an online community. In: 8th international conference on collaborative computing: networking, applications and Worksharing (CollaborateCom). IEEE, pp 611–618

  20. Bista SK, Nepal S, Paris C (2012) Engagement and cooperation in social networks: do benefits and rewards help? In: 2012 IEEE 11th international conference on trust, security and privacy in computing and communications. IEEE, pp 1405–1410

  21. Blohm, I. J.M. Leimeister, Design of IT-based enhancing services for motivational support and behavioral change. 2013.

  22. Borzsony S, Kossmann D, Stocker K (2001) The skyline operator. In: Proceedings 17th international conference on data engineering. IEEE

    Google Scholar 

  23. Bradley RA, Terry ME (1952) Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika 39(3/4):324–345

    MathSciNet  Google Scholar 

  24. Busa-Fekete R, Szorenyi B, Cheng W, Weng P, Hüllermeier E (2013) Top-k selection based on adaptive sampling of noisy preferences. In: International conference on machine learning. PMLR, pp 1094–1102

  25. Cramer H, Rost M, Holmquist LE (2011) Performing a check-in: emerging practices, norms and conflicts; in location sharing using foursquare. In: Proceedings of the 13th international conference on human computer interaction with mobile devices and services, pp 57–66

  26. Csikszentmihalyi M, Csikzentmihaly M (1990) Flow: The psychology of optimal experience. Vol. 1990. Harper & Row New York

    Google Scholar 

  27. Davidson SB, Khanna S, Milo T, Roy S (2013) Using the crowd for top-k and group-by queries. In: Proceedings of the 16th international conference on database theory, pp 225–236

  28. Deci EL, Koestner R, Ryan RM (1999) A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychol Bull 125(6):627

    CAS  PubMed  Google Scholar 

  29. Demartini G, Difallah DE, Cudré-Mauroux P (2012) Zencrowd: leveraging probabilistic reasoning and crowdsourcing techniques for large-scale entity linking. In: Proceedings of the 21st international conference on world wide web, pp 469–478

  30. Deterding S (2011) Situated motivational affordances of game elements: a conceptual model. In Gamification: using game design elements in non-gaming contexts, a workshop at CHI 2011 10, 1979742.1979575

  31. Elo AE (1978) The rating of chessplayers, past and present. Arco Pub

    Google Scholar 

  32. Eriksson B (2013) Learning to top-k search using pairwise comparisons. In: Artificial intelligence and statistics. PMLR, pp 265–273

  33. Fan J et al (2014) A hybrid machine-crowdsourcing system for matching web tables. In: 2014 IEEE 30th international conference on data engineering. IEEE

    Google Scholar 

  34. Fang Y et al (2016) Effective result inference for context-sensitive tasks in crowdsourcing. In: International conference on database systems for advanced applications. Springer

    Google Scholar 

  35. Feige U et al (1994) Computing with noisy information. SIAM J Comput 23(5):1001–1018

    MathSciNet  Google Scholar 

  36. Frith JH (2012) Constructing location, one check-in at a time: examining the practices of foursquare users. North Carolina State University

  37. Gåsland MM (2011) Game mechanic based e-learning: A case study (Master thesis, Instituttor datateknikk og informasjonsvitenskap)

  38. Ghadiyaram D, Bovik AC (2015) Massive online crowdsourced study of subjective and objective picture quality. IEEE Trans Image Process 25(1):372–387

    ADS  MathSciNet  PubMed  Google Scholar 

  39. Ghallab M, Nau D, Traverso P (2004) Automated planning: theory and practice. Elsevier

    Google Scholar 

  40. Gnauk B, Dannecker L, Hahmann M (2012) Leveraging gamification in demand dispatch systems. In: Proceedings of the 2012 joint EDBT/ICDT workshops, pp 103–110

  41. Gokhale C, Das S, Doan A, Naughton JF, Rampalli N, Shavlik J, Zhu X (2014) Corleone: hands-off crowdsourcing for entity matching. In: Proceedings of the 2014 ACM SIGMOD international conference on management of data, pp 601–612

  42. Grote A, Schaadt NS, Forestier G, Wemmert C, Feuerhake F (2018) Crowdsourcing of histological image labeling and object delineation by medical students. IEEE Trans Med Imaging 38(5):1284–1294

    PubMed  Google Scholar 

  43. Groz B, Milo T (2015) Skyline queries with noisy comparisons. In: Proceedings of the 34th ACM sigmod-sigact-sigai symposium on principles of database systems, pp 185–198

  44. Gruenheid A et al (2012) Crowdsourcing entity resolution: when is a= b? Technical Report/ETH Zurich. Department of Computer Science, p 785

    Google Scholar 

  45. Guin TD-L et al (2012) Myths and realities of respondent engagement in online surveys. Int J Mark Res 54(5):613–633

    Google Scholar 

  46. Guo S, Parameswaran A, Garcia-Molina H (2012) So who won? Dynamic max discovery with the crowd. In: Proceedings of the 2012 ACM SIGMOD international conference on management of data, pp 385–396

  47. Hajarian M, Bastanfard A, Mohammadzadeh J, Khalilian M (2019) A personalized gamification method for increasing user engagement in social networks. Soc Netw Anal Min 9(1):1–14

    Google Scholar 

  48. Hajarian M, Bastanfard A, Mohammadzadeh J, Khalilian M (2019) SNEFL: social network explicit fuzzy like dataset and its application for Incel detection. Multimed Tools Appl 78:33457–33486. https://doi.org/10.1007/s11042-019-08057-3

    Article  Google Scholar 

  49. Heikinheimo H, Ukkonen A (2013) The crowd-median algorithm. In: Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, vol 1, pp 69–77

  50. Herbrich R, Minka T, Graepel T (2006) TrueSkill™: a Bayesian skill rating system. Adv Neural Inf Proces Syst 19

  51. Hu L, Qian Y, Chen J, Shi X, Zhang J, Mao S (2017) Photo crowdsourcing based privacy-protected healthcare. IEEE Trans Sustain Comput 4(2):168–177

    Google Scholar 

  52. Jeffery SR, Franklin MJ, Halevy AY (2008) Pay-as-you-go user feedback for dataspace systems. In: Proceedings of the 2008 ACM SIGMOD international conference on management of data, pp 847–860

  53. Jiang X et al (2011) Statistical ranking and combinatorial Hodge theory. Math Program 127(1):203–244

    MathSciNet  Google Scholar 

  54. Kaplan H et al (2013) Answering planning queries with the crowd. Proc VLDB Endow 6(9):697–708

    Google Scholar 

  55. Kapp KM (2012) The gamification of learning and instruction: game-based methods and strategies for training and education. John Wiley & Sons

    Google Scholar 

  56. Khan AR, Garcia-Molina H (2014) Hybrid strategies for finding the max with the crowd: technical report. Stanford InfoLab

    Google Scholar 

  57. Lee H, Doh YY (2012) A study on the relationship between educational achievement and emotional engagement in a gameful interface for video lecture systems. In: 2012 international symposium on ubiquitous virtual reality. IEEE

    Google Scholar 

  58. Li G, Deng D, Wang J, Feng J (2011) Pass-join: a partition-based method for similarity joins. arXiv preprint arXiv:11117171

  59. Li G, Zheng Y, Fan J, Wang J, Cheng R (2017) Crowdsourced data management: overview and challenges. In: Proceedings of the 2017 ACM international conference on Management of Data, pp 1711–1716

  60. Liu Y, Alexandrova T, Nakajima T (2011) Gamifying intelligent environments. In: Proceedings of the 2011 international ACM workshop on ubiquitous meta user interfaces, pp 7–12

  61. Lofi C, El Maarry K, Balke WT (2013) Skyline queries in crowd-enabled databases. In: Proceedings of the 16th international conference on extending database technology, pp 465–476

  62. Lofi C, El Maarry K, Balke W-T (2013) Skyline queries over incomplete data-error models for focused crowd-sourcing. In: International conference on conceptual modeling. Springer

    Google Scholar 

  63. Malone TW (1981) Toward a theory of intrinsically motivating instruction. Cogn Sci 5(4):333–369

    Google Scholar 

  64. Malone TW (1982) Heuristics for designing enjoyable user interfaces: lessons from computer games. In: Proceedings of the 1982 conference on human factors in computing systems, pp 63–68

  65. Marcus A, Wu E, Karger D, Madden S, Miller R (2011) Human-powered sorts and joins. arXiv preprint arXiv:11096881

  66. Marcus A, Karger D, Madden S, Miller R, Oh S (2012) Counting with the crowd. Proc VLDB Endow 6(2):109–120

  67. Mason AD, Michalakidis G, Krause PJ (2012) Tiger nation: empowering citizen scientists. In: 2012 6th IEEE international conference on digital ecosystems and technologies (DEST). IEEE

    Google Scholar 

  68. Massung E, Coyle D, Cater KF, Jay M, Preist C (2013) Using crowdsourcing to support pro-environmental community activism. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 371–380

  69. McDaniel R, Lindgren R, Friskics J (2012) Using badges for shaping interactions in online learning environments. In: 2012 IEEE international professional communication conference. IEEE

    Google Scholar 

  70. McGonigal J, Broken RI (2011) Why games make us better and how they can change the world. The Penguin Group

    Google Scholar 

  71. Mekler ED et al (2013) Do points, levels and leaderboards harm intrinsic motivation? An empirical analysis of common gamification elements. In: Proceedings of the first international conference on gameful design, research, and applications

    Google Scholar 

  72. Minoofam SA, Bastanfard A (2010) Square Kufic pattern formation by asynchronous cellular automata. In: International conference on cellular automata. Springer, Berlin, Heidelberg, pp 79–82

    Google Scholar 

  73. MirMashhouri A, Bastanfard A, Amirkhani D (2022) Collecting a database for emotional responses to simple and patterned two-color images. Multimed Tools Appl 81(13):18935–18953

    Google Scholar 

  74. Morneau RA, Van Herreweghe WG, Little JW, Lefebvre DB (2012) Energy company perspective on virtual worlds/3-D immersive environments. In: SPE intelligent energy international. OnePetro

  75. Movahedi Z, Bastanfard A (2021) Toward competitive multi-agents in polo game based on reinforcement learning. Multimed Tools Appl 80:26773–26793. https://doi.org/10.1007/s11042-021-10968-z

  76. Mozafari B et al (2014) Scaling up crowd-sourcing to very large datasets: a case for active learning. Proc VLDB Endow 8(2):125–136

    Google Scholar 

  77. Negahban S, Oh S, Shah D (2012) Iterative ranking from pair-wise comparisons. In: Advances in neural information processing systems

    Google Scholar 

  78. Nguyen QV, Nguyen TT, Miklós Z, Aberer K, Gal A, Weidlich M (2014) Pay-as-you-go reconciliation in schema matching networks. In: 2014 IEEE 30th international conference on data engineering. IEEE, pp 220–231

  79. Nicholson S (2012) A user-centered theoretical framework for meaningful gamification. In: Paper presented at the games+ learning+ society. 8.0, Madison, USA

    Google Scholar 

  80. Parameswaran A, Sarma AD, Garcia-Molina H, Polyzotis N, Widom J (2011) Human-assisted graph search: it's okay to ask questions. arXiv preprint arXiv:11033102

  81. Parameswaran AG, Garcia-Molina H, Park H, Polyzotis N, Ramesh A, Widom J (2012) Crowdscreen: algorithms for filtering data with humans. In: Proceedings of the 2012 ACM SIGMOD international conference on Management of Data, pp 361–372

  82. Park H, Widom J (2014) Crowdfill: collecting structured data from the crowd. In: Proceedings of the 2014 ACM SIGMOD international conference on management of data, pp 577–588

  83. Passos EB et al (2011) Turning real-world software development into a game. In: 2011 Brazilian symposium on games and digital entertainment. IEEE

    Google Scholar 

  84. Pfeiffer T, Gao X, Chen Y, Mao A, Rand D (2012) Adaptive polling for information aggregation. In: Proceedings of the AAAI conference on artificial intelligence vol 26, no 1, pp 122–128

  85. Pomerol J-C, Barba-Romero S (2012) Multicriterion decision in management: principles and practice. Vol. 25. Springer Science & Business Media

    Google Scholar 

  86. Rahm E, Bernstein PA (2001) A survey of approaches to automatic schema matching. VLDB J 10(4):334–350

    Google Scholar 

  87. Rudinac S, Larson M, Hanjalic A (2013) Learning crowdsourced user preferences for visual summarization of image collections. IEEE Trans Multimed 15(6):1231–1243

    Google Scholar 

  88. Ryan RM, Deci EL (2000) Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol 55(1):68

    CAS  PubMed  Google Scholar 

  89. Sakamoto M, Nakajima T, Alexandrova T (2012) Value-based design for gamifying daily activities. In: International conference on entertainment computing. Springer

    Google Scholar 

  90. Sarawagi S, Bhamidipaty A (2002) Interactive deduplication using active learning. In: Proceedings of the eighth ACM SIGKDD international conference on knowledge discovery and data mining, pp 269–278

  91. Sarma AD et al (2014) Crowd-powered find algorithms. In: 2014 IEEE 30th international conference on data engineering. IEEE

    Google Scholar 

  92. Sarma AD et al (2015) Jellybean: crowd-powered image counting algorithms. HCOMP

    Google Scholar 

  93. Seaborn K, Fels DI (2015) Gamification in theory and action: a survey. Int J Human-Comput Stud 74:14–31

    Google Scholar 

  94. Skinner BF (1965) Science and human behavior. Simon and Schuster

    Google Scholar 

  95. Takbiri Y, Bastanfard A, Amini A (2023) A gamified approach for improving the learning performance of K-6 students using Easter eggs. Multimed Tools Appl 82:20683–20701. https://doi.org/10.1007/s11042-023-14356-7

  96. Thom J, Millen D, DiMicco J (2012) Removing gamification from an enterprise SNS. In: Proceedings of the ACM 2012 conference on computer supported cooperative work, pp 1067–1070

  97. Trushkowsky B et al (2013) Crowdsourced enumeration queries. In: 2013 IEEE 29th international conference on data engineering (ICDE). IEEE

    Google Scholar 

  98. Tseng W-Y, Chen K-H, Huang J-W (2019) Crowdsourced object-labeling based on a game-based mobile application. Multimed Tools Appl 78(13):18137–18168

    Google Scholar 

  99. Venetis P, Garcia-Molina H, Huang K, Polyzotis N (2012) Max algorithms in crowdsourcing environments. In: Proceedings of the 21st international conference on world wide web, pp 989–998

  100. Verroios V, Garcia-Molina H (2015) Entity resolution with crowd errors. In: 2015 IEEE 31st international conference on data engineering. IEEE

    Google Scholar 

  101. Vesdapunt N, Bellare K, Dalvi N (2014) Crowdsourcing algorithms for entity resolution. Proc VLDB Endow 7(12):1071–1082

    Google Scholar 

  102. Wang J, Kraska T, Franklin MJ, Feng J (2012) Crowder: crowdsourcing entity resolution. arXiv preprint arXiv:12081927

  103. Wang J, Li G, Feng J (2012) Can we beat the prefix filtering? An adaptive framework for similarity join and search. In: Proceedings of the 2012 ACM SIGMOD international conference on management of data, pp 85–96

  104. Wang J, Li G, Kraska T, Franklin MJ, Feng J (2013) Leveraging transitive relations for crowdsourced joins. In: Proceedings of the 2013 ACM SIGMOD international conference on Management of Data, pp 229–240

  105. Wang S, Xiao X, Lee CH (2015) Crowd-based deduplication: an adaptive approach. In: Proceedings of the 2015 ACM SIGMOD international conference on Management of Data, pp 1263–1277

  106. Wauthier F, Jordan M, Jojic N (2013) Efficient ranking from pairwise comparisons. In: International conference on machine learning. PMLR, pp 109–117

  107. Whang SE, McAuley J, Garcia-Molina H (2012) Compare me maybe: crowd entity resolution interfaces. Stanford InfoLab

    Google Scholar 

  108. Whang SE, Lofgren P, Garcia-Molina H (2013) Question selection for crowd entity resolution. Proc VLDB Endow 6(6):349–360

    Google Scholar 

  109. Witt M, Scheiner CW, Robra-Bissantz S (2011) Gamification of online idea competitions: insights from an explorative case. In: GI-Jahrestagung, p 392

  110. Yan T, Kumar V, Ganesan D (2010) Crowdsearch: exploiting crowds for accurate real-time image search on mobile phones. In: Proceedings of the 8th international conference on Mobile systems, applications, and services, pp 77–90

  111. Zhang CJ, Chen L, Jagadish HV, Cao CC (2013) Reducing uncertainty of schema matching via crowdsourcing. Proc VLDB Endow 6(9):757–768

  112. Zhang X, Li G, Feng J (2016) Crowdsourced top-k algorithms: an experimental evaluation. Proc VLDB Endow 9(8):612–623

    MathSciNet  Google Scholar 

  113. Zichermann G, Cunningham C (2011) Gamification by design: implementing game mechanics in web and mobile apps. O'Reilly Media, Inc.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Karaj Branch, Islamic Azad University, Karaj, Iran

    Azam Bastanfard & Mohammad Shahabipour

  2. Department of Computer Science and Engineering, University of Quebec in Outaouais, Gatineau, Canada

    Dariush Amirkhani

Authors
  1. Azam Bastanfard
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Shahabipour
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dariush Amirkhani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Azam Bastanfard.

Ethics declarations

Conflict of interests

Authors declare that there is no Conflict of Interests or Competing of Interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bastanfard, A., Shahabipour, M. & Amirkhani, D. Crowdsourcing of labeling image objects: an online gamification application for data collection. Multimed Tools Appl 83, 20827–20860 (2024). https://doi.org/10.1007/s11042-023-16325-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16325-6

Keywords

Search

Navigation