Abstract
Sports highlight selection has traditionally required expert opinions and manual labor of video editors. To automate this laborious task, crowdsourcing viewers’ live comments has recently emerged as a promising tool, which can remove the burden of extracting semantic information by computer vision. However, popular crowdsourcing methods based on peak-finding are sensitive to noise and may produce deviant highlights from the expert choice. To increase the accuracy of automated selection of sports highlight, we introduce a statistically sound crowdsourcing method, SportLight. In this work, we take a statistical approach that combines multiple hypothesis testing and \(\ell _1\)-trend filtering (fused lasso), supported by a computationally inexpensive algorithm. By analyzing 29 baseball games played in the 2016 and 2017 seasons, we demonstrate that our approach properly reduces the risk of false alarm and generates the results closer to expert-chosen highlights than that of the peak-finding method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Adding the \(\ell _1\) penalties to the Poisson log-likelihood function may be plausible, but the theory of Son and Lim (2019) only supports normal models, and our empirical experience favors the transformation approach.
References
Assfalg, J., Bertini, M., Colombo, C., Del Bimbo, A., & Nunziati, W. (2003). Semantic annotation of soccer videos: Automatic highlights identification. Computer Vision and Image Understanding, 92(2–3), 285–305.
Assfalg, J., Bertini, M., Del Bimbo, A., Nunziati, W., & Pala, P. (2002). Soccer highlights detection and recognition using HMMs. In Proc. 2002 IEEE international conference on multimedia and expo (ICME’02) (Vol. 1, pp. 825–828). IEEE.
Babaguchi, N., Kawai, Y., Ogura, T., & Kitahashi, T. (2004). Personalized abstraction of broadcasted American football video by highlight selection. IEEE Transactions on Multimedia, 6(4), 575–586.
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 57, 289–300.
Bernstein, M.S., Little, G., Miller, R.C., Hartmann, B., Ackerman, M.S., Karger, D.R., Crowell, D., & Panovich, K. (2010). Soylent: A word processor with a crowd inside. In Proceedings of the 23nd annual ACM symposium on User interface software and technology (pp. 313–322). ACM.
Bettadapura, V., Pantofaru, C., & Essa, I. (2016). Leveraging contextual cues for generating basketball highlights. In: Proceedings of the 2016 ACM on Multimedia Conference, MM ’16 (pp. 908–917). ACM. https://doi.org/10.1145/2964284.2964286
Bland, J. M., & Altman, D. G. (1995). Multiple significance tests: The bonferroni method. BMJ, 310(6973), 170.
Chao, C.Y., Shih, H.C., & Huang, C.L. (2005). Semantics-based highlight extraction of soccer program using DBN. In Proc. 2005 IEEE international conference on acoustics, speech, and signal processing (ICASSP’05) (Vol. 2, p. ii-1057). IEEE.
Chu, W. T., & Chou, Y. C. (2017). On broadcasted game video analysis: Event detection, highlight detection, and highlight forecast. Multimedia Tools and Applications, 76(7), 9735–9758.
Cormen, T. H., Leiserson, C. E., Rivest, R. L., & Stein, C. (2009). Introduction to algorithms. Cambridge: MIT Press.
D’Orazio, T., & Leo, M. (2010). A review of vision-based systems for soccer video analysis. Pattern Recognition, 43(8), 2911–2926.
Ha, S., Kim, D., & Lee, J. (2013). Crowdsourcing as a method for digital media interaction. In HCI 2013 (pp. 153–154). The HCI Society of Korea.
Ha, S., Kim, D., & Lee, J. (2013). Crowdsourcing as a method for indexing digital media. In CHI’13 extended abstracts on human factors in computing systems (pp. 931–936). ACM.
Hannon, J., McCarthy, K., Lynch, J., & Smyth, B. (2011). Personalized and automatic social summarization of events in video. In Proceedings of the 16th international conference on intelligent user interfaces (pp. 335–338). ACM
Hoefling, H. (2010). A path algorithm for the fused lasso signal approximator. Journal of Computational and Graphical Statistics, 19(4), 984–1006.
Jacobson, V. (1988). Congestion avoidance and control. In ACM SIGCOMM computer communication review (Vol. 18, pp. 314–329). ACM.
Kapetanakis, A. (2018). IBM Watson: Inside the ’black box’. US Open News. Accessed 29 Apr 2019.
Liu, C., Huang, Q., Jiang, S., Xing, L., Ye, Q., & Gao, W. (2009). A framework for flexible summarization of racquet sports video using multiple modalities. Computer Vision and Image Understanding, 113(3), 415–424.
Marcus, A., Bernstein, M.S., Badar, O., Karger, D.R., Madden, S., & Miller, R.C. (2011). Twitinfo: Aggregating and visualizing microblogs for event exploration. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 227–236). ACM.
Metulini, R. (2017). Filtering procedures for sensor data in basketball. Statistica & Applicazioni, 15(2), 133–150.
Moyer, S. (2013). In America’s pastime, baseball players pass a lot of time. The Wall Street Journal. https://www.wsj.com/articles/SB10001424127887323740804578597932341903720.
Paxson, V., Allman, M., Chu, J., & Sargent, M. (2011). Computing TCP’s retransmission timer. RFC 6298.
Qian, X., Wang, H., Liu, G., & Hou, X. (2012). HMM based soccer video event detection using enhanced mid-level semantic. Multimedia Tools and Applications, 60(1), 233–255.
Quinn, A.J., & Bederson, B.B. (2011) Human computation: A survey and taxonomy of a growing field. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1403–1412). ACM.
Rui, Y., Gupta, A., & Acero, A. (2000). Automatically extracting highlights for TV baseball programs. In Proceedings of the eighth ACM international conference on Multimedia (pp. 105–115). ACM.
Shih, H. C. (2018). A survey of content-aware video analysis for sports. IEEE Transactions on Circuits and Systems for Video Technology, 28(5), 1212–1231.
Son, W., & Lim, J. (2019). Modified path algorithm of fused lasso signal approximator for consistent recovery of change points. Journal of Statistical Planning and Inference, 200, 223–238.
Tang, A., & Boring, S. (2012). #EpicPlay: Crowd-sourcing sports video highlights. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1569–1572). ACM.
Thompson, C. (2010). What is I.B.M.’s Watson? The New York Times Magazine. https://www.nytimes.com/2010/06/20/magazine/20Computer-t.html. Accessed 29 Apr 2019.
Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., & Knight, K. (2005). Sparsity and smoothness via the fused lasso. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(1), 91–108.
Von Ahn, L., & Dabbish, L. (2004). Labeling images with a computer game. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 319–326). ACM.
Von Ahn, L., Maurer, B., McMillen, C., Abraham, D., & Blum, M. (2008). recaptcha: Human-based character recognition via web security measures. Science, 321(5895), 1465–1468.
Xiong, Z., Radhakrishnan, R., & Divakaran, A. (2004). Method and system for extracting sports highlights from audio signals. US Patent App. 10/374,017
Xiong, Z., Radhakrishnan, R., Divakaran, A., & Huang, T.S. (2003). Audio events detection based highlights extraction from baseball, golf and soccer games in a unified framework. In Proc. 2003 IEEE International Conference on Multimedia and Expo (ICME’03) (Vol. 3, p. III-401). IEEE.
Acknowledgements
Won Son was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government (MSIT) (No. 2020R1F1A1A01051039). Joong-Ho Won was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2019R1A2C1007126).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Jung, J., Ha, S., Son, W. et al. SportLight: statistically principled crowdsourcing method for sports highlight selection. J. Korean Stat. Soc. 51, 127–148 (2022). https://doi.org/10.1007/s42952-021-00128-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42952-021-00128-2