Abstract
In the current world, individuals are faced with decision making problems and opinion formation processes on a daily basis. Nevertheless, answering a comparative question by retrieving documents based only on traditional measures (such as TF-IDF and BM25) does not always satisfy the need. In this paper, we propose a multi-layer architecture to answer comparative questions based on arguments. Our approach consists of a pipeline of query expansion, argument mining model, and sorting of the documents by a combination of different ranking criteria. Given the crucial role of the argument mining step, we examined two models: DistilBERT and an ensemble learning approach using stacking of SVM and DistilBERT. We compare the results of both models using two argumentation corpora on the level of argument identification task, and further using the dataset of CLEF 2021 Touché Lab shared task 2 on the level of answering comparative questions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
We used Transformers from huggingface.com for our experiments.
- 7.
References
Van Eemeren, F., Grootendorst, R.: The Pragma-Dialectical Approach. Cambridge University Press, A systematic theory of argumentation (2004)
Chernodub, A., et al.: Targer: neural argument mining at your fingertips. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 195–200 (2019)
Sanh, V., Debut, L., Chaumond, J., Wolf, T.: DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019)
Alhamzeh, A., Bouhaouel, M., Egyed-Zsigmond, E., Mitrović, J., Brunie, L., Kosch, H.: A stacking approach for cross-domain argument identification. In: Strauss, C., Kotsis, G., Tjoa, A.M., Khalil, I. (eds.) DEXA 2021. LNCS, vol. 12923, pp. 361–373. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86472-9_33
Wambsganss, T., Molyndris, N., Söllner, M.: Unlocking transfer learning in argumentation mining: a domain-independent modelling approach. In: 15th International Conference on Wirtschaftsinformatik (2020)
Reimers, N., Schiller, B., Beck, T., Daxenberger, J., Stab, C., Gurevych, I. Classification and clustering of arguments with contextualized word embeddings. arXiv preprint arXiv:1906.09821 (2019)
Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2009)
Devlin, J., Chang, M. W., Lee, K., Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Panchenko, A., Bondarenko, A., Franzek, M., Hagen, M., Biemann, C.: Categorizing comparative sentences. arXiv preprint arXiv:1809.06152 (2018)
Schilwächter, M., Bondarenko, A., Zenker, J., Hagen, M., Biemann, C., Panchenko, A.: Answering comparative questions: better than ten-blue-links? In: Proceedings of the 2019 Conference on Human Information Interaction and Retrieval, pp. 361–365 (2019)
Bondarenko, A., Panchenko, A., Beloucif, M., Biemann, C., Hagen, M.: Answering comparative questions with arguments. Datenbank-Spektrum 20(2), 155–160 (2020)
Daxenberger, J., Schiller, B., Stahlhut, C., Kaiser, E., Gurevych, I.: Argumentext: argument classification and clustering in a generalized search scenario. Datenbank-Spektrum 20(2), 115–121 (2020)
Bondarenko, A., et al.: Overview of touché 2021: argument retrieval. In: Hiemstra, D., Moens, M.-F., Mothe, J., Perego, R., Potthast, M., Sebastiani, F. (eds.) ECIR 2021. LNCS, vol. 12657, pp. 574–582. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72240-1_67
Abye, T., Sager, T., Triebel, A.J.: An open-domain web search engine for answering comparative questions. In: Cappellato, L., Eickhoff, C., Ferro, N., Névéol, A. (eds.), Working Notes of CLEF 2020-Conference and Labs of the Evaluation Forum, Thessaloniki, Greece, 22–25 September 2020, volume 2696 of CEUR Workshop Proceedings. CEUR-WS.org (2020)
Chekalina, V., Panchenko, A.: Retrieving comparative arguments using ensemble methods and neural information retrieval. Working Notes of CLEF (2021)
Potthast, M., Gollub, T., Wiegmann, M., Stein, B.: TIRA integrated research architecture. In: Information Retrieval Evaluation in a Changing World. TIRS, vol. 41, pp. 123–160. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22948-1_5
Azad, H.K., Deepak, A.: Query expansion techniques for information retrieval: a survey. Inf. Process. Manag. 56(5), 1698–1735 (2019)
Alhamzeh, A., Bouhaouel, M., Egyed-Zsigmond, E., Mitrović, J.: DistilBERT-based argumentation retrieval for answering comparative questions. Working Notes of CLEF (2021)
Pasi, G., Piwowarski, B., Azzopardi, L., Hanbury, A. (eds.): ECIR 2018. LNCS, vol. 10772. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-76941-7
Potthast, M., et al.: A search engine for the ClueWeb09 corpus. In: Hersh, B., Callan, J., Maarek, Y., Sanderson, M. (eds.), 35th International ACM Conference on Research and Development in Information Retrieval (SIGIR 2012), p. 1004. ACM, August 2012
Stab, C., Gurevych, I.: Annotating argument components and relations in persuasive essays. In: Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pp. 1501–1510 (2014)
Habernal, I., Gurevych, I.: Argumentation mining in user-generated web discourse. Comput. Linguist. 43(1), 125–179 (2017)
Toulmin, S.E.: The Uses of Argument, 2nd edn. Cambridge University Press, Cambridge (2003)
Niven, T., Kao, H.Y.: Probing neural network comprehension of natural language arguments. arXiv preprint arXiv:1907.07355 (2019)
Liu, Y., et al.: Roberta: a robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using siamese BERT-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, p. 11 (2019)
Bondarenko, A., et al.: Overview of touché 2021: argument retrieval. In: Candan, K.S., et al. (eds.) CLEF 2021. LNCS, vol. 12880, pp. 450–467. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85251-1_28
Acknowledgements
The project on which this report is based was partly funded by the German Federal Ministry of Education and Research (BMBF) under the funding code 01|S20049. The author is responsible for the content of this publication.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Alhamzeh, A., Bouhaouel, M., Egyed-Zsigmond, E., Mitrović, J., Brunie, L., Kosch, H. (2022). Query Expansion, Argument Mining and Document Scoring for an Efficient Question Answering System. In: Barrón-Cedeño, A., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2022. Lecture Notes in Computer Science, vol 13390. Springer, Cham. https://doi.org/10.1007/978-3-031-13643-6_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-13643-6_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-13642-9
Online ISBN: 978-3-031-13643-6
eBook Packages: Computer ScienceComputer Science (R0)