Abstract
We develop a question-answering system for questions that ask about a conversational agent’s personality based on large-scale question-answer pairs created by hand. In casual dialogues, the speaker sometimes asks his conversation partner questions about favorites or experiences. Since this behavior also appears in conversational dialogues with a dialogue system, systems must be developed to respond to such questions. However, the effectiveness of personality-question-answering for conversational agents has not been investigated. Our user-machine chat experiments show that our question-answering system, which estimates appropriate answers with 60.7 % accuracy for the personality questions in our conversation corpus, significantly improves user’s subjective evaluations.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Shibata, M., Nishiguchi, T., Tomiura, Y.: Dialog system for open-ended conversation using web documents. Informatica 33, 277–284 (2009), http://www.informatica.si/PDF/Informatica_2009_3.pdf#page=37
Ritter, A., Cherry, C., Dolan, W.: Data-driven response generation in social media. In: Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pp. 583–593 (2011). http://ai.cs.washington.edu/www/media/papers/paper2.pdf
Wong, W., Cavedon, L., Thangarajah, J., Padgham, L.: Strategies for mixed-initiative conversation management using question-answer pairs. In: Proceedings of the 24th International Conference on Computational Linguistics. pp. 2821–2834 (2012). https://www.aclweb.org/anthology-new/C/C12/C12-1172.pdf
Meguro, T., Higashinaka, R., Minami, Y., Dohsaka, K.: Controlling listening-oriented dialogue using partially observable markov decision processes. In: Proceedings of the 23rd International Conference on Computational Linguistics, pp. 761–769 (2010)
Higashinaka, R., Imamura, K., Meguro, T., Miyazaki, C., Kobayashi, N., Sugiyama, H., Hirano, T., Makino, T., Matsuo, Y.: Towards an open-domain conversational system fully based on natural language processing. In: Proceedings of the 25th International Conference on Computational Linguistics, pp. 928–939 (2014)
Bickmore, T., Cassell, J.: Relational agents: a model and implementation of building user trust. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 396–403 (2001). http://dl.acm.org/citation.cfm?id=365304
Tidwell, L.C., Walther, J.B.: Computer-mediated communication effects on disclosure, impressions, and interpersonal evaluations. Hum. Commun. Res. 28(3), 317–348 (2002)
Nisimura, R., Nishihara, Y., Tsurumi, R., Lee, A., Saruwatari, H., Shikano, K.: Takemaru-kun.: Speech-oriented information system for real world research platform. In: Proceedings of the First International Workshop on Language Understanding and Agents for Real World Interaction, pp. 70–78 (2003). http://library.naist.jp:12180/dspace/handle/10061/8226
Weizenbaum, J.: ELIZA—a computer program for the study of natural language communication between man and machine. Commun. ACM 9(1), 36–45 (Jan 1966). http://portal.acm.org/citation.cfm?doid=365153.365168
Caspi, A., Roberts, B.W., Shiner, R.L.: Personality development: stability and change. Ann. Rev. Psychol. 56, 453–484 (Jan 2005). http://www.ncbi.nlm.nih.gov/pubmed/15709943
John, O.P., Srivastava, S.: The Big Five trait taxonomy: history, measurement, and theoretical perspectives. No. 510 (1999). http://scholar.google.com/scholar?hl=en&btnG=Search&q=intitle:The+Big-Five+Trait+Taxonomy:+History,+Measurement,+and+Theoretical+Perspectives#0
Mairesse, F., Walker, M.: PERSONAGE: Personality generation for dialogue. In: Proceedings of the Annual Meeting of the Association For Computational Linguistics, pp. 496–503 (2007). http://acl.ldc.upenn.edu/P/P07/P07-1063.pdf
Batacharia, B., Levy, D., Catizone, R., Krotov, A., Wilks, Y.: CONVERSE: a conversational companion. Machine conversations, pp. 205–215 (1999). http://link.springer.com/chapter/10.1007/978-1-4757-5687-6_17
Traum, D., Georgila, K., Artstein, R., Leuski, A.: Evaluating spoken dialogue processing for time-offset interaction. In: Proceedings of the 16th Annual SIGdial Meeting on Discourse and Dialogue, pp. 199–208 (2015)
Leuski, A., Traum, D.: Creating virtual human dialogue using information retrieval techniques. AI Mag. 32(2), 42–56 (2011). http://www.aaai.org/ojs/index.php/aimagazine/article/view/2347
Sugiyama, H., Meguro, T., Higashinaka, R.: Large-scale collection and analysis of personal question-answer pairs for conversational agents. In: Proceedings of Intelligent Virtual Agents, pp. 420–433 (2014)
Sugiyama, H., Meguro, T., Higashinaka, R., Minami, Y.: Open-domain utterance generation using phrase pairs based on dependency relations. In: Proceedings of Spoken Language Technology Workshop, pp. 60–65 (2014)
Turian, J., Ratinov, L., Bengio, Y.: Word representations: A simple and general method for semi-supervised learning. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linquistics. pp. 384–394. July (2010)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Proceedings of the 27th Annual Conference on Neural Information Processing Systems, pp. 1–9 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer Science+Business Media Singapore
About this chapter
Cite this chapter
Sugiyama, H., Meguro, T., Higashinaka, R. (2017). Evaluation of Question-Answering System About Conversational Agent’s Personality. In: Jokinen, K., Wilcock, G. (eds) Dialogues with Social Robots. Lecture Notes in Electrical Engineering, vol 427. Springer, Singapore. https://doi.org/10.1007/978-981-10-2585-3_14
Download citation
DOI: https://doi.org/10.1007/978-981-10-2585-3_14
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-2584-6
Online ISBN: 978-981-10-2585-3
eBook Packages: EngineeringEngineering (R0)