[PDF][PDF] Question Answering Challenge (QAC-1): An Evaluation of Question Answering Task at NTCIRWorkshop 3.

J Fukumoto, T Kato, F Masui�- New Directions in Question Answering, 2003 - cdn.aaai.org
J Fukumoto, T Kato, F Masui
New Directions in Question Answering, 2003cdn.aaai.org
In this paper we describe the Question Answering Challenge (QAC), a question answering
task, and its first evaluation (QAC1). The project was carried out as a task of the NTCIR
Workshop 3 in October 2002. One objective of the QAC was to develop practical QA systems
in a general domain by focusing on research relating to user interaction and information
extraction. Our second objective was to develop an evaluation method for the question
answering system and information resources for evaluation. We defined three kinds of tasks�…
Abstract
In this paper we describe the Question Answering Challenge (QAC), a question answering task, and its first evaluation (QAC1). The project was carried out as a task of the NTCIR Workshop 3 in October 2002. One objective of the QAC was to develop practical QA systems in a general domain by focusing on research relating to user interaction and information extraction. Our second objective was to develop an evaluation method for the question answering system and information resources for evaluation. We defined three kinds of tasks in the QAC: Task 1, where questions required five possible answers;, Task 2, where questions had a single answer; and Task 3, where there was one answer to a question related to a question in Task 2. We prepared 200 questions for Task 1 and Task 2 and 40 questions for Task 3 at the Formal Run and about 900 questions for the additional run. We conducted a Dry Run and a Formal Run evaluation. There were 16 participants (two of them from among the task organizers) at the QAC1.
cdn.aaai.org
Showing the best result for this search. See all results