Embedding web-based statistical translation models in cross-language information retrieval. (English) Zbl 1234.68416
Summary: Although more and more language pairs are covered by machine translation (MT) services, there are still many pairs that lack translation resources. Cross-language information retrieval (CLIR) is an application that needs translation functionality of a relatively low level of sophistication, since current models for information retrieval (IR) are still based on a bag of words. The Web provides a vast resource for the automatic construction of parallel corpora that can be used to train statistical translation models automatically. The resulting translation models can be embedded in several ways in a retrieval model. In this article, we will investigate the problem of automatically mining parallel texts from the Web and different ways of integrating the translation models within the retrieval process. Our experiments on standard test collections for CLIR show that the Web-based translation models can surpass commercial MT systems in CLIR tasks. These results open the perspective of constructing a fully automatic query translation device for CLIR at a very low cost.
MSC:
68T50 | Natural language processing |
68M11 | Internet topics |
68T05 | Learning and adaptive systems in artificial intelligence |
References:
[1] | Baum L. E., Inequalities 3 pp 1– (1972) |
[2] | Broglio John, National Institute of Standards and Technology Special Publication 500-236, Gaithersburg, MD, pages pp 29– (1995) |
[3] | Brown Peter F, Computational Linguistics 16 (2) pp 79– (1990) |
[4] | Harman Donna K, National Institute of Standards and Technology Special Publication pp 500– (1995) |
[5] | DOI: 10.1002/(SICI)1097-4571(199601)47:1<70::AID-ASI7>3.0.CO;2-# · doi:10.1002/(SICI)1097-4571(199601)47:1<70::AID-ASI7>3.0.CO;2-# |
[6] | Jansen Bernard J, Journal of the American Society for Information Science and Technology 53 (3) pp 226– (2001) |
[7] | DOI: 10.1075/target.4.1.03laf · doi:10.1075/target.4.1.03laf |
[8] | Nie Jian-Yun, Journal of the American Society for Information Science and Technology 54 (4) pp 340– (2002) |
[9] | DOI: 10.1108/eb046814 · doi:10.1108/eb046814 |
[10] | Tague-Sutcliffe Jean, National Institute of Standards and Technology Special Publication 500-236, Gaithersburg, MD, pages pp 385– (1995) |
[11] | DOI: 10.1016/S0004-3702(98)00063-0 · Zbl 0910.68068 · doi:10.1016/S0004-3702(98)00063-0 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.