Skip to main content
Springer Nature Link
Log in

Neural Machine Translation with Diversity-Enabled Translation Memory

  • Conference paper
  • First Online:
Intelligent Information and Database Systems (ACIIDS 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13995))

Included in the following conference series:

  • 428 Accesses

Abstract

Neural machine translation (NMT) using translation memory (TM) has been introduced as an emergent technique for improving machine translation systems (MTS). In this study, we propose an end-to-end NMT model with TM by exploiting the diversity of the retrieval-augmented phase using maximal marginal relevance (MMR). In particular, the proposed model is designed with monolingual TM, which is able to support low-resource scenarios. Furthermore, the memory retriever and translation models are jointly trained to improve translation performance. For the experiment, we use IWSLT15 (En \(\longleftrightarrow \) Vi) as a benchmark dataset to evaluate the performance of the proposed method. Accordingly, the experiential results show the effectiveness of the proposed method compared with strong baselines in this research field.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
eBook
USD 59.99
Price excludes VAT (USA)
Softcover Book
USD 79.99
Price excludes VAT (USA)

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/dorianbrown/rank_bm25.

References

  1. Agrawal, R., Gollapudi, S., Halverson, A., Ieong, S.: Diversifying search results. In: Proceedings of the Second ACM International Conference on Web Search and Data Mining, WSDM 2009, pp. 5–14. Association for Computing Machinery, New York (2009). https://doi.org/10.1145/1498759.1498766

  2. Artetxe, M., Labaka, G., Agirre, E., Cho, K.: Unsupervised neural machine translation (2018)

    Google Scholar 

  3. Borodin, A., Lee, H.C., Ye, Y.: Max-sum diversification, monotone submodular functions and dynamic updates. In: Proceedings of the 31st ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems. PODS 2012, pp. 155–166. Association for Computing Machinery, New York (2012). https://doi.org/10.1145/2213556.2213580

  4. Cai, D., Wang, Y., Li, H., Lam, W., Liu, L.: Neural machine translation with monolingual translation memory. In: Zong, C., Xia, F., Li, W., Navigli, R. (eds.) Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, 1–6 August 2021, pp. 7307–7318. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.acl-long.567

  5. Cao, Q., Kuang, S., Xiong, D.: Learning to reuse translations: guiding neural machine translation with examples. In: Giacomo, G.D., et al. (eds.) 24th European Conference on Artificial Intelligence, ECAI 2020, Santiago de Compostela, Spain, 29 August–8 September 2020, Including 10th Conference on Prestigious Applications of Artificial Intelligence, PAIS 2020. Frontiers in Artificial Intelligence and Applications, vol. 325, pp. 1982–1989. IOS Press (2020). https://doi.org/10.3233/FAIA200318

  6. Cao, Q., Xiong, D.: Encoding gated translation memory into neural machine translation. In: Riloff, E., Chiang, D., Hockenmaier, J., Tsujii, J. (eds.) Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 31 October–4 November 2018, pp. 3042–3047. Association for Computational Linguistics (2018). https://doi.org/10.18653/v1/d18-1340

  7. Carbonell, J., Goldstein, J.: The use of MMR, diversity-based reranking for reordering documents and producing summaries. In: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 1998, pp. 335–336. Association for Computing Machinery, New York (1998). https://doi.org/10.1145/290941.291025

  8. Cettolo, M., Niehues, J., Stüker, S., Bentivogli, L., Cattoni, R., Federico, M.: The IWSLT 2015 evaluation campaign. In: Proceedings of the 12th International Workshop on Spoken Language Translation: Evaluation Campaign, Da Nang, Vietnam, 3–4 December 2015, pp. 2–14 (2015). https://aclanthology.org/2015.iwslt-evaluation.1

  9. Cheng, X., Gao, S., Liu, L., Zhao, D., Yan, R.: Neural machine translation with contrastive translation memories. In: Goldberg, Y., Kozareva, Z., Zhang, Y. (eds.) Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, 7–11 December 2022, pp. 3591–3601. Association for Computational Linguistics (2022). https://aclanthology.org/2022.emnlp-main.235

  10. Cheng, Y., Yang, Q., Liu, Y., Sun, M., Xu, W.: Joint training for pivot-based neural machine translation. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, pp. 3974–3980 (2017). https://doi.org/10.24963/ijcai.2017/555

  11. Fan, A., et al.: Beyond English-centric multilingual machine translation. J. Mach. Learn. Res. 22, 1–48 (2020)

    MathSciNet  Google Scholar 

  12. Feng, Y., Zhang, S., Zhang, A., Wang, D., Abel, A.: Memory-augmented neural machine translation. In: Palmer, M., Hwa, R., Riedel, S. (eds.) Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, 9–11 September 2017, pp. 1390–1399. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/d17-1146

  13. Feng, Y., Zhang, S., Zhang, A., Wang, D., Abel, A.: Memory-augmented neural machine translation. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark, September 2017, pp. 1390–1399. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/D17-1146

  14. Gu, J., Wang, Y., Cho, K., Li, V.: Search engine guided neural machine translation. In: McIlraith, S.A., Weinberger, K.Q. (eds.) Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, AAAI 2018, The 30th Innovative Applications of Artificial Intelligence, IAAI 2018, and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2018, New Orleans, Louisiana, USA, 2–7 February 2018, pp. 5133–5140. AAAI Press (2018). https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/17282

  15. He, J., Gu, J., Shen, J., Ranzato, M.: Revisiting self-training for neural sequence generation (2020)

    Google Scholar 

  16. Johnson, J., Douze, M., Jégou, H.: Billion-scale similarity search with GPUs. IEEE Trans. Big Data 7(3), 535–547 (2021). https://doi.org/10.1109/TBDATA.2019.2921572

    Article  Google Scholar 

  17. Khandelwal, U., Fan, A., Jurafsky, D., Zettlemoyer, L., Lewis, M.: Nearest neighbor machine translation. CoRR abs/2010.00710 (2020). https://arxiv.org/abs/2010.00710

  18. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015). http://arxiv.org/abs/1412.6980

  19. Luong, M.T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation (2015)

    Google Scholar 

  20. Neubig, G., Hu, J.: Rapid adaptation of neural machine translation to new languages. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium October–November 2018, pp. 875–880. Association for Computational Linguistics (2018). https://doi.org/10.18653/v1/D18-1103. https://aclanthology.org/D18-1103

  21. Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 311–318. Association for Computational Linguistics (2002)

    Google Scholar 

  22. Pham, M.Q., Xu, J., Crego, J., Yvon, F., Senellart, J.: Priming neural machine translation. In: Proceedings of the Fifth Conference on Machine Translation, November 2020, Online, pp. 516–527. Association for Computational Linguistics (2020). https://aclanthology.org/2020.wmt-1.63

  23. Sennrich, R., Haddow, B., Birch, A.: Improving neural machine translation models with monolingual data (2016)

    Google Scholar 

  24. Sennrich, R., Haddow, B., Birch, A.: Neural machine translation of rare words with subword units. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Berlin, Germany, August 2016, pp. 1715–1725. Association for Computational Linguistics (2016). https://doi.org/10.18653/v1/P16-1162. https://aclanthology.org/P16-1162

  25. Vaswani, A., et al.: Attention is all you need. CoRR abs/1706.03762 (2017). http://arxiv.org/abs/1706.03762

  26. Xia, M., Huang, G., Liu, L., Shi, S.: Graph based translation memory for neural machine translation. In: The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, 27 January–1 February 2019, pp. 7297–7304. AAAI Press (2019). https://doi.org/10.1609/aaai.v33i01.33017297

  27. Xu, J., Crego, J.M., Senellart, J.: Boosting neural machine translation with similar translations. In: Jurafsky, D., Chai, J., Schluter, N., Tetreault, J.R. (eds.) Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, 5–10 July 2020, pp. 1580–1590. Association for Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.acl-main.144

  28. Zhang, J., Utiyama, M., Sumita, E., Neubig, G., Nakamura, S.: Guiding neural machine translation with retrieved translation pieces (2018). https://doi.org/10.48550/ARXIV.1804.02559. https://arxiv.org/abs/1804.02559

  29. Zoph, B., Yuret, D., May, J., Knight, K.: Transfer learning for low-resource neural machine translation (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Viettel Cyberspace Center, Viettel Group, Hanoi, Vietnam

    Quang Chieu Nguyen, Xuan Dung Doan & Khac-Hoai Nam Bui

  2. Vietnam National University of Hanoi, Hanoi, Vietnam

    Quang Chieu Nguyen & Van-Vinh Nguyen

Authors
  1. Quang Chieu Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuan Dung Doan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Van-Vinh Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Khac-Hoai Nam Bui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Khac-Hoai Nam Bui .

Editor information

Editors and Affiliations

  1. Wrocław University of Science and Technology, Wrocław, Poland

    Ngoc Thanh Nguyen

  2. King Mongkut’s Institute of Technology Ladkrabang, Bangkok, Thailand

    Siridech Boonsang

  3. Iwate Prefectural University Iwate, Iwate, Japan

    Hamido Fujita

  4. Wroclaw University of Science and Technology, Wrocław, Poland

    Bogumiła Hnatkowska

  5. National University of Kaohsiung, Kaohsiung, Taiwan

    Tzung-Pei Hong

  6. King Mongkut’s Institute of Technology Ladkrabang, Bangkok, Thailand

    Kitsuchart Pasupa

  7. Malaysia Japan International Institute of Technology, Kuala Lumpur, Malaysia

    Ali Selamat

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Nguyen, Q.C., Doan, X.D., Nguyen, VV., Bui, KH.N. (2023). Neural Machine Translation with Diversity-Enabled Translation Memory. In: Nguyen, N.T., et al. Intelligent Information and Database Systems. ACIIDS 2023. Lecture Notes in Computer Science(), vol 13995. Springer, Singapore. https://doi.org/10.1007/978-981-99-5834-4_26

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-5834-4_26

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-5833-7

  • Online ISBN: 978-981-99-5834-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation