BioNMT: A Biomedical Neural Machine Translation System
Keywords:
neural machine translation, Transformer, self-attention, semantic disambiguationAbstract
To solve the problem of translation of professional vocabulary in the biomedical field and help biological researchers to translate and understand foreign language documents, we proposed a semantic disambiguation model and external dictionaries to build a novel translation model for biomedical texts based on the transformer model. The proposed biomedical neural machine translation system (BioNMT) adopts the sequence-to-sequence translation framework, which is based on deep neural networks. To construct the specialized vocabulary of biology and medicine, a hybrid corpus was obtained using a crawler system extracting from universal corpus and biomedical corpus. The experimental results showed that BioNMT which composed by professional biological dictionary and Transformer model increased the bilingual evaluation understudy (BLEU) value by 14.14%, and the perplexity was reduced by 40%. And compared with Google Translation System and Baidu Translation System, BioNMT achieved better translations about paragraphs and resolve the ambiguity of biomedical name entities to greatly improved.
References
[2] Bazzi, I.; Glass, J.R. (2000). Modeling Out-of-vocabulary Words for Robust Speech Recognition. Proc. of ISCA ASR2000, 401-404, 2000.
[3] Blei, D.M.; Ng, A.Y.; Jordan, M.I. (2003). Latent Dirichlet Allocation. Journal of Machine Learning Research, 3, 993-1022, 2003.
[4] Bo, Q.; Xiong, N.; Zou J. et al. (2007). Internationally agreed medical terminology: Medical Dictionary for Regulatory Activities. Chinese Journal of Clinical Pharmacology and Therapeutics, 2007.
[5] Brazill, S. (2016). Chinese to English Translation: Identifying Problems and Providing Solutions. Graduate Theses & Non-Theses. 71, 2016.
[6] Bu, F.; Gharajeh, M.S.(2019). Intelligent and Vision-based Fire Detection Systems: a Survey. Image and Vision Computing, 91, 2019. https://doi.org/10.1016/j.imavis.2019.08.007
[7] Chen, H.B.; Hsen, H.H.; Chang, H.A. (2017). A Simplification Translation Restoration Framework for Domain Adaptation in Statistical Machine Translation: A Case Study in Medical Record Translation. Computer Speech & Language, 42:59-80, 2017. https://doi.org/10.1016/j.csl.2016.08.003
[8] Cheng, J.; Dong, L.; Lapata, M. (2016). Long Short-Term Memory-Networks for Machine Reading. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 551-561, 2016. https://doi.org/10.18653/v1/D16-1053
[9] Cho, K.; van Merrienboer, B.; Gulcehre, C. et al. (2014). Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1724-1734, 2014. https://doi.org/10.3115/v1/D14-1179
[10] Devlin, J.; Chang, M.W.; Lee, K. et al. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of NAACL-HLT 2019, 1, 4171-4186, 2019.
[11] Garcia-Castillo, D.;Fetters, M.D. (2007). Quality in Medical Translations: A Review. Journal of Health Care for the Poor and Underserved, 18(1): 2007. https://doi.org/10.1353/hpu.2007.0009
[12] Guan, R.;Zhang, H.; Liang, Y. et al. (2020). Deep feature-based text clustering and its explanation. IEEE Transaction on Knowledge and Data Engineering, 2020. https://doi.org/10.1109/TKDE.2020.3028943
[13] Gulcehre, C.; Ahn, S.; Nallapati, R. et al. (2016). Pointing the Unknown Words. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2016. https://doi.org/10.18653/v1/P16-1014
[14] Khanmohammadi, S.; Gharajeh, M.S. (2017). A Routing Protocol for Data Transferring in Wireless Sensor Networks Using Predictive Fuzzy Inference System and Neural Node, Ad Hoc & Sensor Wireless Networks, 38(1-4), 103-124, 2017.
[15] Kuzmina, O. D., Fominykh, A.D; Abrosimova, N.A. (2015). Problems of the English Abbreviations in Medical Translation. Procedia-Social and Behavioral Sciences, 199, 548-554, 2015. https://doi.org/10.1016/j.sbspro.2015.07.545
[16] Luong, M.T.; Pham, H.; Manning, C.D. (2015). Effective Approaches to Attention-based Neural Machine Translation. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 1412-1421, 2015. https://doi.org/10.18653/v1/D15-1166
[17] Manning, C.D.; Raghavan, P.; Schutze, H. (2008). Introduction to information retrieval, vol. 1, Cambridge University Press, Cambridge, 2008. https://doi.org/10.1017/CBO9780511809071
[18] Meng, X.; Liu, X.Z. Li, Y.Y. et al. (2020). Correlation between Genotype and Phenotype in 69 Chinese Patients with USH2A Mutations: A comparative study of the patients with Usher Syndrome and Nonsyndromic Retinitis Pigmentosa. Acta Ophthalmologica, 2020. https://doi.org/10.1111/aos.14626
[19] Mercy, O.E. (2006). English-Edo Medical Translation. Perspectives: Studies in Translatology, 13(4), 268-277, 2006. https://doi.org/10.1080/09076760608668997
[20] Mi, H.; Wang, Z.; Ittycheriah, A. (2016). Supervised Attentions for Neural Machine Translation. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2283- 2288, 2016. https://doi.org/10.18653/v1/D16-1249
[21] Mikolov, T.; Karafiát, M.; Burget, L. et al. (2010). Recurrent Neural Network based Language Model. INTERSPEECH, 1045-1048, 2010.
[22] Mitka, M. (2001). Tearing down the Tower of Babel: Medical Translation in today's world. Journal of the American Medical Association, 285(6), 722-3, 2001. https://doi.org/10.1001/jama.285.6.722-JMN0214-2-1
[23] Neubig, G. (2017). Neural Machine Translation and Sequence-to-sequence Models: A Tutorial. arXiv:1703.01619 [cs.CL], 2017.
[24] Papineni, K.; Roukos, S.; Ward, T. et al. (2002). BLEU: a Method for Automatic Evaluation of Machine Translation. Computational Linguistics, 40, 311-318, 2002. https://doi.org/10.3115/1073083.1073135
[25] Pascanu, R.; Mikolov, T.; Bengio, Y. (2013). On the Difficulty of Training Recurrent Neural Networks. Proceedings of the 30 th International Conference on Machine Learning, Atlanta, Georgia, USA, 2013, (3), 1310-1318, 2013.
[26] Peters, P., Qian, Y. Ding, J. (2018). Translating medical terminology and bilingual terminography. Lexicography ASIALEX, 3, 99-113, 2018. https://doi.org/10.1007/s40607-018-0037-y
[27] Rocktí¤schel, T.; Grefenstette, E.; Hermann, K.M. et al. (2015). Reasoning about Entailment with Neural Attention. Proceedings of ICLR2015, 2015.
[28] Sennrich, R.; Haddow, B.; Birch, A. (2016). Neural Machine Translation of Rate Words with Subword Units. The 54th Annual Meeting of the Association for Computational Linguistics, 1715- 1725, 2016. https://doi.org/10.18653/v1/P16-1162
[29] Shen, L.; Chen, H.; Yu, Z. et al. (2016). Evolving support vector machines using fruit fly optimization for medical data classification. Knowledge-Based Systems,96, 61-75. 2016. https://doi.org/10.1016/j.knosys.2016.01.002
[30] Slimani, T.(2013). Description and Evaluation of Semantic Similarity Measures Approaches International Journal of Computer Applications, 80(10), 25-33, 2013. https://doi.org/10.5120/13897-1851
[31] Vaswani, A.; Shazeer, N.; Parmar, N. et al. (2017). Attention is All You Need. 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA, 5998-6008, 2017.
[32] Wu, Y.; Schuster, M.; Chen, Z. et al. (2016). Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. arXiv:1609.08144, 2016.
[33] Yandell, M.D.; Majoros, W.H. (2002). Genomics and Natural Language Processing. Nature Reviews Genetics, 601-610, 2002. https://doi.org/10.1038/nrg861
[34] Ziemski, M.; Junczys-Dowmunt, M.; Pouliquen, B. (2016). The United Nations Parallel Corpus. Language Resources and Evaluation (LREC'16), 2016.
[35] [Online]. Science Fund Shared Service Network. Available: https://output.nsfc.gov.cn/
[36] [Online]. Chinese Medical Journal Network. Available: https://medjournals.cn/index.do
Additional Files
Published
Issue
Section
License
ONLINE OPEN ACCES: Acces to full text of each article and each issue are allowed for free in respect of Attribution-NonCommercial 4.0 International (CC BY-NC 4.0.
You are free to:
-Share: copy and redistribute the material in any medium or format;
-Adapt: remix, transform, and build upon the material.
The licensor cannot revoke these freedoms as long as you follow the license terms.
DISCLAIMER: The author(s) of each article appearing in International Journal of Computers Communications & Control is/are solely responsible for the content thereof; the publication of an article shall not constitute or be deemed to constitute any representation by the Editors or Agora University Press that the data presented therein are original, correct or sufficient to support the conclusions reached or that the experiment design or methodology is adequate.
