Interactive Malayalam Question Answering System: A Neural Word Embedding And Similarity Measure Based Approach.
Main Article Content
Abstract
This innovative system operates as an automated, domain-specific knowledge repository designed specifically to furnish reliable Malayalam responses to inquiries pertaining to COVID-19. Leveraging advanced Natural Language Processing (NLP) algorithms, both Malayalam documents and questions undergo meticulous processing. The semantic modelling and document conversion stages employ the Word Embedding approach, specifically Continuous Bag of Words (CBOW), to enhance the system's understanding of the language nuances. Subsequently, the retrieved results for a given query are meticulously ranked using the cosine similarity measure, ensuring that the most relevant and accurate information is presented to the user. Integral to the system's efficacy is our proprietary Malayalam question-answering dataset. This dataset has been meticulously curated, drawing from reliable and publicly accessible sources related to COVID-19. It serves as the foundation for experimentation, reflecting the system's ability to provide accurate responses. The system's performance is quantified using the F1 score, a metric that combines precision and recall, yielding a comprehensive evaluation. In our experimentation, the F1 score of the Semantic Malayalam Question-Answering System is found to be 76%, attesting to its robustness and effectiveness in delivering trustworthy information in the Malayalam language within the context of COVID-19.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
Green BF, Wolf AK, Chomsky C, and Laughery K. “Baseball: An automatic question answerer”. In Proceedings of Western Computing Conference, Vol. 19, 1961, pp. 219–224. Proceedings of AFIPS Conference, Vol.42, 1973, pp. 441–450.
YuanZhang, Dong Wang, Yan Zang, “Neural IRM eets Graph Embedding: A Ranking Model for Product Search” The Web Conference, May 2019, USA, ACM.
Piyush Mital, Saurabh Agrawal, Bhargavi Neti, Yashodhara Haribhakta, Vibhavari Kamble, Krishnanjan Bhattacharjee, Debashri Das, Swati Mehta, Ajai Kumar. “Graph-based Question Answering System”, ICACCI Dec 2018 IEEE
Tom Young , Devamanyu Hazarika , Soujanya Poria , Erik Cambria. “Recent Trends in Deep Learning Based Natural Language Processing”, arXiv, Nov 2018, August 2018, IEEE Computational intelligence magazine.
Fan fang, Bo-wen zhang, and Xu-cheng yin. “Semantic Sequential Query Expansion for Bio- medical Article Search”, 2169-3536 2018 IEEE.
Bo Xu, Hongfei Lin, Yuan Lin. “Learning to Refine Expansion Terms for Bio-medical Information Retrieval Using Semantic Resources” , 10.1109/TCBB.2018.2801303, IEEE/ACM.
Xu, B., Lin, H., Lin, Y. (2016). “Assessment of learning to rank methods for query expansion”. Journal of the Association for Information Science and Technology, 2016, 67(6): 1345- 1357.
DwaipayanRoy, Debasis Ganguly ,sumit Bhatia ,Srikanta Bedathur,Mandar Mitra, “Using Word Embeddings for Information Retrieval: How Collection and Term Normalization Choices Affect Performance”, 3269206.3269277 CIKM ’18, October 22–26, 2018, Torino, Italy, ACM.
Shomi Khan , Khadiza Tul Kubra, Md Mahadi Hasan Nahid, “Improving Answer Extraction For Bangali Q/A System Using Anaphora-Cataphora Resolution”, International Conference on Innovation in Engineering and Technology (ICIET) 27-29 December, 2018 IEEE.
Archana S.M. , Naima Vahab , Rekha Thankappa , C. Raseek,”A Rule Based Question Answering System in Malayalam corpus using Vibhakthi and POS Tag Analysis”,International Conference on Emerging Trends in Engineering, Science and Technology (ICETEST – 2015) .
Arjun Babu, Sindhu L. “An Information Retrieval System for Malayalam Using Query Expansion Technique”,978-1-4799-8792-4/15/$31.00 c 2015 IEEE
Vaishali Singh,Sanjay K. Dwivedi .”Personalized approach for automated question answering in restricted domain”.International Journal of Information Technology, Springer.https://doi.org/10.1007/s41870-018-0200-6. 2018.
Sheetal S. Sonawane , Parag Kulkarni. Concept based document similarity using graph model International Journal of Information Technology,Springer. https://doi.org/10.1007/s41870-019-00314-w. 2019.
Swathilakshmi Venkatachalam , Lakshmana Pandian Subbiah et al[3].”An ontologybased information extraction and summarization of multiple news articles”. International Journal of Information Technology,Springer. 2019.
Navjot Kaur, Himanshu Aggarwal.”Query reformulation approach using domain specific ontology for semantic information retrieval”. International Journal of Information Technology, Springer. https://doi.org/10.1007/s41870-020-00464-2. 2020.
Shickel, B., Tighe, P. J., Bihorac, A., & Rashidi “Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis”, JBHI.2017.2767063,IEEE.
T. Kawamura, K. Kozaki, T. Kushida, K. Watanabe, and K. Matsumura, ‘‘Expanding science and technology thesauri from bibliographic datasets using word embedding,’’ in Proc. IEEE Int. Conf.Tools Artif. Intell., Nov. 2017, pp. 857–864
Liji S K and Lajish V L., “An Efficient Malayalam Query Processing System for University Enquiry “, Proceedings of the Eight National Conference on Indian Language Computing (NCILC), 2018, March 2018, CUSAT, Kerala.
Roberto Passailaigue Baquerizo ,Hubert Viltres Sala , Paúl Rodríguez Leyva , Vivian Estrada. “Sentí :Model for semantic processing in information retrieval systems”, International Research Journal of Engineering and Technology (IRJET),Volume: 04 Issue: 05 ,May -2017.
Liji S K, Muhamed Ilyas P, "Review and Analysis of Different Approaches to Semantic Level Question Answering and Information Retrieval", International Journal of Science and Research (IJSR),https://www.ijsr.net/search_index_results_paperid.php?id=SR21121141135, Volume 10 Issue 1, January 2021, 1238 – 1244.