A Retrieval-Augmented Generation Framework for a Study Program Accreditation Question-Answering System
Received: 17 November 2025 | Revised: 16 January 2026 | Accepted: 2 February 2026 | Online: 4 April 2026
Corresponding author: Sri Hartati
Abstract
Accreditation agencies play a crucial role in ensuring the quality of higher-education institutions by evaluating programs based on established standards and guidelines. However, accreditation processes often involve complex documents that applicants may find difficult to interpret, leading to repetitive inquiries and administrative burden. This study proposes an Artificial Intelligence (AI)-based Question-Answering (QA) system built using a Retrieval-Augmented Generation (RAG) architecture to support accreditation-related information services. The system is developed specifically in the Indonesian language, enabling users to understand accreditation documents more easily by retrieving authoritative content and generating contextually appropriate explanations. As a case study, we apply this approach to LAM INFOKOM, Indonesia's Independent Accreditation Agency for Informatics and Computing. Experimental results show strong retrieval performance, with a Mean Reciprocal Rank (MRR) of 0.88, and high-quality response generation, with a BERTScore F1 of 0.76 and a manual accuracy of 83.67%. These results demonstrate the effectiveness of the Indonesian RAG-based system in improving accreditation information services with the potential to reduce manual workloads.
Keywords:
accreditation, Question-Answering (QA), Retrieval-Augmented Generation (RAG), IndonesianDownloads
References
P. Kumar, B. Shukla, and D. Passey, "Impact of Accreditation on Quality and Excellence of Higher Education Institutions," Revista Investigacion Operacional, vol. 41, no. 2, pp. 151–167, Feb. 2020.
N. Sousa et al., "A hitchhikers' guide to the terminology of accreditation processes for health professionals and institutions," MedEdPublish, vol. 13, Feb. 2023, Art. no. 11. DOI: https://doi.org/10.12688/mep.19566.1
P. Jadhav and S. M. Jadhav, "Survey on Question Answering System with Emphasis on Method and Evaluation Metrics," in 2025 5th International Conference on Soft Computing for Security Applications, Salem, India, 2025, pp. 2135–2140. DOI: https://doi.org/10.1109/ICSCSA66339.2025.11170976
R. Khan, S. P. Khan, and S. A. Ali, "Conversational AI," in Conversational Artificial Intelligence, R. Rawat, R. K. Chakrawarti, S. K. Sarangi, P. Vyas, M. S. Alamanda, K. Srividya, and K. S. Sankaran, Eds. Hoboken, NJ, USA: John Wiley & Sons, Ltd, 2024, pp. 249–268.
D. Collarana et al., "A Question Answering System on Regulatory Documents," in 31st International Conference on Legal Knowledge and Information Systems, Groningen, The Netherlands, 2018, pp. 41–50. DOI: https://doi.org/10.3233/978-1-61499-935-5-41
Z. Jiang et al., "Active Retrieval Augmented Generation," in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore, 2023, pp. 7969–7992. DOI: https://doi.org/10.18653/v1/2023.emnlp-main.495
N. Chirkova, D. Rau, H. Déjean, T. Formal, S. Clinchant, and V. Nikoulina, "Retrieval-augmented generation in multilingual settings," in Proceedings of the 1st Workshop on Towards Knowledgeable Language Models (KnowLLM 2024), Bangkok, Thailand, 2024, pp. 177–188. DOI: https://doi.org/10.18653/v1/2024.knowllm-1.15
"Lembaga Akreditasi Mandiri Informatika dan Komputer." LAM INFOKOM. https://laminfokom.or.id/official/.
R. Kumar, G. B. Singh, P. Shailendra, and N. Sharma, "A Brief Review on Question Answering System: Techniques, Challenges and Future Directions," in 2024 10th International Conference on Advanced Computing and Communication Systems, Coimbatore, India, 2024, pp. 407–414. DOI: https://doi.org/10.1109/ICACCS60874.2024.10717204
E. Adamopoulou and L. Moussiades, "Chatbots: History, technology, and applications," Machine Learning with Applications, vol. 2, Dec. 2020, Art. no. 100006. DOI: https://doi.org/10.1016/j.mlwa.2020.100006
Z. Abbasiantaeb and S. Momtazi, "Text-based question answering from information retrieval and deep neural network perspectives: A survey," WIREs Data Mining and Knowledge Discovery, vol. 11, no. 6, Nov. 2021, Art. no. e1412. DOI: https://doi.org/10.1002/widm.1412
A. Raza, M. Latif, M. U. Farooq, M. A. Baig, M. A. Akhtar, and Waseemullah, "Enabling Context-based AI in Chatbots for conveying Personalized Interdisciplinary Knowledge to Users," Engineering, Technology & Applied Science Research, vol. 13, no. 6, pp. 12231–12236, Dec. 2023. DOI: https://doi.org/10.48084/etasr.6313
S. Ahriz, H. Gharbaoui, N. Benmoussa, A. Chahid, and K. Mansouri, "Enhancing Information Technology Governance in Universities: A Smart Chatbot System based on Information Technology Infrastructure Library," Engineering, Technology & Applied Science Research, vol. 14, no. 6, pp. 17876–17882, Dec. 2024. DOI: https://doi.org/10.48084/etasr.8878
A. T. Neumann, Y. Yin, S. Sowe, S. Decker, and M. Jarke, "An LLM-Driven Chatbot in Higher Education for Databases and Information Systems," IEEE Transactions on Education, vol. 68, no. 1, pp. 103–116, Feb. 2025. DOI: https://doi.org/10.1109/TE.2024.3467912
A. J. Oche, A. G. Folashade, T. Ghosal, and A. Biswas, "A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions." arXiv, July 25, 2025.
J. Malhotra, S. Sharma, and Er. A. S. Rana, "Improving Retrieval-Augmented Generation for Low-Resource Languages," in 2025 International Conference on Modern Sustainable Systems, Shah Alam, Malaysia, 2025, pp. 1164–1169. DOI: https://doi.org/10.1109/CMSS66566.2025.11182476
K. Muludi, K. M. Fitria, J. Triloka, and Sutedi, "Retrieval-Augmented Generation Approach: Document Question Answering using Large Language Model," International Journal of Advanced Computer Science and Applications, vol. 15, no. 3, pp. 776–785, Mar. 2024. DOI: https://doi.org/10.14569/IJACSA.2024.0150379
N. Reimers and I. Gurevych, "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks," in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, 2019, pp. 3982–3992. DOI: https://doi.org/10.18653/v1/D19-1410
V. Karpukhin et al., "Dense Passage Retrieval for Open-Domain Question Answering," in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, Online, 2020, pp. 6769–6781. DOI: https://doi.org/10.18653/v1/2020.emnlp-main.550
S. Agarwal et al., "gpt-oss-120b & gpt-oss-20b Model Card." arXiv, Aug. 08, 2025.
R. Ferreira, M. Canesche, P. Jamieson, O. P. V. Neto, and J. A. M. Nacif, "Examples and tutorials on using Google Colab and Gradio to create online interactive student-learning modules," Computer Applications in Engineering Education, vol. 32, no. 4, July 2024, Art. no. e22729. DOI: https://doi.org/10.1002/cae.22729
Y. Andriyani et al., "Improving University Community Service Communication with Kukerti's Fuzzy String Matching Chatbot," in 2023 International Conference on Informatics, Multimedia, Cyber and Informations System (ICIMCIS), Jakarta Selatan, Indonesia, 2023, pp. 398–403. DOI: https://doi.org/10.1109/ICIMCIS60089.2023.10348968
P. Janardhana Rao, K. Nageswara Rao, S. Gokuruboyina, and K. N. Neeraja, "An Efficient Methodology for Identifying the Similarity Between Languages with Levenshtein Distance," in Proceedings of the 6th International Conference on Communications and Cyber Physical Engineering, Hyderabad, India, 2023, pp. 161–174. DOI: https://doi.org/10.1007/978-981-99-7137-4_15
S. Reddy, "Evaluating Retrieval-Based Chatbot Systems: A Focus on Speed and Performance," in 2025 International Conference on Computing and Communication Technologies, Chennai, India, 2025, pp. 1–6. DOI: https://doi.org/10.1109/ICCCT63501.2025.11019950
T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, "BERTScore: Evaluating Text Generation with BERT." arXiv, Feb. 24, 2020.
Downloads
How to Cite
License
Copyright (c) 2026 Rifki Afina Putri, Helena I. Nurramdhani, Sri Hartati
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
