Implementation and Evaluation of a Proactive Context Reading Mechanism in a Web-Based AI Chatbot Application

Authors

  • Nadiya Ghefira El Firsi✉ Department of Informatics Management, Politeknik Negeri Lampung, Bandar Lampung, Indonesia Author
  • Nandy Thaher Ulga Department of Informatics Management, Politeknik Negeri Lampung, Bandar Lampung, Indonesia Author
  • Elya Dianis Department of Informatics Management, Politeknik Negeri Lampung, Bandar Lampung, Indonesia Author
  • Ahmad Muhajir Department of Informatics Management, Politeknik Negeri Lampung, Bandar Lampung, Indonesia Author
  • Muhammad Reza Redo Islami Department of Informatics Management, Politeknik Negeri Lampung, Bandar Lampung, Indonesia Author
Diajukan 10 Feb 2025 Diterbitkan 30 Jun 2025
DOI: https://doi.org/10.59025/9365e6b9

Keywords:

AI chatbot, proactive context, context injection, LLM, Flask

Abstract

Large Language Model (LLM)-based chatbots widely deployed today typically operate reactively: users must explicitly restate uploaded document content for every query. This pattern adds cognitive overhead and degrades answer quality when users forget to attach the relevant context. This study implements and evaluates a proactive context reading mechanism in a web-based AI chatbot built with Python Flask and integrated with the multi-model OpenRouter API. The proactive mechanism injects the full uploaded file content into the system message before the user query is sent to the API. Development followed a modified Waterfall method extended with a comparative evaluation phase. The resulting system contains 525 lines of Python code, 14 REST endpoints, and supports five file formats. Black-box testing across 32 scenarios recorded 31 successes (96.9 percent). A comparative evaluation across four quality metrics showed the proactive mode outperformed the reactive baseline: relevance reached 91.2 percent versus 68.5 percent, answer completeness 88.7 percent versus 62.4 percent, factual accuracy 89.4 percent versus 71.8 percent, and average tokens per request 3,420 versus 1,180. An End-User Computing Satisfaction survey of 15 respondents produced a mean score of 4.1 out of 5 for the proactive mode and 3.2 for the reactive mode. The contribution is empirical evidence that proactive context injection improves relevance by 22.7 percentage points with a manageable token cost trade-off.

Author Biographies

  • Nadiya Ghefira El Firsi, Department of Informatics Management, Politeknik Negeri Lampung, Bandar Lampung, Indonesia

    Nadiya Ghefira El Firsi is a student in the Department of Informatics Management, Politeknik Negeri Lampung. Research interests include web application development, information systems, and artificial intelligence.

  • Nandy Thaher Ulga, Department of Informatics Management, Politeknik Negeri Lampung, Bandar Lampung, Indonesia

    Nandy Thaher Ulga is a student in the Department of Informatics Management, Politeknik Negeri Lampung. Research interests include web application development, information systems, and artificial intelligence.

  • Elya Dianis, Department of Informatics Management, Politeknik Negeri Lampung, Bandar Lampung, Indonesia

    Elya Dianis is a student in the Department of Informatics Management, Politeknik Negeri Lampung. Research interests include web application development, information systems, and artificial intelligence.

  • Ahmad Muhajir, Department of Informatics Management, Politeknik Negeri Lampung, Bandar Lampung, Indonesia

    Ahmad Muhajir is a student in the Department of Informatics Management, Politeknik Negeri Lampung. Research interests include web application development, information systems, and artificial intelligence.

  • Muhammad Reza Redo Islami, Department of Informatics Management, Politeknik Negeri Lampung, Bandar Lampung, Indonesia

    Muhammad Reza Redo Islami is a faculty member in the Department of Informatics Management, Politeknik Negeri Lampung. Research interests include web application development, information systems, and artificial intelligence.

References

  1. M. A. Roshani et al., "Generative Large Language Model–Powered Conversational AI App for Personalized Risk Assessment: Case Study in COVID-19," JMIR Preprints, 2024, doi: 10.2196/preprints.67363.
  2. E. M. Bender, T. Gebru, A. McMillan-Major, dan S. Shmitchell, "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?" dalam Proc. 2021 ACM Conf. Fairness, Accountability, Transparency (FAccT), Virtual Event, 2021, hal. 610-623, doi: 10.1145/3442188.3445922.
  3. S. Ahmed, A. Rahman, dan M. Ashrafuzzaman, "A Systematic Review of AI and Machine Learning-Driven IT Support Systems: Enhancing Efficiency and Automation in Technical Service Management," Am. J. Scholarly Res. Innov., vol. 2, no. 2, hal. 75-101, 2023, doi: 10.63125/fd34sr03.
  4. A. T. Neumann, Y. Yin, S. Sowe, S. Decker, dan M. Jarke, "An LLM-Driven Chatbot in Higher Education for Databases and Information Systems," IEEE Trans. Educ., vol. 68, no. 1, hal. 103-116, Feb. 2025, doi: 10.1109/te.2024.3467912.
  5. Z. Gong et al., "Enhancing Trust in LLM Chatbots for Workplace Support Through User Experience Design and Prompt Engineering," dalam The Human Side of Service Engineering, vol. 143, AHFE Open Access, 2024, hal. 139-149, doi: 10.54941/ahfe1005092.
  6. S. Abedu, A. Abdellatif, dan E. Shihab, "LLM-Based Chatbots for Mining Software Repositories: Challenges and Opportunities," dalam Proc. 28th Int. Conf. Eval. Assess. Softw. Eng. (EASE), Salerno, Italy, 2024, hal. 201-210, doi: 10.1145/3661167.3661218.
  7. W. C. Choi dan C. I. Chang, "Enhancing Education with ChatGPT 4o and Microsoft Copilot: A Review of Opportunities, Challenges, and Student Perspectives on LLM-Based Text-to-Image Generation Models," Preprints, 2025, doi: 10.20944/preprints202503.2387.v1.
  8. M. Kaif, S. Sharma, dan S. Rana, "Gemini MultiPDF Chatbot: Multiple Document RAG Chatbot Using Gemini Large Language Model," Int. J. Res. Appl. Sci. Eng. Technol., vol. 12, no. 5, hal. 24-30, 2024, doi: 10.22214/ijraset.2024.61195.
  9. Z. Liu, P. Agrawal, S. Singhal, V. Madaan, dan K. Kanmani, "LPITutor: An LLM-Based Personalized Intelligent Tutoring System Using RAG and Prompt Engineering," PeerJ Comput. Sci., vol. 11, 2025, doi: 10.7717/peerj-cs.2991.
  10. H. Modran, I. C. Bogdan, D. Ursutiu, C. Samoila, dan H. Modran, "LLM Intelligent Agent Tutoring in Higher Education Courses Using a RAG Approach," Preprints, 2024, doi: 10.20944/preprints202407.0519.v1.
  11. P. V. Khanh dan P. V. A. Tuan, "Building a Vietnamese Math Chatbot Based on RAG and LLM: System Design, Implementation and Experimental Evaluation," HNUE J. Sci. Nat. Sci., vol. 70, no. 4, hal. 43-56, 2025, doi: 10.18173/2354-1059.2025-0053.
  12. A. Ghosh, N. Heffernan, dan A. S. Lan, "Context-Aware Attentive Knowledge Tracing," dalam Proc. 26th ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining (KDD), 2020, hal. 2330-2339, doi: 10.1145/3394486.3403282.
  13. A. Mukanova, M. Milosz, A. Dauletkaliyeva, A. Nazyrova, dan G. Yelibayeva, "LLM-Powered Natural Language Text Processing for Ontology Enrichment," Appl. Sci., vol. 14, no. 13, Art. no. 5860, 2024, doi: 10.3390/app14135860.
  14. K. Angkananon, R. Khunchamnan, B. Kongchana, dan P. Ploadaksorn, "Tourist Decision Support System for Hot Springs, Thailand," GeoJournal Tourism Geosites, vol. 53, no. 2, hal. 611-621, 2024, doi: 10.30892/gtg.53224-1236.
  15. P. Temniranrat, K. Kiratiratanapruk, A. Kitvimonrat, W. Sinthupinyo, dan S. Patarapuwadol, "A System for Automatic Rice Disease Detection from Rice Paddy Images Serviced via a Chatbot," Comput. Electron. Agric., vol. 185, Art. no. 106156, 2021, doi: 10.1016/j.compag.2021.106156.
  16. V. Apriana dan U. Nurhasanah, "Implementasi Metode Waterfall Pada Sistem Informasi Penjualan Berbasis Web," Artikel Ilm. Sist. Inf. Akuntansi (AKASIA), vol. 1, no. 2, hal. 106-111, Okt. 2021, doi: 10.31294/akasia.v1i2.585.
  17. F. Widianto, "Implementasi Model SDLC Dalam Perancangan Sistem Informasi Manajemen Perpustakaan Berbasis Web," Sistematis: J. Ilm. Sist. Inf., vol. 1, no. 1, hal. 60-68, Okt. 2024, doi: 10.69533/4w86eq90.
  18. D. S. H. Putra dan M. A. Darmawan, "Analisis Kepuasan Pengguna Sistem Informasi Administrasi Rumah Sakit (SIARS) dengan Model DeLone dan McLean," J. Sist. Inf. Bisnis, vol. 11, no. 1, hal. 78-85, 2021, doi: 10.21456/vol11iss1pp78-85.
  19. B. B. Akbar, "Analisis Kepuasan Pengguna Sistem Informasi Manajemen Surat Menggunakan Metode End User Computing Satisfaction," J. Pengembangan Sist. Inf. Inform., vol. 3, no. 1, hal. 24-32, Jan. 2022, doi: 10.47747/jpsii.v3i1.613.
  20. A. Nurseptaji dan Y. Ramdhani, "Sistem Informasi Perpustakaan dengan Implementasi Model Waterfall," INFORMASI: J. Inform. Sist. Inf., vol. 13, no. 1, hal. 61-79, 2021, doi: 10.37424/informasi.v13i1.68.
Cover JTDS Vol 3 No 1 (2025)

Downloads

How to Cite

Implementation and Evaluation of a Proactive Context Reading Mechanism in a Web-Based AI Chatbot Application. (2025). Journal of Technology and Data Science, 3(1), 120-130. https://doi.org/10.59025/9365e6b9