DEVELOPMENT OF A MULTIPLE-CHOICE ITEM ANALYSIS APPLICATION TO ENHANCE LEARNING ASSESSMENT INSTRUMENTS

DOI: https://doi.org/10.33650/edureligia.v9i2.10366
Authors

(1) * Hilyah Ashoumi   (Universitas KH. A. Wahab Hasbullah)  
        Indonesia
(2)  Siti Citra Aulia Aprina   (Universitas KH. A. Wahab Hasbullah)  
        Indonesia
(3)  Muhammad Kris Yuan Hidayatulloh   (Universitas Negeri Surabaya)  
        Indonesia
(4)  Munifah Munifah   (Universitas Islam Negeri Syekh Wasil Kediri)  
        Indonesia
(5)  Iskandar Tsani   (Universitas Islam Negeri Syekh Wasil Kediri)  
        Indonesia
(*) Corresponding Author

Abstract


The research aimed to ascertain the viability, receptivity of teachers to the implementation of multiple-choice item analysis and to develop a multiple-choice item analysis application in desktop form. This type of research was developed using the research and development (R&D) method. This research employs the ADDIE development model, which comprised five stages. The phases of analysis, design, development, implementation, and evaluation were employed. The results of this research indicated that the level of feasibility, based on the first media expert with a percentage value of 85%, was declared "Very Eligible"; the second media expert, with a percentage value of 86.66%, was declared "Very Eligible"; and the third media expert, with a percentage value of 95%, was declared "Very Appropriate." In contrast, the response results from teachers received a percentage value of 80.5%. It indicated the application was suitable for use in educational institutions. The findings showed that the multiple-choice item analysis application had great potential to improve the quality of test questions in schools. The application overcame the weaknesses of existing software by offering a reliable, easy-to-use tool, supported by high ratings from experts and positive feedback from teachers. This innovation helped educators create better assessments, improving learning outcomes and overall education quality. The implication of this research was that the developed application can serve as a practical solution for schools to strengthen assessment quality, enhance teachers’ competency in test construction, and support policy-making in education by providing accurate data-driven insights into student learning outcomes.

Keywords

Develompent Assessment Instruments, Question Item Analysis, Multiple Choice



Full Text: PDF



References


Abuhassna, H., Alnawajha, S., Awae, F., Adnan, M., & Edwards, B. I. (2024). Synthesizing Technology Integration Within the Addie Model for Instructional Design: a Comprehensive Systematic Literature Review. Journal of Autonomous Intelligence, 7(5), 1–28. 10.32629/jai.v7i5.1546

Astiza, D. A., Hidayatulloh, M. K. Y., & Ashoumi, H. (2023). The Influence of the SQ4R Learning Model on Learning Outcomes Student. APPLICATION: Applied Science in Learning Research, 3(2), 33–37. https://doi.org/10.32764/application.v3i2.4750

Balasopoulou, A., Κokkinos, P., Pagoulatos, D., Plotas, P., Makri, O. E., Georgakopoulos, C. D., Vantarakis, A., Li, Y., Liu, J. J., Qi, P., Rapoport, Y., Wayman, L. L., Chomsky, A. S., Joshi, R. S., Press, D., Rung, L., Ademola-popoola, D., Africa, S., Article, O., … Loukovaara, S. (2017). Symposium Recent Advances and Challenges in the Management of Retinoblastoma Globe‑Saving Treatments. BMC Ophthalmology, 17(1), 1. https://doi.org/10.4103/ijo.IJO

Beerepoot, M. T. P. (2023). Formative and Summative Automated Assessment with Multiple-Choice Question Banks. Journal of Chemical Education, 100(8), 2947–2955. https://doi.org/10.1021/acs.jchemed.3c00120

Belay, L. M., Sendekie, T. Y., & Eyowas, F. A. (2022). Quality of Multiple-Choice Questions in Medical Internship Qualification Examination Determined by Item Response Theory at Debre Tabor University, Ethiopia. BMC Medical Education, 22(1), 1–11. https://doi.org/10.1186/s12909-022-03687-y

Belkhir, F. Z. (2024). Challenges and Opportunities of AI-Assisted Learning: A Systematic Literature Review on the Impact of ChatGPT Usage in Higher Education. IJLTER, 22(7), 25–39. https://doi.org/https://www.nature.com/articles/s41586-023-06221-2

Chin, H., Chew, C. M., Lim, H. L., & Thien, L. M. (2022). Development and Validation of a Cognitive Diagnostic Assessment with Ordered Multiple-Choice Items for Addition of Time. International Journal of Science and Mathematics Education, 20(4), 817–837. https://doi.org/10.1007/s10763-021-10170-5

Crawford, J., Cowling, M., & Allen, K. A. (2023). Leadership is Needed for Ethical Chatgpt: Character, Assessment, and Learning Using Artificial Intelligence (AI). Journal of University Teaching and Learning Practice, 20(3), 1–21. https://doi.org/10.53761/1.20.3.02

Darvin, R., & Norton, B. (2023). Investment and Mmotivation in Language Learning: What’s the Difference? Language Teaching, 56(1), 29–40. https://doi.org/10.1017/S0261444821000057

Forsblom, L., Pekrun, R., Loderer, K., & Peixoto, F. (2022). Cognitive Appraisals, Achievement Emotions, and Students’ Math Achievement: A Longitudinal Analysis. Journal of Educational Psychology, 114(2), 346–367. https://doi.org/10.1037/edu0000671

Garcia, A. R., Filipe, S. B., Fernandes, C., Estevão, C., & Ramos, G. (2022). Detection of Bipolar Disorder in the Prodromal Phase: a Systematic Review Of Assessment Instruments. Study Selection Criteria, 1(1), 1–79. https://doi.org/https://www.nature.com/articles/s41586-023-06221-2

Halverson, L., Graham, C. R., & Henrie, C. (2020). Learning, Design, and Technology. in Learning, Design, and Technology (Issue February 2021). https://doi.org/10.1007/978-3-319-17727-4

Harahap, H. N., & Ritonga, A. A. (2023). The Effect of Using Project Based Learning Model with Autoplay Media Studio on Learning Achievement. Edureligia: Jurnal Pendidikan Agama Islam, 07(01), 99. https://doi.org/10.1016/j.mjafi.2020.11.007

Hidayatulloh, M. K. Y., & Ashoumi, H. (2022). Creativity and Entrepreneur Knowledge to Increase Entrepreneurial Intent Among Vocational School Students. Journal of Education and Learning (EduLearn), 16(4), 434–439. https://doi.org/10.11591/edulearn.v16i4.19771

Ismail, M. I. (2020). Asesmen dan Evaluasi Pembelajaran (1st ed.). Cendekia Publisher.

Kadaskar, H. R. (2024). Enhancing User Experience in Mobile Application Design Through Gestural Interaction: a Human-Computer Interaction Perspective. International Journal of Scientific Research in Modern Science and Technology, 3(8), 1–6. https://doi.org/10.59828/ijsrmst.v3i8.239

Karim, S. A., Sudiro, S., & Sakinah, S. (2021). Utilizing Test Items Analysis to Examine the Level of Difficulty and Discriminating Power in a Teacher-Made Test. EduLite: Journal of English Education, Literature and Culture, 6(2), 256. https://doi.org/10.30659/e.6.2.256-269

Khoirunnisa, S. (2020). Aplikasi Penilaian Kinerja Soal Pilihan Ganda Dengan Microsoft Excel 2013.

Kumar, D., Jaipurkar, R., Shekhar, A., Sikri, G., & Srinivas, V. (2021). Item Analysis of Multiple Choice Questions: a Quality Assurance Test for an Assessment Tool. Medical Journal Armed Forces India, 77(1), S85–S89. https://doi.org/10.1016/j.mjafi.2020.11.007

Laato, S., Morschheuser, B., Hamari, J., & Bjorne, J. (2023). AI-Assisted Learning with ChatGPT and Large Language Models: Implications for Higher Education. Proceedings - 2023 IEEE International Conference on Advanced Learning Technologies, ICALT 2023, 12(November 2022), 226–230. https://doi.org/10.1109/ICALT58122.2023.00072

Lahza, H., Smith, T. G., & Khosravi, H. (2023). Beyond Item Analysis: Connecting Student Behaviour and Performance Using E-Assessment Logs. British Journal of Educational Technology, 54(1), 335–354. https://doi.org/10.1111/bjet.13270

Liu, W. (2021). Does Teacher Immediacy Affect Students? A Systematic Review of the Association Between Teacher Verbal and Non-verbal Immediacy and Student Motivation. Frontiers in Psychology, 12(June), 1–13. https://doi.org/10.3389/fpsyg.2021.713978

Liu, Y., Tan, H., Cao, G., & Xu, Y. (2024). Enhancing User Engagement Through Adaptive UI/UX Design: a Study on Personalized Mobile App Interfaces. Computer Science & IT Research Journal, 5(8), 1942–1962. : 10.53469/wjimt.2024.07(05).01

Mai, D. T. T., Da, C. Van, & Hanh, N. Van. (2024). The Use of ChatGPT in Teaching and Learning: a Systematic Review Through SWOT Analysis Approach. Frontiers in Education, 9(February), 1–17. https://doi.org/10.3389/feduc.2024.1328769

Mesra, R. (2023). Research & Development dalam Pendidikan. In PT. Mifandi Mandiri Digital. PT. Mifandi Mandiri Digital.

Nurbayan, Y., & Anwar, S. (2022). Digital Library Utilization ; Strategies to Improve Digital Islamic Digital Literacy for Religion Teachers. Edureligia: Jurnal Pendidikan Agama Islam, 6(December), 150–160. https://doi.org/10.33650/edureligia.v6i2.4536

Peters, S. J. (2022). The Challenges of Achieving Equity Within Public School Gifted and Talented Programs [University of Wisconsin – Whitewater]. In Gifted Child Quarterly (Vol. 66, Issue 2). https://doi.org/10.1177/00169862211002535

Rezigalla, A. A., Eleragi, A. M. E. S. A., Elhussein, A. B., Alfaifi, J., ALGhamdi, M. A., Al Ameer, A. Y., Yahia, A. I. O., Mohammed, O. A., & Adam, M. I. E. (2024). Item Analysis: the Impact of Distractor Efficiency on the Difficulty Index and Discrimination Power of Multiple-Choice Items. BMC Medical Education, 24(1), 1–7. https://doi.org/10.1186/s12909-024-05433-y

Ribeiro-Silva, E., Amorim, C., Aparicio-Herguedas, J. L., & Batista, P. (2022). Trends of Active Learning in Higher Education and Students’ Well-Being: A Literature Review. Frontiers in Psychology, 13(April), 1–10. https://doi.org/10.3389/fpsyg.2022.844236

Rozi, F., Zahiro, R., & Hotimah, A. S. (2022). Training on the Utilization of Used Goods as APE Based on Natural Potential. Indonesian Journal of Community Research & Engagement, 01(01), 1–7. https://doi.org/http://doi.org/10.15408/ijocore.vxix.xxxx

Sicilia, A., Alcaraz-Ibáñez, M., Paterna, A., & Griffiths, M. D. (2022). A Review of the Components of Problematic Exercise in Psychometric Assessment Instruments. Frontiers in Public Health, 10(March), 1–13. https://doi.org/10.3389/fpubh.2022.839902

Slepkov, A. D., Van Bussel, M. L., Fitze, K. M., & Burr, W. S. (2021). A Baseline for Multiple-Choice Testing in the University Classroom. SAGE Open, 11(2), 1–12. https://doi.org/10.1177/21582440211016838

Vaughn, S. R., Bos, C. S., & Schumm, J. S. (2023). Teaching Students Who are Exceptional, Diverse, and at Risk in the General Educational Classroom (pp. 1–7). Pearson Education. https://doi.org/10.1016/j.tate.2022.103945

You, K., Liu, Y., Wang, J., & Long, M. (2021). LogME: Practical Assessment of Pre-trained Models for Transfer Learning. Proceedings of Machine Learning Research, 139(1), 12133–12143. https://proceedings.mlr.press/v139/you21b.html


Dimensions, PlumX, and Google Scholar Metrics

10.33650/edureligia.v9i2.10366


Refbacks

  • There are currently no refbacks.


Copyright (c) 2025 Hilyah Ashoumi, Siti Citra Aulia Aprina, Muhammad Kris Yuan Hidayatulloh, Munifah, Iskandar Tsani

Creative Commons License
 

Edureligia : Jurnal Pendidikan Agama Islam
Published by Lembaga Penerbitan, Penelitian dan Pengabdian kepada Masyarakat (LP3M) of Nurul Jadid University, Probolinggo, East Java, Indonesia.