COMPUTERIZED ADAPTIVE TEST (CAT) OPTIMIZATION USING THE EFFICIENCY BALANCED INFORMATION (EBI) METHOD



Authors (s)


(1) * Aji Joko Budi Pramono   (Institut Agama Islam Negeri Ternate)  
        Indonesia
(*) Corresponding Author

Abstract


This study aims to optimize Efficiency Balanced Information (EBI) for selecting items in the computer-based adaptive exam model (CAT), usually used in education. This optimization is used to reduce problems that often occur in CAT based on item response theory (TRB), namely the emergence of ability bias, large Mean Absolute Error (MAE), too many test lengths, and uncontrolled Standard Error Estimation (SEE). The success of the CAT-based exam is strongly influenced by the item selection method; therefore, an appropriate item selection strategy is needed. This study proposes selecting items using the EBI method to solve this problem. The simulation results show that the grain bias is slightly around zero, and CMAE is consistent in the allowable theta area. The average number of items presented is less than 20 when -2<θ<2. The average test length increases to 25 seconds | θ | > 2. The items presented show the different power parameters of questions (a) at the beginning of the test more than at the end of the test; therefore, this method can be the right solution for optimizing accuracy in selecting items

Full Text: PDF

References


Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46(4). https://doi.org/10.1007/BF02293801

Bock, R. D., & Mislevy, R. J. (1982). Adaptive EAP Estimation of Ability in a Microcomputer Environment. Applied Psychological Measurement, 6(4). https://doi.org/10.1177/014662168200600405

Bunderson, C. V., Inouye, D. K., & Olsen, J. B. (1988). THE FOUR GENERATIONS OF COMPUTERIZED EDUCATIONAL MEASUREMENT. ETS Research Report Series, 1988(1), i–148. https://doi.org/10.1002/j.2330-8516.1988.tb00291.x

Chrysafiadi, K., Troussas, C., & Virvou, M. (2020). Combination of fuzzy and cognitive theories for adaptive e-assessment. Expert Systems with Applications, 161. https://doi.org/10.1016/j.eswa.2020.113614

Deng, H., Ansley, T., & Chang, H. H. (2010). Stratified and maximum information item selection procedures in computer adaptive testing. Journal of Educational Measurement, 47(2). https://doi.org/10.1111/j.1745-3984.2010.00109.x

Diana, H., & Maula, R. (2022). Manajemen Sumber Daya Manusia Dalam Meningkatkan Kinerja Organisasi Panji Pelopor di Pondok Pesantren. JUMPA: Jurnal Manajemen Pendidikan, 3(1).

Dunkel, P. A. (1998). Considerations in developing or using second/foreign language proficiency computer-adaptive tests. Language Learning and Technology, 2(2).

Falk, C. F., & Feuerstahler, L. M. (2022). On the Performance of Semi- and Nonparametric Item Response Functions in Computer Adaptive Tests. Educational and Psychological Measurement, 82(1). https://doi.org/10.1177/00131644211014261

Gawliczek, P., Krykun, V., Tarasenko, N., Tyshchenko, M., & Shapran, O. (2021). COMPUTER ADAPTIVE LANGUAGE TESTING ACCORDING TO NATO STANAG 6001 REQUIREMENTS. Advanced Education, 8(18). https://doi.org/10.20535/2410-8286.225018

Gibbons, C., Bachulis, A., & Alien, G. (1999). A comparison of a computer and pencil and paper assignment. Computers in Nursing, 17(6).

Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of Item Response Theory. Sage Publications.

Han, K. C. T. (2018a). Components of the item selection algorithm in computerized adaptive testing. In Journal of educational evaluation for health professions (Vol. 15). https://doi.org/10.3352/jeehp.2018.15.7

Han, K. C. T. (2018b). Conducting simulation studies for computerized adaptive testing using SimulCAT: an instructional piece. Journal of Educational Evaluation for Health Professions, 15. https://doi.org/10.3352/jeehp.2018.15.20

Hussain, J., Chawla, G., Rafiqzad, H., Huang, S., Bartlett, S. J., Li, M., Howell, D., Peipert, J. D., Novak, M., & Mucsi, I. (2022). Validation of the PROMIS sleep disturbance item bank computer adaptive test (CAT) in patients on renal replacement therapy. Sleep Medicine, 90. https://doi.org/10.1016/j.sleep.2022.01.001

International Test Comission. (2005). Guidelines for Translating and Adapting Tests. International Test Commission (ITC).

Kingsbury, G. G., & Zara, A. R. (1989). Procedures for selecting items for computerized adaptive tests. Applied Measurement in Education, 2(4). https://doi.org/10.1207/s15324818ame0204_6

LAN, X., YU, H., & LI, J. (2017). Optimization of Moodle for Adaptive Testing. DEStech Transactions on Computer Science and Engineering, cmsam. https://doi.org/10.12783/dtcse/cmsam2017/16420

Li, G., Cai, Y., Gao, X., Wang, D., & Tu, D. (2021). Automated Test Assembly for Multistage Testing With Cognitive Diagnosis. Frontiers in Psychology, 12. https://doi.org/10.3389/fpsyg.2021.509844

Lord, F. M. (2012). Applications of item response theory to practical testing problems. In Applications of Item Response Theory To Practical Testing Problems. https://doi.org/10.4324/9780203056615

Marastuti, A., Anggoro, W. J., Marvianto, R. D., & Al Afghani, A. A. (2020). Perbandingan Properti Psikometri antara Tes PAPs Berbentuk Computer-Based dan Paper and Pencil Test. Gadjah Mada Journal of Psychology (GamaJoP), 6(1). https://doi.org/10.22146/gamajop.51852

Maravić Čisar, S., Čisar, P., & Pinter, R. (2016). Evaluation of knowledge in Object Oriented Programming course with computer adaptive tests. Computers and Education, 92–93. https://doi.org/10.1016/j.compedu.2015.10.016

Naeem, F. (2019). Cultural adaptations of CBT: A summary and discussion of the Special Issue on Cultural Adaptation of CBT. Cognitive Behaviour Therapist. https://doi.org/10.1017/S1754470X19000278

Oud, M., de Winter, L., Vermeulen-Smit, E., Bodden, D., Nauta, M., Stone, L., van den Heuvel, M., Taher, R. Al, de Graaf, I., Kendall, T., Engels, R., & Stikkelbroek, Y. (2019). Effectiveness of CBT for children and adolescents with depression: A systematic review and meta-regression analysis. In European Psychiatry (Vol. 57). https://doi.org/10.1016/j.eurpsy.2018.12.008

Owen, R. J. (1975). A bayesian sequential procedure for quantal response in the context of adaptive mental testing. Journal of the American Statistical Association, 70(350). https://doi.org/10.1080/01621459.1975.10479871

Popova, Y. B. (2019). Learning Management System CATS. Science & Technique, 18(4). https://doi.org/10.21122/2227-1031-2019-18-4-339-349

Postigo, Á., Cuesta, M., Pedrosa, I., Muñiz, J., & García-Cueto, E. (2020). Development of a computerized adaptive test to assess entrepreneurial personality. Psicologia: Reflexao e Critica, 33(1). https://doi.org/10.1186/s41155-020-00144-x

Quiñones, A. C., & Humphrey, G. E. (2007). Addressing the pharmacist shortage through a cooperative internship program for foreign pharmacy graduates. Journal of the American Pharmacists Association, 47(2). https://doi.org/10.1331/QJ0N-4278-738U-0061

Rezaie, M., & Golshan, M. (2015). Computer Adaptive Test (CAT): Advantages and Limitations. International Journal of Educational Investigations Available Online @ Www.Ijeionline.Com, 2(5).

Solberg, B. L. (2015). Critical Thinking as a Predictor of Certification Exam Performance in Medical Laboratory Science. American Society for Clinical Laboratory Science, 28(2). https://doi.org/10.29074/ascls.28.2.76

Sudaryono. (2011). Implementasi Teori Responsi Butir (Item Response Theory) Pada Penilaian Hasil Belajar Akhir di Sekolah. Jurnal Pendidikan Dan Kebudayaan, 17(6). https://doi.org/10.24832/jpnk.v17i6.62

Tisocco, F., & Liporace, M. F. (2021). The tuckman procrastination scale: Psychometric features among Buenos Aires undergraduates. Psychological Thought, 14(2). https://doi.org/10.37708/psyct.v14i2.603


Article View

Abstract views : 286 times | PDF files viewed : 96 times



Refbacks

  • There are currently no refbacks.