COMPUTERIZED ADAPTIVE TEST (CAT) OPTIMIZATION USING THE EFFICIENCY BALANCED INFORMATION (EBI) METHOD
Abstract
This study aims to optimize Efficiency Balanced Information (EBI) for selecting items in the computer-based adaptive exam model (CAT), usually used in education. This optimization is used to reduce problems that often occur in CAT based on item response theory (TRB), namely the emergence of ability bias, large Mean Absolute Error (MAE), too many test lengths, and uncontrolled Standard Error Estimation (SEE). The success of the CAT-based exam is strongly influenced by the item selection method; therefore, an appropriate item selection strategy is needed. This study proposes selecting items using the EBI method to solve this problem. The simulation results show that the grain bias is slightly around zero, and CMAE is consistent in the allowable theta area. The average number of items presented is less than 20 when -2<θ<2. The average test length increases to 25 seconds | θ | > 2. The items presented show the different power parameters of questions (a) at the beginning of the test more than at the end of the test; therefore, this method can be the right solution for optimizing accuracy in selecting items
References
Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46(4). https://doi.org/10.1007/BF02293801
Bock, R. D., & Mislevy, R. J. (1982). Adaptive EAP Estimation of Ability in a Microcomputer Environment. Applied Psychological Measurement, 6(4). https://doi.org/10.1177/014662168200600405
Bunderson, C. V., Inouye, D. K., & Olsen, J. B. (1988). THE FOUR GENERATIONS OF COMPUTERIZED EDUCATIONAL MEASUREMENT. ETS Research Report Series, 1988(1), i–148. https://doi.org/10.1002/j.2330-8516.1988.tb00291.x
Chrysafiadi, K., Troussas, C., & Virvou, M. (2020). Combination of fuzzy and cognitive theories for adaptive e-assessment. Expert Systems with Applications, 161. https://doi.org/10.1016/j.eswa.2020.113614
Deng, H., Ansley, T., & Chang, H. H. (2010). Stratified and maximum information item selection procedures in computer adaptive testing. Journal of Educational Measurement, 47(2). https://doi.org/10.1111/j.1745-3984.2010.00109.x
Diana, H., & Maula, R. (2022). Manajemen Sumber Daya Manusia Dalam Meningkatkan Kinerja Organisasi Panji Pelopor di Pondok Pesantren. JUMPA: Jurnal Manajemen Pendidikan, 3(1).
Dunkel, P. A. (1998). Considerations in developing or using second/foreign language proficiency computer-adaptive tests. Language Learning and Technology, 2(2).
Falk, C. F., & Feuerstahler, L. M. (2022). On the Performance of Semi- and Nonparametric Item Response Functions in Computer Adaptive Tests. Educational and Psychological Measurement, 82(1). https://doi.org/10.1177/00131644211014261
Gawliczek, P., Krykun, V., Tarasenko, N., Tyshchenko, M., & Shapran, O. (2021). COMPUTER ADAPTIVE LANGUAGE TESTING ACCORDING TO NATO STANAG 6001 REQUIREMENTS. Advanced Education, 8(18). https://doi.org/10.20535/2410-8286.225018
Gibbons, C., Bachulis, A., & Alien, G. (1999). A comparison of a computer and pencil and paper assignment. Computers in Nursing, 17(6).
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of Item Response Theory. Sage Publications.
Han, K. C. T. (2018a). Components of the item selection algorithm in computerized adaptive testing. In Journal of educational evaluation for health professions (Vol. 15). https://doi.org/10.3352/jeehp.2018.15.7
Han, K. C. T. (2018b). Conducting simulation studies for computerized adaptive testing using SimulCAT: an instructional piece. Journal of Educational Evaluation for Health Professions, 15. https://doi.org/10.3352/jeehp.2018.15.20
Hussain, J., Chawla, G., Rafiqzad, H., Huang, S., Bartlett, S. J., Li, M., Howell, D., Peipert, J. D., Novak, M., & Mucsi, I. (2022). Validation of the PROMIS sleep disturbance item bank computer adaptive test (CAT) in patients on renal replacement therapy. Sleep Medicine, 90. https://doi.org/10.1016/j.sleep.2022.01.001
International Test Comission. (2005). Guidelines for Translating and Adapting Tests. International Test Commission (ITC).
Kingsbury, G. G., & Zara, A. R. (1989). Procedures for selecting items for computerized adaptive tests. Applied Measurement in Education, 2(4). https://doi.org/10.1207/s15324818ame0204_6
LAN, X., YU, H., & LI, J. (2017). Optimization of Moodle for Adaptive Testing. DEStech Transactions on Computer Science and Engineering, cmsam. https://doi.org/10.12783/dtcse/cmsam2017/16420
Li, G., Cai, Y., Gao, X., Wang, D., & Tu, D. (2021). Automated Test Assembly for Multistage Testing With Cognitive Diagnosis. Frontiers in Psychology, 12. https://doi.org/10.3389/fpsyg.2021.509844
Lord, F. M. (2012). Applications of item response theory to practical testing problems. In Applications of Item Response Theory To Practical Testing Problems. https://doi.org/10.4324/9780203056615
Marastuti, A., Anggoro, W. J., Marvianto, R. D., & Al Afghani, A. A. (2020). Perbandingan Properti Psikometri antara Tes PAPs Berbentuk Computer-Based dan Paper and Pencil Test. Gadjah Mada Journal of Psychology (GamaJoP), 6(1). https://doi.org/10.22146/gamajop.51852
Maravić Čisar, S., Čisar, P., & Pinter, R. (2016). Evaluation of knowledge in Object Oriented Programming course with computer adaptive tests. Computers and Education, 92–93. https://doi.org/10.1016/j.compedu.2015.10.016
Naeem, F. (2019). Cultural adaptations of CBT: A summary and discussion of the Special Issue on Cultural Adaptation of CBT. Cognitive Behaviour Therapist. https://doi.org/10.1017/S1754470X19000278
Oud, M., de Winter, L., Vermeulen-Smit, E., Bodden, D., Nauta, M., Stone, L., van den Heuvel, M., Taher, R. Al, de Graaf, I., Kendall, T., Engels, R., & Stikkelbroek, Y. (2019). Effectiveness of CBT for children and adolescents with depression: A systematic review and meta-regression analysis. In European Psychiatry (Vol. 57). https://doi.org/10.1016/j.eurpsy.2018.12.008
Owen, R. J. (1975). A bayesian sequential procedure for quantal response in the context of adaptive mental testing. Journal of the American Statistical Association, 70(350). https://doi.org/10.1080/01621459.1975.10479871
Popova, Y. B. (2019). Learning Management System CATS. Science & Technique, 18(4). https://doi.org/10.21122/2227-1031-2019-18-4-339-349
Postigo, Á., Cuesta, M., Pedrosa, I., Muñiz, J., & García-Cueto, E. (2020). Development of a computerized adaptive test to assess entrepreneurial personality. Psicologia: Reflexao e Critica, 33(1). https://doi.org/10.1186/s41155-020-00144-x
Quiñones, A. C., & Humphrey, G. E. (2007). Addressing the pharmacist shortage through a cooperative internship program for foreign pharmacy graduates. Journal of the American Pharmacists Association, 47(2). https://doi.org/10.1331/QJ0N-4278-738U-0061
Rezaie, M., & Golshan, M. (2015). Computer Adaptive Test (CAT): Advantages and Limitations. International Journal of Educational Investigations Available Online @ Www.Ijeionline.Com, 2(5).
Solberg, B. L. (2015). Critical Thinking as a Predictor of Certification Exam Performance in Medical Laboratory Science. American Society for Clinical Laboratory Science, 28(2). https://doi.org/10.29074/ascls.28.2.76
Sudaryono. (2011). Implementasi Teori Responsi Butir (Item Response Theory) Pada Penilaian Hasil Belajar Akhir di Sekolah. Jurnal Pendidikan Dan Kebudayaan, 17(6). https://doi.org/10.24832/jpnk.v17i6.62
Tisocco, F., & Liporace, M. F. (2021). The tuckman procrastination scale: Psychometric features among Buenos Aires undergraduates. Psychological Thought, 14(2). https://doi.org/10.37708/psyct.v14i2.603
Article Statistic
Abstract view : 199 timesPDF views : 59 times
Dimensions Metrics
How To Cite This :
Refbacks
- There are currently no refbacks.