IMITATIONAL MODELING AND ANALYSIS OF MATRIXES CONTAINING PRIMARY GRADING OBTAINED IN EDUCATIONAL TESTING BY THE MEANS OF LANGUAGE R

Cover Image
Oleksandr O. Dykhovychnyi, Nataliia V. Kruglova

Abstract

The article researches methods of imitational modeling of matrixes containing primary grading obtained in educational testing by the means of statistical programming language R. Unique algorithms and functions were developed to allow generating of matrix of the primary grades according to corresponding test of the defined structure. The importance of this approach is defined by several reasons, specifically the needs to: create reference samples; analyze primary grades by means of CTT (Clasic Test Theory) and IRT; predict basic statistical test characteristics; clarify parameters for the calibrated tasks; model independent parameters for the test takers; increase and development educator’s competency. It should be noted that input parameters could be generated or set up manually. Comparable analysis was conducted for created functions against already existing function packages, such as eRm, ltm, mcIRT, as well as statistical analysis of the generated matrixes. This analysis took into consideration the following procedures: verification of hypothesis about compatibility between generated matrixes and set parameters for the testing tasks per criteria; verification of hypothesis about equivalence of average grades in entire matrix according Student criteriа; verification of hypothesis about equality for vectors of correct answers relative frequencies by columns according to Hotteling criteriа; comparison of theoretical characteristic curves with empirical probabilities; comparison of set parameters for task complexity and parameters graded by generated matrixes with consideration of errors in grading. An experimental system of imitational modeling and analysis for testing results was created. Such system combines contemporary methods of IRT and methods of Classical Testing Theory (CTT). It allows generating matrixes of primary testing grades and performing test results analysis; permits computation of basic statistical characteristics of the test, estimation of the latent parameters, construction of characteristic curves and informational functions. The system graphic shell was generated with the help of the package Shiny. The system utilizes modeling and analysis for testing results according to basic IRT models: Rasch, Birnbaum, Suh-Bolt, Rasch-Masters. A performance verification for algorithms and functions implemented into the system has been done by utilizing several noted statistical methods and procedures; and correct execution of these algorithms has been confirmed.

Keywords

IRT; Rasch’s model; Suh-Bolt’s model; Rasch-Masters model; matrix’ modeling of the primary grades; testing



References

W. Linden and R. Hambleton, Handbook of modern item response theory. New York: Springer, 1997. 503 с.

О. О. Диховичний, А. Ф. Дудко, "Комплексна методика аналізу якості тестів з вищої математики", Науковий часопис НПУ імені М. П. Драгоманова. Серія 2 : Комп'ютерно- орієнтовані системи навчання., №. 15, с. 139-144, 2015.

"Що передбачає апробація інструментів дослідження? Київський регіональний центр оцінювання якості освіти: веб-сайт". [Електронний ресурс]. Режим доступу:https://kievtest.org.ua/2017/04/моніторинг-початкової-освіти-запита/.

И. Н. Елисеев, "Калибровка заданий теста с использованием бутстреп-метода" Программные продукты и системы. № 2. С. 96-99. 2010.

И. Н. Елисеев, И. И. Елисеев, "Уточненный расчет параметров латентных переменных по выборкам малого объѐма", Развитие тестовых технологий в России: тезисы доклада на IX Всероссийской научно-методической конференции. М., С. 117–118, 2007.

И. Н. Елисеев, "Модель дихотомической матрицы результатов тестирования", Программные продукты и системы, № 3. С. 80-86, 2011.

Р. Х. Сафаров, О. Ю. Панищев, " Численное моделирование инвариантности оценки знанияотносительно трудности тестовых заданий в рамках модели Г. Раша" , Образовательные технологии и общество,Т.15,№1, С. 424-435, 2012

А. В. Саяпин, К. В. Сафонов, "Оценка дифференцирующей способности компьютерного теста методами имитационного моделирования" КиберЛенинка.[Електронний ресурс]. Режим доступу: https://cyberleninka.ru/article/n/otsenka-differentsiruyuschey-sposobnosti-kompyuternogo-testa- metodami-imitatsionnogo-modelirovaniya, 2012.

Available Packages. The R Project for Statistical Computing: Website. .[Електронний ресурс]. Режим доступу: https://cran.us.r-project.org.

С. М. Ермаков, Г. А. Михайлов, Статистическое моделирование, М.: Наука, 1982,296с.

Ю. М. Нейман, В. А. Хлебников , Ведение в теорию моделирования и параметризации педагогических тестов, М., 2000, 168с.

С. Рао , Линейные статистические методы и их применения. пер. с англ., М. .: Наука,, 1968, 547 с.

M. Matsumoto, T. Nishimura, "Mersenne twister: A 623-dimensionally equidistributed uniform pseudorandom number generator»" ACM Trans. on Modeling and Computer Simulations, Vol. 8, № 1, pp. 3-30, 2017.

A. J Walker, "New fast method for generating discrete random numbers with arbitrary frequencydistributions",. Electronics Letters. 10 (8), p . 127, 1974.

Г. А. Михайлов, А. В. Войтишек. Численное статистическое моделирование. Методы Монте-Карло. М.: «Академия», 2006, 368с.


REFERENCES (TRANSLATED AND TRANSLITERATED)

W. Linden and R. Hambleton, Handbook of modern item response theory. New York: Springer, 1997,503p.(in English).

O. Dykhovychnyi and A. Dudko, “The comprehensive procedure of analysis of tests in higher mathematics “, Kompiuterno-oriientovani systemy navchannia: Zb. nauk. prats NPUimeni M.P.Drahomanova, Volume 2, №15, P.139-144,2015 (in Ukrainian).

What does testing research tools mean? Kyiv Regional Center for Educational Quality Assessment: Website. . [Online].Available: https://kievtest.org.ua/2017/04/моніторинг-початкової-освіти-запита/(in Ukrainian).

I.N. Eliseev, «Calibration of items by the means Bootstrap methods», Program products and systems., № 2. P. 96-99. 2010 (in Russian).

I.N. Eliseev , I.I. Eliseev «Refined calculation of latent variables parameters by small samplies», Razvitie testovyh tehnologij v Rossii: tezisy doklada na IX Vserossijskoj nauchno-metodicheskoj konferenci. М., P.117–118, 2007 (in Russian).

I.N. Eliseev, “ Model of the dichotomius matrix of test results”, Program products and systems., № 3. P. 80-86, 2011 (in Russian).

R.H. Safarov and O.Ju. Panishhev, “ Numerical simulation of the invariance of the estimation of the Knowledges relative to difficulty of the test items within the model of G. Rasch”, Obrazovatel'nye tehnologii i obshhestvo, Т. 15, № 1, P. 424-435, 2012 (in Russian).

A.V. Sajapin and K.V. Safonov,“Measuring of differentiation ability of computer testing by methods of imitation modelling”, Cyberleninka. [Online]. Available: https://cyberleninka.ru/article/n/otsenka-differentsiruyuschey-sposobnosti-kompyuternogo-testa- metodami-imitatsionnogo-modelirovaniya, 2012. (in Russian)

Available Packages. The R Project for Statistical Computing: Website. [Online]. Available: https://cran.us.r-project.org.

S.M. Ermakov, G.A. Mihajlov, Statistical modeling, M., Nauka, 1982, (in Russian)

Iu.M. Neiman and V.A. Khlebnykov ,Introduction in the theory of modeling and parametrization of pedagogical tests, М. ,2000, 168p. (in Russian).

С. Rao , Linear statistical inference and its application, М. Nauka, 1968, 547p. (in Russian).

Matsumoto, T. Nishimura, «Mersenne twister: A 623-dimensionally equidistributed uniform pseudorandom number generator», ACM Trans. on Modeling and Computer Simulations, Vol.8, № 1, pp. 3—30, 2017, (in English).

A. J.Walker, «New fast method for generating discrete random numbers with arbitrary frequency distributions». Electronics Letters. 10 (8), p.127, 1974(in English).

H.A. Mykhailov and A.V. Voityshek, Numerical statistical modeling. Monte-Carlo methods,М.: «Аcademia», 2006, 368p. (in Russian).





Copyright (c) 2018 Oleksandr O. Dykhovychnyi, Nataliia V. Kruglova


Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.