Speed and Power Indices of tests and items
DOI:
https://doi.org/10.23882/rmd.23136Keywords:
Error scores, Unattempted items, Random guessing, Speed test, Power testAbstract
A simple index is proposed to measure speed or power components of a test. The index is independent of position of the items and provides necessary and sufficient condition for pure speed test and pure power test and enables testing of statistical hypothesis to infer that the test can be taken as a speed test or a power test. Similar index of an item is also proposed to reflect whether the item is a speed item or a power item. The proposed index C is a ratio such that C = 0 ⟺Pure power test and C = 1⟺Pure speed test facilitating computation of similar index of each item and statistical test of significance. Properties of the index discussed. Operational method outlined to modify a test to speed or power test. Items can be ranked with respect to such item-wise index. Identification of power items and speed items help to modify the test to a speed or power test by deleting items in stages, if speediness (or power) is not intended. Relationship between index for the test and item-wise indices derived.
References
Attali, Y. (2005). Reliability of speeded number-right multiple-choice tests. Applied Psychological Measurement, 29, 357-368. https://doi.org/10.1177/0146621605276676
Bejar, I. I. (1985). Test speededness under number-right scoring: An analysis of the Test of English as a Foreign Language. Report No. ETS-RR-85-11, Educational Testing Service. Princeton, NJ
Cintron, D. (2021). Methods for Measuring Speededness: Chronology, Classification, and Ensuing Research and Development. ETS Research Report Series, 2021(1). https://doi.org/10.1002/ets2.12337
College Entrance Examination Board (1984). The College Board technical handbook for the scholastic aptitude test and achievement tests. New York
Cronbach, L. J., & Warrington, W. G. (1951). Time limit tests: Estimating their reliability and degree of speeding. Psychometrika, 14, 167–188.
Estrada, E., Román, F. J., Abad, F. J., & Colom, R. (2017). Separating power and speed components of standardized intelligence measures. Intelligence, 61, 159-168.
Ferrando, P. J., & Lorenzo-Seva, U. (2007). An item response theory model for incorporating response time data in binary personality items. Applied Psychological Measurement, 31(6), 525–543. https://doi.org/10.1177/0146621606295197
Flinn, L., Braham, L., & Das Nair, R. (2015). How reliable are case formulations? A systematic literature review. British Journal of Clinical Psychology, 54, 266-290. https://doi.org/10.1111/bjc.12073
Gulliksen, H. (1950). Theory of Mental Tests. New York, Wiley, 177 – 198
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory (Vol. 2). SAGE Publications, Inc. Newbury Park. California 91320
Hong, M. R., and Cheng, Y. (2019). Clarifying the Effect of Test Speededness. Applied Psychological Measurement, 43(8), 611–623. https://doi.org/10.1177/0146621618817783
Huang, J. L., Curran, P. G., Keeney, J., Poposki, E. M., & DeShon, R. P. (2012): Detecting and deterring insufficient effort responding to surveys. Journal of Business and Psychology, 27, 99-114. https://doi.org/10.1007/s10869-011-9231-8
Klein Entink, R. H., van der Linden, W. J., & Fox, J. P. (2009): A Box-Cox normal model for response times. British Journal of Mathematical and Statistical Psychology, 62, 621-640.
Lord, F. (1956). A study of speed factors in tests and academic grades. Psychometrika, 21, 31-50.
Lu, Y., & Sireci, S. G. (2007). Validity issues in test speededness. Educational Measurement: Issues and Practice, 26(4), 29–37. https://doi.org/10.1111/j.1745-3992.2007.00106.x
Marcus S. Hjärne (2021). Just Enough Time to Level the Playing Field: Time Adaptation in a College Admission Test, Scandinavian Journal of Educational Research, 65(6), 941-955. https://doi.org/10.1080/00313831.2020.1788143
Maris, E. (1993). Additive and multiplicative models for gamma distributed random variables, and their application as psychometric models for response times. Psychometrika, 58, 445-469.
Meade, A. W., & Craig, S. B. (2012). Identifying careless responses in survey data. Psychological Methods, 17, 437-455. https://doi.org/10.1037/a0028085
Partchev, I., De Boeck, P., & Steyer, R. (2013). How much power and speed is measured in this test? Assessment, 20(2), 242–252. https://doi.org/10.1177/1073191111411658
Ranger, J., & Kuhn, J.T. (2012). Improving item response theory model calibration by considering response times in psychological tests. Applied Psychological Measurement, 36(3), 214–231. https://doi.org/10.1177/0146621612439796
Rindler, S. E. (1979). Pitfalls in assessing test speededness. Journal of Educational Measurement, 16(4), 261–270.
Rouder, J., Sun, D., Speckman, P., Lu, J., & Zhou, D. (2003). A hierarchical Bayesian statistical framework for response time distributions. Psychometrika, 68, 589-606.
Stafford, R. E. (1971). The speed quotient: A new descriptive statistic for tests. Journal of Educational Measurement, 8, 275–278.
Swineford, F. (1974). The test analysis manual (SR-74-06). Princeton, NJ: Educational Testing Service.
Van der Linden, W. J. (2009). Conceptual issues in response-time modeling. Journal of Educational Measurement, 46(3), 247–272. https://doi.org/10.1111/j.1745-3984.2009.00080.x
Van der Linden, W. J. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72, 287-308.
Wise, S. L., & DeMars, C. E. (2009). A clarification of the effects of rapid guessing on coefficient a: A note on Attali’s reliability of speeded number-right multiple-choice tests. Applied Psychological Measurement, 33, 488-490. https://doi.org/10.1177/0146621607304655
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Satyendra Chakrabartty
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
