Yogyakarta, August 25, 2025 – Not all respondents complete questionnaires seriously as expected by researchers. Some provide answers that appear random or careless. Various statistical methods are available to detect such response patterns. Respondents identified through these statistics as giving careless responses are referred to as misfit persons or aberrant individuals. From a psychometric perspective, what should be done with this kind of response data?
Two researchers from the Faculty of Psychology, Universitas Gadjah Mada (UGM), Muhammad Dwirifqi Kharisma Putra, S.Psi., M.Si. and Prof. Dr. Faturochman, M.A., , have just published a research article in the international journal Romanian Journal of Applied Psychology Volume 27 Number 1. The study reveals an interesting finding: the behavior of giving “careless responses” appears consistently when the same respondent answers two different questionnaires within the same test administration. The Romanian Journal of Applied Psychology is indexed by Scopus with a 2024 SJR of 0.159.
In this study, the research team used two questionnaires: MSARPW (which measures metacognitive skills in research-proposal writing) and RSES (the Rosenberg Self-Esteem Scale), both administered simultaneously through Google Form.
The analysis showed that respondents identified as misfit persons in one instrument (MSARPW) also tended to be identified as misfit persons in the second instrument (RSES), when administered at the same time (within the same Google Form). In other words, respondents who were unmotivated from the start to provide serious answers were likely to give careless responses throughout the entire test battery.
According to Muhammad Dwirifqi Kharisma Putra, this finding is significant for psychology research.
“We found that aberrant response behavior across multiple instruments is not merely coincidental. This is closely related to data collection practices in psychological research, which often involve several instruments in a single Google Form, leading to a very large number of items to answer (sometimes more than 100!). Clearly, some respondents are unmotivated from the beginning to complete the task seriously because of the sheer number of items. Therefore, we formalized a procedure we call the five-step procedure for detecting and eliminating aberrant individuals,” he explained.
The study’s findings can help applied psychology researchers detect and eliminate individuals who consistently display aberrant responding across two questionnaires. This, in turn, makes test interpretation and research conclusions more accurate by removing such aberrant respondents. This is especially valuable given that most psychology researchers often administer multiple questionnaires simultaneously.
Here are the abstract, keywords, and link to the article:
Abstract. Person-fit statistics are important statistics in modern test theory, used to identify respondents with aberrant response behaviors, statistically classified as misfit persons. This study aims to determine whether misfit persons are consistent across two different questionnaires administered simultaneously. The respondents in this study were 723 students in Indonesia. Respondents completed two instruments, namely the 24-item Metacognitive Skills Assessment in Research-Proposal Writing (MSARPW) and the 10-item Rosenberg Self-Esteem Scale (RSES), where the MSARPW was administered in the same Google Form as the RSES. Both instruments were analyzed using the Multidimensional Partial Credit Model (MPCM). The MPCM produced two person-fit statistics, which were the focus of this research, namely person Outfit z and Zh statistics. The results of the regression analysis show that the regression coefficients between Zh MSARPW and Zh RSES are 0.375 (p < .001) and 0.424 (p < .001) for Person Outfit z MSARPW and RSES, respectively. To conclude, with a significant regression coefficient, this finding shows that misfit persons in responding to the MSARPW tend to be misfit persons in responding to the RSES, which indicates that the misfit persons are consistent between different measuring instruments administered at the same administration time. Limitations and implications for future research were also noted.
Keywords: aberrant response, person-fit, outfit statistics, Rasch measurement, Zh statistic
This publication can be accessed at: https://doi.org/10.2478/rjap-2025-0006
Based on the keywords, this article is a research output that contributes to SDG 4: Quality Education.
Congratulations to Mas Rifqi and Prof. Faturochman.
Writer: Fauzi
Editor: Zufar