This article advances a simple conception of test validity: A test is valid for measuring an attribute if (a) the attribute exists and (b) variations in the attribute causally produce variation in the measurement outcomes. This conception is shown to diverge from current validity theory in several respects. In particular, the emphasis in the proposed conception is on ontology, reference, and causality, whereas current validity theory focuses on epistemology, meaning, and correlation. It is argued that the proposed conception is not only simpler but also theoretically superior to the position taken in the existing literature. Further, it has clear theoretical and practical implications for validation research. Most important, validation research must not be directed at the relation between the measured attribute and other attributes but at the processes that convey the effect of the measured attribute on the test scores.
Beliefs can be highly resilient in the sense that they are not easily abandoned in the face of counterevidence. This has the advantage of guiding consistent behavior and judgments but may also have destructive consequences for individuals, nature, and society. For instance, pathological beliefs can sustain psychiatric disorders, the belief that rhinoceros horn is an aphrodisiac may drive a species extinct, beliefs about gender or race may fuel discrimination, and belief in conspiracy theories can undermine democracy. Here, we present a unifying framework of how self-amplifying feedbacks shape the inertia of beliefs on levels ranging from neuronal networks to social systems. Sustained exposure to counterevidence can destabilize rigid beliefs but requires organized rational override as in cognitive behavioral therapy for pathological beliefs or institutional control of discrimination to reduce racial biases. Black-and-white thinking is a major risk factor for the formation of resilient beliefs associated with psychiatric disorders as well as prejudices and conspiracy thinking. Such dichotomous thinking is characteristic of a lack of cognitive resources, which may be exacerbated by stress. This could help explain why conspiracy thinking and psychiatric disorders tend to peak during crises. A corollary is that addressing social factors such as poverty, social cleavage, and lack of education may be the most effective way to prevent the emergence of rigid beliefs, and thus of problems ranging from psychiatric disorders to prejudices, conspiracy theories, and posttruth politics.
This article shows that measurement invariance (defined in terms of an invariant measurement model in different groups) is generally inconsistent with selection invariance (defined in terms of equal sensitivity and specificity across groups). In particular, when a unidimensional measurement instrument is used and group differences are present in the location but not in the variance of the latent distribution, sensitivity and positive predictive value will be higher in the group at the higher end of the latent dimension, whereas specificity and negative predictive value will be higher in the group at the lower end of the latent dimension. When latent variances are unequal, the differences in these quantities depend on the size of group differences in variances relative to the size of group differences in means. The effect originates as a special case of Simpson's paradox, which arises because the observed score distribution is collapsed into an accept-reject dichotomy. Simulations show the effect can be substantial in realistic situations. It is suggested that the effect may be partly responsible for overprediction in minority groups as typically found in empirical studies on differential academic performance. A methodological solution to the problem is suggested, and social policy implications are discussed.
In this longitudinal research, we adopt a complexity approach to examine the temporal dynamics of variables related to compliance with behavioural measures during the COVID-19 pandemic. Dutch participants (N = 2399) completed surveys with COVID-19-related variables five times over a period of 10 weeks (23 April-30 June 2020). With these data, we estimated within-person COVID-19 attitude networks containing a broad set of psychological variables and their relations. These networks display variables' predictive effects over time between measurements and contemporaneous effects during measurements. Results show (1) bidirectional effects between multiple variables relevant for compliance, forming potential feedback loops, and (2) a positive reinforcing structure between compliance, support for behavioural measures, involvement in the pandemic and vaccination intention. These results can explain why levels of these variables decreased throughout the course of the study. The reinforcing structure points towards potentially amplifying effects of interventions on these variables and might inform processes of polarization. We conclude that adopting a complexity approach might contribute to understanding protective behaviour in the initial phase of pandemics by combining different theoretical models and modelling bidirectional effects between variables. Future research could build upon this research by studying causality with interventions and including additional variables in the networks.
No abstract is provided for this article.
Previous research and methodological advice has focused on the importance of accounting for measurement error in psychological data. That perspective assumes that psychological variables conform to a common factor model. We explore what happens when data that are not generated from a common factor model are nonetheless modeled as reflecting a common factor. Through a series of hypothetical examples and an empirical reanalysis, we show that when a common factor model is misused, structural parameter estimates that indicate the relations among psychological constructs can be severely biased. Moreover, this bias can arise even when model fit is perfect. In some situations, composite models perform better than common factor models. These demonstrations point to a need for models to be justified on substantive, theoretical bases in addition to statistical ones. (PsycINFO Database Record (c) 2020 APA, all rights reserved).
Jones & Love (J&L) suggest that Bayesian approaches to the explanation of human behavior should be constrained by mechanistic theories. We argue that their proposal misconstrues the relation between process models, such as the Bayesian model, and mechanisms. While mechanistic theories can answer specific issues that arise from the study of processes, one cannot expect them to provide constraints in general.
Network models like the Ising model are increasingly used in psychological research. In a recent article published in this journal, Brusco, Steinley, Hoffman, Davis-Stober, and Wasserman (in press) provide a critical assessment of the conditions that underlie the Ising model and the eLasso method that is commonly used to estimate it. In this commentary, we show that their main criticisms are unfounded. First, where Brusco et al. suggest that Ising models have little to do with classical network models such as random graphs, we show that they can be fruitfully connected. Second, if one makes this connection it is immediately evident that Brusco et al.’s second criticism – that the Ising model requires complete population homogeneity and does not allow for individual differences in network structure – is incorrect. In particular, we establish that if every individual has their own topology, and these individual differences instantiate a random graph model, the Ising model will hold in the population. Hence, population homogeneity is sufficient for the Ising model, but it is not necessary, as Brusco et al. suggest. Third, we address Brusco et al.’s criticism regarding the sparsity assumption that is made in common uses of the Ising model. We show that this criticism is misdirected, as it targets a particular estimation algorithm for the Ising model rather than the model itself. We also describe various established and validated approaches for estimating the Ising model for networks that violate the sparsity assumption. Finally, we outline important avenues for future research.
Measurement invariance is an important prerequisite for the adequate comparison of group differences in test scores. In psychology, measurement invariance is typically investigated by means of linear factor analyses of subtest scores. These subtest scores typically result from summing the item scores. In this paper, we discuss 4 possible problems related to this common practice. Specifically, we discuss (a) nonlinearity of the latent variable to subtest relation; (b) suboptimality of the total score as a proxy for the latent variable measured through the item scores; (c) non-normality of the subtest score; and (d) differences in the nature of the latent variable at the item level as compared to the latent variable at the subtest level. Additionally, we give guidelines to overcome these problems and illustrate the issues by analysing data that pertain to a performal IQ data set.
No abstract is provided for this article.
In this article, a distinction is made between absolute and relative measurement. Absolute measurement refers to the measurement of traits on a group-invariant scale, and relative measurement refers to the within-group measurement of traits, where the scale of measurement is expressed in terms of the within-group position on a trait. Relative measurement occurs, for example, if an item induces a within-group comparison in respondents. These distinctions are discussed within the framework of measurement invariance, differentiating between absolute and relative forms of measurement invariance and bias. It is shown that items for relative measurement will produce bias as classically defined if the mean and/or variance of the trait distribution differ between groups. This form of bias, however, does not result from multidimensionality but from the fact that measurement is on a relative scale. A logistic regression procedure for the detection of relative measurement invariance and bias is proposed, as well as a model that allows for the incorporation of items for relative measurement in test analysis. Implications of the distinction between absolute and relative measurement are discussed and prove to be especially relevant for the domain of personality research.
