Bots (i.e., automated software programs that perform various tasks) and fraudulent responders pose a growing and costly threat to psychological research as well as affect data integrity. However, few studies have been published on this topic. (a) Describe our experience with bots and fraudulent responders using a case study, (b) present various bot and fraud detection tactics (BFDTs) and identify the

Intensive longitudinal designs allow researchers to study the dynamics of psychological processes in daily life. Yet, because these methods are usually observational, they do not allow strong causal inferences. A promising solution is to incorporate (micro-)randomized interventions within intensive longitudinal designs to uncover within-person (Wp) causal effects. However, it remains unclear whether

Psychological network models often feature communities: subsets of nodes that are more densely connected to themselves than to other nodes. The Spinglass algorithm is a popular method of detecting communities within a network, but it is a nondeterministic algorithm, meaning that the results can vary from one iteration to the next. There is no established method for determining the optimal solution

When multivariate intensive longitudinal data are collected from a sample of individuals, the model-based clustering (e.g., vector autoregressive [VAR] based) approach can be used to cluster the individuals based on the (dis)similarity of their person-specific dynamics of the studied processes. To implement such clustering procedures, one needs to set the temporal order to be identical for all individuals;

How to evaluate how well a psychological scale measures just one construct is a recurring problem in assessment. We introduce an index, u, of the unidimensionality and homogeneity of a scale. u is just the product of two other indices: τ (a measure of τ equivalence) and ρc (a measure of congeneric fit). By combining these two indices into one, we provide a simple index of the unidimensionality and

Reliability generalization (RG) studies frequently interpret meta-analytic heterogeneity in score reliability as evidence of differences in an instrument's measurement quality across administrations. However, such interpretations ignore the fact that, under classical test theory, score reliability depends on two parameters: true score variance and error score variance. True score variance refers to

Many psychological theories imply that the degree of congruence between two variables (e.g., self-rated and objectively measured intelligence) is related to some psychological outcome (e.g., life satisfaction). Such congruence hypotheses can be tested with response surface analysis (RSA), in which a second-order polynomial regression model is estimated and suitably interpreted. Whereas several strategies

With the advancement of artificial intelligence (AI), many AI-derived techniques have been adapted into psychological and behavioral science research, including measure development, which is a key task for psychometricians and methodologists. Ant colony optimization (ACO) is an AI-derived metaheuristic algorithm that has been integrated into the structural equation modeling framework to search for

Confirmatory bifactor models have become very popular in psychological applications, but they are increasingly criticized for statistical pitfalls such as tendency to overfit, tendency to produce anomalous results, instability of solutions, and underidentification problems. In part to combat this state of affairs, many different reliability and dimensionality measures have been proposed to help researchers

Questionnaire validation is indispensable in psychology and medicine and is essential for understanding differences across diverse populations in the measured construct. While traditional latent factor models have long dominated psychometric validation, recent advancements have introduced alternative methodologies, such as the "network framework." This study presents a pioneering approach integrating

Meta-regression is an essential meta-analytic tool for investigating sources of heterogeneity and assessing the impact of moderators. However, existing methods for meta-regression have limitations, such as inadequate consideration of model uncertainty and poor performance under publication bias. To overcome these limitations, we extend robust Bayesian meta-analysis (RoBMA) to meta-regression (RoBMA-regression)

Determining the number of factors is a decisive, yet very difficult decision a researcher faces when conducting an exploratory factor analysis (EFA). Over the last decades, numerous so-called factor retention criteria have been developed to infer the latent dimensionality from empirical data. While some tutorials and review articles on EFA exist which give recommendations on how to determine the number

Classic confirmatory factor analysis (CFA) models are theoretically superior to exploratory factor analysis (EFA) models because they specify that each indicator only measures one factor. In contrast, in EFA, all loadings are permitted to be nonzero. In this article, we show that when fit to EFA structures and other models with many cross-loadings, classic CFA models often produce excellent fit. A

Mediation analysis is widely used in psychological research to identify the relationship between independent and dependent variables through mediators. Assessing the relative importance of mediators in parallel mediator models can help researchers better understand mediation effects and guide interventions. The traditional coefficient-based measures of indirect effect merely focus on the partial effect

For cluster randomized trials and individually randomized group treatment trials that compare two treatments on a continuous outcome, designs are presented that minimize the number of subjects or the amount of research budget, when aiming for a desired power level. These designs optimize the treatment-to-control allocation ratio of study participants but also optimize the choice between the number

This essay discusses a peculiarity in institutionalized psychological measurement practices. Namely, an inherent contradiction between guidelines for how scales/tests are developed and how those scales/tests are typically analyzed. Best practices for developing scales/tests emphasize developing individual items or subsets of items to capture unique aspects of constructs, such that the full construct

Model complexity is a critical consideration when evaluating a statistical model. To quantify complexity, one can examine fitting propensity (FP), or the ability of the model to fit well to diverse patterns of data. The scant foundational research on FP has focused primarily on proof of concept rather than practical application. To address this oversight, the present work joins a recently published

To study the dimensional structure of psychological phenomena, a precise definition of unidimensionality is essential. Most definitions of unidimensionality rely on factor analysis. However, the reliability of factor analysis depends on the input data, which primarily consists of Pearson correlations. A significant issue with Pearson correlations is that they are almost guaranteed to underestimate

This study challenges three common methodological beliefs and practices. The first question examines whether ordinal reliability estimators are more accurate than continuous estimators for unidimensional data with uncorrelated errors. Continuous estimators (e.g., coefficient alpha) can be applied to both continuous and ordinal data, while ordinal estimators (e.g., ordinal alpha and categorical omega)

Intensive longitudinal designs are increasingly popular for assessing moment-to-moment changes in mood, affect, and interpersonal or health behavior. Compliance in these studies is never perfect given the high frequency of data collection, so missing data are unavoidable. Nonetheless, there is relatively little existing research on missing data within dynamic structural equation models, a recently

Compositional data are multivariate data made up of components that sum to a fixed value. Often the data are presented as proportions of a whole, where the value of each component is constrained to be between 0 and 1 and the sum of the components is 1. There are many applications in psychology and other disciplines that yield compositional data sets including Morris water maze experiments, psychological

Intensive longitudinal data, increasingly common in social and behavioral sciences, often consist of multivariate time series from multiple individuals. Dynamic factor analysis, combining factor analysis and time series analysis, has been used to uncover individual-specific processes from single-individual time series. However, integrating these processes across individuals is challenging due to estimation

Intensive longitudinal data analysis, commonly used in psychological studies, often concerns outcomes that have strong floor effects, that is, a large percentage at its lowest value. Ignoring a strong floor effect, using regular analysis with modeling assumptions suitable for a continuous-normal outcome, is likely to give misleading results. This article suggests that two-part modeling may provide

Longitudinal randomized controlled trials (RCTs) have been commonly used in psychological studies to evaluate the effectiveness of treatment or intervention strategies. Outcomes in longitudinal RCTs may follow either straight-line or curvilinear change trajectories over time, and missing data are almost inevitable in such trials. The current study aims to investigate (a) whether the estimate of average

In psychology, we often want to know whether or not an effect exists. The traditional way of answering this question is to use frequentist statistics. However, a significance test against a null hypothesis of no effect cannot distinguish between two states of affairs: evidence of absence of an effect and the absence of evidence for or against an effect. Bayes factors can make this distinction; however

During an experimental session, participants adapt and change due to learning, fatigue, fluctuations in attention, or other physiological or environmental changes. This temporal variation affects measurement, potentially reducing statistical power. We introduce a restricted randomization algorithm, permuted-subblock randomization (PSR), that boosts power by balancing experimental conditions over the

This article walks through steps to conduct a meta-analysis of Monte Carlo simulation studies. The selected Monte Carlo simulation studies focused on mixture modeling, which is becoming increasingly popular in the social and behavioral sciences. We provide details for the following steps in a meta-analysis: (a) formulating a research question; (b) identifying the relevant literature; (c) screening

Experiments with fully crossed designs are often used in experimental psychology spanning several fields, from cognitive psychology to social cognition. These experiments consist in the presentation of stimuli representing super-ordinate categories, which have to be sorted into the correct category in two contrasting conditions. This tutorial presents a linear mixed-effects model approach for obtaining

Moderated nonlinear factor analysis (MNLFA) has emerged as an important and flexible data analysis tool, particularly in integrative data analysis setting and psychometric studies of measurement invariance and differential item functioning. Substantive applications abound in the literature and span a broad range of disciplines. MNLFA unifies item response theory, multiple group, and multiple indicator

In generalized linear models, the effect of a treatment or intervention is often expressed as a ratio (e.g., risk ratio and odds ratio). There is discussion about when ratio effect measures can be interpreted in a causal way. For example, ratio effect measures suffer from noncollapsibility, that is, even in randomized experiments, the average over individual ratio effects is not identical to the (unconditional)

It is unappreciated that there are four different approaches to power analysis for detecting misspecification by testing overall fit of structural equation models (SEMs) and, moreover, that common approaches can yield radically diverging results for SEMs with many items (high p). Here we newly relate these four approaches. Analytical power analysis methods using theoretical null and theoretical alternative

sequential stopping rule (SSR) can generate a confidence interval (CI) for a standardized mean difference d that has an exact standardized width, ω. Two methods were tested using a broad range of ω and standardized effect sizes δ. A noncentral t (NCt) CI used with normally distributed data had coverages that were nominal at narrow widths but were slightly inflated at wider widths. A distribution-free

Most scientific disciplines use significance testing to draw conclusions about experimental or observational data. This classical approach provides a theoretical guarantee for controlling the number of false positives across a set of hypothesis tests, making it an appealing framework for scientists seeking to limit the number of false effects or associations that they claim to observe. Unfortunately

Simulation studies are widely used for evaluating the performance of statistical methods in psychology. However, the quality of simulation studies can vary widely in terms of their design, execution, and reporting. In order to assess the quality of typical simulation studies in psychology, we reviewed 321 articles published in Psychological Methods, Behavior Research Methods, and Multivariate Behavioral

In norm-referenced psychological testing, an individual's performance is expressed in relation to a reference population using a standardized score, like an intelligence quotient score. The reference population can depend on a continuous variable, like age. Current continuous norming methods transform the raw score into an age-dependent standardized score. Such methods have the shortcoming to solely

Bayesian analysis of variance (BANOVA), implemented through R packages, offers a Bayesian approach to analyzing experimental data. A tutorial in Psychological Methods extensively documents BANOVA. This note critically examines a method for evaluating mediation using partial eta-squared as an effect size measure within the BANOVA framework. We first identify an error in the formula for partial eta-squared

Study preregistration has become increasingly popular in psychology, but its potential to restrict researcher degrees of freedom has not yet been empirically verified. We used an extensive protocol to assess the producibility (i.e., the degree to which a study can be properly conducted based on the available information) of preregistrations and the consistency between preregistrations and their corresponding

The current article introduces lagged multidimensional recurrence quantification analysis. The method is an extension of multidimensional recurrence quantification analysis and allows to quantify the joint dynamics of multivariate time series and to investigate leader-follower relationships in behavioral and physiological data. Moreover, the method enables the quantification of the joint dynamics of

The heterogeneity of outcomes in behavioral research has long been perceived as a challenge for the validity of various theoretical models. More recently, however, researchers have started perceiving heterogeneity as something that needs to be not only acknowledged but also actively addressed, particularly in applied research. A serious challenge, however, is that classical psychological methods are

This article is a guide on how to conduct mixed methods meta-analyses (sometimes called mixed methods systematic reviews, integrative meta-analyses, or integrative meta-syntheses), using an integrative approach. These aggregative methods allow researchers to synthesize qualitative and quantitative findings from a research literature in order to benefit from the strengths of both forms of analysis.

Multiple imputation (MI) is one of the most popular methods for handling missing data in psychological research. However, many imputation approaches are poorly equipped to handle a large number of variables, which are a common sight in studies that employ questionnaires to assess psychological constructs. In such a case, conventional imputation approaches often become unstable and require that the

Idiographic network models are estimated on time series data of a single individual and allow researchers to investigate person-specific associations between multiple variables over time. The most common approach for fitting graphical vector autoregressive (GVAR) models uses least absolute shrinkage and selection operator (LASSO) regularization to estimate a contemporaneous and a temporal network.

This article delves into the often-overlooked metric of percentage of variance accounted for in structural equation models (SEM). The goodness of fit index (GFI) provides the percentage of variance of the sum of squared covariances explained by the model. Despite being introduced over four decades ago, the GFI has been overshadowed in favor of fit indices that prioritize distinctions between close

A critical assumption in exploratory factor analysis (EFA) is that manifest variables are no longer correlated after the influences of the common factors are controlled. The assumption may not be valid in some EFA applications; for example, questionnaire items share other characteristics in addition to their relations to common factors. We present a computationally efficient and robust method to estimate

tatistical power is a topic of intense interest as part of proposed methodological reforms to improve the defensibility of psychological findings. Power has been used in disparate ways-some that follow and some that do not follow from definitional features of statistical power. We introduce a taxonomy on three uses of power (comparing the performance of different procedures, designing or planning studies

Despite their popularity and flexibility, questions remain regarding how to optimally solve particular unknown variables of interest using Monte Carlo simulation experiments. This article reviews two common approaches based on either performing deterministic iterative searches with noisy objective functions or by constructing interpolation estimates given fitted surrogate functions, highlighting the

Many statistical models have been proposed to examine reciprocal cross-lagged causal effects from panel data. The present article aims to clarify how these various statistical models control for unmeasured time-invariant confounders, helping researchers understand the differences in the statistical models from a causal inference perspective. Assuming that the true data generation model (i.e., causal

To date, cross-lagged panel modeling has been studied only for continuous outcomes. This article presents methods that are suitable also when there are binary and ordinal outcomes. Modeling, testing, identification, and estimation are discussed. A two-part ordinal model is proposed for ordinal variables with strong floor effects often seen in applications. An example considers the interaction between

There have been long and bitter debates between those who advocate for the use of residualized change as the foundation of longitudinal models versus those who utilize difference scores. However, these debates have focused primarily on modeling change in the outcome variable. Here, we extend these same ideas to the covariate side of the change equation, finding similar issues arise when using lagged

When starting to formalize psychological constructs, researchers traditionally rely on two distinct approaches: the quantitative approach, which defines constructs as part of a testable theory based on prior research and domain knowledge often deploying self-report questionnaires, or the qualitative approach, which gathers data mostly in the form of text and bases construct definitions on exploratory

Although the interest in latent growth models (LGMs) with categorical indicator variables has recently increased, there are still difficulties regarding the selection of estimation methods and the interpretation of model estimates. However, difficulties in estimating and interpreting categorical LGMs can be avoided by understanding the scaling process. Depending on which parameter constraint methods

Many clustering problems are associated with a particular objective criterion that is sought to be optimized. There are often several methods that can be used to tackle the optimization problem, and one or more of them might guarantee a globally optimal solution. However, it is quite possible that, relative to one or more suboptimal solutions, a globally optimal solution might be less interpretable

Response surface analysis (RSA) allows researchers to study whether the degree of congruence between two predictor variables is related to a potential psychological outcome. Here, we adapt RSA to the case in which the two predictor variables whose congruence is of interest refer to individual differences in within-person associations (WPAs) between variables that fluctuate over time. For example, a

Mediation analyses in randomized controlled trials (RCTs) can unpack potential causal pathways between interventions and outcomes and help the iterative improvement of interventions. When designing RCTs investigating these mechanisms, two key considerations are (a) the sample size needed to achieve adequate statistical power and (b) the efficient use of resources. The current study has developed closed-form

Behavioral scientists often examine the relations between two or more latent variables (e.g., how emotions relate to life satisfaction), and structural equation modeling (SEM) is the state-of-the-art for doing so. When comparing these "structural relations" among many groups, they likely differ across the groups. However, it is equally likely that some groups share the same relations so that clusters

Social and behavioral scientists are increasingly interested the dynamics of the processes they study. Despite the wide array of processes studied, a fairly narrow set of models are applied to characterize dynamics within these processes. For social and behavioral research to take the next step in modeling dynamics, a wider variety of models need to be considered. The reservoir model is one model of

McNeish et al. argue for the general use of covariance pattern growth mixture models because these models do not involve the assumption of random effects, demonstrate high rates of convergence, and are most likely to identify the correct number of latent subgroups. We argue that the covariance pattern growth mixture model is a single random intercept model. It and other models considered in their article

Machine learning explainability techniques have been proposed as a means for psychologists to "explain" or interrogate a model in order to gain an understanding of a phenomenon of interest. Researchers concerned with imposing overly restrictive functional form (e.g., as would be the case in a linear regression) may be motivated to use machine learning algorithms in conjunction with explainability techniques

The characterization of an effect size is best made in reference to effect sizes found in the literature. A random-effects meta-analysis is the systematic synthesis of related effects from across a literature, producing an estimate of the distribution of effects in the population. We propose using the estimated mean and variance from a random-effects meta-analysis to inform the characterization of

Unidimensionality is fundamental to psychometrics. Despite the recent focus on dimensionality assessment in network psychometrics, unidimensionality assessment remains a challenge. Community detection algorithms are the most common approach to estimate dimensionality in networks. Many community detection algorithms maximize an objective criterion called modularity. A limitation of modularity is that