Simulation studies are widely used for evaluating the performance of statistical methods in psychology. However, the quality of simulation studies can vary widely in terms of their design, execution, and reporting. In order to assess the quality of typical simulation studies in psychology, we reviewed 321 articles published in Psychological Methods, Behavior Research Methods, and Multivariate Behavioral

In norm-referenced psychological testing, an individual's performance is expressed in relation to a reference population using a standardized score, like an intelligence quotient score. The reference population can depend on a continuous variable, like age. Current continuous norming methods transform the raw score into an age-dependent standardized score. Such methods have the shortcoming to solely

Bayesian analysis of variance (BANOVA), implemented through R packages, offers a Bayesian approach to analyzing experimental data. A tutorial in Psychological Methods extensively documents BANOVA. This note critically examines a method for evaluating mediation using partial eta-squared as an effect size measure within the BANOVA framework. We first identify an error in the formula for partial eta-squared

Study preregistration has become increasingly popular in psychology, but its potential to restrict researcher degrees of freedom has not yet been empirically verified. We used an extensive protocol to assess the producibility (i.e., the degree to which a study can be properly conducted based on the available information) of preregistrations and the consistency between preregistrations and their corresponding

The current article introduces lagged multidimensional recurrence quantification analysis. The method is an extension of multidimensional recurrence quantification analysis and allows to quantify the joint dynamics of multivariate time series and to investigate leader-follower relationships in behavioral and physiological data. Moreover, the method enables the quantification of the joint dynamics of

The heterogeneity of outcomes in behavioral research has long been perceived as a challenge for the validity of various theoretical models. More recently, however, researchers have started perceiving heterogeneity as something that needs to be not only acknowledged but also actively addressed, particularly in applied research. A serious challenge, however, is that classical psychological methods are

This article is a guide on how to conduct mixed methods meta-analyses (sometimes called mixed methods systematic reviews, integrative meta-analyses, or integrative meta-syntheses), using an integrative approach. These aggregative methods allow researchers to synthesize qualitative and quantitative findings from a research literature in order to benefit from the strengths of both forms of analysis.

Multiple imputation (MI) is one of the most popular methods for handling missing data in psychological research. However, many imputation approaches are poorly equipped to handle a large number of variables, which are a common sight in studies that employ questionnaires to assess psychological constructs. In such a case, conventional imputation approaches often become unstable and require that the

Idiographic network models are estimated on time series data of a single individual and allow researchers to investigate person-specific associations between multiple variables over time. The most common approach for fitting graphical vector autoregressive (GVAR) models uses least absolute shrinkage and selection operator (LASSO) regularization to estimate a contemporaneous and a temporal network.

This article delves into the often-overlooked metric of percentage of variance accounted for in structural equation models (SEM). The goodness of fit index (GFI) provides the percentage of variance of the sum of squared covariances explained by the model. Despite being introduced over four decades ago, the GFI has been overshadowed in favor of fit indices that prioritize distinctions between close

A critical assumption in exploratory factor analysis (EFA) is that manifest variables are no longer correlated after the influences of the common factors are controlled. The assumption may not be valid in some EFA applications; for example, questionnaire items share other characteristics in addition to their relations to common factors. We present a computationally efficient and robust method to estimate

tatistical power is a topic of intense interest as part of proposed methodological reforms to improve the defensibility of psychological findings. Power has been used in disparate ways-some that follow and some that do not follow from definitional features of statistical power. We introduce a taxonomy on three uses of power (comparing the performance of different procedures, designing or planning studies

Despite their popularity and flexibility, questions remain regarding how to optimally solve particular unknown variables of interest using Monte Carlo simulation experiments. This article reviews two common approaches based on either performing deterministic iterative searches with noisy objective functions or by constructing interpolation estimates given fitted surrogate functions, highlighting the

Many statistical models have been proposed to examine reciprocal cross-lagged causal effects from panel data. The present article aims to clarify how these various statistical models control for unmeasured time-invariant confounders, helping researchers understand the differences in the statistical models from a causal inference perspective. Assuming that the true data generation model (i.e., causal

To date, cross-lagged panel modeling has been studied only for continuous outcomes. This article presents methods that are suitable also when there are binary and ordinal outcomes. Modeling, testing, identification, and estimation are discussed. A two-part ordinal model is proposed for ordinal variables with strong floor effects often seen in applications. An example considers the interaction between

There have been long and bitter debates between those who advocate for the use of residualized change as the foundation of longitudinal models versus those who utilize difference scores. However, these debates have focused primarily on modeling change in the outcome variable. Here, we extend these same ideas to the covariate side of the change equation, finding similar issues arise when using lagged

When starting to formalize psychological constructs, researchers traditionally rely on two distinct approaches: the quantitative approach, which defines constructs as part of a testable theory based on prior research and domain knowledge often deploying self-report questionnaires, or the qualitative approach, which gathers data mostly in the form of text and bases construct definitions on exploratory

Although the interest in latent growth models (LGMs) with categorical indicator variables has recently increased, there are still difficulties regarding the selection of estimation methods and the interpretation of model estimates. However, difficulties in estimating and interpreting categorical LGMs can be avoided by understanding the scaling process. Depending on which parameter constraint methods

Many clustering problems are associated with a particular objective criterion that is sought to be optimized. There are often several methods that can be used to tackle the optimization problem, and one or more of them might guarantee a globally optimal solution. However, it is quite possible that, relative to one or more suboptimal solutions, a globally optimal solution might be less interpretable

Response surface analysis (RSA) allows researchers to study whether the degree of congruence between two predictor variables is related to a potential psychological outcome. Here, we adapt RSA to the case in which the two predictor variables whose congruence is of interest refer to individual differences in within-person associations (WPAs) between variables that fluctuate over time. For example, a

Mediation analyses in randomized controlled trials (RCTs) can unpack potential causal pathways between interventions and outcomes and help the iterative improvement of interventions. When designing RCTs investigating these mechanisms, two key considerations are (a) the sample size needed to achieve adequate statistical power and (b) the efficient use of resources. The current study has developed closed-form

Behavioral scientists often examine the relations between two or more latent variables (e.g., how emotions relate to life satisfaction), and structural equation modeling (SEM) is the state-of-the-art for doing so. When comparing these "structural relations" among many groups, they likely differ across the groups. However, it is equally likely that some groups share the same relations so that clusters

Social and behavioral scientists are increasingly interested the dynamics of the processes they study. Despite the wide array of processes studied, a fairly narrow set of models are applied to characterize dynamics within these processes. For social and behavioral research to take the next step in modeling dynamics, a wider variety of models need to be considered. The reservoir model is one model of

McNeish et al. argue for the general use of covariance pattern growth mixture models because these models do not involve the assumption of random effects, demonstrate high rates of convergence, and are most likely to identify the correct number of latent subgroups. We argue that the covariance pattern growth mixture model is a single random intercept model. It and other models considered in their article

Machine learning explainability techniques have been proposed as a means for psychologists to "explain" or interrogate a model in order to gain an understanding of a phenomenon of interest. Researchers concerned with imposing overly restrictive functional form (e.g., as would be the case in a linear regression) may be motivated to use machine learning algorithms in conjunction with explainability techniques

The characterization of an effect size is best made in reference to effect sizes found in the literature. A random-effects meta-analysis is the systematic synthesis of related effects from across a literature, producing an estimate of the distribution of effects in the population. We propose using the estimated mean and variance from a random-effects meta-analysis to inform the characterization of

Unidimensionality is fundamental to psychometrics. Despite the recent focus on dimensionality assessment in network psychometrics, unidimensionality assessment remains a challenge. Community detection algorithms are the most common approach to estimate dimensionality in networks. Many community detection algorithms maximize an objective criterion called modularity. A limitation of modularity is that

Internet-based studies are widely used in social and behavioral health research, yet bots and fraud from "survey farming" bring significant threats to data integrity. For research centering marginalized communities, data integrity is an ethical imperative, as fraudulent data at a minimum poses a threat to scientific integrity, and worse could even promulgate false, negative stereotypes about the population

Factorial designs lend themselves to a variety of analyses with Bayesian methodology. The de facto standard is Bayesian analysis of variance (ANOVA) with Monte Carlo integration. Alternative, and readily available methods, are Bayesian ANOVA with Laplace approximation as well as Bayesian t tests for individual effects. This simulation study compared the three approaches regarding ordinal and metric

The standard statistical procedure for researchers comprises a two-step process. Before data collection, researchers perform power analyses, and after data collection, they perform significance tests. Many have proffered arguments that significance tests are unsound, but that issue will not be rehashed here. It is sufficient that even for aficionados, there is the usual disclaimer that null hypothesis

Researchers commonly use analysis of variance (ANOVA) to statistically test results of factorial designs. Performing an a priori power analysis is crucial to ensure that the ANOVA is sufficiently powered, however, it often poses a challenge and can result in large sample sizes, especially if the expected effect size is small. Due to the high prevalence of small effect sizes in psychology, studies are

In analyzing longitudinal data with growth curve models, a critical assumption is that changes in the observed measures reflect construct changes and not changes in the manifestation of the construct over time. However, growth curve models are often fit to a repeated measure constructed as a sum or mean of scale items, making an implicit assumption of constancy of measurement. This practice risks confounding

The coefficient of determination, R², also called the explained variance, is often taken as a proportional measure of the relative determination of model on outcome. However, while R² has some attractive statistical properties, its reliance on squared variations (variances) may limit its use as an easily interpretable descriptive statistic of that determination. Here, the properties of this coefficient

The interpretation of cross-effects from vector autoregressive models to infer structure and causality among constructs is widespread and sometimes problematic. I describe problems in the interpretation of cross-effects when processes that are thought to fluctuate continuously in time are, as is typically done, modeled as changing only in discrete steps (as in e.g., structural equation modeling)-zeroes

Psychologists tend to rely on verbal descriptions of the environment over time, using terms like "unpredictable," "variable," and "unstable." These terms are often open to different interpretations. This ambiguity blurs the match between constructs and measures, which creates confusion and inconsistency across studies. To better characterize the environment, the field needs a shared framework that

Joint modeling of decisions and neural activation poses the potential to provide significant advances in linking brain and behavior. However, methods of joint modeling have been limited by difficulties in estimation, often due to high dimensionality and simultaneous estimation challenges. In the current article, we propose a method of model estimation that draws on state-of-the-art Bayesian hierarchical

Propensity score analysis (PSA) is a prominent method to alleviate selection bias in observational studies, but missing data in covariates is prevalent and must be dealt with during propensity score estimation. Through Monte Carlo simulations, this study evaluates the use of imputation methods based on multiple random forests algorithms to handle missing data in covariates: multivariate imputation

We demonstrate that all conventional meta-analyses of correlation coefficients are biased, explain why, and offer solutions. Because the standard errors of the correlation coefficients depend on the size of the coefficient, inverse-variance weighted averages will be biased even under ideal meta-analytical conditions (i.e., absence of publication bias, p-hacking, or other biases). Transformation to

A currently overlooked application of the latent curve model (LCM) is its use in assessing the consequences of development patterns of change-that is as a predictor of distal outcomes. However, there are additional complications for appropriately specifying and interpreting the distal outcome LCM. Here, we develop a general framework for understanding the sensitivity of the distal outcome LCM to the

Wilson et al. (2022) compared the Likert response format to an alternative format, which they called the Guttman response format. Using a Rasch modeling approach, they found that the Guttman response format had better properties relative to the Likert response format. We agree with their analyses and conclusions. However, they have failed to mention many existing articles that have sought to overcome

Reports an error in "Comparing theories with the Ising model of explanatory coherence" by Maximilian Maier, Noah van Dongen and Denny Borsboom (Psychological Methods, Advanced Online Publication, Mar 02, 2023, np). In the article, the copyright attribution was incorrectly listed, and the Creative Commons CC BY 4.0 license disclaimer was incorrectly omitted from the author note. The correct copyright

Linde et al. (2021) compared the "two one-sided tests" the "highest density interval-region of practical equivalence", and the "interval Bayes factor" approaches to establishing equivalence in terms of power and Type I error rate using typical decision thresholds. They found that the interval Bayes factor approach exhibited a higher power but also a higher Type I error rate than the other approaches

In this response to Valente et al. (2022), I am discussing the plausibility and applicability of the proposed mediation model and its causal effects estimation for single case experimental designs (SCEDs). I will focus on the underlying assumptions that the authors use to identify the causal effects. These assumptions include the particularly problematic assumption of sequential ignorability or no-unmeasured

This article considers identification, estimation, and model fit issues for models with contemporaneous and reciprocal effects. It explores how well the models work in practice using Monte Carlo studies as well as real-data examples. Furthermore, by using models that allow contemporaneous and reciprocal effects, the paper raises a fundamental question about current practice for cross-lagged panel modeling

The use of ambulatory assessments (AAs) as an approach to gather self-reported questionnaires or self-collected biochemical data is constantly increasing to investigate the experiences, states, and behaviors of individuals and their interaction with external situational factors during everyday life. It is often implicitly assumed that data from different sampling protocols can be used interchangeably

The Bayesian highest-density interval plus region of practical equivalence (HDI + ROPE) decision rule is an increasingly common approach to testing null parameter values. The decision procedure involves a comparison between a posterior highest-density interval (HDI) and a prespecified region of practical equivalence. One then accepts or rejects the null parameter value depending on the overlap (or

Testing the presence of mediation effects is important in social science research. Recently, Bayesian hypothesis testing with Bayes factors (BFs) has become increasingly popular. However, the use of BFs for testing mediation effects is still under-studied, despite the growing literature on Bayesian mediation analysis. In this study, we systematically examine the performance of the BF for testing the

Latent change score (LCS) models within a continuous-time state-space modeling framework provide a convenient statistical approach for analyzing developmental data. In this study, we evaluate the robustness of such an approach in the context of accelerated longitudinal designs (ALDs). ALDs are especially interesting because they imply a very high rate of planned data missingness. Additionally, most

Much research in psychology relies on data from observational studies that traditionally do not allow for causal interpretation. However, a range of approaches in statistics and computational sciences have been developed to infer causality from correlational data. Based on conceptual and theoretical considerations on the integration of interventional and time-restrainment notions of causality, we set

Increasingly, psychologists make use of modern configurational comparative methods (CCMs), such as qualitative comparative analysis (QCA) and coincidence analysis (CNA), to infer regularity-theoretic causal structures from psychological data. At the same time, existing CCMs remain unable to reveal such structures in the presence of complex effects. Given the strong emphasis configurational methodology

Dishop (see record 2022-78260-001) identifies the consensus emergence model (CEM) as a useful tool for future research on emergence but argues that autoregressive models with positive autoregressive effects are an important alternative data-generating mechanism that researchers need to rule out. Here, we acknowledge that alternative data-generating mechanisms are possibility for most, if not all, nonexperimental

The comparison of idiographic network structures to determine the presence of heterogeneity is a challenging endeavor in many applied settings. Previously, researchers eyeballed idiographic networks, computed correlations, and used techniques that make use of the multilevel structure of the data (e.g., group iterative multiple model estimation and multilevel vector autoregressive) to investigate individual

Some researchers and clinicians may feel hesitant to assess sexual orientation and gender-related characteristics in youth surveys because they are unsure if youth will respond to these questions or are concerned the questions will cause discomfort or offense. This can result in missed opportunities to identify LGBTQ+ youth and address health inequities among this population. The aim of this study

Human decision behavior entails a graded awareness of its certainty, known as a feeling of confidence. Until now, considerable interest has been paid to behavioral and computational dissociations of decision and confidence, which has raised an urgent need for measurement frameworks that can quantify the efficiency of confidence rating relative to decision accuracy (metacognitive efficiency). As a unique

Latent moderated structural equation (LMS) is one of the most common techniques for estimating interaction effects involving latent variables (i.e., XWITH command in Mplus). However, empirical applications of LMS often overlook that this estimation technique assumes normally distributed variables and that violations of this assumption may lead to seriously biased parameter estimates. Against this backdrop

Methods that measure the association between two intensively measured time series are of interest to researchers studying the symmetry of behaviors during social interaction. Such methods have historically focused on aggregating the amount of symmetry across all measurement occasions. However, it is rarely expected that symmetry is present at all measurement occasions. The current method, the pairwise

Measurement invariance is an assumption underlying the regression of a latent variable on a background variable. It requires the measurement model parameters of the latent variable to be equal across the levels of the background variable. Item-specific violations of this assumption are referred to as differential item functioning and are ideally substantively explainable to warrant theoretically valid

Colliders, variables that serve as a common outcome of an independent and dependent variable, pose a major challenge in psychological research. Collider variables can induce bias in the estimation of a population relationship of interest when (a) the composition of a research sample is restricted by scores on a collider variable or (b) researchers adjust for a collider variable in their statistical

Following an extensive simulation study comparing the operating characteristics of three different procedures used for establishing equivalence (the frequentist "TOST," the Bayesian "HDI-ROPE," and the Bayes factor interval null procedure), Linde et al. (2021) conclude with the recommendation that "researchers rely more on the Bayes factor interval null approach for quantifying evidence for equivalence"

Individual differences are studied with a multitude of test instruments. Meta-analysis of tests is useful to understand whether individual differences in certain populations can be detected with the help of a class of tests. A method for the quantitative meta-analytical evaluation of test instruments with dichotomous items is introduced. The method assumes beta-binomially distributed test scores, an