https://doi.org/10.24265/liberabit.2020.v26n2.05

ARTÍCULOS DE INVESTIGACIÓN

Initial psychometric evaluation of the Perseverative Thinking Questionnaire in Peruvian undergraduates

Evaluación psicométrica inicial del Cuestionario de Pensamiento Perseverativo en universitarios peruanos

Pablo D. Valencia^a

^aAsociación Peruana Contextual-Conductual de Psicología, Perú

Autor corresponsal:*pabvalenciam@gmail.com

Para citar este artículo:

Valencia, P. D. (2020). Initial psychometric evaluation of the Perseverative Thinking Questionnaire in Peruvian undergraduates. Liberabit, 26(2), e404. https://doi.org/10.24265/liberabit.2020.v 26n2.05

Abstract

Background: the Perseverative Thinking Questionnaire (PTQ) is a global measure of repetitive negative thinking, a core transdiagnostic dimension of mental health. Objectives: this study sought to examine the factor structure, reliability, and evidence of associative validity of the PTQ in Peruvian undergraduates. Method: data from 240 undergraduates (I = 20.33, 59% female) were evaluated with confirmatory factor analysis. After finding the best factor structure, reliability was estimated with coefficient omega. The correlation between the PTQ’s latent variable and cognitive fusion was examined as evidence of associative validity. Results: the PTQ proved to be essentially unidimensional since acceptable fit was obtained by the one-factor model with three correlations between errors (CFI = .945; RMSEA = .079). Reliability was high (ω = .927). The correlation between the PTQ and cognitive fusion was very large (φ= .876). Conclusion: the PTQ is an essentially unidimensional measure, thus only a global score should be computed. Associative validity should be further examined in future studies.

Keywords:factor analysis; mental health; mental processes; psychological tests; validation study.

Resumen

Antecedentes: el Cuestionario de Pensamiento Perseverativo (PTQ) es una medida global del pensamiento negativo repetitivo, una dimensión transdiagnóstica central de la salud mental. Objetivos: este estudio buscó examinar la estructura factorial, la confiabilidad y la evidencia de validez asociativa del PTQ en estudiantes universitarios peruanos. Método: los datos de 240 estudiantes universitarios (M_edad = 20.33, 59% mujeres) se evaluaron con análisis factorial confirmatorio. Después de encontrar la mejor estructura factorial, se estimó la confiabilidad con el coeficiente omega. La correlación entre la variable latente del PTQ y la fusión cognitiva se examinó como evidencia de validez asociativa. Resultados: el PTQ demostró ser esencialmente unidimensional, ya que se obtuvo un ajuste aceptable mediante el modelo de un factor con tres correlaciones entre errores (CFI = .945; RMSEA = .079). La fiabilidad fue alta (ω = .927). La correlación entre el PTQ y la fusión cognitiva fue muy grande (φ = .876). Conclusión: el PTQ es una medida esencialmente unidimensional; por lo tanto, solo se debe calcular una puntuación global. La validez asociativa debe examinarse más a fondo en futuros estudios.

Palabras clave: análisis factorial; salud mental; procesos mentales; pruebas psicológicas; estudio de validación.

Introduction

For people who suffer from emotional disorders, dwelling on the ‘quicksand of thought’ is an all-toocommon experience. Indeed, researchers have long acknowledged the fact that repetitive negative thinking (RNT) is a core feature of human emotional suffering (Ehring & Watkins, 2008; Harvey et al., 2004). RNT is characterised by the predominance of verbal processing (as opposed to imagery), negative affective valence, and abstractness (as opposed to concreteness) (Ehring & Watkins, 2008; Watkins, 2008). This thinking style hence promotes and prolongs states of discomfort, while also impairing problem-solving and interpersonal relations (Erickson et al., 2020).

Traditionally, RNT has been studied as different, diagnosis-specific constructs (Erickson et al., 2020). For example, in relation to depressive disorders, RNT has been mostly studied as rumination, which is characterised by thinking repetitively and passively about the symptoms of distress, as well as the possible causes and consequences of such symptoms (Nolen-Hoeksema et al., 2008). Rumination, as conceptualised in such a way, is usually measured using the Ruminative Response Scale (RRS), which consists of two subscales: brooding and reflection, the latter arguably being a more adaptive form of rumination (Treynor et al., 2003). Similarly, RNT in the context of generalised anxiety disorder has frequently been studied as worry, which is defined as ‘an issue whose outcome is uncertain but contains the possibility of one or more negative outcomes’ (Borkovec et al., 1983, p. 10). In the same vein as rumination, most research on worry has been conducted using a specific measure: the Penn State Worry Questionnaire (PSWQ; Meyer et al., 1990).

Despite the extensive evidence of RNT in specific diagnoses, researchers have begun to argue for a broader, transdiagnostic approach to RNT (Ehring et al., 2011; Ehring & Watkins, 2008). It has been shown, for example, that the RRS and the PSWQ largely overlap, and their items show better fit when modelled as part of a global RNT factor in addition to the specific rumination and worry factors (Hur et al., 2016; Topper et al., 2014). Moreover, this global factor shows robust associations with various forms of depression and anxiety (Arditte et al., 2016; Hur et al., 2016; McEvoy & Brans, 2013; Spinhoven et al., 2015; Topper et al., 2014). Some studies have found that the specific factors (i.e., rumination and worry controlling for global RNT) also predict depression and anxiety: Rumination is more associated with depression (Arditte et al., 2016; McEvoy & Brans, 2013; Spinhoven et al., 2015), and worry with anxiety (Hur et al., 2016; Spinhoven et al., 2015). However, these specific factors do not predict depressive symptoms when studied longitudinally (Topper et al., 2014). From a slightly different perspective, it has also been shown that RNT accounts for the overlap between depression and anxiety, both at the symptom (McLaughlin & Nolen-Hoeksema, 2011) and at the diagnosis level (Drost et al., 2014). This suggests that one possible reason why many psychological disorders overlap is their sharing of RNT as a common process.

RNT has also been studied in different populations, one of them being university students. There is increasing interest in identifying mental health predictors in this population (Auerbach et al., 2018; Duffy et al., 2019). RNT, studied as worry, predicts anxiety, depression and insomnia in students (Zvolensky et al., 2019). It is also considered as a significant mediator of the relationship between exposure to stressors and emotional distress (Bauer et al., 2020). Furthermore, when studied as rumination, RNT mediates the relationship between cyberbullying victimisation and depression (Liu et al., 2020). It can be appreciated, then, that RNT is an important variable for contemporary research on students’ mental health.

One important aspect related to the transdiagnostic approach to psychopathology is that of discriminant validity. That is, it is important to test whether RNT is distinguishable from other psychological variables. If RNT has very strong correlations with these variables (say, around .90 or higher), it could be that they are all part of a broader, overarching construct (Hong & Cheung, 2015; Mansell & McEvoy, 2017). A relevant construct for testing discriminant validity of RNT measures is cognitive fusion. It is defined as the tendency to engage in thoughts as if they were actual (direct, contingent) stimuli, so that behaviour ends up being governed by such verbal rules (Blackledge, 2015; Hayes et al., 2012). Theoretically, RNT and cognitive fusion are related yet distinct constructs, because somebody’s engaging in RNT implies that their overt behaviour is being inflexibly affected by their verbal behaviour (Ruiz et al., 2016). Thus, we should expect a moderate to strong correlation between these constructs, but no so high that they become indistinguishable.

Due to the importance of RNT research, there is a need to go beyond diagnosis-specific measures (e.g., RRS, PSWQ), and consider others that conceptualise RNT as a whole. Based on the existing evidence, Ehring et al., (2011) developed a measure that encompassed the common features rather than the differences between RNT types. They defined RNT as

a style of thinking about one’s problems (current, past, or future) or negative experiences (past or anticipated) that shows three key characteristics: (1a) the thinking is repetitive, (1b) it is at least partly intrusive, and (1c) it is difficult to disengage from. Two additional features of RNT are that (2) individuals perceive it as unproductive and (3) it captures mental capacity. (Ehring et al., 2011, p. 226)

Thus, the Perseverative Thinking Questionnaire (PTQ) contained items that covered the full scope of this definition. The original validation study showed that the best factor structure of the PTQ was that of a second-order RNT factor and three first-order factors corresponding to the three features described before: (1) Core RNT Features, (2) Perceived Unproductiveness of RNT, and (3) RNT Capturing Mental Capacity (CFI = .950 - .980, RMSEA = .038 - .057; Ehring et al., 2011). This second-order structure was also replicated in Dutch-speaking undergraduates (Ehring et al., 2012), British patients with persecutory delusions (Cernis et al., 2016), Turkish non-clinical adults (Altan-Atalay & Saritas- Atalar, 2018), and Iranian clinical and non-clinical samples (Kami et al., 2019).

It should be noted that the structure proposed by Ehring et al. (2011) was developed theoretically, not with empirical (i.e., exploratory) methods. Therefore, it is still possible that less factors are needed to explain the inter-item correlations. Take, for example, the results of Kornacka et al. (2016). Although they extracted three factors that approximately resembled the original PTQ factors, the fact that only two eigenvalues were larger than one suggests that less than three factors are needed for the sake of parsimony. Indeed, using data from a Portuguese student sample, Chaves et al. (2013) extracted only two components in their study: (1) Repetitive Thought, and (2) Cognitive Interference and Unproductiveness. This two-dimensional structure was later replicated with acceptable fit in both student and community samples (CFI_student = .958, RMSEA_student = .075, CFI_community = .955, RMSEA_community = .072; Azevedo et al., 2017).

In spite of the evidence that supports a multidimensional structure for the PTQ (be it with two or three lower-order factors), some additional data suggest that this measure is essentially unidimensional. That is, only the general RNT factor seems to be important and interpretable. For instance, the sum scores of the two components identified by Chaves et al. (2013) had a large correlation (rs = .67), suggesting that there is one general factor which underlies both dimensions. Moreover, when the strictly unidimensional model was tested in some studies, its fit was only marginally unacceptable (CFI = .920 - .960, RMSEA = .061 - .130; Altan-Atalay & Saritas-Atalar, 2018; Ehring et al., 2011). It is possible that the additional factors needed to achieve acceptable fit are of residual nature, and therefore non-substantial. Models that are composed of one general factor and several group factors are called bifactor models, and they have become widespread in recent years (Reise, 2012; Reise et al., 2010). To the best of my knowledge, only one study has applied bifactor modelling to the PTQ, which showed that, as hypothesised, this questionnaire is essentially unidimensional (Devynck et al., 2017). However, one limitation of this study is that the PTQ was shortened before being analysed with bifactor modelling. For this reason, Devynck et al. (2017) results cannot be directly extrapolated to the full PTQ.

The aim of the present study was to assess three competing models of the PTQ’s factor structure. First, I tested a bifactor model with one general RNT and three specific factors based on the original structure proposed by Ehring et al. (2011). The second model tested was also a bifactor model, but with only two specific factors as suggested by Chaves et al. (2013). For completion, a strictly unidimensional model was also tested. In case none of these models achieved acceptable fit, theoretically coherent respecifications based on modification indices would be added in an exploratory way. Finally, in order to examine associative validity, the latent correlation between the final RNT factor and a cognitive fusion latent variable was assessed. From a methodological point of view, this study’s novel approach to unidimensionality should help better understand the factor structure of the PTQ. Also, from a practical perspective, this should set the path for more sophisticated studies that, in the long run, will provide researchers and practitioners with better measures of global RNT.

Method

Participants

The sample comprised 240 psychology students (59% female) from one large public university in Lima, Peru. Ages ranged from 17 to 29 (M = 20.33, SD = 1.85). Thirty-one students (13%) came from San Juan de Lurigancho, the most populated district in Lima City. Two hundred and eleven people (89%) said that they had lived in this city for the greater part of their lives. Most participants were second (43%) or third (53%) year students. Even though this was a convenience sample whose size was not determined a priori, it was larger than the minimum sample size of 200 suggested by some popular rules of thumb (Kline, 2016).

Measures

Perseverative Thinking Questionnaire (PTQ; Ehring et al., 2011). As already mentioned, the PTQ was created as a disorder-independent measure of RNT. It comprises 15 items; however, items 2 (Thoughts intrude into my mind) and 12 (Thoughts just pop up into my mind) had almost identical phrasings when translated into Spanish. For this reason, I decided to exclude item 2 from this study, even though the original numeration was maintained to facilitate comparison. Each item is answered using a 5-point Likert scale, which ranges from 0 (never) to 4 (almost always). An existing Spanish version was used, which was developed in Colombia (F. J. Ruiz, personal communication, October 16, 2017). Since the wording of this translation was very clear and comparable to the Spanish spoken in Peru, no modification was made to the items. Details about the psychometric properties of the PTQ are presented throughout this article.

Cognitive Fusion Questionnaire (CFQ; Gillanders et al., 2014). This questionnaire was constructed as a unidimensional measure of cognitive fusion. It consists of seven items that are measured on a 7-point Likert scale (1 = never true, 7 = always true). There are no reversed items, so that all of them are averaged in order to get a total score. An existing Spanish version (Ruiz et al., 2017), which had been previously tested in Peruvian undergraduates (Valencia & Falcón, 2019), was used in this study. Reliability was high in the present data (ω = .922).

Procedure

Participants were approached collectively in their classrooms. They were given a questionnaire booklet, at the start of which an informed consent note was included. Both orally and in written form, it was stated that their participation was voluntary and anonymous, and that there would be no negative consequences in case they decided not to participate. After they finished, the booklets were collected, and participants were thanked for their time. No financial or academic reward was delivered.

Data analysis

All items were treated as continuous variables even when, strictly speaking, they are categorical. This is justified by the fact that the PTQ’s items have five response options, which is the minimum number suggested for treating a categorical variable as continuous (Rhemtulla et al., 2012). Moreover, as will be shown in the Results section, all items had low skewness and kurtosis (< 1 in absolute values). This supports the decision to analyse them as continuous variables using linear factor analysis (Ferrando & Anguiano-Carrasco, 2010).

Data were analysed in R 4.0.3 using the following packages: psych 2.0.8 (Revelle, 2020), lavaan .6-7 (Rosseel, 2012), semTools .5-3 (Jorgensen et al., 2020), semPlot 1.1.2 (Epskamp, 2019), and boot 1.3- 25 (Canty & Ripley, 2020). The operating system of the computer used for the analyses was Ubuntu 18.04.5 LTS 64-bit. All items were treated as continuous variables even when, strictly speaking, they are categorical. This is justified by the fact that the PTQ’s items have five response options, which is the minimum number suggested for treating a categorical variable as continuous (Rhemtulla et al., 2012). Moreover, as will be shown in the Results section, all items had low skewness and kurtosis (< 1 in absolute values). This supports the decision to analyse them as continuous variables using linear factor analysis (Ferrando & Anguiano-Carrasco, 2010). The first step of the analysis was to examine the descriptive statistics of each item. Specifically, the mean, standard deviation, skewness, and kurtosis were calculated. Also, since a global RNT dimension is hypothesised, the corrected item-total correlations were also computed. Next, the factor structure of the PTQ was examined via confirmatory factor analysis (CFA). The three competing models (bifactor with three specific factors, bifactor with two specific factors, and strictly unidimensional) were compared to one another. Since multivariate normality cannot be expected from Likert variables, a robust modification of the maximum likelihood estimator (MLR; Yuan & Bentler, 2000) was used for the CFA.

Apart from the χ² statistic, which is usually considered overly conservative (Byrne, 2012), two approximate fit indices were used to evaluate the model (criteria for good fit are presented in brackets): The comparative fit index (CFI > .95), and the root mean square error of approximation (RMSEA < .06; Hu & Bentler, 1999). In both cases, special formulae were used to account for the fact that they were based on a modified, robust estimator (Brosseau- Liard et al., 2012; Brosseau-Liard & Savalei, 2014). Since none of the three models was able to achieve good fit, modification indices (MI) were examined to alter the model in an exploratory way. Each modified model was compared to the previous one (Δχ²) using Satorra and Bentler’s (2001) method.

Even though internal consistency reliability has traditionally been estimated using coefficient alpha, several critics have pointed out the limitations of this approach (McNeish, 2018; Viladrich et al., 2017; Yang & Green, 2011). Instead, model-based coefficients (most prominently, coefficient omega; Dunn et al., 2014) have been proposed, which can be estimated from the results of a CFA (Savalei & Reise, 2019; Viladrich et al., 2017). Importantly, a correction is routinely applied to coefficient omega whenever there are correlated errors in the CFA model (McNeish, 2018; Viladrich et al., 2017). In this study, I calculated both a point-estimate of coefficient omega, and 95% bias-corrected and accelerated confidence intervals (BCa CI). Since correlated errors were included, the reliability coefficient was corrected accordingly.

Next, evidence of associative validity was sought. Starting from the final CFA model of the PTQ, I added a cognitive fusion latent variable measured by the CFQ’s indicators and allowed it to correlate with the RNT latent variable. Model fit was assessed as before, and the correlation between both latent variables (RNT and cognitive fusion) was examined.

Two additional procedures were performed to further analyse discriminant validity. First, I calculated the average variance extracted (AVE) of both the PTQ and the CFQ and compared these values with the squared factor correlation. Each AVE had to be larger than the squared factor correlation to assume discriminant validity (Hair et al., 2019). The second procedure I did consisted of a bifactor model with one general factor (comprising all items) and two specific factors (corresponding to the PTQ and the CFQ). The strength of the general factor was evaluated using the explained common variance (ECV) index. It has been suggested that values larger than .60 (Reise et al., 2013), .70 or .80 (Rodriguez et al., 2016b) are evidence of essential unidimensionality. Finally, the three factors were evaluated estimating their construct reliability (H) and factor determinacy (FD). Only factors with H > .70 (Rodriguez et al., 2016a) or FD > .90 (Brown, 2015; Grice, 2001) are considered important enough to be included in a structural equation model. If the ECV was large, and the specific factors’ H and FD were small, then discriminant validity could not be supported.

Results

Descriptive statistics

Table 1 presents the descriptive statistics of the items. Taking into consideration the fact that the minimum and maximum scores of each item are 0 and 4, respectively, it can be noticed that all items’ means are around the theoretical central value (i.e., 2). Also, all items have similar standard deviations, as well as low skewness and kurtosis values. Table 1 also presents the corrected item-total correlations, all of which are higher than .60.

Factor structure

I intended to test the first bifactor model (i.e., one RNT factor and the three specific factors). However, this model presented an improper solution (negative estimated variances). Similarly, the second bifactor model (i.e., one RNT factor and two specific factors) did not converge and, consequently, it was not possible to assess its fit. Given these problems, I focused on the strictly unidimensional model and modified it in an exploratory way.

The strictly unidimensional model showed mediocre fit: MLRχ²(77) = 273.55, p < .001.; Robust CFI = .892; Robust RMSEA = .109. Thus, I explored the MI and concluded that adding the correlation between the errors of items 7 (Thoughts come to my mind without me wanting them to) and 12 (Thoughts just pop into my mind) was theoretically defensible. After adding this correlation, the fit of the model improved significantly: MLRχ²(76) = 215.67, p < .001; Robust CFI = .923; Robust RMSEA = .092; Δχ²(1) = 50.01, p < .001.

Next, I examined the MI again and decided to add yet another correlation between errors—those of items 1 (The same thoughts keep going through my mind again and again) and 3 (I can’t stop dwelling on them). This is justified by the fact that item 3 refers to item 1; therefore, these items become functionally dependent. The new model thus derived again had significantly better fit: MLRχ²(75) = 196.49, p < .001; Robust CFI = .934; Robust RMSEA = .086; Δχ²(1) = 14.12, p < .001.

Even though the previous modification improved model fit to an important degree, examination of the MI suggested that there was still room for improvement. Specifically, I added the correlation between the errors of items 5 (I can’t do anything else while thinking about my problems) and 10 (My thoughts prevent me from focusing on other things). This decision made sense because both items contained words related to not being able to do things because of RNT. After this modification, model fit again showed a significant improvement: MLRχ²(74) = 175.92, p < .001; Robust CFI = .945; Robust RMSEA = .079; Δχ²(1) = 14.30, p < .001. No further modifications could be theoretically supported; therefore, the final model is the one presented in Figure 1

Reliability estimation

Model-based internal consistency reliability was first estimated for the full 14-item PTQ. This model, which was depicted in Figure 1, presents correlated errors; thus, the formula of coefficient omega was corrected accordingly. Using this modified formula, reliability was very high for the PTQ: ω = .927, 95% BCa CI [.909, .941].

Evidence of associative validity

In order to examine associative validity, I extended the model presented in Figure 1 to include a cognitive fusion factor using the CFQ’s items as indicators. This extended model had acceptable fit: MLRχ²(185) = 311.37, p < .001; Robust CFI = .958; Robust RMSEA = .056. Of special interest was the correlation between both latent variables (RNT and cognitive fusion), which was very large: φ = .876.

Further examination of discriminant validity

Given the large factor correlation, further inquiry into the PTQ’s discriminant validity was needed. The AVEs of the PTQ and the CFQ’s latent variables were .518 and .628, respectively. Both values were smaller than the squared factor correlation φ² = .876² = .767. Thus, there was not sufficient evidence of discriminant validity.

Similarly, when a bifactor model was examined, evidence for essential unidimensionality was found (ECV = .865). Moreover, construct reliability and factor determinacy were low for specific factors (H_PTQ = .387, FD_PTQ = .752, H_CFQ = .571, FD_CFQ = .825) but high for the general factor (H = .957, FD = .975).

Discussion

This study examined the factor structure of the PTQ, a general measure of RNT. None of the three hypothesised models showed good fit, either because they did not converge, presented impossible values, or because the fit itself was mediocre. However when the strictly unidimensional model was modified in an exploratory way, marginally acceptable fit was achieved after adding three correlations between errors. Moreover, the PTQ had a large latent correlation with cognitive fusion.

The fact that none of the two bifactor models achieved realistic solutions suggests two possible explanations. First, it could be that the sample size was not large enough for such complex models (Morgan et al., 2015). Second, it could also be that they were indeed wrong models, so it was impossible for the implied covariance matrix not to show unrealistic values (e.g., negative variances) (Chen et al., 2001). In any case, essential unidimensionality could not be tested through bifactor models. Therefore, I turned to the strictly unidimensional model and made modifications upon it. Specifically, I added error correlations, which is akin to modelling residual factors in a bifactor approach (Rodriguez et al., 2016a). The three error correlations added to the model were all theoretically justified.

One possible implication of the results is that, although the original lower-order (here treated as residual) factors do not hold in this sample, the global RNT factor is still predominant, just as was proposed by the PTQ’s creators (Ehring et al., 2011). Furthermore, this aligns well with the fact that, even when different factors are extracted, the large overlap between them hints at the existence of a general RNT factor (e.g., Chaves et al., 2013). Similarly, the only existing study that applied bifactor modelling to the PTQ showed that this measure is essentially unidimensional (Devynck et al., 2017). From a conceptual perspective, this implies that RNT may be best understood as a single latent variable, which goes beyond intricate distinctions between constructs such as brooding, reflection, and worry (Ehring & Watkins, 2008). In practical terms, this means that, when modelling a latent variable or computing a composite score of the PTQ, it makes more sense to focus on global RNT rather than on any specific ‘component’.

Acknowledging the fact that the PTQ is essentially unidimensional allows for further refinement. For instance, it is possible to select the items that have larger loadings on the global RNT factor and use them to create a short version of the PTQ that is clearly (strictly) unidimensional. Using shorter scales has not only the advantage of reduced administration time but also helps to define more clearly the variable being measured (Hayduk & Littvay, 2012). However, this was not done in this study due to the limited sample size, which would not have allowed cross-validation of the brief version.

One result that warrants attention is the large correlation between the PTQ (which, admittedly, measures RNT) and the CFQ (a cognitive fusion measure). Even though a large correlation was expected, the one observed (> .85) does seem to pose a threat to the discriminant validity of the PTQ. It is possible, then, that the PTQ and the CFQ are both measuring a broader construct that goes beyond the specific variables for which they were developed (Hong & Cheung, 2015; Mansell & McEvoy, 2017). This is also supported by the fact that both factors’ AVEs were lower than the squared correlation, a finding that indicates lack of discriminant validity (Hair et al., 2019). Moreover, when both latent variables were included in a bifactor model, it was observed that a global factor was the only construct of practical importance. Future studies should examine the conceptual limits of this overarching construct, being personality factors like neuroticism plausible focus for analysis (Barlow et al., 2014; Kotov et al., 2010).

This study has several limitations. First, even though RNT is a relevant variable associated to students’ well-being (Bauer et al., 2020; Liu et al., 2020; Zvolensky et al., 2019), the fact that this study did not address a more diverse sample limits the generalisability of the results. In fact, all the data collected came from psychology undergraduates only, which can lead to biased inferences if the PTQ is to be used with students from other academic fields. Second, even though the sample size was ‘large enough’ according to popular rules o thumb (Kline, 2016), it should be larger in future studies in order to obtain more stable estimates. Moreover, a larger sample size would allow tests of measurement invariance (e.g., with regard to gender), which were not possible in this study due to the limited sample size. Third, evidence of associative validity was limited to the CFQ. Future studies that examine the psychometric properties of the PTQ in Peruvian population should consider including well-known measures of disorder-specific RNT (i.e., the RRS for rumination, and the PSWQ for worry). Fourth, I want to emphasise the fact that these results are exploratory, since I did not include a confirmation sample. The final model was based on the data, not on a priori hypotheses. Thus, it is necessary that these results be replicated before use of the PTQ can be recommended.

These limitations notwithstanding, this study has some potential practical implications. First, it is expected that the PTQ will eventually help practitioners monitor changes in psychological interventions focused on RNT (e.g., Ruiz et al., 2018). Second, the PTQ should also help researchers study a global RNT construct, rather than disorder-specific constructs such as rumination and worry. This aligns well with the transdiagnostic approach to psychopathology, which seeks to identify etiological or maintaining variables that are common across disorders (Dalgleish et al., 2020).

In conclusion, these results suggest that the PTQ in essentially unidimensional, so that only a global RNT score is justified. The large correlation between the PTQ and the CFQ may point to problems regarding discriminant validity. Consequently, future studies should examine in greater detail what inferences can be made based on the PTQ (i.e., can we really interpret the PTQ’s scores as being RNT measures?)

Conflict of interest

The author declares that there are no existing or potential conflicts of interest that could affect the present study.

Ethical responsibility

This non-experimental study was conducted on a sample of human subjects. It was explained to the that the data would be used for research on ‘human emotions and thoughts’, as well as for the improvement of psychological tests. Participants read an informed consent form before answering the questionnaires. All the data were collected anonymously and voluntarily.

References

Altan-Atalay, A., & Saritas-Atalar, D. (2018). Psychometric Qualities of Turkish Version of Perseverative Thinking Questionnaire (PTQ). Journal of Rational-Emotive & Cognitive-Behavior Therapy, 36, 252-266. https://doi.org/10.1007/s10942-018-0285-7

Arditte, K. A., Shaw, A. M., & Timpano, K. R. (2016). Repetitive Negative Thinking: A Transdiagnostic Correlate of Affective Disorders. Journal of Social and Clinical Psychology, 35(3), 181-201. https://doi.org/10.1521/jscp.2016.35.3.181

Auerbach, R. P., Mortier, P., Bruffaerts, R., Alonso, J., Benjet, C., Cuijpers, P., Demyttenaere, K., Ebert, D. D., Green, J. G., Hasking, P., Murray, E., Nock, M. K., Pinder-Amaker, S., Sampson, N. A., Stein, D. J., Vilagut, G., Zaslavsky, A. M., & Kessler, R. C. (2018). WHO World Mental Health Surveys International College Student Project: Prevalence and distribution of mental disorders. Journal of Abnormal Psychology, 127(7), 623-638. https://doi.org/10.1037/abn0000362

Azevedo, J., Pereira, A. T., Amaral, A. P., Nogueira, V., Soares, M. J., Mota, D., & Macedo, A. (2017). Perseverative Thinking Questionnaire: Confirmatory factor analysis with two different samples. European Psychiatry, 41(S1), S377-S378. https://doi.org/10.1016/J.EURPSY.2017.02.401

Barlow, D. H., Sauer-Zavala, S., Carl, J. R., Bullis, J. R., & Ellard, K. K. (2014). The Nature, Diagnosis, and Treatment of Neuroticism: Back to the Future. Clinical Psychological Science, 2(3), 344-365. https://doi.org/10.1177/2167702613505532

Bauer, E. A., Braitman, A. L., Judah, M. R., & Cigularov, K. P. (2020). Worry as a mediator between psychosocial stressors and emotional sequelae: Moderation by contrast avoidance. Journal of Affective Disorders, 266, 456-464. https://doi.org/10.1016/j.jad.2020.01.092

Blackledge, J. T. (2015). Cognitive defusion in practice: a clinician’s guide to assessing, observing, supporting change in your client. Context Press.

Borkovec, T. D., Robinson, E., Pruzinsky, T., & DePree, J. A. (1983). Preliminary exploration of worry: Some characteristics and processes. Behaviour Research and Therapy, 21(1), 9-16. https://doi.org/10.1016/0005-7967(83)90121-3

Brosseau-Liard, P. E., & Savalei, V. (2014). Adjusting Incremental Fit Indices for Nonnormality. Multivariate Behavioral Research, 49(5), 460-470. https://doi.org/10.1080/00273171.2014.933697

Brosseau-Liard, P. E., Savalei, V., & Li, L. (2012). An Investigation of the Sample Performance of Two Nonnormality Corrections for RMSEA. Multivariate Behavioral Research, 47(6), 904-930. https://doi.org/10.1080/00273171.2012.715252

Brown, T. A. (2015). Confirmatory factor analysis for applied research (2.a ed.). Guilford Press.

Byrne, B. M. (2012). Structural equation modeling with Mplus: Basic concepts, applications, and programming. Routledge.

Canty, A., & Ripley, B. (2020). Boot: Bootstrap R (S-Plus) Functions (1.3-25). https://cran.r-project.org/package=boot

Cernis, E., Dunn, G., Startup, H. M., Kingdon, D., Wingham, G., Evans, N., Lister, R., Pugh, K., Cordwell, J., Mander, H., & Freeman, D. (2016). The Perseverative Thinking Questionnaire in Patients with Persecutory Delusions. Behavioural and Cognitive Psychotherapy, 44(4), 472- 481. https://doi.org/10.1017/S1352465815000533

Chaves, B., Castro, J., Pereira, A. T., Soares, M. J., Amaral, A. P., Bos, S., Madeira, N., Nogueira, V., Roque, C., & Macedo, A. (2013). Preservative Thinking Questionnaire: Validation of the Portuguese version. I Congresso Mundial de Comportamentos de Saúde Infanto-Juvenil.

Chen, F., Bollen, K. A., Paxton, P., Curran, P. J., & Kirby, J. B. (2001). Improper Solutions in Structural Equation Models: Causes, Consequences, and Strategies. Sociological Methods & Research, 29(4), 468-508. https://doi.org/10.1177/0049124101029004003

Dalgleish, T., Black, M., Johnston, D., & Bevan, A. (2020). Transdiagnostic Approaches to Mental Health Problems: Current Status and Future Directions. Journal of Consulting and Clinical Psychology, 88(3), 179-195. https://doi.org/10.1037/ccp0000482

Devynck, F., Kornacka, M., Baeyens, C., Serra, É., das Neves, J. F., Gaudrat, B., Delille, C., Taquet, P., Depraete, O., Tison, P., Sgard, F., Rousseau, A., & Romo, L. (2017). Perseverative Thinking Questionnaire (PTQ): French Validation of a Transdiagnostic Measure of Repetitive Negative Thinking. Frontiers in Psychology, 8, 2159. https://doi.org/10.3389/fpsyg.2017.02159

Drost, J., van der Does, W., van Hemert, A. M., Penninx, B. W., & Spinhoven, P. (2014). Repetitive negative thinking as a transdiagnostic factor in depression and anxiety: A conceptual replication. Behaviour Research and Therapy, 63, 177-183. https://doi.org/10.1016/J.BRAT.2014.06.004

Duffy, A., Saunders, K. E. A., Malhi, G. S., Patten, S., Cipriani, A., McNevin, S. H., MacDonald, E., & Geddes, J. (2019). Mental health care for university students: a way forward? The Lancet Psychiatry, 6(11), 885-887. https://doi.org/10.1016/S2215-0366(19)30275-5

Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399-412. https://doi.org/10.1111/bjop.12046

Ehring, T., Raes, F., Weidacker, K., & Emmelkamp, P. M. G. (2012). Validation of the Dutch Version of the Perseverative Thinking Questionnaire (PTQ-NL). European Journal of Psychological Assessment, 28(2), 102-108. https://doi.org/10.1027/1015-5759/a000097

Ehring, T., & Watkins, E. R. (2008). Repetitive Negative Thinking as a Transdiagnostic Process. International Journal of Cognitive Therapy, 1(3), 192-205. https://doi.org/10.1521/ijct.2008.1.3.192

Ehring, T., Zetsche, U., Weidacker, K., Wahl, K., Schönfeld, S., & Ehlers, A. (2011). The Perseverative Thinking Questionnaire (PTQ): Validation of a contentindependent measure of repetitive negative thinking. Journal of Behavior Therapy and Experimental Psychiatry, 42(2), 225-232. https://doi.org/10.1016/J.JBTEP.2010.12.003

Epskamp, S. (2019). SemPlot: Path Diagrams and Visual Analysis of Various SEM Packages’ Output (1.1.2). https://cran.r-project.org/package=semPlot

Erickson, T. M., Newman, M. G., & Tingey, J. L. (2020). Worry and rumination. In J. S. Abramowitz & S. M. Blakey (eds.), Clinical handbook of fear and anxiety: Maintenance processes and treatment mechanisms (pp. 133-151). American Psychological Association. https://doi.org/10.1037/0000150-008

Ferrando, P. J., & Anguiano-Carrasco, C. (2010). El análisis factorial como técnica de investigación en psicología. Papeles del Psicólogo, 31(1), 18-33.

Gillanders, D. T., Bolderston, H., Bond, F. W., Dempster, M., Flaxman, P. E., Campbell, L., Kerr, S., Tansey, L., Noel, P., Ferenbach, C., Masley, S., Roach, L., Lloyd, J., May, L., Clarke, S., & Remington, B. (2014). The Development and Initial Validation of the Cognitive Fusion Questionnaire. Behavior Therapy, 45(1), 83-101. https://doi.org/10.1016/J.BETH.2013.09.001

Grice, J. W. (2001). Computing and evaluating factor scores. Psychological Methods, 6(4), 430-450. https://doi.org/10.1037/1082-989X.6.4.430

Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis (8th ed.). Cengage Learning.

Harvey, A. G., Watkins, E. R., Mansell, W., & Shafran, R. (2004). Cognitive behavioural processes across psychological disorders. Oxford University Press.

Hayduk, L. A., & Littvay, L. (2012). Should researchers use single indicators, best indicators, or multiple indicators in structural equation models? BMC Medical Research Methodology, 12(1), 159. https://doi.org/10.1186/1471-2288-12-159

Hayes, S. C., Strosahl, K. D., & Wilson, K. G. (2012). Acceptance and Commitment Therapy: The Process and Practice of Mindful Change (2.a ed.). Guilford Press.

Hong, R. Y., & Cheung, M. (2015). The Structure of Cognitive Vulnerabilities to Depression and Anxiety: Evidence for a Common Core Etiologic Process Based on a Meta- Analytic Review. Clinical Psychological Science, 3(6), 892-912. https://doi.org/10.1177/2167702614553789

Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118

Hur, J., Heller, W., Kern, J., & Berenbaum, H. (2016). A Bi- Factor Approach to Modeling the Structure of Worry and Rumination. Journal of Experimental Psychopathology, 8(3), 252-264. https://doi.org/10.5127/jep.057116

Jorgensen, T. D., Pornprasertmanit, S., Schoemann, A. M., & Rosseel, Y. (2020). SemTools: Useful Tools for Structural Equation Modeling (0.5-3). https://cran.rproject.org/package=semTools

Kami, M., Moloodi, R., Mazidi, M., Ehring, T., Mansoori, A. K., Nodooshan, M. B., Mazinani, Z., Molavi, M.-R., & Momeni, F. (2019). Measuring repetitive thinking in Iran: Psychometric properties of Persian version of Perseverative Thinking Questionnaire. Personality and Individual Differences, 148, 101-109. https://doi.org/10.1016/J.PAID.2019.05.012

Kline, R. B. (2016). Principles and Practice of Structural Equation Modeling (4th ed.). Guilford Press.

Kornacka, M., Buczny, J., & Layton, R. L. (2016). Assessing Repetitive Negative Thinking Using Categorical and Transdiagnostic Approaches: A Comparison and Validation of Three Polish Language Adaptations of Self-Report Questionnaires. Frontiers in Psychology, 7, 322. https://doi.org/10.3389/fpsyg.2016.00322

Kotov, R., Gamez, W., Schmidt, F., & Watson, D. (2010). Linking «big» personality traits to anxiety, depressive, and substance use disorders: A meta-analysis. Psychological Bulletin, 136(5), 768-821. https://doi.org/10.1037/a0020327

Liu, C., Liu, Z., & Yuan, G. (2020). The longitudinal influence of cyberbullying victimization on depression and posttraumatic stress symptoms: The mediation role of rumination. Archives of Psychiatric Nursing, 34(4), 206-210. https://doi.org/10.1016/j.apnu.2020.05.002

Mansell, W., & McEvoy, P. M. (2017). A test of the core process account of psychopathology in a heterogenous clinical sample of anxiety and depression: A case of the blind men and the elephant? Journal of Anxiety Disorders, 46, 4-10. https://doi.org/ 10.1016/J.JANXDIS.2016.06.008

McEvoy, P. M., & Brans, S. (2013). Common Versus Unique Variance Across Measures of Worry and Rumination: Predictive Utility and Mediational Models for Anxiety and Depression. Cognitive Therapy and Research, 37(1), 183-196. https://doi.org/10.1007/s10608-012-9448-5

McLaughlin, K. A., & Nolen-Hoeksema, S. (2011). Rumination as a transdiagnostic factor in depression and anxiety. Behaviour Research and Therapy, 49(3), 186-193. https://doi.org/10.1016/j.brat.2010.12.006

McNeish, D. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412-433. https://doi.org/10.1037/met0000144

Meyer, T. J., Miller, M. L., Metzger, R. L., & Borkovec, T. D. (1990). Development and validation of the penn state worry questionnaire. Behaviour Research and Therapy, 28(6), 487-495. https://doi.org/10.1016/0005-7967(90)90135-6

Morgan, G., Hodge, K., Wells, K., & Watkins, M. (2015). Are Fit Indices Biased in Favor of Bi-Factor Models in Cognitive Ability Research?: A Comparison of Fit in Correlated Factors, Higher-Order, and Bi-Factor Models Via Monte Carlo Simulations. Journal of Intelligence, 3(1), 2-20. https://doi.org/10.3390/jintelligence3010002

Nolen-Hoeksema, S., Wisco, B. E., & Lyubomirsky, S. (2008). Rethinking Rumination. Perspectives on Psychological Science, 3(5), 400-424. https://doi.org/10.1111/j.1745-6924.2008.00088.x

Reise, S. P. (2012). The Rediscovery of Bifactor Measurement Models. Multivariate Behavioral Research, 47(5), 667-696. https://doi.org/10.1080/00273171.2012.715555

Reise, S. P., Moore, T. M., & Haviland, M. G. (2010). Bifactor Models and Rotations: Exploring the Extent to Which Multidimensional Data Yield Univocal Scale Scores. Journal of Personality Assessment, 92(6), 544- 559. https://doi.org/10.1080/00223891.2010.496477

Reise, S. P., Scheines, R., Widaman, K. F., & Haviland, M. G. (2013). Multidimensionality and Structural Coefficient Bias in Structural Equation Modeling: A Bifactor Perspective. Educational and Psychological Measurement, 73(1), 5-26.https://doi.org/10.1177/0013164412449831

Revelle, W. (2020). Psych: Procedures for Personality and Psychological Research (2.0.8). https://cran.rproject.org/package=psych

Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354-373. https://doi.org/10.1037/a0029315

Rodríguez, A., Reise, S. P., & Haviland, M. G. (2016a). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21(2), 137- 150. https://doi.org/10.1037/met0000045

Rodríguez, A., Reise, S. P., & Haviland, M. G. (2016b). Applying Bifactor Statistical Indices in the Evaluation of Psychological Measures. Journal of Personality Assessment, 98(3), 223-237. https://doi.org/10.1080/00223891.2015.1089249

Rosseel, Y. (2012). Lavaan: An R Package for Structural Equation Modeling. Journal of Statistical Software, 48(2), 1-36. https://doi.org/10.18637/jss.v048.i02

Ruiz, F. J., Flórez, C. L., García-Martín, M. B., Monroy- Cifuentes, A., Barreto-Montero, K., García-Beltrán, D. M., Riaño-Hernández, D., Sierra, M. A., Suárez-Falcón, J. C., Cardona-Betancourt, V., & Gil-Luciano, B. (2018). A multiple-baseline evaluation of a brief acceptance and commitment therapy protocol focused on repetitive negative thinking for moderate emotional disorders. Journal of Contextual Behavioral Science, 9, 1-14. https://doi.org/10.1016/j.jcbs.2018.04.004

Ruiz, F. J., Riaño-Hernández, D., Suárez-Falcón, J. C., & Luciano, C. (2016). Effect of a One-Session ACT Protocol in Disrupting Repetitive Negative Thinking: A Randomized Multiple-Baseline Design. International Journal of Psychology and Psychological Therapy, 16(3), 213-233.

Ruiz, F. J., Suárez-Falcón, J. C., Riaño-Hernández, D., & Gillanders, D. T. (2017). Psychometric Properties of the Cognitive Fusion Questionnaire in Colombia. Revista Latinoamericana de Psicología, 49(1), 80-87. https://doi.org/10.1016/J.RLP.2016.09.006

Satorra, A., & Bentler, P. M. (2001). A Scaled Difference Chi-Square Test Statistic for Moment Structure Analysis. Psychometrika, 66(4), 507-514. https://doi.org/10.1007/BF02296192

Savalei, V., & Reise, S. P. (2019). Don’t Forget the Model in your Model-Based Reliability Coefficients: A Reply to McNeish (2018). Collabra: Psychology, 5(1), 36. https://doi.org/10.1525/collabra.247

Spinhoven, P., Drost, J., van Hemert, B., & Penninx, B. W. (2015). Common Rather than Unique Aspects of Repetitive Negative Thinking are Related to Depressive and Anxiety Disorders and Symptoms. Journal of Anxiety Disorders, 33, 45-52. https://doi.org/10.1016/j.janxdis.2015.05.001

Topper, M., Molenaar, D., Emmelkamp, P. M. G., & Ehring, T. (2014). Are Rumination and Worry Two Sides of the Same Coin? A Structural Equation Modelling Approach. Journal of Experimental Psychopathology, 5(3), 363- 381. https://doi.org/10.5127/jep.038813

Treynor, W., Gonzalez, R., & Nolen-Hoeksema, S. (2003). Rumination Reconsidered: A Psychometric Analysis. Cognitive Therapy and Research, 27(3), 247-259. https://doi.org/10.1023/A:1023910315561

Valencia, P. D., & Falcón, C. (2019). Estructura factorial del Cuestionario de Fusión Cognitiva en universitarios de Lima. Interacciones. Revista de Avances en Psicología, 5(2), e167. https://doi.org/10.24016/2019.v5n2.167

Viladrich, C., Angulo-Brunet, A., & Doval, E. (2017). A Journey Around Alpha and Omega to Estimate Internal Consistency Reliability. Anales de Psicología, 33(3), 755-782. https://doi.org/10.6018/analesps.33.3.268401

Watkins, E. R. (2008). Constructive and Unconstructive Repetitive Thought. Psychological Bulletin, 134(2), 163-206. https://doi.org/10.1037/0033-2909.134.2.163

Yang, Y., & Green, S. B. (2011). Coefficient Alpha: A Reliability Coefficient for the 21st Century? Journal of Psychoeducational Assessment, 29(4), 377-392. https://doi.org/10.1177/0734282911406668

Yuan, K-H., & Bentler, P. M. (2000). Three Likelihood- Based Methods for Mean and Covariance Structure Analysis with Nonnormal Missing Data. Sociological Methodology, 30(1), 165-200. https://doi.org/10.1111/0081-1750.00078

Zvolensky, M. J., Kauffman, B. Y., Bogiaizian, D., Viana, A. G., Bakhshaie, J., & Peraza, N. (2019). Worry among Latinx college students: relations to anxious arousal, social anxiety, general depression, and insomnia. Journal of American College Health, 1-8. https://doi.org/10.1080/07448481.2019.1686004

Pablo D. Valencia

Asociación Peruana Contextual-Conductual de Psicología, Perú.

Psicólogo enfocado en la investigación en salud y en la psicometría.

ORCID: https://orcid.org/0000-0002-6809-1805

Autor corresponsal: pabvalenciam@gmail.com

Recibido: 02 de junio de 2020

Aceptado: 30 de octubre de 2020

Este es un artículo Open Access publicado bajo la licencia Creative Commons Atribución 4.0 Internacional. (CC-BY 4.0)