Measuring shame and guilt by self-report questionnaires: A validation study

Abstract

Quantitative assessment of shame and guilt using self-report questionnaires can help to understand the role of these emotions in various mental disorders. However, shame and guilt measures have predominantly been tested among healthy subjects that usually show low levels of guilt and shame. Thus, little is known about the comparative validity of different shame and guilt questionnaires in a population of shame- and guilt-prone persons with mental illness as compared to healthy subjects.

This study used the Test of Self-Conscious Affect (TOSCA-3), the Personal Feelings Questionnaire (PFQ-2) and the Experiential Shame Scale (ESS) among 60 women with borderline personality disorder (BPD) and 60 healthy women. Intercorrelations of shame-proneness, guilt-proneness and state shame as well as their correlations with self-efficacy, empowerment, state and trait-anxiety, experiential avoidance, depression, and general psychopathology were assessed.

In both groups, shame-proneness was moderately related to guilt-proneness, both as assessed by the TOSCA-3 and the PFQ-2. For the TOSCA-3, among healthy subjects shame-proneness was significantly correlated with other constructs while guilt-proneness was not. This difference turned largely insignificant among women with BPD. For the PFQ-2, shame- and guilt-proneness showed similar correlational patterns with other constructs in both groups. The guilt-proneness scale of the TOSCA-3 showed poor internal consistency. State shame (ESS) was strongly related to state anxiety in both groups, and its correlations with other constructs were similar to state anxiety.

The discriminant validity of the TOSCA-3 to distinguish between shame- and guilt-proneness may be diminished in clinical samples. The measure of state shame (ESS) showed a large overlap with state anxiety.

Introduction

Lewis (1971) in her influential contribution “Shame and Guilt in Neurosis" introduced the distinction that in shame the focus is on the global self, while in guilt the focus is on a specific behavior. Ever since researchers have remained interested in measuring shame and guilt. Different types of shame and guilt questionnaires have been developed that have specific qualities: First, shame- and guilt-proneness can be assessed by global self-report wordlist-questionnaires such as the Personal Feelings Questionnaire-2 (PFQ-2; Harder and Zalma, 1990). Subjects read words like “remorse" or “feeling ridiculous" and indicate how often they experience these feelings. These word-list measures have high face validity but have been criticized for two reasons (Tangney and Dearing, 2002a): First, they heavily depend on the subject's abstract verbal skills who should be able to distinguish between shame and guilt in order to understand the scales' content; second, by filling in the wordlist subjects make global statements about themselves without a situational context. This may be adequate to measure shame-proneness, but may be problematic in measuring guilt-proneness because guilt is usually associated with behavior in a specific situation and therefore likely limits the potential of word-list measures to differentiate between shame-proneness and guilt-proneness.

A second way to assess shame- and guilt-proneness is scenario-based self-report measures such as the Test of Self-Conscious Affect (TOSCA-3; Tangney et al., 2000). Here, several everyday-life situations are described in which things went wrong. For each situation, possible reactions are described, one each for a shame- or guilt-reaction, and subjects rate how likely they were to react in each way. An advantage is that scenario-based measures rely less heavily on a subject's verbal skills because one does not need to distinguish “shame" and “guilt" in the abstract — in fact, the words shame and guilt are not used in the TOSCA-3. This also lowers the threshold to report shame and reduces denial of shame or self-representational strategies. However, scenario-based measures were criticized for their lower internal consistency as compared to wordlist-questionnaires and the limited choice of guilt- and shame-inducing situations (Tangney and Dearing, 2002a) that may be less appropriate especially for people with mental illness who may encounter them less frequently, e.g. scenarios taken from a work environment. In addition, the TOSCA's construct validity has been criticized because it conceptualizes guilt-proneness as a very adaptive, reparative reaction style (Harder, 1995) and its guilt-proneness scale was not related to a latent guilt construct as were other guilt measures (Ferguson and Crowley, 1997).

The above mentioned instruments measure trait emotions, i.e. shame- and guilt-proneness. On the other hand, state shame may be measured by self-report questionnaires such as the Experiential Shame Scale (ESS; Turner, 1998). The ESS is a brief questionnaire that aims to capture a momentary shame reaction and its physical, emotional and cognitive components. The ESS does not name shame explicitly and may therefore reduce avoidance to report shame and the impact of self-representational distortions. Since the ESS is designed to measure state shame, it is important to assess its discriminative validity to differentiate state shame from other related negative emotional states such as state anxiety. To our knowledge the ESS has not been compared to a measure of state anxiety so far.

Lewis' distinction between shame and guilt has been supported by numerous qualitative and quantitative studies over the last decades (Tangney and Dearing, 2002e). Accordingly, guilt often leads to reparative, adaptive, more empathic behavior while shame has been linked to anger attacks as well as avoidant and less empathic behavior (Eisenberg, 2000, Tangney and Dearing, 2002d). The relation of guilt and shame to psychopathology has been controversially debated (Harder et al., 1992, Tangney and Dearing, 2002c). In most empirical studies, both shame and guilt were positively related to psychopathology and interpersonal difficulties. However, since shame and guilt are closely related emotions, researchers have often analyzed correlations of “shame-free" guilt, i.e. guilt residuals controlled for shame; and vice versa “guilt-free" shame, i.e. shame residuals controlled for guilt. In these analyses, “shame-free" guilt was weakly or not at all related to psychopathology, while “guilt-free" shame was positively correlated with psychopathology (Harder and Greenwald, 1999, Tangney and Dearing, 2002c).

So far, shame and guilt measures were mainly studied among populations of healthy college students. This is surprising given the fact that shame-proneness is a central negative emotion and cornerstone of their psychopathology for many people with mental illness. Women with borderline personality disorder (BPD) are such a group especially high in shame-proneness (Rüsch et al., in press). Since shame and guilt are, to different degrees, related to many aspects of psychopathology, self-esteem and various dysfunctional behaviors, the validity of shame and guilt measures needs to be assessed in samples of both healthy controls and persons with psychiatric disorders. We are aware of only one previous study that compared the TOSCA and the PFQ-2 to assess shame- and guilt-proneness among individuals with mental illness (Averill et al., 2002). However, that study investigated a diagnostically heterogeneous sample and did not include a comparison group of healthy subjects, and further used the now obsolete first version of the TOSCA. For our study, we chose women with BPD as an exemplary group of highly shame-prone persons with mental illness who react with feelings of shame or guilt in a broad range of situations. In our view, it was preferable to study a diagnostically homogeneous group since individuals with different diagnoses may not only differ in their shame- and guilt-proneness but also in responding to questionnaires, e.g. due to cognitive deficits in subjects with schizophrenia.

In our study, we compared the differential correlations of shame- versus guilt-proneness and state shame with different constructs among both healthy women and women with borderline personality disorder as a comparison group of relatively more shame-prone persons with mental illness. The superordinate aim of this study was to assess the construct validity of different shame and guilt measures in women with BPD as compared to healthy controls.

Specifically, we investigated the following constructs:

Self-efficacy, which implies the belief of being able to lead an active and self-determined life and of being able to control challenging environmental demands (Schwarzer, 1993) and therefore was expected to be inversely related to shame measures. In a recent study, “guilt-free" shame-proneness as assessed by the TOSCA's first version was negatively related to self-efficacy in problem-solving while “shame-free" guilt-proneness was positively related to self-efficacy (Covert et al., 2003).

Empowerment, a construct that includes self-worth, self-efficacy, and the optimism to exert control over one's life (Rogers et al., 1997). Since shame-proneness implies the tendency to devalue the self and is usually accompanied by a feeling of helplessness, we expected shame measures to be negatively related to empowerment.

Shame measures were expected to be positively correlated with state and trait-anxiety, general psychopathology, and experiential avoidance since clinical and empirical literature has linked shame to different aspects of psychopathology (Lewis, 1971, Harder and Zalma, 1990, Eisenberg, 2000, Tangney and Dearing, 2002c). Experiential avoidance can be conceptualized as the unwillingness of a person to remain in contact with private experiences such as emotions, thoughts or memories (Hayes et al., 2004). Since shame is usually linked to the impulse to leave the shame-inducing situation or to “disappear”, we expected it to be positively related to experiential avoidance.

In previous research, shame was more strongly related to psychopathology and dysfunctional behavior than guilt. Therefore we expected shame-proneness to be related to the mentioned constructs to a higher degree than guilt-proneness (Eisenberg, 2000). For the assessment of correlations with shame- versus guilt-proneness, it is relevant to consider residual correlations with “shame-free guilt-proneness” and “guilt-free shame-proneness” mentioned above since shame-proneness and guilt-proneness were expected to show a considerable overlap.

This study aimed to assess the internal consistency of the shame and guilt measures and to test the following three hypotheses, separately among healthy women and among women with BPD:

• That the self-report measures used in this study (TOSCA-3 and PFQ-2) can discriminate between shame-proneness and guilt-proneness, i.e.

  • Shame-proneness is only moderately related to guilt-proneness in each scale.

  • Shame-proneness is related more negatively than guilt-proneness to self-efficacy and empowerment.

  • Shame-proneness is related more positively than guilt-proneness to trait-anxiety, experiential avoidance, depression, and general psychopathology.

• That a scenario-based measure of shame- and guilt-proneness (TOSCA-3) discriminates more consistently between shame- and guilt-proneness than a word-list measure of shame- and guilt-proneness (PFQ-2).

• That state shame (ESS) can be distinguished from state anxiety (STAI-X1).