Reliability and Validity of the Haitian Creole PHQ-9

INTRODUCTION

Little is known about depressive symptomatology amongst Haitian men who have sex with men (MSM) in the Republic of Haiti. Haitian MSM have been a hidden and stigmatized population since the early days of the acquired immunodeficiency syndrome (AIDS) epidemic,¹ and despite 30 years of funded research on human immunodeficiency virus (HIV)/AIDS in Haiti, the emphasis has been on preventing mother-to-child transmission and clinical surveillance of the general population.² However, recent work on Haitian MSM shows that 30 % of Haitian men living with AIDS in the United States self-report male-to-male sexual contact as their primary route of HIV transmission,³ and the majority of MSM living in Haiti report a bisexual orientation, which means they are able to contribute to a generalized HIV epidemic through bridges between them and heterosexual women in Haiti.⁴ While both studies provide previously unknown information about Haitian MSM, neither study reports on the mental health status of this population.

A review of the literature shows that depression is highly prevalent amongst MSM when compared to men in the general population;⁵8 and MSM are known to be at-high risk for infection with HIV. A meta-analysis reported that HIV-positive gay men are nearly twice as likely to have experienced a recent episode of major depressive disorder as their HIV-negative counterparts,⁹ and other literature reports that depression is associated with sexual risk behavior,¹⁰16 underutilization of HIV primary care services, ¹⁷ poor adherence to antiretroviral therapy,¹⁸27 and lower self-efficacy in one’s ability to take antiretroviral therapy.²⁸29 Hence, measurement of depression amongst both MSM and HIV infected populations is warranted.

There is little information on the clinical manifestation of depression in Haitians infected with HIV and scant information on the prevalence of depression amongst persons of Haitian ancestry in general ³⁰36. The paucity of studies may be partly attributable to the absence of culturally and linguistically adapted instruments in the Haitian-Creole language ³⁷38. The few studies examining the mental health of Haitians have been primarily conducted in the US with questions administered in English ³⁰3339.

STUDY OBJECTIVES

In this study, the authors conducted a psychometric evaluation of a culturally and linguistically adapted Haitian-Creole version of the PHQ-9. The PHQ-9 is a brief self-report screening instrument for depressive symptomatology that focuses on the nine diagnostic criteria for DSM-IV depressive disorders:⁴⁰anhedonia, depressed, sleep problems, low energy, appetite problems, low self-esteem, trouble concentrating, psychomotor problem and suicide ideation. In a meta-analytical review, the PHQ-9 has been deemed acceptable for depression screening in a range of settings, countries and populations,⁴¹46 has been rigorously evaluated in multi-ethnic populations showing strong evidence of validity, reliability and utility;⁴⁷54 and its psychometric properties have been evaluated in HIV infected populations.⁴⁷5556

Therefore, the authors addressed three major research questions:

1. What is the reliability of the PHQ-9 amongst a sample of Haitian MSM?

2. What is the construct validity of the PHQ-9 in relation to HIV status in this population?

3. Are there any observed differences in item responses across sub-groups (age, educational level, HIV status and sexual orientation) that may be due to differential item functioning (DIF)?

METHODS

Study Design, Recruitment Methodology and Personnel

The Integrated Behavioral and Biological HIV Survey (IBBS) conducted between October and December 2011 used a cross-sectional design and respondent driven sampling (RDS) to recruit the hidden population of MSM throughout Haiti. Biological samples were collected by study personnel trained according to national standards in Haiti,² and training on the collection of behavioral data was conducted by PSI, a global social marketing company in Haiti. The RDS approach used a system of coupons for recruiting eligible participants, which was initialized with specifically chosen study participants (“seeds”) who assisted in the recruitment at each study site.⁵⁷ Study personnel evaluated the eligibility of potential study participants when they presented with coupons; then read to participants the informed consent script, the in-depth description of the research procedures (interview, blood sample, HIV testing) and the specific ethical considerations of the study prior to obtaining an informed consent.

Setting: Study Sites

The study was conducted across all ten administratively defined departments in Haiti, with 11 study sites across the nation where networks of MSM existed, or near an accessible safe house where screening and collection of bio-behavioral data were possible.

Participants: Inclusion/Exclusion Criteria

Study participants had to be male, aged 18 years or older, report sexual relations with a male sexual partner in the last 12 months, have lived in Haiti during the past 3 months, be in possession of a recruitment coupon, and know the first and last name of the person (“seed”) who gave them the coupon. However, study personnel did not capture the first or last name of study participants, and did not keep a list of identifiers other than for the 11 “seeds” chosen for each study site.

Study recruits were excluded from the study enrollment for the following reasons: incapable of understanding or giving clear consent (e.g., suspected of being under the influence of drugs or alcohol); already enrolled in the study; currently a peer educator; or having received a coupon from a stranger (does not know his recruiter).

An honorarium [200 Haitian Gourdes ($5 USD)] was given to all study participants who enrolled in the study even if they did not provide a blood sample, to offset their time and travel. Each enrolled participant could recruit up to three other participants; in this case, secondary compensation [100 Haitian Gourdes ($2.50 USD)] was offered for each participant recruited.

Measurement

A structured questionnaire was verbally administered to study participants, capturing information on sociodemographics, sexual behaviors, sexual orientation, number and type of sexual partners, alcohol or drug use, utilization of health services, knowledge and attitudes related to HIV, and rapid HIV testing was performed on all participants who consented. Length of time to complete the survey was approximately 45 min.

Depressive Symptomatology—PHQ-9

Depressive symptomatology was measured using the PHQ-9, which asks whether a symptom has been present more than half the time, over the past two weeks. The instrument uses a Likert type response format ranging from 0 (not at all) to 3 (everyday), and the total score ranges from 0 to 27 with five severity categories: minimal (0–4), mild (5–9), moderate (10–14), moderately severe (15–19) and severe (20–27).⁴⁰

Translation and Content Validity of the PHQ-9

Forward-backward translation of the PHQ-9 [English to Haitian-Creole] and a content validity analysis were performed by a bilingual/bicultural panel of Haitian experts (professional translator, psychiatrist, social epidemiologist and social worker). Panel members had native knowledge of the cultural beliefs of depression, and the psychiatrist and social worker were familiar with the expressions of depression in the Haitian context based on their clinical practice.³⁰ The bilingual translator performed the initial forward translation of the PHQ-9 from English to Haitian-Creole. Backward translation was performed by the social worker. Differences in translation between the original English and translated English versions were reviewed, and agreement was achieved by consensus with the psychiatrist and social epidemiologist. The vocabulary selected for the translated items incorporated expressions of depressive symptomatology that are familiar and routinely used with Haitian clients. In the final selection of terms, the panel performed a content validity analysis by examining the correspondence between items and the symptom content describing depression.

Statistical Analyses

Descriptive Statistics

Univariate statistics were generated and ceiling/floor effects were considered substantial if ≥ 20 % of subjects scored at the upper-most or bottom-most end of the PHQ-9 scale.⁵⁸59 Internal consistency reliability was assessed using Cronbach's alpha statistic.⁶⁰ Ordinary least squares and multivariable regression were used to determine the association between the PHQ-9 score and HIV status. Ordinal logistic regression, using the proportional odds model, was used to predict PHQ-9 severity categories using HIV status, while adjusting for age, education and sexual orientation. The measure of magnitude consisted of the odds ratio with 95 % confidence intervals. Stata v12 ⁶¹ and Microsoft Excel ⁶² were used to analyze the data.

Factor Analysis

The principal factor method, with promax rotation, was used to identify the unobservable ‘latent’ factors that underlie or explain the set of observed variables.⁶³65 A scree plot of the positive eigenvalues of the adjusted correlation matrix was generated to determine if one or more factors accounted for most of the common variances among the nine items.⁶⁶ Item discrimination was assessed with corrected item-total correlations.⁶⁷

Differential Item Functioning

Drawing a parallel from the field of epidemiology to the field of psychometrics, the evaluation of uniform DIF and nonuniform DIF are analogous to confounding and effect modification, respectively (Personal Communication, Crane P, 2003). Evidence of DIF was defined using optimal cutoff values of an odd’s ratio > 2.0 or < 0.50.⁶⁸70

DIF was evaluated using logistic regression to predict item responses, controlling for the total scale score (unidimensional underlying construct) across the following: dichotomous categories for HIV Status (0 = negative; 1 = positive); polychotomous categories for sexual orientation (0 = homosexual, 1 = bisexual, 2 = heterosexual and 3 = transsexual); ordinal categories for age groups (0 = 18-24 years; 1 = 25-29 years; 2 = 30-44 years) and educational levels (0 = alphabetization; 1 = primary; 2 = secondary and 3 = superior). For example, the uniform DIF model for HIV status included an individual item of the PHQ-9 as the dependent variable, with the total PHQ-9 score and HIV status as independent variables, having 2° of freedom [PHQ9-Item_i = PHQ9-Score + HIV Status]; whereas the nonuniform DIF model added an interaction term to this equation [PHQ9-Item_i = PHQ9-Score + HIV Status + (PHQ9-Score* HIV Status)], having 3 of freedom.

BIOETHICS

The study protocol was approved by the Haitian Bioethics Committee in Port-au-Prince, Haiti. The Office of Human Research Protection at the Harvard School of Public Health granted an institutional review board (IRB) exemption waiver for analyses of the de-identified data set.

RESULTS

Factor Analysis and Differential Item Functioning

A scree plot was used to establish the number of distinct factors (Fig. 1). Results show one dominant dimension with a large decrease between the first and second eigenvalues and small decreases thereafter, suggesting unidimensionality, required to assess DIF. Factor loadings ranged from 0.45 to 0.60 (i.e., above the 0.40 cutoff) (Table 2). The corrected item-total correlations show how each individual item is correlated with the scale computed only from the other eight items, and this ranges from 0.38 to 0.52. “Low energy” was the most highly endorsed item (mean 1.14).

Figure 1

Scree Plot of Eigenvalues after Factor Analysis

Table 2

Factor Analysis

Item number	PHQ-9 Questions	Factor Loading	Item-total Correlation	Item Mean	Item SD
1	Anhedonia	0.45	0.40	0.59	0.76
2	Depressed	0.59	0.53	0.73	0.87
3	Sleep problems	0.50	0.46	0.82	0.91
4	Low energy	0.53	0.48	0.91	0.95
5	Appetite problems	0.49	0.45	0.89	0.97
6	Low self esteem	0.60	0.53	0.84	0.91
7	Trouble concentrating	0.55	0.50	0.69	0.86
8	Psychomotor problems	0.55	0.50	0.62	0.89
9	Suicide ideation	0.55	0.44	0.47	0.80

Results show no evidence of DIF across HIV status, age, education or sexual orientation (Table 3). We purposefully selected an effect size cut-point (odds ratio > 2.0 or < 0.50) to define a meaningful level of item bias, similar to other authors.⁶⁸70 We did so to take the emphasis off the p value and focus on the measure of association; and the results in Table 3 show no odds ratio that meets the effect size cut-point, which suggests no evidence of DIF.

Table 3

Differential Item Functioning Across Age, Education, Sexual Orientation and HIV Status

Age							Education
	Uniform			Nonuniform			Uniform			Nonuniform
	OR	CI	p	OR	CI	p	OR	CI	p	OR	CI	p
Q1	1.09	(0.91–1.32)	0.35	1.00	(0.96–1.05)	0.48	1.71	(1.37–2.13)	< 0.001	1.05	(1.00–1.09)	< 0.0
Q2	1.19	(0.99–1.43)	0.06	1.02	(0.98–1.07)	0.31	0.96	(0.74–1.23)	0.73	1.03	(0.97–1.10)	0.26
Q3	1.04	(0.87–1.25)	0.65	1.00	(0.96–1.04)	0.91	1.13	(0.91–1.39)	0.26	1.00	(0.96–1.05)	0.88
Q4	1.00	(0.83–1.21)	0.94	1.03	(0.98–1.08)	0.21	0.83	(0.67–1.02)	0.08	1.00	(0.95–1.15)	0.91
Q5	1.03	(0.86–1.23)	0.72	1.03	(0.99–1.08)	0.11	1.12	(0.91–1.38)	0.28	1.02	(0.97–1.06)	0.45
Q6	1.00	(0.82–1.22)	0.96	0.95	(0.91–0.99)	0.02	0.90	(0.71–1.12)	0.34	1.00	(0.96–1.05)	0.85
Q7	0.84	(0.68–1.04)	0.12	0.95	(0.90–0.99)	<0.03	1.22	(0.99–1.51)	0.06	0.99	(0.94–1.03)	0.6
Q8	1.01	(0.82–1.24)	0.92	0.98	(0.93–1.03)	0.54	0.81	(0.64–1.01)	0.06	0.98	(0.93–1.04)	0.62
Q9	1.12	(0.92-1.37)	0.23	1	(0.95–1.04)	0.97	0.72	(056–0.93)	< 0.01	0.96	(0.91–1.01)	0.19
Sexual Orientation							HIV Status
	Uniform			Nonuniform			Uniform			Nonuniform
	OR	CI	p	OR	CI	p	OR	CI	p	OR	CI	p
Q1	1.16	(0.88–1.53)	0.29	0.96	(0.90–1.02)	0.22	1.00	(0.69–1.45)	0.99	1.00	(0.90–1.05)	0.53
Q2	1.11	(0.84–1.46)	0.46	1.00	(0.94–1.08)	0.84	1.00	(0.69–1.46)	0.96	1.01	(0.92–1.10)	0.81
Q3	0.86	(0.66–1.12)	0.27	1.02	(0.96–1.08)	0.58	1.51	(1.08–2.12)	0.02	0.97	(0.89–1.06)	0.49
Q4	0.81	(0.62–1.04)	0.1	0.96	(0.90–1.02)	0.16	1.02	(0.71–1.48)	0.89	0.97	(0.89–1.05)	0.49
Q5	0.90	(0.69–1.17)	0.4	0.98	(0.92–1.04)	0.59	0.85	(0.61–1.19)	0.36	1.02	(0.94–1.09)	0.64
Q6	0.85	(0.65–1.11)	0.24	0.99	(0.93–1.05)	0.75	1.07	(0.76–1.52)	0.69	0.98	(0.90–1.67)	0.68
Q7	1.02	(0.77–1.34)	0.90	0.99	(0.92–1.05)	0.72	0.66	(0.44–0.98)	0.04	0.94	(0.87–1.01)	0.12
Q8	1.08	(0.81–1.44)	0.61	0.96	(0.90–1.03)	0.31	0.98	(0.69–1.40)	0.93	1.04	(0.95–1.14)	0.41
Q9	1.24	(0.91–1.69)	0.17	0.96	(0.89–1.02)	0.24	1.03	(0.67–1.59)	0.87	0.92	(0.84–1.00)	0.07

Uniform DIF results have 2° of freedom. Nonuniform DIF results have 3° of freedom

Participant Characteristics

A total of 1,080 men who have sex with men participated in the study. After data entry, 52 questionnaires were removed from analysis, either because of very incomplete data or non-eligibility. Of the remaining 1,028 MSM, the mean age is 23.2 years (SD ± 5.2, range 18–45). Almost 60 % (N = 615) of the study participants completed secondary education; and sexual orientation is reported as homosexual (45.2 %), bisexual (47.4 %), heterosexual (4.7 %) and transsexual (1.9 %).

Scale Statistics, Reliability and Differential Item Functioning

Univariate statistics show a mean depression score (=6.6) and the prevalence of moderately severe and severe symptomatology is 7.1 % (higher score indicating greater severity of depression) (Table 1). There were no ceiling or floor effects observed, with 8.9 % (n = 91) of study participants having a minimum score of 0, and 0.001 % (n = 1) with a maximum score of 27. The Cronbach coefficient alpha for the PHQ-9 is α = 0.78, confirming a moderately high level of internal consistency. The reliability coefficient suggests that a large proportion of the scale’s total variance is attributed to a common source.

A stratified analysis across age, education, and sexual orientation assessed representative reliability. Reliability across age groups (18–24, 25–29, and 30–44) ranges from α = 0.75 to α = 0.79. Across education, for those receiving more formal education (primary, secondary and superior), reliability ranges from α = 0.77 to α = 0.80. For the subgroup achieving less than formal education (alphabetization), reliability is lowest at α = 0.48; however, this is a sample of only 13 persons. Across sexual orientation, reliability ranges from α = 0.61 to α = 0.82.

Factor Analysis and Differential Item Functioning

A scree plot was used to establish the number of distinct factors (Fig. 1). Results show one dominant dimension with a large decrease between the first and second eigenvalues and small decreases thereafter, suggesting unidimensionality, required to assess DIF. Factor loadings ranged from 0.45 to 0.60 (i.e., above the 0.40 cutoff) (Table 2). The corrected item-total correlations show how each individual item is correlated with the scale computed only from the other eight items, and this ranges from 0.38 to 0.52. “Low energy” was the most highly endorsed item (mean 1.14).

Figure 1

Scree Plot of Eigenvalues after Factor Analysis

Results show no evidence of DIF across HIV status, age, education or sexual orientation (Table 3). We purposefully selected an effect size cut-point (odds ratio > 2.0 or < 0.50) to define a meaningful level of item bias, similar to other authors.⁶⁸70 We did so to take the emphasis off the p value and focus on the measure of association; and the results in Table 3 show no odds ratio that meets the effect size cut-point, which suggests no evidence of DIF.

Construct Validity: Relationship Between PHQ-9, HIV Status and Sexual Orientation

Using ordinary least squares regression, results showed that the PHQ-9 depression score was significantly higher for participants testing positive for HIV (p = 0.002), and this remained significant in a multivariable model adjusting for age and educational attainment. The depression score was also noted to have an inverse relationship with education (p = 0.04), suggesting that participants with lower educational attainment have higher depression scores.

When combined categories for the PHQ-9 of moderately severe and severe symptoms are stratified across HIV status, the prevalence of depressive symptomatology is 6.2 % vs. 12.4 % for HIV-negative versus HIV-positive study participants (OR = 2.2, 95 % CI 1.3–3.7), respectively (Table 1). These findings suggests that HIV-positive Haitian MSM are twice as likely to screen for symptoms that are indicative of major depression, compared to their HIV-negative counterparts in this study population.

DISCUSSION

To our knowledge, this is the first study to investigate the prevalence of depressive symptoms amongst Haitian MSM and also the first study to examine the psychometric properties of the translated version of the Haitian-Creole Patient Health Questionnaire (PHQ-9) for use with Haitian populations. There are three major findings.

First, there is strong evidence of psychometric adequacy for the translated measure. In our study, the properties of the PHQ-9 are similar to those reported for the PHQ-9 administered in western Kenya.⁴⁷ The scree plot indicates one dominant dimension between first and second eigenvalues; low energy is the most highly endorsed item; factor loadings range from 0.45 to 0.60; internal consistency reliability is α = 0.78; and the instrument shows construct validity across HIV status. However, differences in floor and ceiling effects are negligible, which is similar to the findings in the study conducted in Kenya,⁴⁷ but differs significantly from results reported by Crane, Gibbons, Willig et al., (2010) who found prominent floor effects.⁵⁵ Also, there was no evidence of DIF across participant subgroups in the sample of Haitian MSM by age, education and sexual orientation. This also differs from the results reported by Crane, Gibbons, et al. (2010), whose findings suggested there are three individual items showing evidence of DIF with respect to race (African-American vs. White), sex and age.⁵⁵

Second, HIV-positive Haitian MSM are twice as likely to screen positive for moderately severe and severe depressive symptoms compared to their HIV-negative counterparts, similar to the meta-analytical findings reported by Ciesla and Roberts (2001).⁹ Implications of these findings suggest that prioritization of funding to support psychological services for HIV-positive Haitian men is needed in this setting. To date, few tailored programs have been rigorously designed to address the special needs of MSM in Haiti.

Finally, the prevalence of moderately severe and severe depressive symptomatology is 7.1 % in this study population (Table 1), and this rate is remarkably similar to the 12-month prevalence (7.3 %) of any psychiatric disorder identified amongst Haitians living in the US, which was measured using the World Mental Health Composite International Diagnostic Interview (WMH-CIDI).⁷¹ These findings suggest that susceptibility to mental illness amongst Haitians in the homeland may be similar to that of Haitians in the Diaspora. Thus, results highlight the importance of future research to enhance our understanding and the variations within the respective larger social contexts of these populations.

LIMITATIONS

There are several study limitations. First, due to the highly selected nature of the sample (and the recruitment strategy using respondent-drive sampling), study results cannot be generalized to the larger Haitian population that would include women, older adults, individuals without HIV and/or who are heterosexual.

Second, criterion validity was not tested because this is the first translation of a Haitian-Creole instrument measuring depression. However, there is no translated version of a structured psychiatric interview for Haitian-Creole speakers. Given the dearth of research on the mental health of Haitians, future studies should identify the operating characteristics (optimal cutoff values, sensitivity and specificity) of the translated version of this instrument.

Third, construct validity was weakly tested against only one variable—HIV status—rather than comparing with constructs more robustly aligned with depression. To our knowledge, no other validated measures of depression exist in the Haitian-Creole language to assess convergence, and data on other domains aligned with depression, such as social/work/role functioning, were not captured in this study to assess correlations.

CONCLUDING REMARKS

This manuscript provides evidence supporting the PHQ-9 as a valid measure for assessing depressive symptomatology in a sample of Haitian MSM. Given the unique health risk factors and social vulnerabilities in this population, future research is necessary to examine the predictive validity of depression for subsequent health behaviors or clinical outcomes among Haitian MSM. For example, depressive symptomatology as assessed by the PHQ-9 might predict future engagement in sexual risk behaviors, HIV or sexually transmitted infections (STI) acquisition, substance use, or psychiatric outcomes in this population. This measure might thus be an appropriate tool for screening and targeting preventive interventions for members of this population. Furthermore, interventions to address and reduce depressive symptomatology among MSM in Haiti are warranted. In this sample, nearly one-quarter of MSM reported signs of moderate to severe depressive symptoms, and HIV-positive MSM had a twofold greater odds of reporting depressive symptoms compared with HIV-negative MSM. Therefore, validation of this measure provides a useful tool for future research, which is needed to screen, identify, and reduce depression in this highly vulnerable population. The study authors recommend that the measure be evaluated for its adequacy with the general population in Haiti, which is much needed for both research and clinical services.