Co-calibration of two self-reported measures of adherence to antiretroviral therapy

Introduction

Detecting and addressing sub-optimal adherence to antiretroviral therapy (ART) is a crucial aspect of HIV clinical care. Identifying a widely accepted, easily implemented, standard approach to assessment of ART adherence in HIV care settings and clinical studies has remained challenging(Bangsberg, 2006), leading to the use of multiple instruments across studies.

Harmonizing self-reported ART adherence data from multiple studies that use different measures would allow data to be combined to address questions strengthened by having larger sample sizes. However, harmonizing studies that use different measures is difficult without a co-calibration formula to convert among validated scales. Co-calibration would also facilitate interpretation of findings across studies that used different measures by enhancing our understanding of the properties of different individual measures relative to each other.

Patient self-reported measures remain the most common approach to assessing adherence but vary in their question phrasing, recall periods, and response items(Stirratt et al., 2015). We conducted this study to co-calibrate data from two of the most accepted and widely used approaches to measuring ART adherence among persons living with HIV (PLWH); items from the AIDS Clinical Trials Group (ACTG) questionnaire and the Visual Analog Scale (VAS) adherence measure(Amico et al., 2006; Chesney et al., 2000; Simoni et al., 2006). The primary aim of this study was to co-calibrate ART adherence data among PLWH from multiple studies to the same metric to evaluate the validity of the conversion formula in disparate vulnerable populations.

Methods

Study settings

This study combines data from two sources. The first source is data from five studies within the Seek Test Treat and Retain (STTR) consortium funded by the National Institute of Drug Abuse(Chandler et al., 2015). The 5 studies included in these analyses were selected based on simultaneous collection of both adherence measures (described below) and were grouped into categories based on their study populations: criminal justice (CJ) (Carda-Auten et al., 2014; Gordon, Kinlock, McKenzie, Wilson, & Rich, 2013), international (Tomori et al., 2014; Wechsberg et al., 2014), or other vulnerable populations (VP) (Glenn et al., In press). The second data source is the University of Washington site of the Centers for AIDS Research (CFAR) Network of Integrated Clinical Systems (CNICS)(Kitahata et al., 2008) cohort, a longitudinal observational study of PLWH in primary care. Assessments for this study were completed between 7/2009 and 6/2013.

Adherence measures

We included responses from several Adult AIDS Clinical Trial Group (AACTG) items(Chesney et al., 2000; Simoni et al., 2006), including the last time any HIV medications were missed; whether any ART doses were missed during the previous weekend; and how many doses of ART were missed over the last 14 days. Data were also available from a 30-day VAS adherence item(Amico et al., 2006; Simoni et al., 2006). The VAS was scored as a percentage from 0 to 100%. ART medication data were used to identify the number of doses of ART per day. Study procedures for all sites were approved by local Institutional Review Boards.

Statistical analyses

To create a formula for co-calibration between the two instruments, decision rules were first created using CNICS data. We made decision rules based on the last missed dose item or the 14-day item versus the VAS. The first decision rule was that if the response on the last missed dose item was ≥1 months ago the predicted VAS was set to 100%. The second decision rule was to set the predicted VAS to 100% if the number of missed doses in the last 14 days was zero.

Robust linear regression, which assigns weights to each observation in order to de-emphasize outliers, was then used to fit a VAS prediction equation for the participants who did not fit into the decision rules. The predictors included doses per day and results from the weekend and the 14-day missed dose items. This equation to estimate VAS scores obtained after application of the decision rules is called the co-calibration formula.

The formula was used to predict a VAS score for each person based on their responses to ACTG items. Then the performance of the formula was evaluated by comparing the predicted VAS to the reported VAS. We calculated Pearson’s correlation both with and without outliers, which were defined as those with a predicted and reported VAS which were more than 50% apart.

Data heterogeneity issues make it difficult to combine all of the STTR studies into a single decision rule estimate and so we present results stratified by the context of the population (CJ, international, other VP).

Results

Among 1,045 CNICS participants the mean age was 45 years and 87% were male (see Table 1). Rates of 100% VAS adherence varied across groups categorized on the basis of the AACTG item assessing timing of the last missed dose (data not shown). We set predicted VAS to 100% for those who responded that their last missed dose was 1 or more months ago (N=582), or that the number of missed doses in the last 14 days was zero (N=104).

When we fit a robust linear regression line to predict VAS among the 359 remaining people, the coefficients for the missed dose last weekend item and doses/day were not statistically significant and so these were dropped as predictors. The robust linear regression using only the 14-day item had an r ^{of 0.17, and estimated VAS as:}

Finally, the predicted VAS was set to zero if the equation predicted a value of less than zero to preserve the correct range for the VAS.

Applying the co-calibration formula to all 1045 in the CNICS sample, the correlation between predicted and reported VAS was 0.61 (Table 3). There were 20 outliers (2%), and without these outliers the correlation was 0.73. Many of the outliers were people who apparently inverted the VAS scale and reported 0–1% on the VAS despite also reporting “never skipping” medications (data not shown).

The STTR validation sample included 446 people in the CJ group, 311 people in the international group, and 63 people in the other VP group. The age distribution is similar for CNICS and CJ, while the international group is younger and the other VP group is older (Table 1).

The correlation between predicted and reported VAS for STTR overall was 0.66, and was 0.81 for CJ, 0.46 for international, and 0.32 for other VP (Table 3). The percentage of outliers was 9% overall, 2% for CJ and international groups, and 19% for other VP. Removing the outliers increased the correlations overall and for CJ to 0.86 and 0.88 respectively, and more for international and other VP groups to 0.78 and 0.80 respectively.

For comparison, the robust linear regression model which predicts the reported VAS with the 14-day ACTG item was fit in the STTR CJ group (r ^{of 0.27). The number of missed doses coefficient was 0.8% smaller than in CNICS at −2.1% per missed dose, while the intercept was 2.8% higher at 99.5% (}Table 2).

Discussion

This study demonstrates the feasibility of generating an equation to co-calibrate ART adherence instruments across diverse patient populations, thereby allowing adherence data from multiple studies with different approaches to adherence measurement to be combined and compared to answer questions requiring larger sample sizes. This study also demonstrates the negative impact of outliers and particularly individuals who appear to have inverted the VAS scale with the greatest impact among the international and other VP groups.

There has been an increasing push to harmonize data sources to address important research questions. By co-calibrating just a small number of select instruments or items, domains for many STTR studies can be on the same metric facilitating stronger methodological approaches to combining data across studies.

This study is also an important example of some of the limitations and barriers to co-calibration. The similarity of data can be compromised even when the same or similar measures are used. When combining data sets from across the world, these differences can be magnified for reasons including differences in language, culture, and method of administration. We found similar co-calibration performance across the different groups but, as described above, this was the case only when the outliers were excluded. This impact raises questions as to the appropriateness of the VAS as a single item of adherence.

Strengths and Limitations

This study included PLWH from six diverse studies including those in routine clinical care, treated and untreated drug users, and individuals in prison, on parole, or on probation. However, the sample size for the international and other VP studies were smaller than ideal thus limiting the strength of the findings. In addition, studies and sites used different approaches to collect the adherence information.

This study calibrated two adherence measures that differed in timeframe (e.g. 14 vs. 30 day items) which may have contributed to discrepancies between the measures.

Conclusion

A simple decision rule and equation allowed us to co-calibrate between two widely used measures of ART adherence among groups of PLWH and allowed for these measures to be placed on the same scale. Effective co-calibration allows results across studies that used different instruments to be combined to facilitate the development of larger pooled data sets to address high-priority research questions.

Acknowledgments