METHODOLOGICAL PLURALISM AND MIXED METHODOLOGY TO STRENGTHEN COMMUNITY PSYCHOLOGY RESEARCH: AN EXAMPLE FROM OXFORD HOUSE
Methodological Pluralism in Community Psychology
Methodological pluralism involves finding value in a variety of sources of information, including believing that no research method is inherently superior to any other (Barker & Pistrang, 2005; Barker, Pistrang, & Elliott, 2002). Community psychology researchers in particular aspire to methodological pluralism, as it aligns with the field’s values of diversity, empowerment, and attention to context (Banyard & Miller, 1998; Barker & Pistrang, 2005; 2012; Orford, 2008; Rappaport, 1990). The research topics that community psychologists study are often complex and contextually situated, and a plurality of research methods are best suited to address this complexity (Campbell, Gregory, Patterson, & Bybee, 2012).
Mixed methods research can occur within the confines of a single study or can exist at multiple levels (Campbell et al., 2012). First, a pluralistic study, also known as a mixed methods study, is the first level at which pluralism can occur. A mixed methods study incorporates both quantitative and qualitative data collection and analysis. Next, a pluralistic research program reflects pluralism across a program of studies conducted by the same team on one topic. Finally, pluralistic research disciplines are fields of science or scholarship in which a balance of study designs and analysis techniques is used in publications (Barker & Pistrang, 2012). Community psychology is an example of a pluralistic research discipline, particularly since a call to increase the use of qualitative research methods in the 1990s (Banyard & Miller, 1998; Luke, 2005).
Although nearly all community psychologists would endorse pluralism in principle, there have been few attempts to examine how the pluralist position works in practice (Barker & Pistrang, 2005). An evaluation of the research methodologies employed by community psychology researchers is an important task for the field. This article reviews and describes how methodological pluralism has functioned in a much-needed area of research: the role of housing and recovery residences in addiction recovery.
Recovery Residences and Oxford House
Substance-related disorders pose serious public health threats and exact costly social and economic tolls on users, their families and friends, neighborhoods, and the larger society (Jason, Olson, & Foli, 2008). A small but rapidly growing body of research supports the effectiveness of recovery residences for long-term addiction recovery, and a white policy paper on this topic has been published by the Society for Community Research and Action (SCRA; Jason, Mericle, Polcin, & White, 2013). Recovery residencies are safe, sober, and healthy living environments that promote recovery from substance use disorders and associated problems (Jason, Mericle, Polcin, White, & The National Association of Recovery Residences, 2012, September 20). By providing a physical place and a social world in which to recover, the recovery residence model offers promising mechanisms of change that extend beyond the arenas of traditional addiction treatment to increase the probability of long-term abstinence (Jason, Mericle, et al., 2013).
Since the advent of mutually and self-supported recovery residences in the 1960s and 1970s, they have continued to grow in the United States (Jason et al., 2012, September 20). During this time period, researchers and practitioners called for a shift from short-term treatment models toward recovery models that emphasize sustained, long-term well-being (Jason, Mericle, et al., 2013). However, there are still many challenges for researchers who want to study after-care recovery residences, given the predominant funding emphasis on more formal substance use treatment issues as well as the neuroscience of addiction (Jason, Mericle, et al., 2013). Furthermore, efforts to open and fund new recovery residences may be met with political and community resistance due to stigma or misunderstandings (Jason, Groh, et al., 2008).
The largest and most empirically supported network of recovery residences is the Oxford House organization. Since its founding in 1975, the Oxford House organization has grown to over 1,700 homes in the United States, housing more than 10,000 individuals (Oxford House, 2014). Similar to other recovery residences, Oxford Houses offer sober, safe, healthy living environments that promote recovery from substance use by promoting abstinence and gains in life domains, such as education, employment, and family relationships. Oxford Houses are unique residential recovery settings in that they are democratically run with no staff and emphasize sobriety, employment, chores, and opportunities to develop responsibility and leadership skills (Jason, Ferrari, Freeland, Danielewicz, & Olson, 2005; Jason, Schober, & Olson, 2008).
Current Review
Community psychology values emphasize the importance of the relationships between community researchers and oppressed groups, and research should be driven by the needs of the community with their active participation (Foster-Fishman, Nowell, Deacon, Nievar, & McCann, 2005; Orford, 2008; SCRA, 2014). In accordance with these values, it is important to consider whether research methods include, empower, and ultimately benefit community members (Nelson, Prilleltensky, & MacGillivary, 2001; Suarez-Balcazar et al., 2004). The SCRA asserts that the use of multiple methodologies is aligned with these values of community psychology (SCRA, 2014). However, there has been no comprehensive examination of the diverse research methodologies or data analysis techniques used with Oxford Houses or any other recovery residence.
Additionally, although mixed methods and methodological pluralism are highly regarded among community psychology researchers, and although competency in multiple research methods is considered an important aspect of training for community psychologists, few attempts have been made to explain how the pluralist position works in practice (Barker & Pistrang, 2012; Christens, Connell, Faust, Haber, & the Council of Education Programs, 2015; Jason & Glenwick, 2015). There are few evaluations of research methodologies within any long-standing, community-based, research program.
Following recommendations in the literature, we reviewed the research methods used in partnership with Oxford House. We assessed study designs and data analysis techniques both quantitatively, by coding each study, and qualitatively, by subjectively assessing how exemplars of each design have contributed to this pluralistic research program. We hope to better understand how a plurality of methods has served this research program in terms of both advancing knowledge and actively including community members. Further, we illustrate the many ways in which recovery residences may be assessed to aid future research development in this important area of research.
METHOD
Article Selection and Coding
Articles were located through a PsycInfo search (search terms: “Oxford House;” National Institute on Drug Abuse grants “DA13231,” “DA19935,” and DA16037”; National Institute of Health, Institute on Minority Health and Health Disparities grant “#5R24MD002748”) and were cross-referenced with records maintained by XX University. Articles were included in the current study if they focused in whole or in part on Oxford Houses or their residents. We included articles from peer-reviewed journals and one book chapter that had empirical data. Magazine articles and non-peer reviewed journal articles were excluded. The aforementioned inclusion criteria resulted in a total sample size of 126 peer-reviewed articles (including the one book chapter). These articles were published between 1995 and July 2015. For the purpose of coding study design and types of data analysis, we only included articles that reported data analyses; thus, 12 theoretical or review articles were excluded from further analysis, resulting in a subsample of 114 empirical articles with data. We had hoped to find articles from different research teams, but instead, we found that all of the articles included authors who were currently or previously associated with the DePaul University team conducting the present review.
The study design coding scheme was created using study designs presented in previous research and advanced psychology research methods textbooks (e.g., Bodner, 2006; Goodwin, 2005; Montero & Leon, 2007). The research team integrated the study designs presented in these resources to create the following categories: (a) descriptive studies using systematic observation, (b) cross-sectional descriptive studies through survey research, (c) longitudinal studies through survey research, (d) experiments, between-subjects designs, (e) experiments, within-subjects designs, (f) quasi-experiments, (g) ex post facto, retrospective studies, (h) ex post facto, prospective studies, (i) measure development studies, (j) ethnographic studies, (k) case studies, (l) action research studies, (m) qualitative interviews, and (n) mixed methods studies.
In some cases, a guideline set by one resource was modified to parallel guidelines from other resources and capture the extent of the research articles surveyed. For example, Montero and Leon (2007) advise that cross-sectional and longitudinal descriptive studies are solely descriptive and should not have hypotheses; however, many Oxford House articles were descriptive in nature but still proposed hypotheses based on existing theory and research. With consultation from the third author, the first and second authors developed this coding scheme. The first author coded all of the articles, and the first and second authors coded a subsample of articles, with high levels of agreement. If a question about coding arose, a discussion between all authors took place to make a final coding decision.
For coding data analyses, every analysis employed in a research article was counted to obtain the total number of times the analysis was used across all Oxford House research studies. Additionally, the main or most important analysis was coded in every study to create a count of the number of times the analysis technique was used as the main analysis among all Oxford House research articles.
Article Selection and Coding
Articles were located through a PsycInfo search (search terms: “Oxford House;” National Institute on Drug Abuse grants “DA13231,” “DA19935,” and DA16037”; National Institute of Health, Institute on Minority Health and Health Disparities grant “#5R24MD002748”) and were cross-referenced with records maintained by XX University. Articles were included in the current study if they focused in whole or in part on Oxford Houses or their residents. We included articles from peer-reviewed journals and one book chapter that had empirical data. Magazine articles and non-peer reviewed journal articles were excluded. The aforementioned inclusion criteria resulted in a total sample size of 126 peer-reviewed articles (including the one book chapter). These articles were published between 1995 and July 2015. For the purpose of coding study design and types of data analysis, we only included articles that reported data analyses; thus, 12 theoretical or review articles were excluded from further analysis, resulting in a subsample of 114 empirical articles with data. We had hoped to find articles from different research teams, but instead, we found that all of the articles included authors who were currently or previously associated with the DePaul University team conducting the present review.
The study design coding scheme was created using study designs presented in previous research and advanced psychology research methods textbooks (e.g., Bodner, 2006; Goodwin, 2005; Montero & Leon, 2007). The research team integrated the study designs presented in these resources to create the following categories: (a) descriptive studies using systematic observation, (b) cross-sectional descriptive studies through survey research, (c) longitudinal studies through survey research, (d) experiments, between-subjects designs, (e) experiments, within-subjects designs, (f) quasi-experiments, (g) ex post facto, retrospective studies, (h) ex post facto, prospective studies, (i) measure development studies, (j) ethnographic studies, (k) case studies, (l) action research studies, (m) qualitative interviews, and (n) mixed methods studies.
In some cases, a guideline set by one resource was modified to parallel guidelines from other resources and capture the extent of the research articles surveyed. For example, Montero and Leon (2007) advise that cross-sectional and longitudinal descriptive studies are solely descriptive and should not have hypotheses; however, many Oxford House articles were descriptive in nature but still proposed hypotheses based on existing theory and research. With consultation from the third author, the first and second authors developed this coding scheme. The first author coded all of the articles, and the first and second authors coded a subsample of articles, with high levels of agreement. If a question about coding arose, a discussion between all authors took place to make a final coding decision.
For coding data analyses, every analysis employed in a research article was counted to obtain the total number of times the analysis was used across all Oxford House research studies. Additionally, the main or most important analysis was coded in every study to create a count of the number of times the analysis technique was used as the main analysis among all Oxford House research articles.
RESULTS
Study Designs
Table 1 displays the types of study designs and the number of articles coded as each. Of the 114 studies that included data, there were 96 (84%) quantitative only designs, 8 (7%) qualitative only designs, and 10 (11%) mixed quantitative and qualitative designs.
Table 1
Summary of Study Designs Used in Oxford House Research
| Type | Study design | N(%) |
|---|---|---|
| Quantitative | ||
| Descriptive study using systematic observation | 2(2) | |
| Descriptive study of populations, survey research | ||
| Cross-sectional | 40(32) | |
| Longitudinal | 31(25) | |
| Experiments with different groups | 3(2) | |
| Operant experiments | 1(1) | |
| Quasi experiments | ||
| Pretest-posttest | 1(1) | |
| Posttest only design with duplicated groups | 2(2) | |
| Ex post facto studies | ||
| Retrospective studies | 4(3) | |
| Prospective studies | 3(2) | |
| Instrumental studies | 9(7) | |
| Qualitative | ||
| Ethnography | 3(2) | |
| Case study | 1(1) | |
| Action-research | 1(1) | |
| Qualitative interviewing | 3(2) | |
| Mixed methods | ||
| Mixed quantitative and qualitative methods | 10(8) | |
| Theory | ||
| Literature review, theoretical paper, or meta-analysis | 12(10) | |
| Total | 126 | |
Quantitative Study Designs
Descriptive study using systematic observation
Observational studies are best for examining features of residences or behaviors of residents as they naturally occur (Goodwin, 2005; Montero & Leon, 2007). This design has been used to study interactions and behaviors of Oxford House residents during regularly scheduled business meetings (Jason, Ferrari et al, 2005) as well as the physical properties of Oxford Houses and the neighborhoods in which they are located (Ferrari, Jason, Blake, Davis, Olson, 2006).
Cross-sectional descriptive studies through survey research
In a cross-sectional study, a description of a population is made at a single time point (Montero & Leon, 2007). Cross-sectional descriptive studies have been used to measure within-group (within Oxford House) differences, such as differences in outcomes based on gender and ethnicity (Belyaev-Glantsman, Jason, & Ferrari, 2009; Brown, Davis, Jason, & Ferrari, 2006). This design can also be used to examine the prevalence of a comorbid disorder, such as an eating disorder, within a population of recovery home residents (Czarlinski, Aase, & Jason, 2011). The findings of cross-sectional descriptive survey studies may lead to the design and implementation of longitudinal, descriptive studies.
Longitudinal descriptive studies through survey research
In a longitudinal study, data are collected across multiple time points with a descriptive goal (Montero & Leon, 2007). Longitudinal, descriptive studies have been used to examine how length of time in Oxford House relates to specific phenomena such as anxiety symptoms (Aase et al., 2005) or social support (Groh, Jason, Ferrari, & Halpert, 2005–2006; Ferrari, Stevens, & Jason, 2010). Additionally, longitudinal descriptive studies have supported the long-term positive effects of traits such as hope, self-esteem, and self-efficacy on abstinence and other outcomes while living in Oxford Houses (Chavarria, Stevens, Jason, & Ferrari, 2012; Dekhtyar, Beasley, Jason, & Ferrari, 2012; Ferrari, Stevens, Legler, & Jason, 2012; May, Hunter, Ferrari, Noel, & Jason, 2015).
Experiments, between-subjects designs
In an experiment, the independent variable is manipulated by the researcher; by randomly assigning participants to experimental and control groups, the researcher can eliminate potential causal factors that are not related to the independent variable and infer causality (Goodwin, 2005; Montero & Leon, 2007; Shadish, Cook, & Campbell, 2002). In a controlled trial, Jason, Olson, and Harvey (2014) randomly assigned 270 participants who had been released from the criminal justice system to a therapeutic community, Oxford House, or a usual aftercare condition.
Results showed that longer lengths of stay in therapeutic communities and Oxford House were related to reduced drug and alcohol use. Individuals assigned to the Oxford House condition worked more days, received more money from employment, achieved higher continuous alcohol abstinence rates, and had more favorable cost-benefit ratios than individuals assigned to the therapeutic community or usual aftercare conditions. Such experimental designs are best for evaluating the effectiveness of an intervention or treatment. The number of these types of experiments on recovery residences is small. De Leon, Inciardi, and Martin (1995) suggest that recovery residences are difficult to evaluate using randomized study designs because the decision to live in a recovery residence may be an integral step in recovery.
Experiment, within-subjects designs, including operant experiments and repeated-measures designs
Operant and repeated-measures experiments involve data collected over time for individuals or settings, with the intervention being introduced and taken away (reversal designs) or being applied to different groups at different times (multiple baseline designs) to examine the effects of the intervention (Goodwin, 2005; Montero & Leon, 2007). For example, using a multiple baseline design, Jason, Braciszewski, Olson, and Ferrari (2005) examined the impact of an Oxford House outreach program on the opening of new houses within different states. A significant increase in the opening of new houses occurred in states immediately following the employment of an outreach worker to provide technical assistance in that state.
Quasi-experiments
These designs are suitable for the evaluation of outcomes associated with a treatment without random assignment (Goodwin, 2005; Montero & Leon, 2007). For example, compared traditional recovery homes to culturally modified recovery homes for Spanish-speaking residents and found that residents in both types of recovery homes experienced decreases in substance use over time. Because recovery residence research may depend on ones’ desire to join a mutually supportive environment, quasi-experiments are useful when random assignment to conditions is not appropriate or feasible. However, self-selection into treatment conditions can serve as a threat to the validity of findings (Shadish, Cook, & Campbell, 2002).
Additionally, quasi-experiments with posttest-only designs may be used when measures can only be taken after the treatment or intervention has been applied (Montero & Leon, 2007). For example, the crime rates in areas surrounding 42 Oxford Houses and 42 random control houses in the same city were examined, and results suggest that well-managed Oxford Houses pose little to no risk to the neighborhood in terms of criminal behavior (Deaner, Jason, Aase, Mueller, & 2009).
Ex post facto, retrospective study
In a retrospective study, participants or units are grouped (postrecruitment) based on their independent variable status (i.e., presence or absence of a condition, personal circumstance, trait, or an experience of a specific past event; Montero & Leon, 2007). This study design differs from quasi-experimental designs in that at least one independent variable is a condition that cannot be manipulated and is unrelated to an intervention or treatment. For example, Jason et al. (2011) examined how the presence of post-traumatic stress disorder (as well as type of treatment) affects employment, self-regulation, and abstinence. Similarly, differences in psychological sense of community were examined based on the presence of eating disorders and past trauma among women Oxford House residents (Curtis, Jason, Olson, & Ferrari, 2005).
Ex post facto, prospective study
Prospective studies are designed with the intention of comparing groups, and participants are based on their group status a priori (Montero & Leon, 2007). Similarly to retrospective studies, at least one independent variable is a condition that cannot be manipulated. For example, Oxford House residents who were deaf and hearing were compared across outcomes, and both groups had comparable scores for sense of community and abstinence self-efficacy (Alvarez, Adebanjo, Davidson, Jason, & Davis, 2006).
Measure development studies
Measure development studies include the development of new scales, measures, tests, or procedures, as well as examining their psychometric properties (Goodwin, 2005; Montero & Leon, 2007). For example, a sense of community scale has been developed (Bishop, Chertok, & Jason, 1997) and validated with a sample of Oxford House residents (Stevens, Jason, Ferrari, Olson, & Legler, 2012).
Qualitative Study Designs
Ethnographic studies
Ethnographic studies are often characterized by subjective phenomenology, unstructured techniques, and participant observation (Montero & Leon, 2007). Ethnography may be best distinguished from other types of qualitative approaches by the collection of extensive fieldnotes based on the researcher’s observations and interactions “in the field” (Giles, 2002). This approach to studying human experience allows researchers to value diversity and emphasize the importance of culture in treatment, including culture-related barriers to treatment (Alvarez, Jason, Davis, Ferrari, & Olson, 2004) and cultural strengths that facilitate recovery (Alvarez, Jason, Davis, Olson, & Ferrari, 2009).
Case studies
This descriptive study design refers to a unique element such as a person or organization (Montero & Leon, 2007). Jason et al. (2003) used case studies of recovery residences to provide a more comprehensive understanding of the rules and norms regarding how the residences function.
Action-research studies
Montero and Leon (2007) describe action-research as a social context that is studied in such a way that both the intervention and the scientific description are sequenced in a spiral form so as to improve the context. Olson and Jason’s (2011) development of the Community Narration approach used personal and community narratives as an entryway into the evaluation process of the organization. The guidelines developed from the Community Narration approach led to further member-guided research and discussions. Action-research and the Community Narration approach may be used as a way of simultaneously understanding and improving residences by empowering residents.
Qualitative interviews
In the above-mentioned qualitative study designs the main form of data collection is structured or unstructured interviews with participants. Such methods are ideal for providing rich descriptions to capture diverse experiences with specificity (Banyard, & Miller, 1998). Qualitative interviews have been employed to understand facets of recovery residences that have rarely been studied, such as the effects of children on the process of recovery in Oxford Houses (Legler et al., 2012) and the experiences of Oxford House residents who have chronic illnesses, such as the Hepatitis C virus (Contreras & Jason, 2013).
Mixed Methods Studies
Mixed methods research can involve a single study that incorporates both quantitative and qualitative data collection and data analysis techniques. However, mixed methods studies can be more complex than an A + B operation (Campbell et al., 2012) because each method has the potential to bring clarity and insight to the other. For example, given the paucity of literature on the role of natural mentoring relationships in substance abuse recovery, Lawlor, Hunter, Jason, and Rosing (2014) used focus groups paired with a series of empirically validated quantitative measures to better understand natural mentoring relationships. Quantitative results showed that mentoring characteristics and activities predicted social support and helping behaviors.
Focus group questions were created with the input of Oxford House residents, and the nature of these mentoring relationships was described during focus groups. As an example of the richness of this approach, one male participant explained: “I usually go to any one of the guys. That’s why I put five on there (on the survey, regarding the number of mentors in the house). I didn’t just put one ‘cause we all have different personalities, so there’s certain things that you can speak to, certain individuals that will help you out that will lead you the right way” (Lawlor et al., 2014, p. 136). The use of both quantitative and qualitative methods in this study provides rich empirical support for the characteristics of natural mentoring relationships and their importance for Oxford House residents.
The methods and results of one Oxford House study often shed light on the results of another study or inspire the design of a new study. For example, while conducting a nationwide longitudinal study on Oxford House (Jason, Davis, Ferrari, & Anderson, 2007), the research team recognized through qualitative findings and discussions with members of the Oxford House organization that Latinas/os were underrepresented among Oxford House residents. This awareness stirred curiosity about Latinos’ perceptions of Oxford Houses and recovery and potential barriers that Latinos encounter when entering an Oxford House. To address these questions, the researchers, with help from the Oxford House organization, conducted a series of qualitative studies that revealed that Latinos had a positive recovery experience in Oxford House and that more Latinos could potentially benefit from participating (Alvarez et al., 2004; Alvarez et al., 2009). The findings of these two qualitative studies influenced the development of a quasi-experiment in which Latino residents of traditional recovery homes were compared to Latino residents of culturally modified homes (Jason, DiGangi, et al., 2013).
Data Analyses
Different data analysis techniques have been utilized in research with Oxford Houses and Oxford House residents. This list is not intended to be exhaustive, but rather illustrates how diverse techniques have been used in studies with Oxford House. Table 2 contains the frequency at which each data analysis technique was used as well as the main analysis for each article. Below, we highlight a few select examples. The following analysis techniques, while not necessarily unique to the field of community psychology, have added interesting and important contributions to the area of recovery residence research and community psychology in general.
Table 2
Statistical Analyses Used in Oxford House Research Articles With Data
| Type | Analysis | Main analysis N(%) | Total count N(%) |
|---|---|---|---|
| Quantitative | Descriptive statistics | 5(4) | 105(92) |
| Group differences tests | 36(32) | 63(55) | |
| T-test | 33(29) | ||
| chi-square analysis | 44(39) | ||
| ANOVA/MANOVA/MANCOVA | 47(41) | ||
| Repeated measures test (t-test, ANOVA, MANCOVA, etc.) | 1(1) | 4(4) | |
| Factor analysis | 9(8) | 12(11) | |
| Correlations | 2(2) | 20(18) | |
| Linear regressions | 22(19) | 26(23) | |
| Logistic regressions | 6(5) | 11(10) | |
| Hierarchical linear modeling/multilevel modeling | 9(8) | 11(10) | |
| Structural equation model | 5(4) | 7(6) | |
| Latent growth curve analysis | 1(1) | 1(1) | |
| Cost-benefit analysis | 1(1) | 2(2) | |
| Survival analysis | 1(1) | 1(1) | |
| Stochastic actor-based model | 1(1) | 1(1) | |
| Generalized linear model | 2(2) | 2(2) | |
| Generalized estimating equation | 0.00(0) | 1(1) | |
| Qualitative | Grounded theory | 3(3) | 3(3) |
| NVIVO/coding | 3(3) | 3(3) | |
| Community narration | 1(1) | 1(1) | |
| Content analysis | 2(2) | 2(2) | |
| Summary of qualitative responses/anecdotal information | 2(2) | 8(7) |
Note. ANOVA = analysis of variance; MANOVA = multivariate analysis of variance; MANCOVA = multivariate analysis of covariance.
N = 114.
Survival analysis
A defining feature of survival analysis is the measurement of a behavior (such as drop out or recidivism) over time and what might be influencing that behavior. This method may be particularly useful because the delay or reduction of a specific event (e.g., relapse) is the desired outcome (Connell, 2012). Bishop, Jason, Ferrari, and Huang (1998) used survival analysis to understand predictors at the individual and house levels that were related to residents leaving Oxford House. They found that older age and older age of fellow residents were associated with being more likely to continue residence and being less pessimistic.
Multilevel modeling
Multilevel modeling, often referred to as hierarchical linear modeling, is a regression based method that is particularly useful for community psychology researchers because it captures context by investigating levels of analysis beyond the individual level (Luke, 2005; Shinn & Rapkin, 2000). Multilevel modeling allows for an understanding of how both individual and group characteristics may predict individual level outcomes (Todd, Allen, & Javdani, 2012). For example, multilevel modeling has been used to understand longitudinal changes in substance use of Oxford House residents compared to a usual aftercare condition (Jason, Olson et al., 2007). The effects of the predictor variables on each of four outcomes were examined within each condition over time. At the 24-month follow-up, there was less substance abuse for residents living in Oxford Houses for 6 or more months (16%), compared to participants both with less than 6 months (46%) and assigned to the usual after-care condition (65%).
Geographic information systems (GIS)
GIS supports the detailed analysis of spatial information. GIS enables community researchers to analyze individuals and groups in context, including details about the physical environment, the socioeconomic environment, and the way people move through the environment (Morton, Peterson, Speer, Reid, & Hughey, 2012). The use of publicly available databases analyzed with GIS has revealed that the crime rates immediately surrounding Oxford Houses are no different than the crime rate of a group of control houses, suggesting that recovery homes pose minimal risk to neighbors and communities (Deaner et al., 2009).
Cost-benefit analysis
Cost-benefit analysis is used to evaluate the cost-effectiveness of a program or treatment. The ability to demonstrate that the value of a program’s benefits exceed its costs is important for community psychologists, especially in terms of funding. Cost-benefit analyses show that Oxford House fares favorably in comparison to usual aftercare in terms of cost-effectiveness (Lo Sasso, Byro, Jason, Ferrari, & Olson, 2012; Lo Sasso & Jason, 2012). Using the public payer perspective, which does not include the rent and expenses associated with the Oxford House treatment modality as the residents paid these expenses out of their own pocket, the net benefit for Oxford House over usual aftercare was $31,043 per person.
Structural equation modeling (SEM)
SEM is a technique used to test theoretical relations among variables, including latent variables. While causal relationships may not be tested in nonexperimental studies, SEM is useful for community researchers to test theoretical models, including directional hypotheses. An additional advantage of SEM is the ability to construct latent variables or estimated variables that comprise several measured variables (Bartholomew & Knott, 1999). With Oxford House residents, SEM has been used to test models of theoretically related constructs including mediated paths; for example, Aase, Jason, Ferrari, Li, and Scott (2014) reported that time spent in Oxford House and a 12-step affiliation predicted abstinence outcomes, and this relationship was mediated by increased social support.
Grounded theory
Grounded theory is a qualitative data analysis method that involves coding and categorization of responses based on the researcher’s recognition of emergent themes that are “grounded” in the data (i.e., not predetermined or theoretically based). The use of grounded theory allows the researcher to think about the data in new ways, including finding new insights and rendering hidden assumptions visible (Charmaz, 2006). In the context of Oxford House, grounded theory has been used to understand how participants think about leadership in a women’s Oxford House (Davis et al., 2006; Timpo et al., 2014) and Latinos’ perceptions of living in an Oxford House (Alvarez et al., 2009).
Study Designs
Table 1 displays the types of study designs and the number of articles coded as each. Of the 114 studies that included data, there were 96 (84%) quantitative only designs, 8 (7%) qualitative only designs, and 10 (11%) mixed quantitative and qualitative designs.
Table 1
Summary of Study Designs Used in Oxford House Research
| Type | Study design | N(%) |
|---|---|---|
| Quantitative | ||
| Descriptive study using systematic observation | 2(2) | |
| Descriptive study of populations, survey research | ||
| Cross-sectional | 40(32) | |
| Longitudinal | 31(25) | |
| Experiments with different groups | 3(2) | |
| Operant experiments | 1(1) | |
| Quasi experiments | ||
| Pretest-posttest | 1(1) | |
| Posttest only design with duplicated groups | 2(2) | |
| Ex post facto studies | ||
| Retrospective studies | 4(3) | |
| Prospective studies | 3(2) | |
| Instrumental studies | 9(7) | |
| Qualitative | ||
| Ethnography | 3(2) | |
| Case study | 1(1) | |
| Action-research | 1(1) | |
| Qualitative interviewing | 3(2) | |
| Mixed methods | ||
| Mixed quantitative and qualitative methods | 10(8) | |
| Theory | ||
| Literature review, theoretical paper, or meta-analysis | 12(10) | |
| Total | 126 | |
Quantitative Study Designs
Descriptive study using systematic observation
Observational studies are best for examining features of residences or behaviors of residents as they naturally occur (Goodwin, 2005; Montero & Leon, 2007). This design has been used to study interactions and behaviors of Oxford House residents during regularly scheduled business meetings (Jason, Ferrari et al, 2005) as well as the physical properties of Oxford Houses and the neighborhoods in which they are located (Ferrari, Jason, Blake, Davis, Olson, 2006).
Cross-sectional descriptive studies through survey research
In a cross-sectional study, a description of a population is made at a single time point (Montero & Leon, 2007). Cross-sectional descriptive studies have been used to measure within-group (within Oxford House) differences, such as differences in outcomes based on gender and ethnicity (Belyaev-Glantsman, Jason, & Ferrari, 2009; Brown, Davis, Jason, & Ferrari, 2006). This design can also be used to examine the prevalence of a comorbid disorder, such as an eating disorder, within a population of recovery home residents (Czarlinski, Aase, & Jason, 2011). The findings of cross-sectional descriptive survey studies may lead to the design and implementation of longitudinal, descriptive studies.
Longitudinal descriptive studies through survey research
In a longitudinal study, data are collected across multiple time points with a descriptive goal (Montero & Leon, 2007). Longitudinal, descriptive studies have been used to examine how length of time in Oxford House relates to specific phenomena such as anxiety symptoms (Aase et al., 2005) or social support (Groh, Jason, Ferrari, & Halpert, 2005–2006; Ferrari, Stevens, & Jason, 2010). Additionally, longitudinal descriptive studies have supported the long-term positive effects of traits such as hope, self-esteem, and self-efficacy on abstinence and other outcomes while living in Oxford Houses (Chavarria, Stevens, Jason, & Ferrari, 2012; Dekhtyar, Beasley, Jason, & Ferrari, 2012; Ferrari, Stevens, Legler, & Jason, 2012; May, Hunter, Ferrari, Noel, & Jason, 2015).
Experiments, between-subjects designs
In an experiment, the independent variable is manipulated by the researcher; by randomly assigning participants to experimental and control groups, the researcher can eliminate potential causal factors that are not related to the independent variable and infer causality (Goodwin, 2005; Montero & Leon, 2007; Shadish, Cook, & Campbell, 2002). In a controlled trial, Jason, Olson, and Harvey (2014) randomly assigned 270 participants who had been released from the criminal justice system to a therapeutic community, Oxford House, or a usual aftercare condition.
Results showed that longer lengths of stay in therapeutic communities and Oxford House were related to reduced drug and alcohol use. Individuals assigned to the Oxford House condition worked more days, received more money from employment, achieved higher continuous alcohol abstinence rates, and had more favorable cost-benefit ratios than individuals assigned to the therapeutic community or usual aftercare conditions. Such experimental designs are best for evaluating the effectiveness of an intervention or treatment. The number of these types of experiments on recovery residences is small. De Leon, Inciardi, and Martin (1995) suggest that recovery residences are difficult to evaluate using randomized study designs because the decision to live in a recovery residence may be an integral step in recovery.
Experiment, within-subjects designs, including operant experiments and repeated-measures designs
Operant and repeated-measures experiments involve data collected over time for individuals or settings, with the intervention being introduced and taken away (reversal designs) or being applied to different groups at different times (multiple baseline designs) to examine the effects of the intervention (Goodwin, 2005; Montero & Leon, 2007). For example, using a multiple baseline design, Jason, Braciszewski, Olson, and Ferrari (2005) examined the impact of an Oxford House outreach program on the opening of new houses within different states. A significant increase in the opening of new houses occurred in states immediately following the employment of an outreach worker to provide technical assistance in that state.
Quasi-experiments
These designs are suitable for the evaluation of outcomes associated with a treatment without random assignment (Goodwin, 2005; Montero & Leon, 2007). For example, compared traditional recovery homes to culturally modified recovery homes for Spanish-speaking residents and found that residents in both types of recovery homes experienced decreases in substance use over time. Because recovery residence research may depend on ones’ desire to join a mutually supportive environment, quasi-experiments are useful when random assignment to conditions is not appropriate or feasible. However, self-selection into treatment conditions can serve as a threat to the validity of findings (Shadish, Cook, & Campbell, 2002).
Additionally, quasi-experiments with posttest-only designs may be used when measures can only be taken after the treatment or intervention has been applied (Montero & Leon, 2007). For example, the crime rates in areas surrounding 42 Oxford Houses and 42 random control houses in the same city were examined, and results suggest that well-managed Oxford Houses pose little to no risk to the neighborhood in terms of criminal behavior (Deaner, Jason, Aase, Mueller, & 2009).
Ex post facto, retrospective study
In a retrospective study, participants or units are grouped (postrecruitment) based on their independent variable status (i.e., presence or absence of a condition, personal circumstance, trait, or an experience of a specific past event; Montero & Leon, 2007). This study design differs from quasi-experimental designs in that at least one independent variable is a condition that cannot be manipulated and is unrelated to an intervention or treatment. For example, Jason et al. (2011) examined how the presence of post-traumatic stress disorder (as well as type of treatment) affects employment, self-regulation, and abstinence. Similarly, differences in psychological sense of community were examined based on the presence of eating disorders and past trauma among women Oxford House residents (Curtis, Jason, Olson, & Ferrari, 2005).
Ex post facto, prospective study
Prospective studies are designed with the intention of comparing groups, and participants are based on their group status a priori (Montero & Leon, 2007). Similarly to retrospective studies, at least one independent variable is a condition that cannot be manipulated. For example, Oxford House residents who were deaf and hearing were compared across outcomes, and both groups had comparable scores for sense of community and abstinence self-efficacy (Alvarez, Adebanjo, Davidson, Jason, & Davis, 2006).
Measure development studies
Measure development studies include the development of new scales, measures, tests, or procedures, as well as examining their psychometric properties (Goodwin, 2005; Montero & Leon, 2007). For example, a sense of community scale has been developed (Bishop, Chertok, & Jason, 1997) and validated with a sample of Oxford House residents (Stevens, Jason, Ferrari, Olson, & Legler, 2012).
Descriptive study using systematic observation
Observational studies are best for examining features of residences or behaviors of residents as they naturally occur (Goodwin, 2005; Montero & Leon, 2007). This design has been used to study interactions and behaviors of Oxford House residents during regularly scheduled business meetings (Jason, Ferrari et al, 2005) as well as the physical properties of Oxford Houses and the neighborhoods in which they are located (Ferrari, Jason, Blake, Davis, Olson, 2006).
Cross-sectional descriptive studies through survey research
In a cross-sectional study, a description of a population is made at a single time point (Montero & Leon, 2007). Cross-sectional descriptive studies have been used to measure within-group (within Oxford House) differences, such as differences in outcomes based on gender and ethnicity (Belyaev-Glantsman, Jason, & Ferrari, 2009; Brown, Davis, Jason, & Ferrari, 2006). This design can also be used to examine the prevalence of a comorbid disorder, such as an eating disorder, within a population of recovery home residents (Czarlinski, Aase, & Jason, 2011). The findings of cross-sectional descriptive survey studies may lead to the design and implementation of longitudinal, descriptive studies.
Longitudinal descriptive studies through survey research
In a longitudinal study, data are collected across multiple time points with a descriptive goal (Montero & Leon, 2007). Longitudinal, descriptive studies have been used to examine how length of time in Oxford House relates to specific phenomena such as anxiety symptoms (Aase et al., 2005) or social support (Groh, Jason, Ferrari, & Halpert, 2005–2006; Ferrari, Stevens, & Jason, 2010). Additionally, longitudinal descriptive studies have supported the long-term positive effects of traits such as hope, self-esteem, and self-efficacy on abstinence and other outcomes while living in Oxford Houses (Chavarria, Stevens, Jason, & Ferrari, 2012; Dekhtyar, Beasley, Jason, & Ferrari, 2012; Ferrari, Stevens, Legler, & Jason, 2012; May, Hunter, Ferrari, Noel, & Jason, 2015).
Experiments, between-subjects designs
In an experiment, the independent variable is manipulated by the researcher; by randomly assigning participants to experimental and control groups, the researcher can eliminate potential causal factors that are not related to the independent variable and infer causality (Goodwin, 2005; Montero & Leon, 2007; Shadish, Cook, & Campbell, 2002). In a controlled trial, Jason, Olson, and Harvey (2014) randomly assigned 270 participants who had been released from the criminal justice system to a therapeutic community, Oxford House, or a usual aftercare condition.
Results showed that longer lengths of stay in therapeutic communities and Oxford House were related to reduced drug and alcohol use. Individuals assigned to the Oxford House condition worked more days, received more money from employment, achieved higher continuous alcohol abstinence rates, and had more favorable cost-benefit ratios than individuals assigned to the therapeutic community or usual aftercare conditions. Such experimental designs are best for evaluating the effectiveness of an intervention or treatment. The number of these types of experiments on recovery residences is small. De Leon, Inciardi, and Martin (1995) suggest that recovery residences are difficult to evaluate using randomized study designs because the decision to live in a recovery residence may be an integral step in recovery.
Experiment, within-subjects designs, including operant experiments and repeated-measures designs
Operant and repeated-measures experiments involve data collected over time for individuals or settings, with the intervention being introduced and taken away (reversal designs) or being applied to different groups at different times (multiple baseline designs) to examine the effects of the intervention (Goodwin, 2005; Montero & Leon, 2007). For example, using a multiple baseline design, Jason, Braciszewski, Olson, and Ferrari (2005) examined the impact of an Oxford House outreach program on the opening of new houses within different states. A significant increase in the opening of new houses occurred in states immediately following the employment of an outreach worker to provide technical assistance in that state.
Quasi-experiments
These designs are suitable for the evaluation of outcomes associated with a treatment without random assignment (Goodwin, 2005; Montero & Leon, 2007). For example, compared traditional recovery homes to culturally modified recovery homes for Spanish-speaking residents and found that residents in both types of recovery homes experienced decreases in substance use over time. Because recovery residence research may depend on ones’ desire to join a mutually supportive environment, quasi-experiments are useful when random assignment to conditions is not appropriate or feasible. However, self-selection into treatment conditions can serve as a threat to the validity of findings (Shadish, Cook, & Campbell, 2002).
Additionally, quasi-experiments with posttest-only designs may be used when measures can only be taken after the treatment or intervention has been applied (Montero & Leon, 2007). For example, the crime rates in areas surrounding 42 Oxford Houses and 42 random control houses in the same city were examined, and results suggest that well-managed Oxford Houses pose little to no risk to the neighborhood in terms of criminal behavior (Deaner, Jason, Aase, Mueller, & 2009).
Ex post facto, retrospective study
In a retrospective study, participants or units are grouped (postrecruitment) based on their independent variable status (i.e., presence or absence of a condition, personal circumstance, trait, or an experience of a specific past event; Montero & Leon, 2007). This study design differs from quasi-experimental designs in that at least one independent variable is a condition that cannot be manipulated and is unrelated to an intervention or treatment. For example, Jason et al. (2011) examined how the presence of post-traumatic stress disorder (as well as type of treatment) affects employment, self-regulation, and abstinence. Similarly, differences in psychological sense of community were examined based on the presence of eating disorders and past trauma among women Oxford House residents (Curtis, Jason, Olson, & Ferrari, 2005).
Ex post facto, prospective study
Prospective studies are designed with the intention of comparing groups, and participants are based on their group status a priori (Montero & Leon, 2007). Similarly to retrospective studies, at least one independent variable is a condition that cannot be manipulated. For example, Oxford House residents who were deaf and hearing were compared across outcomes, and both groups had comparable scores for sense of community and abstinence self-efficacy (Alvarez, Adebanjo, Davidson, Jason, & Davis, 2006).
Measure development studies
Measure development studies include the development of new scales, measures, tests, or procedures, as well as examining their psychometric properties (Goodwin, 2005; Montero & Leon, 2007). For example, a sense of community scale has been developed (Bishop, Chertok, & Jason, 1997) and validated with a sample of Oxford House residents (Stevens, Jason, Ferrari, Olson, & Legler, 2012).
Qualitative Study Designs
Ethnographic studies
Ethnographic studies are often characterized by subjective phenomenology, unstructured techniques, and participant observation (Montero & Leon, 2007). Ethnography may be best distinguished from other types of qualitative approaches by the collection of extensive fieldnotes based on the researcher’s observations and interactions “in the field” (Giles, 2002). This approach to studying human experience allows researchers to value diversity and emphasize the importance of culture in treatment, including culture-related barriers to treatment (Alvarez, Jason, Davis, Ferrari, & Olson, 2004) and cultural strengths that facilitate recovery (Alvarez, Jason, Davis, Olson, & Ferrari, 2009).
Case studies
This descriptive study design refers to a unique element such as a person or organization (Montero & Leon, 2007). Jason et al. (2003) used case studies of recovery residences to provide a more comprehensive understanding of the rules and norms regarding how the residences function.
Action-research studies
Montero and Leon (2007) describe action-research as a social context that is studied in such a way that both the intervention and the scientific description are sequenced in a spiral form so as to improve the context. Olson and Jason’s (2011) development of the Community Narration approach used personal and community narratives as an entryway into the evaluation process of the organization. The guidelines developed from the Community Narration approach led to further member-guided research and discussions. Action-research and the Community Narration approach may be used as a way of simultaneously understanding and improving residences by empowering residents.
Qualitative interviews
In the above-mentioned qualitative study designs the main form of data collection is structured or unstructured interviews with participants. Such methods are ideal for providing rich descriptions to capture diverse experiences with specificity (Banyard, & Miller, 1998). Qualitative interviews have been employed to understand facets of recovery residences that have rarely been studied, such as the effects of children on the process of recovery in Oxford Houses (Legler et al., 2012) and the experiences of Oxford House residents who have chronic illnesses, such as the Hepatitis C virus (Contreras & Jason, 2013).
Ethnographic studies
Ethnographic studies are often characterized by subjective phenomenology, unstructured techniques, and participant observation (Montero & Leon, 2007). Ethnography may be best distinguished from other types of qualitative approaches by the collection of extensive fieldnotes based on the researcher’s observations and interactions “in the field” (Giles, 2002). This approach to studying human experience allows researchers to value diversity and emphasize the importance of culture in treatment, including culture-related barriers to treatment (Alvarez, Jason, Davis, Ferrari, & Olson, 2004) and cultural strengths that facilitate recovery (Alvarez, Jason, Davis, Olson, & Ferrari, 2009).
Case studies
This descriptive study design refers to a unique element such as a person or organization (Montero & Leon, 2007). Jason et al. (2003) used case studies of recovery residences to provide a more comprehensive understanding of the rules and norms regarding how the residences function.
Action-research studies
Montero and Leon (2007) describe action-research as a social context that is studied in such a way that both the intervention and the scientific description are sequenced in a spiral form so as to improve the context. Olson and Jason’s (2011) development of the Community Narration approach used personal and community narratives as an entryway into the evaluation process of the organization. The guidelines developed from the Community Narration approach led to further member-guided research and discussions. Action-research and the Community Narration approach may be used as a way of simultaneously understanding and improving residences by empowering residents.
Qualitative interviews
In the above-mentioned qualitative study designs the main form of data collection is structured or unstructured interviews with participants. Such methods are ideal for providing rich descriptions to capture diverse experiences with specificity (Banyard, & Miller, 1998). Qualitative interviews have been employed to understand facets of recovery residences that have rarely been studied, such as the effects of children on the process of recovery in Oxford Houses (Legler et al., 2012) and the experiences of Oxford House residents who have chronic illnesses, such as the Hepatitis C virus (Contreras & Jason, 2013).
Mixed Methods Studies
Mixed methods research can involve a single study that incorporates both quantitative and qualitative data collection and data analysis techniques. However, mixed methods studies can be more complex than an A + B operation (Campbell et al., 2012) because each method has the potential to bring clarity and insight to the other. For example, given the paucity of literature on the role of natural mentoring relationships in substance abuse recovery, Lawlor, Hunter, Jason, and Rosing (2014) used focus groups paired with a series of empirically validated quantitative measures to better understand natural mentoring relationships. Quantitative results showed that mentoring characteristics and activities predicted social support and helping behaviors.
Focus group questions were created with the input of Oxford House residents, and the nature of these mentoring relationships was described during focus groups. As an example of the richness of this approach, one male participant explained: “I usually go to any one of the guys. That’s why I put five on there (on the survey, regarding the number of mentors in the house). I didn’t just put one ‘cause we all have different personalities, so there’s certain things that you can speak to, certain individuals that will help you out that will lead you the right way” (Lawlor et al., 2014, p. 136). The use of both quantitative and qualitative methods in this study provides rich empirical support for the characteristics of natural mentoring relationships and their importance for Oxford House residents.
The methods and results of one Oxford House study often shed light on the results of another study or inspire the design of a new study. For example, while conducting a nationwide longitudinal study on Oxford House (Jason, Davis, Ferrari, & Anderson, 2007), the research team recognized through qualitative findings and discussions with members of the Oxford House organization that Latinas/os were underrepresented among Oxford House residents. This awareness stirred curiosity about Latinos’ perceptions of Oxford Houses and recovery and potential barriers that Latinos encounter when entering an Oxford House. To address these questions, the researchers, with help from the Oxford House organization, conducted a series of qualitative studies that revealed that Latinos had a positive recovery experience in Oxford House and that more Latinos could potentially benefit from participating (Alvarez et al., 2004; Alvarez et al., 2009). The findings of these two qualitative studies influenced the development of a quasi-experiment in which Latino residents of traditional recovery homes were compared to Latino residents of culturally modified homes (Jason, DiGangi, et al., 2013).
Data Analyses
Different data analysis techniques have been utilized in research with Oxford Houses and Oxford House residents. This list is not intended to be exhaustive, but rather illustrates how diverse techniques have been used in studies with Oxford House. Table 2 contains the frequency at which each data analysis technique was used as well as the main analysis for each article. Below, we highlight a few select examples. The following analysis techniques, while not necessarily unique to the field of community psychology, have added interesting and important contributions to the area of recovery residence research and community psychology in general.
Table 2
Statistical Analyses Used in Oxford House Research Articles With Data
| Type | Analysis | Main analysis N(%) | Total count N(%) |
|---|---|---|---|
| Quantitative | Descriptive statistics | 5(4) | 105(92) |
| Group differences tests | 36(32) | 63(55) | |
| T-test | 33(29) | ||
| chi-square analysis | 44(39) | ||
| ANOVA/MANOVA/MANCOVA | 47(41) | ||
| Repeated measures test (t-test, ANOVA, MANCOVA, etc.) | 1(1) | 4(4) | |
| Factor analysis | 9(8) | 12(11) | |
| Correlations | 2(2) | 20(18) | |
| Linear regressions | 22(19) | 26(23) | |
| Logistic regressions | 6(5) | 11(10) | |
| Hierarchical linear modeling/multilevel modeling | 9(8) | 11(10) | |
| Structural equation model | 5(4) | 7(6) | |
| Latent growth curve analysis | 1(1) | 1(1) | |
| Cost-benefit analysis | 1(1) | 2(2) | |
| Survival analysis | 1(1) | 1(1) | |
| Stochastic actor-based model | 1(1) | 1(1) | |
| Generalized linear model | 2(2) | 2(2) | |
| Generalized estimating equation | 0.00(0) | 1(1) | |
| Qualitative | Grounded theory | 3(3) | 3(3) |
| NVIVO/coding | 3(3) | 3(3) | |
| Community narration | 1(1) | 1(1) | |
| Content analysis | 2(2) | 2(2) | |
| Summary of qualitative responses/anecdotal information | 2(2) | 8(7) |
Note. ANOVA = analysis of variance; MANOVA = multivariate analysis of variance; MANCOVA = multivariate analysis of covariance.
N = 114.
Survival analysis
A defining feature of survival analysis is the measurement of a behavior (such as drop out or recidivism) over time and what might be influencing that behavior. This method may be particularly useful because the delay or reduction of a specific event (e.g., relapse) is the desired outcome (Connell, 2012). Bishop, Jason, Ferrari, and Huang (1998) used survival analysis to understand predictors at the individual and house levels that were related to residents leaving Oxford House. They found that older age and older age of fellow residents were associated with being more likely to continue residence and being less pessimistic.
Multilevel modeling
Multilevel modeling, often referred to as hierarchical linear modeling, is a regression based method that is particularly useful for community psychology researchers because it captures context by investigating levels of analysis beyond the individual level (Luke, 2005; Shinn & Rapkin, 2000). Multilevel modeling allows for an understanding of how both individual and group characteristics may predict individual level outcomes (Todd, Allen, & Javdani, 2012). For example, multilevel modeling has been used to understand longitudinal changes in substance use of Oxford House residents compared to a usual aftercare condition (Jason, Olson et al., 2007). The effects of the predictor variables on each of four outcomes were examined within each condition over time. At the 24-month follow-up, there was less substance abuse for residents living in Oxford Houses for 6 or more months (16%), compared to participants both with less than 6 months (46%) and assigned to the usual after-care condition (65%).
Geographic information systems (GIS)
GIS supports the detailed analysis of spatial information. GIS enables community researchers to analyze individuals and groups in context, including details about the physical environment, the socioeconomic environment, and the way people move through the environment (Morton, Peterson, Speer, Reid, & Hughey, 2012). The use of publicly available databases analyzed with GIS has revealed that the crime rates immediately surrounding Oxford Houses are no different than the crime rate of a group of control houses, suggesting that recovery homes pose minimal risk to neighbors and communities (Deaner et al., 2009).
Cost-benefit analysis
Cost-benefit analysis is used to evaluate the cost-effectiveness of a program or treatment. The ability to demonstrate that the value of a program’s benefits exceed its costs is important for community psychologists, especially in terms of funding. Cost-benefit analyses show that Oxford House fares favorably in comparison to usual aftercare in terms of cost-effectiveness (Lo Sasso, Byro, Jason, Ferrari, & Olson, 2012; Lo Sasso & Jason, 2012). Using the public payer perspective, which does not include the rent and expenses associated with the Oxford House treatment modality as the residents paid these expenses out of their own pocket, the net benefit for Oxford House over usual aftercare was $31,043 per person.
Structural equation modeling (SEM)
SEM is a technique used to test theoretical relations among variables, including latent variables. While causal relationships may not be tested in nonexperimental studies, SEM is useful for community researchers to test theoretical models, including directional hypotheses. An additional advantage of SEM is the ability to construct latent variables or estimated variables that comprise several measured variables (Bartholomew & Knott, 1999). With Oxford House residents, SEM has been used to test models of theoretically related constructs including mediated paths; for example, Aase, Jason, Ferrari, Li, and Scott (2014) reported that time spent in Oxford House and a 12-step affiliation predicted abstinence outcomes, and this relationship was mediated by increased social support.
Grounded theory
Grounded theory is a qualitative data analysis method that involves coding and categorization of responses based on the researcher’s recognition of emergent themes that are “grounded” in the data (i.e., not predetermined or theoretically based). The use of grounded theory allows the researcher to think about the data in new ways, including finding new insights and rendering hidden assumptions visible (Charmaz, 2006). In the context of Oxford House, grounded theory has been used to understand how participants think about leadership in a women’s Oxford House (Davis et al., 2006; Timpo et al., 2014) and Latinos’ perceptions of living in an Oxford House (Alvarez et al., 2009).
Survival analysis
A defining feature of survival analysis is the measurement of a behavior (such as drop out or recidivism) over time and what might be influencing that behavior. This method may be particularly useful because the delay or reduction of a specific event (e.g., relapse) is the desired outcome (Connell, 2012). Bishop, Jason, Ferrari, and Huang (1998) used survival analysis to understand predictors at the individual and house levels that were related to residents leaving Oxford House. They found that older age and older age of fellow residents were associated with being more likely to continue residence and being less pessimistic.
Multilevel modeling
Multilevel modeling, often referred to as hierarchical linear modeling, is a regression based method that is particularly useful for community psychology researchers because it captures context by investigating levels of analysis beyond the individual level (Luke, 2005; Shinn & Rapkin, 2000). Multilevel modeling allows for an understanding of how both individual and group characteristics may predict individual level outcomes (Todd, Allen, & Javdani, 2012). For example, multilevel modeling has been used to understand longitudinal changes in substance use of Oxford House residents compared to a usual aftercare condition (Jason, Olson et al., 2007). The effects of the predictor variables on each of four outcomes were examined within each condition over time. At the 24-month follow-up, there was less substance abuse for residents living in Oxford Houses for 6 or more months (16%), compared to participants both with less than 6 months (46%) and assigned to the usual after-care condition (65%).
Geographic information systems (GIS)
GIS supports the detailed analysis of spatial information. GIS enables community researchers to analyze individuals and groups in context, including details about the physical environment, the socioeconomic environment, and the way people move through the environment (Morton, Peterson, Speer, Reid, & Hughey, 2012). The use of publicly available databases analyzed with GIS has revealed that the crime rates immediately surrounding Oxford Houses are no different than the crime rate of a group of control houses, suggesting that recovery homes pose minimal risk to neighbors and communities (Deaner et al., 2009).
Cost-benefit analysis
Cost-benefit analysis is used to evaluate the cost-effectiveness of a program or treatment. The ability to demonstrate that the value of a program’s benefits exceed its costs is important for community psychologists, especially in terms of funding. Cost-benefit analyses show that Oxford House fares favorably in comparison to usual aftercare in terms of cost-effectiveness (Lo Sasso, Byro, Jason, Ferrari, & Olson, 2012; Lo Sasso & Jason, 2012). Using the public payer perspective, which does not include the rent and expenses associated with the Oxford House treatment modality as the residents paid these expenses out of their own pocket, the net benefit for Oxford House over usual aftercare was $31,043 per person.
Structural equation modeling (SEM)
SEM is a technique used to test theoretical relations among variables, including latent variables. While causal relationships may not be tested in nonexperimental studies, SEM is useful for community researchers to test theoretical models, including directional hypotheses. An additional advantage of SEM is the ability to construct latent variables or estimated variables that comprise several measured variables (Bartholomew & Knott, 1999). With Oxford House residents, SEM has been used to test models of theoretically related constructs including mediated paths; for example, Aase, Jason, Ferrari, Li, and Scott (2014) reported that time spent in Oxford House and a 12-step affiliation predicted abstinence outcomes, and this relationship was mediated by increased social support.
Grounded theory
Grounded theory is a qualitative data analysis method that involves coding and categorization of responses based on the researcher’s recognition of emergent themes that are “grounded” in the data (i.e., not predetermined or theoretically based). The use of grounded theory allows the researcher to think about the data in new ways, including finding new insights and rendering hidden assumptions visible (Charmaz, 2006). In the context of Oxford House, grounded theory has been used to understand how participants think about leadership in a women’s Oxford House (Davis et al., 2006; Timpo et al., 2014) and Latinos’ perceptions of living in an Oxford House (Alvarez et al., 2009).
DISCUSSION
The current review outlined the study designs and data analyses that were used in a community psychology research program involving the Oxford House organization and its members. Methodological pluralism can exist at different levels, and research on the Oxford House organization exemplifies a pluralistic research program (i.e., when pluralism occurs across a program of studies conducted by the same team on one topic). Overall, studies on Oxford Houses over the past 20 years have demonstrated their benefits for residents and communities. Oxford House residents, in comparison to controls, are more likely to maintain sobriety and have improved health outcomes (Jason, Olson, et al., 2008). Studies on Oxford House have supported its benefits for men, women, individuals of different racial and ethnic groups, individuals with different types of substance use problems, and individuals with a number of co-occurring conditions (Alvarez et al., 2006; Belyaev-Glantsman et al., 2009; Contreras & Jason, 2013; Czarlinski et al., 2011; Jason, Mileviciute et al., 2011; Jason, DiGangi, et al., 2013).
A multitude of quantitative, qualitative, and mixed methods study designs and data analyses were used in these studies to understand the many facets of Oxford Houses and their residents. Specifically, 16% of published empirical studies used either qualitative or mixed method study designs. Of the studies with quantitative designs, a variety of methods, including longitudinal, observational, experimental, and quasi-experimental, were used. This review of methods and their contributions illustrated the notion that different methodologies are best suited to answer different questions.
Quantitative methods are useful for examining patterns of behavior, studying change across time with large samples, and developing causal models that can be used for understanding the effect of interventions (Banyard & Miller, 1998). Quantitative research methods provided support for the Oxford House organizations’ effectiveness and cost-effectiveness for a variety of subpopulations. In contrast, qualitative methods are particularly useful approaches for valuing diversity by allowing for rich description that may be missed by asking people to answer more forced choice answers (Banyard & Miller, 1998). Qualitative methods used with Oxford House yielded ecologically sophisticated descriptions about facets of recovery residences.
Finally, the use of both quantitative and qualitative methods, within a single study or a pluralistic research program, provides a more complete picture than either type of method alone. In the Oxford House research program, quantitative and qualitative data were collected and analyzed in ways that made them mutually informative. Moreover, the use of multiple methods reduces the potential effects of the methods themselves on the findings. When multiple methods yield similar findings on a singular topic, the findings are strengthened. In this way, methodological pluralism has contributed to the rigor of this research program in advancing knowledge on recovery residences.
The pluralist approach, which emphasizes the value of diverse sources of information, has also facilitated the active participation of community members in the spirit of community psychology. Researchers from the DePaul University constantly sought ideas from the Oxford House organization, and organization members frequently provided ideas that formed the basis of our work. For example, at an Oxford House convention, a member told the third author that the development of tolerance that occurred in recovery residences was critical to recovery, and he encouraged our team to study this in future research. Olson, Jason, Davidson, and Ferrari (2009) subsequently incorporated this idea into a larger research project, and findings demonstrated that Oxford House members had larger changes in tolerance over time compared to the usual care group.
As an example of how data can be used to affect policy, the third author was called by a lawyer who asked for help with a dispute involving a town that was trying to close down the local Oxford House. Several town residents cited an ordinance that prohibited more than five unrelated individuals from living in one home. Some communities oppose sharing their neighborhood with group homes, including Oxford House, and laws are occasionally passed that make it illegal for more than five unrelated people to live in a house. These efforts deliberately target recovery homes, which usually need 6 to 10 house members to make rent affordable. Our research team used one of our national Oxford House data sets to investigate whether the number of residents in Oxford House affected residents’ individual outlooks for recovery. We found that larger house sizes, of 8 to 10 residents, corresponded with less criminal and aggressive behavior.
These results were used in this court case, and in others, to successfully argue against closing Oxford Houses that had six or more nonrelated residents (Jason, Groh, et al., 2008). Research has also shown that Oxford Houses do not decrease neighborhood property values and may even exert a positive influence on their surrounding neighbors and communities, despite frequent political and neighborhood opposition (Jason, Roberts, & Olson, 2005). Finally, based on our work with this organization, the federal government listed Oxford Houses within the National Registry of Evidenced-Based Programs and Practices (SAMSHA’s National Registry of Evidence-Based Programs and Practices, 2011). The Oxford House organization has been able to use this designation to gain more support and expand the number of houses.
By reflecting on the research methods that have been used in collaboration with the Oxford House organization, the present article outlines the many ways to empirically evaluate and investigate recovery residences. Until recently, research on substance abuse has largely focused on formal treatment and the genetics and neuroscience of addiction. Although studying these individual factors is important for addiction research, investigating environmental factors and how they interact with individuals is equally fundamental for recovery research. In fact, the Obama administration, through the National Drug Control Strategy (2013), has emphasized the importance of working with stakeholders to develop services that support sustained recovery, such as recovery residences and other peer support services that can be provided by individuals who themselves are in recovery. Further research on the role of housing and recovery residences in substance abuse will enable policy makers and practitioners to better understand the legitimacy and importance of recovery residences and mutually supportive recovery organizations (Jason et al., 2013).
Limitations
This article has several limitations. Current or former members of the DePaul University research team conducted all of the studies included in this article, because other research teams have not yet conducted empirical evaluations of Oxford House. Although there are strengths associated with having a pluralistic research program, there are also many benefits associated with having findings replicated by other research teams. Thus, as with all scientific endeavors, it is of importance for other research teams to replicate the findings summarized in this article. Another limitation may be that the researchers conducting the current review are also associated with the aforementioned research team. As demonstrated in research on psychotherapy, researcher allegiance may introduce a source of bias in treatment outcome evaluations. However, the aims of the current review were not focused on evaluating the effectiveness of Oxford House as a treatment, but rather evaluating the research processes that have contributed to existing knowledge on Oxford Houses in hopes of stimulating further research on recovery residences.
There are certainly more questions about recovery residences that need to be examined. For example, there is a need for more studies evaluating the effectiveness of Oxford House versus other types of recovery homes. Recovery residences have been classified into “Levels of Support” by the National Association of Recovery Residences based on the intensity and duration of support that are offered to the residents (Jason et al., 2013), where a “Level 1” classification reflects minimal structure and supervision (such as an Oxford House) and a “Level 4” classification reflects a highly structured and staffed program.
Conclusion
There is a need for research that examines how many recovery residences exist in the United States, the geographic distributions of these residences, the factors that promote sustainability of these residences, the physical and social characteristics of residences, their long-term effects for residents, and the neighborhood’s effects on the residence and vice versa. Additionally, research is needed on which level of support is most appropriate for specific individuals. There is also a need for comparisons of long-term residential treatment programs, shelters, safe havens, transitional housing, incarceration, and homelessness programs. We hope that the quantitative and qualitative study designs and data analysis techniques delineated here can provide a framework for researchers to answer these critical questions in future research with recovery residences.
Limitations
This article has several limitations. Current or former members of the DePaul University research team conducted all of the studies included in this article, because other research teams have not yet conducted empirical evaluations of Oxford House. Although there are strengths associated with having a pluralistic research program, there are also many benefits associated with having findings replicated by other research teams. Thus, as with all scientific endeavors, it is of importance for other research teams to replicate the findings summarized in this article. Another limitation may be that the researchers conducting the current review are also associated with the aforementioned research team. As demonstrated in research on psychotherapy, researcher allegiance may introduce a source of bias in treatment outcome evaluations. However, the aims of the current review were not focused on evaluating the effectiveness of Oxford House as a treatment, but rather evaluating the research processes that have contributed to existing knowledge on Oxford Houses in hopes of stimulating further research on recovery residences.
There are certainly more questions about recovery residences that need to be examined. For example, there is a need for more studies evaluating the effectiveness of Oxford House versus other types of recovery homes. Recovery residences have been classified into “Levels of Support” by the National Association of Recovery Residences based on the intensity and duration of support that are offered to the residents (Jason et al., 2013), where a “Level 1” classification reflects minimal structure and supervision (such as an Oxford House) and a “Level 4” classification reflects a highly structured and staffed program.
Conclusion
There is a need for research that examines how many recovery residences exist in the United States, the geographic distributions of these residences, the factors that promote sustainability of these residences, the physical and social characteristics of residences, their long-term effects for residents, and the neighborhood’s effects on the residence and vice versa. Additionally, research is needed on which level of support is most appropriate for specific individuals. There is also a need for comparisons of long-term residential treatment programs, shelters, safe havens, transitional housing, incarceration, and homelessness programs. We hope that the quantitative and qualitative study designs and data analysis techniques delineated here can provide a framework for researchers to answer these critical questions in future research with recovery residences.
Acknowledgments
We appreciate the support of Paul Molloy and Leon Venable and the many Oxford House members who have collaborated with our team for over two decades.
We appreciate the financial support from the National Institute on Alcohol Abuse and Alcoholism (NIAAA grant numbers AA12218 and AA16973), the National Institute on Drug Abuse (NIDA grant numbers DA13231 and DA19935), and the National Institute on Minority Health and Health Disparities (grant MD002748). We also acknowledge the National Institute on Drug Abuse research training grant (T32 DA 019426: J.K. Tebes, P.I.).
Abstract
This article evaluates how a plurality of research methods has served a research program that has functioned in a much-needed area of research: the role of housing and recovery residences in addiction recovery. The review focuses on one mutually supportive recovery residence model, called Oxford House, which represents more than 1,700 democratic, self-governing residences. To date, there has been no comprehensive evaluation of the research methods used with Oxford House or any other recovery residence. In this article, research methods, including study designs and data analyses, are summarized for 114 peer-reviewed empirical studies that included data on Oxford Houses or Oxford House residents. This review of a pluralistic research program can inform community researchers about the value of recovery residences, the many ways in which recovery residences may be assessed, and the benefits of using multiple methods. Implications for future recovery residence research are discussed.
Methodological Pluralism in Community Psychology
Methodological pluralism involves finding value in a variety of sources of information, including believing that no research method is inherently superior to any other (Barker & Pistrang, 2005; Barker, Pistrang, & Elliott, 2002). Community psychology researchers in particular aspire to methodological pluralism, as it aligns with the field’s values of diversity, empowerment, and attention to context (Banyard & Miller, 1998; Barker & Pistrang, 2005; 2012; Orford, 2008; Rappaport, 1990). The research topics that community psychologists study are often complex and contextually situated, and a plurality of research methods are best suited to address this complexity (Campbell, Gregory, Patterson, & Bybee, 2012).
Mixed methods research can occur within the confines of a single study or can exist at multiple levels (Campbell et al., 2012). First, a pluralistic study, also known as a mixed methods study, is the first level at which pluralism can occur. A mixed methods study incorporates both quantitative and qualitative data collection and analysis. Next, a pluralistic research program reflects pluralism across a program of studies conducted by the same team on one topic. Finally, pluralistic research disciplines are fields of science or scholarship in which a balance of study designs and analysis techniques is used in publications (Barker & Pistrang, 2012). Community psychology is an example of a pluralistic research discipline, particularly since a call to increase the use of qualitative research methods in the 1990s (Banyard & Miller, 1998; Luke, 2005).
Although nearly all community psychologists would endorse pluralism in principle, there have been few attempts to examine how the pluralist position works in practice (Barker & Pistrang, 2005). An evaluation of the research methodologies employed by community psychology researchers is an important task for the field. This article reviews and describes how methodological pluralism has functioned in a much-needed area of research: the role of housing and recovery residences in addiction recovery.
Recovery Residences and Oxford House
Substance-related disorders pose serious public health threats and exact costly social and economic tolls on users, their families and friends, neighborhoods, and the larger society (Jason, Olson, & Foli, 2008). A small but rapidly growing body of research supports the effectiveness of recovery residences for long-term addiction recovery, and a white policy paper on this topic has been published by the Society for Community Research and Action (SCRA; Jason, Mericle, Polcin, & White, 2013). Recovery residencies are safe, sober, and healthy living environments that promote recovery from substance use disorders and associated problems (Jason, Mericle, Polcin, White, & The National Association of Recovery Residences, 2012, September 20). By providing a physical place and a social world in which to recover, the recovery residence model offers promising mechanisms of change that extend beyond the arenas of traditional addiction treatment to increase the probability of long-term abstinence (Jason, Mericle, et al., 2013).
Since the advent of mutually and self-supported recovery residences in the 1960s and 1970s, they have continued to grow in the United States (Jason et al., 2012, September 20). During this time period, researchers and practitioners called for a shift from short-term treatment models toward recovery models that emphasize sustained, long-term well-being (Jason, Mericle, et al., 2013). However, there are still many challenges for researchers who want to study after-care recovery residences, given the predominant funding emphasis on more formal substance use treatment issues as well as the neuroscience of addiction (Jason, Mericle, et al., 2013). Furthermore, efforts to open and fund new recovery residences may be met with political and community resistance due to stigma or misunderstandings (Jason, Groh, et al., 2008).
The largest and most empirically supported network of recovery residences is the Oxford House organization. Since its founding in 1975, the Oxford House organization has grown to over 1,700 homes in the United States, housing more than 10,000 individuals (Oxford House, 2014). Similar to other recovery residences, Oxford Houses offer sober, safe, healthy living environments that promote recovery from substance use by promoting abstinence and gains in life domains, such as education, employment, and family relationships. Oxford Houses are unique residential recovery settings in that they are democratically run with no staff and emphasize sobriety, employment, chores, and opportunities to develop responsibility and leadership skills (Jason, Ferrari, Freeland, Danielewicz, & Olson, 2005; Jason, Schober, & Olson, 2008).
Current Review
Community psychology values emphasize the importance of the relationships between community researchers and oppressed groups, and research should be driven by the needs of the community with their active participation (Foster-Fishman, Nowell, Deacon, Nievar, & McCann, 2005; Orford, 2008; SCRA, 2014). In accordance with these values, it is important to consider whether research methods include, empower, and ultimately benefit community members (Nelson, Prilleltensky, & MacGillivary, 2001; Suarez-Balcazar et al., 2004). The SCRA asserts that the use of multiple methodologies is aligned with these values of community psychology (SCRA, 2014). However, there has been no comprehensive examination of the diverse research methodologies or data analysis techniques used with Oxford Houses or any other recovery residence.
Additionally, although mixed methods and methodological pluralism are highly regarded among community psychology researchers, and although competency in multiple research methods is considered an important aspect of training for community psychologists, few attempts have been made to explain how the pluralist position works in practice (Barker & Pistrang, 2012; Christens, Connell, Faust, Haber, & the Council of Education Programs, 2015; Jason & Glenwick, 2015). There are few evaluations of research methodologies within any long-standing, community-based, research program.
Following recommendations in the literature, we reviewed the research methods used in partnership with Oxford House. We assessed study designs and data analysis techniques both quantitatively, by coding each study, and qualitatively, by subjectively assessing how exemplars of each design have contributed to this pluralistic research program. We hope to better understand how a plurality of methods has served this research program in terms of both advancing knowledge and actively including community members. Further, we illustrate the many ways in which recovery residences may be assessed to aid future research development in this important area of research.
Contributor Information
Emily M. May, Pennsylvania State University.
Bronwyn A. Hunter, University of Maryland, Baltimore County.
Leonard A. Jason, Center for Community Research, DePaul University.