Updating contextualized clinical practice guidelines on stroke rehabilitation and low back pain management using a novel assessment framework that standardizes decisions

Background

The PARM group first published two Filipino-contextualized clinical practice guidelines (CPGs) on stroke rehabilitation, and low back pain, in 2012. They are freely available on the PARM website [1, 2], and have since been the subject of nation-wide baseline audit and implementation activity [3–5]. The PARM clinical practice guidelines for stroke and low back pain have been endorsed by the International Society of Physical and Rehabilitation Medicine (ISPRM) which provided the members’ recommendations for best practice in the field of rehabilitation medicine. The low back pain guideline has been submitted to the Philippine Health Insurance Corporation (PHIC), and now serves as the basis for reimbursement of fees for the management of low back pain which includes rehabilitation consultation, physical therapy treatments, non-surgical interventions such as acupuncture and epidural steroid injections; and diagnostic procedures such as spine X-ray, magnetic resonance imaging and electromyography. This will be ground-breaking in a developing country such as the Philippines because rehabilitation services are presently not being subsidized by the government health insurance agency.

In line with international recommendations regarding the importance of the currency of evidence in clinical guidelines [6, 7], the 2012 Filipino-contextualized stroke and low back pain guidelines were due for revision and updating in 2014. Updating guidelines is an essential process that incorporates the best new evidence in its recommendations, using relevant new scientific research including new technologies in the diagnostic and treatment alternatives, economic differences or changes in values and preferences [8]. The updating process consists of the identification of new evidence, the assessment whether the new evidence warrants an update, and the formulation of new or modified recommendations [8, 9].

However, there are only a few publications that reported methods and approaches for updating guidelines [10–13]. This appears to reflect the greater emphasis placed on methods for developing ‘de novo’ (new) evidence based clinical practice guidelines [14]. International studies have consistently shown that common difficulties encountered in updating guidelines are: (1) the most appropriate timeframe within which guidelines should be updated; (2) lack of guidance in choosing a rigorous, efficient, standardized updating process; (3) the lack of efficient monitoring systems to identify new, potentially-relevant evidence; (4) decisions regarding extent of updating (whether updates should be partial or full); and (5) the cost effectiveness of updating a practice guideline [6, 7, 14].

Thus, as with guideline contextualization, the PARM group found itself without international guidance, and faced with this question: how do we update clinical practice guidelines using contextually-relevant principles? These principles need to provide a transparent, standard and comprehensive platform which all team members could use, when updating the current stroke and low back pain guidelines, and which could be used ongoing to update other contextualized guidelines as necessary.

This paper reports on the approach taken by PARM to develop a standard tool to assist clinicians involved in updating clinical guidelines in developing countries. This tool dovetails with the PARM writing guide [3], which underpinned the development of the foundation PARM contextualized guidelines for stroke and low back pain.

Methods

Ethical approval was obtained from the ethics review board of the Philippine Academy of Rehabilitation Medicine. Informed consent was obtained from each member of the working group involved in this study.

As occurred for the original PARM guidelines, all updating work was voluntarily undertaken by a working party of approximately 35 physiatrists (rehabilitation doctors) who were members of the PARM. Participants were invited for their research background and willingness to contribute. The group ranged in age and clinical experience, and approximately 75 % had not been involved in the foundational contextualization process 3 years before. The group met for an intensive weekend of training, discussion and debate about updating processes, and to develop the updating tool. Then smaller working groups undertook to update specific sections of the stroke and low back pain guidelines. Each group applied the updating tool to one or more recommendations before the weekend workshop finished, so that any concerns regarding the updating process could be identified and clarified.

To assist in standardizing the guideline contextualization process used in the development of the original guidelines, a PARM writing guide was established [3]. This guide establishes a uniform framework for summarizing differently-worded recommendations and differently-reported strengths of the body of evidence for recommendations extracted from the included guidelines, relevant to a particular situation in the Filipino patient journey. The guide is to be used in the event that there are: more than one relevant recommendation extracted from the relevant guidelines, which addresses a particular aspect of the Filipino patient journey, and/or different methods of reporting the underpinning strength of the body of evidence of the relevant recommendations from the included guidelines. All relevant recommendations (to the patient journey) were collated in a table for each element of the journey, along with the underpinning levels of evidence, and the guideline reference from which the recommendation had been extracted. The concept of uniform thought was coined by the PARM group to identify similar intent, from differently worded recommendations from different guidelines. This was found to be a critical step in the contextualizing process, because guidelines formed the source of recommendations and the evidence base, rather than individual literature. This meant that the working parties often needed to resolve the issue of different wording in recommendations, despite the same intent of the recommendation and the same underpinning references. The evidence body was thus described in six different categories by the PARM contextualization process [3] (Table 1).

These categories were then synthesized into the wording that PARM used to present the recommendations which had been distilled from the included guidelines during the 2012 contextualization process. These were PARM Strongly Endorses, PARM Endorses, PARM Recommends, PARM Suggests or PARM Does not endorse.

The new evidence body against which recommendations in the 2012 PARM-contextualized stroke and low back pain guidelines were assessed, consisted of primary and secondary literature published since 2011, and which were reported in the new clinical guidelines. The 2011 date was set, because the 2012 guidelines included in the contextualization process, were based on literature searches up until 2011.

Clinical guidelines in stroke rehabilitation and low back pain management published internationally since 2011 were sought from a comprehensive search of the following electronic databases: PubMed, Google Scholar, National Institute for Health and Clinical Excellence (NICE), Scottish Intercollegiate Guidelines Network (SIGN), National Health and Medical Research Center (NHMRC), New Zealand Guidelines Group (NZGG), National Guidelines Clearinghouse (NGC), using key words of clinical practice guidelines and stroke rehabilitation (or low back pain). Criteria for inclusion in the updating process were: that the guideline included rehabilitation recommendations; the guideline was available in full text; it was published in English language; published since 2010 (the close-off date of literature inclusion for the inaugural PARM guidelines); and not included in the original contextualized PARM guidelines. Moreover, only de novo guidelines, or comprehensive updates of guidelines (which included new literature) were included. Guidelines that re-stated, or adopted the findings of other guidelines, without adding to the body of literature, were excluded.

The 14-item guideline appraisal tool by the International Centre for Allied Health Evidence (iCAHE) was applied to each guideline [15]. Particular attention was paid to the items related to availability of information on the hierarchy of evidence and the quality of studies which underpinned recommendations. If information was inadequate or not available, the guideline could not be included in the updating process. To be included in the study, PARM determined that a score of 10 should be used as the quality cut-off (71 % quality criteria met).

The basis for PARM updating used the four levels proposed by Johnston and his colleagues [10]. The specifications of the PARM writing guide for strength of evidence base, uniformity of thought and volume of references were amalgamated with the Johnston et al. [10] guideline updating approach (Table 2). A number of different descriptions of the synthesized information were presented to the PARM writing group for consideration, and then the preferred options were trialled. Modifications were made based on the utility and reliability of the guide, when it was applied to different evidence examples, and a final version was agreed upon.

A flow diagram of process in updating contextualized guidelines is shown as Fig. 1.

Fig. 1

Flow diagram of process in updating guidelines

Results

Additional file 1: Appendix 1 describes the consort diagrams for the CPG inclusion process for stroke and low back pain. Ten new guidelines on stroke and eleven for low back pain were identified. For the management of low back pain, four of the eleven clinical practice guidelines initially identified did not meet the inclusion criteria: two were not readily available via the internet [16, 17]; one was not published in English [18] one was not a de novo or updated CPG [19]. For the stroke rehabilitation CPGS, of the ten clinical practice guidelines that were initially identified, three were excluded as there was no rehabilitation aspect included in the recommendations [20–22] while another was excluded because it did not rank the quality of the evidence and did not link the hierarchy of evidence to its recommendations [23]. Instead, it used a formal consensus approach in formulating the recommendations. The PARM group agreed that this would not make it possible to compare the quality of evidence with other guidelines.

Seven potentially relevant CPGs for low back pain were critically appraised and three CPGs scored less than 71 % [24–26]. Therefore only four CPGs were included in the revision of the PARM contextualized CPG for low back pain [27–30] (see Table 3). Six potentially-relevant CPGs for stroke were assessed for methodological quality, and one was excluded due to a low methodological score (9/14) [31]. The remaining five guidelines were included in the update [32–36] (see Table 4).

As described in Gonzalez-Suarez et al. [3], designing patient journeys and mapping the steps in them to CPG Recommendations were a critical part of the contextualization process. Thus they were revisited in the 2014 PARM workshop with the focus on assessing generalizability and applicability of the existing guideline recommendations for the two conditions of interest using NHMRC FORM [37] and PARM context points. This process ensured that all the essential steps from initial presentation to discharge from health care services were included, and that new members of the PARM working group were conversant with the patient journey process underpinning the contextualization process of guideline recommendations. This step involved much discussion on the elements of the patient journey, how the previous guidelines’ patient journeys could be improved upon in order to better embed comprehensive and clinically useful recommendations. An example of how the patient journey was modified between 2012 and 2014 is provided in Additional file 1: Appendix 2.

After much debate, the working group agreed that the PARM updating tool should be presented in two different ways (see Table 5; Fig. 2).

Fig. 2

PARM writing guide in revising a recommendation

This guidance was found to be consistently valuable across patient journey points for stroke and low back pain, recommendations and evidence types and sources. This tool assisted the working groups to resolve any dilemma which arose when amalgamating new evidence with old. The most difficult classification was found to be Level 3, where the recommendation itself was not necessarily reworded, but additional statements were sometimes required to incorporate new information which had been reported in recent clinical guidelines. This often related to emergent new technology or refinement of effective treatment techniques (for instance exercise programs). Examples of updated recommendation statements are provided in Table 6.

Discussion

As far as we are aware, the PARM updating tool is the first of its kind, developed to reinforce the updating process of contextualized clinical guidelines in a developing country. The use of the Johnston et al. [10] tool was reported by Grimmer-Somers and Worley (2010) in updating the Australian and New Zealand Acute Pain Management guidelines [38]. This updating framework fitted well with the PARM requirements, as it could be readily understood by the working group members, and applied standardly to any recommendation and its old and new evidence bases.

Vernooij et al. [8] systematically reviewed 35 guideline manuals and found no consistent or clear information about how to update a guideline. Interestingly this paper did not include the Johnson criteria, which are the clearest step by step approach to date. In a systematic review by Martinez Garcia et al. [6] on the strategies for monitoring and updating clinical practice guidelines, there were four studies which considered the updating process [10–13]. All authors agreed that guideline updating processes were neither time nor resource saving. Eccles [11] updated the CPG by exhaustive research, and classified recommendations as follows: new if fresh evidence was identified, refined if supplementary evidence was identified, and unchanged if now evidence was identified. Parmelli et al. [13] updated recommendations using the GRADE framework, where recommendations were classified as strong positive, weak positive, weak negative and strong negative as voted on by the multidisciplinary panel in a series of meetings. A fifth recommendation, ‘no recommendation,’ was eliminated from the categories of strength of evidence. This forced the panelists to take a position on a recommendation even in the absence of strong evidence. Johnston [10] revised recommendations based on the consistency and strength of the evidence. The options reflected the implications of the new evidence on the clinical recommendations. We agreed that this updating process was the most appropriate model in the revision of our contextualized clinical practice guideline.

The PARM updating tool provides a simple, novel and standard approach that considers new evidence’s hierarchy and quality, and revisions of recommendation wordings. It could be used efficiently by other guideline updaters particularly in developing countries, where resources for guideline development and updates are limited. When many people are involved in guideline writing, there is always the possibility of ‘slippage’ in use of wording and interpretation of evidence. The PARM updating tool provides a mechanism for maintaining a standard process for guideline updating processes that can be followed by clinicians with basic training in evidence-based practice principles. The provision of including only high quality reference guidelines in both developing guidelines through contextualization, and updating contextualized guidelines, is one of the strengths of our methodology. A survey by Alonso-Coello [7] showed that the process of updating guidelines is generally not standardized and needs to be more rigorous even if most institutions involved in guideline development have a process for their updates. Our updating approach dovetails with the PARM writing guide [3], which underpinned the development of the foundation contextualized guidelines for stroke and low back pain, rather than a de novo development of a Filipino clinical guideline. We believe that our entire process now offers a resource-efficient process of developing and revising clinical guidelines in order to focus time and resources on evidence dissemination and implementation.

One of the limitations of our approach is that the researchers did not focus on evaluating the currency of the evidence bases of the included CPGs. Shekelle et al. [9] have shown that after 3.5 years, 10 % of the guidelines they reviewed were obsolete, while after 5.8 years, 50 % of the guidelines were outdated. The retrieval of new evidence could either be restricted which would be limited to review, editorials or commentaries of specific journals and expert collaboration [9] or an exhaustive search which was very similar to the process used in the guideline development [10, 39]. In both methods, a methodology group will be needed for skills in systematic search, retrieval and synthesis of evidence which our group did not have. This was largely because of scarcity of resources and time, and voluntary effort from all members. Another limitation of this study would to be potential non-inclusion of high quality guidelines which are not referenced nor published in the internet, as only electronic databases were searched.

Conclusion

PARM has developed a novel framework to assist the process of revising contextualized clinical practice guidelines, dovetailing this with its initial processes to construct a contextualized CPG. Efficiencies of the PARM updating approach included revisiting the patient journey to validate the critical points which required specific recommendations, and a specific writing guide to revise a recommendation when new evidence was available. With this approach, it is envisioned that updating contextualized CPGs processes in the future will be guided by a simple standardized process, and will be effective and efficient in terms of time, finances and manpower.