HomeAboutBackground InformationHow to UseStart HereUseful LinksFeedbackGlossary Contact

Background Information

What is the Framework for Measuring Impact?
The Framework is a decision making framework designed to help health professionals select and use appropriate outcome measures to demonstrate the impact of their practice.  The outcome measures recommended are global, so relevant to most health professionals, and fall in line with the NHSScotland Quality strategy (2010) key drivers.

How were the items included in the Framework selected?
Domain branches and sub-branches
The domain branches (person centred care, effectiveness, and efficiency) were selected based on the NHSScotland Quality Strategy’s (2010) key drivers.  The sub-branches were selected based on existing clinical knowledge, an informal review of the literature surrounding outcome measures, and through formal and informal discussions with clinicians and other key-stakeholders.  These branches and sub-branches were reflexive and subject to change depending on a search of the outcomes measure literature which identified relevant existing outcome measures, key-stakeholder interviews and case study piloting investigating the suitability and usability of the Framework in practice.

Identifying outcome measures
The outcome measures considered for inclusion in the Framework were derived through both empirical and clinical sources. A substantial search of the outcomes measure literature was carried out to identify existing measures that appeared relevant for inclusion in the Framework.  An explicit search strategy was used for each of the four domain branches to search three electronic databases (MEDLINE, CINAHL, PsycInfo).  Over 10,000 papers were initially identified.  Following exclusion criteria, a total of 577 academic papers were scrutinised to identify outcome measures. A group of 282 clinicians ranging from eight AHP professions (physiotherapy, occupational therapy, speech and language therapy, podiatry, dietetics, orthotics, orthoptics, and therapeutic radiography) were also surveyed via email whether they had any particular outcome measure recommendations.  

Developing inclusion criteria for outcome measure selection
Stakeholder meetings
Interviews with key stakeholders were conducted.  These included individuals involved in policy making within Scottish Government, AHP leads, AHP consultants, and clinicians.  These stakeholders were introduced to an early version of the Framework containing the key branch domains, the sub-domains and examples of outcome measures identified as important for inclusion early on in its development. Stakeholders were asked to comment on the draft Framework to identify where relevant information was missing, and on the perceived usefulness of the Framework to their own work.  They were also asked what their requirements of the Framework were.  Following these interviews, any required changes were made to the Framework’s structure.  A group of key stakeholders met to discuss the Framework and develop the structure of the efficiency domain.

Literature Review
A systematic review of the literature was carried out investigating the facilitators and barriers to routine outcome measurement.  The information from the systematic review was used to shape the guidance given to practitioners at each stage of the Framework.  Key facilitators and barriers were identified under the following themes:

  • Knowledge, education and interest in outcome measures
  • Support/priority for outcome measure use
  • Practical considerations
  • Patient considerations

The information obtained through the stakeholder meetings and systematic review was synthesised and used to develop the selection criteria for the outcome measures to be included in the Framework.  All of the identified outcome measures were considered using the selection criteria and were consequently either included or excluded.

Structuring the Framework
Included outcome measures were grouped into descriptive categories.  They were then placed into the Framework under one of the domain branches, and, where appropriate, one of the sub-branches.  This categorisation was conducted by one of the researchers (JM) in the first instance.  Categorisation was confirmed by a second researcher (ED) and any disputes were resolved by a third researcher who acted as arbitrator.

Investigating the suitability and usability of the Framework in clinical practice
Case study methodology was used to pilot the Framework across three clinical sites.  Clinicians were asked to use the Framework as part of their routine work.  During and after the case study time period, clinicians experience on using the Framework was sought.  At the end of the study interviews were conducted with clinicians to obtain a more full account of their experiences while using the Framework.  The feedback obtained during the case studies were then incorporated into the final design of the Framework. 

The CARE project
As part of the Framework project we were also asked to carry out a large scale service evaluation of the Consultation and Relational Empathy (CARE) Measure for use by allied health professionals.  This was a Scotland based study involving four NHS regions and eight allied health professions (physiotherapists, occupational therapists, speech and language therapists, podiatrists, dieticians, orthotists, orthoptists, and therapeutic radiographers).  In total, 265 AHPs from in-patient, out-patient and domiciliary settings, collecting a total of 7328 CARE Measures from patients completed the study, making this one of the largest AHP studies ever carried out in the UK.  While this large amount of data is still undergoing analysis on various elements, we can confirm that the CARE Measure has been assessed as valid and reliable in the majority of settings, and is feasible for most of the allied health professions.  However, some exclusions do apply (due to different ways of working in practice) and the CARE Measure could not be assessed as feasible for every AHP setting and profession.  More details on where the CARE Measure is and is not suitable for use is included under the CARE Measure's profile within the Framework here.  As the CARE Measure is heavily cited in the NHS Quality strategy (2010) and has had such an extensive validation, reliability and feasibility assessment, we sternly recommend its use, where possible, to measure person centred care.

Publications and presentations arising from the Framework and CARE Measure projects
Below is a list of publications and presentations that relate to our Framework and CARE Measure studies.

Publications:
paper preparation in progress

Duncan, E. A. S., & Murray, J. (2012).  The barriers and facilitators to routine outcome measurement by allied health professionals in practice: A systematic reviewBMC Health Services Research, 12:96 (online open access).

Duncan, E. A. S. (2011).  Routine outcome measurement in practice: Overcoming challenges, seeking solutions, demonstrating impactAustralian Occupational Therapy Journal, 58(4), 221.

Presentations:
Renfrew, L., Murray, J., & Duncan, E. A. S. (2012). Capturing person centered care using the CARE Measure with AHPs in NHS Ayrshire & Arran.  Poster presented at NHSScotalnd Event, SECC, Glasgow, June 21-22.

Murray, J., & Duncan, E.A.S. (2011).  Measuring the Impact of AHP Activity: The Development of the EQuiPP.  Allied Health Professions Leadership Programme, NHS Ayrshire & Arran, SAC Ayr, January 20.

Murray, J., & Duncan, E.A.S. (2010).  Measuring the Impact of AHP Activity: The Development of the EQuiPP.  Allied Health Professions Leadership Programme, NHS Ayrshire & Arran, SAC Ayr, December 2.

Murray, J., & Duncan, E.A.S. (2010).  Measuring the Impact of AHP Activity: The Development of the EQuiPP.  Allied Health Professions Leadership Programme, NHS Ayrshire & Arran, SAC Ayr, November 4.

Murray, J., & Duncan, E.A.S. (2010).  Identifying the Facilitators and Barriers to Routine Outcome Measurement for Allied Health Professionals.  Health and Social Research Group Meeting, Department of Psychology, University of Stirling, September 30.

Articles cited in this website:
Bauman, A.E., Fardy, J.H., & Harris, P.G., (2003).  Getting it right: Why bother with patient centered care?  Medical Journal of Australia, 179, 253-256.

Chew, L.D., Bradley, K.A., & Boyko, E.J. (2004).  Brief questions to identify patients with inadequate health literacy. Family Medicine, 36(8), 588–94.

Chew, L.D., Griffin, J.M., Partin, M.R., Noorbaloochi, S., et al. (2008).  Validation of screening questions for limited health literacy in a large VA outpatient population.  Journal of General Internal Medicine, 23(5), 561-566.

Cleary, P.D., Edgman-Levitan, S., Roberts, M., et al. (1991).  Patients evaluate their hospital care: A national survey.  Health Affairs, 10, 254-267.

Derose, S.F., & Petitti, D.B. (2003).  Measuring quality of care and performance from a population health care perspective.  Annual Review of Public Health, 24, 363-384.

Duncan, E.A.S., Entwhistle, V., & Liddle, K. Person Centred Care: Clarifying the Concept. (in preparation)

Fawcett, A.L. (2007). Principles of Assessment and Outcome Measurement for Occupational Therapists and Physiotherapists: THeory, Skills and Application. Chichester: John Wiley and Sons.

Felton, K. (2005).  Meaning-based quality-of-life measurement: A way forward in conceptualizing and measuring client outcomes?  British Journal of Social Work, 35, 221-236.

Holdsworth, L., & Webster, V. (2006).  Patient Self Referral: a Guide for Therapists.  Milton Keynes: Radcliffe Publishing Ltd.

Jenkinson, C., Coulter, A. & Bruster, S. (2002).  The Picker Patient Experience Questionnaire: Development and validation using data from in-patient surveys in five countries.  International Journal for Quality in Health Care, 14(5), 353-358.

Local Government Improvement and Development. (2011).  Measuring Impact in Health Improvement: An Accessible Guide for Health Practitioners.  London: Local government and Improvement.

Mercer, S.W., McConnachie, A., Maxwell, M., Heaney, D.H., & Watt, G.C.M. (2005).  Relevance and performance of the Consultation and Relational Empathy (CARE) Measure in general practiceFamily Practice, 22(3), 328-334.

Mercer, S.W., Watt, G.C.M., Maxwell, M., & Heaney, D.H.. (2004).  The development and preliminary validation of the Consultation and Relational Empathy (CARE) Measure: an empathy-based consultation process measureFamily Practice, 21(6), 699-705.

Mercer, S.W., & Reynolds, W.J. (2002).  Empathy and quality of care. BJGP, 52(Suppplement), S9-S12.

NHSScotland (2010).  The Healthcare Quality Strategy for NHSScotland.  Edinburgh: Scottish Government.

Picker Institute. (2002). Picker Institute Europe. 2002. http://www.pickereurope.org/

Picker institute. (2009).  Using Patient Feedback.  Picker Institute Europe.

Steine, S., Finset, A., & Laerum, E. (2001).  A new, brief questionnaire (PEQ) developed in primary care for measuring patients’ experience of health interaction, emotion and consultation outcome.  Family Practice, 18(4), 410-417.

Ware, J.E., & Sherbourne, C.D. (1992). The MOS 36-item short-form health survey (SF-36). Medical Care, 30(6), 473-483.

 

© Copyright 2012 NMAHP RU, Stirling University