17 April 2019
Development of key performance indicators for patient experience
Patients’ views of the care they receive in New South Wales’ public hospital services provide valuable insights into the state of our healthcare system that help drive improvements.
Through our many surveys, BHI captures and reports on patients’ healthcare experiences, providing the results to local health districts (LHDs) so they can see how their hospital services are performing. This information shows LHDs how they, and their hospitals, compare to others throughout NSW and enables them to identify areas where improvements could be made.
One of the ways that BHI’s survey results are used to monitor and assess performance in LHDs is through key performance indicators (KPIs).
Until recently, patient experience measures used by the NSW Ministry of Health for LHDs were based on the proportion of positive responses to a single survey question about overall ratings of care for admitted patients and emergency department (ED) patients.
However, as a result of work undertaken by BHI, these measures have been enhanced. The single survey question has been replaced by KPIs that focus attention on a broader range of measures of patients’ experiences and create reliable and statistically robust estimates of performance. This has enabled NSW to establish benchmarks for acceptable standards of patients’ experiences, for the first time.
BHI’s work involved analysis of more than 28,000 responses to the Adult Admitted Patient Survey, using a statistical method called factor analysis, to identify experiences important to patients. From this we developed a subset of survey questions that best measure those experiences to form the basis of KPIs.
We then analysed more than 56,000 responses to the Adult Admitted Patient Surveys from 2015 and 2016 to assess the reliability of the measures proposed to form the basis of KPIs. We also assessed the reliability of quarterly measures for smaller LHDs and vulnerable groups.
As a result, BHI successfully demonstrated that the following three measures provide a broad assessment of some of the most important elements of patients’ experiences:
- overall patient experience index, which includes four survey questions
- patient engagement index, which includes six survey questions
- respect and dignity, which is a single question score.
Following a review of BHI’s results, the NSW Ministry of Health adopted the two index scores as KPIs for admitted patients in the 2018–19 Service Agreements with LHDs. A similar process is underway in relation to ED patient experience.
BHI’s work on developing KPIs is detailed in a Measurement Matters report, Development of patient experience key performance indicators for local health districts in NSW.
TagsAccessibility and timeliness 4 Ambulance services 3 Appropriateness of healthcare 2 BHI - general 7 Challenging ideas 1 Chartpack 2 Data 9 Effectiveness of healthcare 3 Efficiency 1 Elective surgery 2 Emergency department 2 Healthcare services 7 Hospital care 6 International data 1 Patient experience 7 Safety and risk 2
Dr Diane Watson is the Chief Executive of the Bureau of Health Information. She has 20 years of senior management experience measuring, monitoring and reporting on the performance of healthcare systems in Australia and internationally.
Lisa Corscadden is a senior researcher at the Bureau of Health Information. She has experience in healthcare research in Australia and Canada, with an interest in measuring equity in healthcare.
Lilian Daly is the Lead for Strategy and Engagement at the Bureau of Health Information. She holds a Master’s degree in public health from the University of New South Wales and has extensive experience as a healthcare clinician, researcher and educator.
Hilary is our Senior Director, Communications and Strategic Relations. She has extensive experience in developing policy and strategy around the role of information in improving healthcare.
Renee Carter is a former senior analyst at the Bureau of Health Information. She holds a Masters degree in health and social policy from the London School of Economics and a Doctorate in epidemiology from McGill University.
Jean-Frederic is the former Chief Executive of BHI. He is now Chief Executive of the Agency for Clinical Innovation.
Kim is the former Senior Director, Performance Measurement & Reporting at BHI. She has extensive experience in health services research in Australia and internationally.