In the previous post I was worried about our tendency to measure structure and process indicators and assume that they relate to outcomes. How well we know that what we do affects outcomes could fill a post of its own (maybe later). But let’s look at outcomes. What should we measure?
Let’s start with life and death – mortality rates – I don’t think any of us would argue that whether they are dead or alive is a matter of fairly major concern to the majority of patients. The cardiothoracic surgeons were the first ones to get going on this in the wake of Bristol. However, there were concerns about this, particularly in elective surgery. If you compare below Mr Scrub 1 (left), who isn’t in all honesty a very good surgeon but only operates on relatively healthy patients with Mr Scrub 2 (right), who is an extremely skilled surgeon with an excellent team but is willing to operate on much sicker patients, their crude mortality rates may be exactly the same. That can’t be fair can it?
Might surgeons shy away from operating on frailer patients (or cherry pick the healthiest ones) in order to improve their figures? This is the point of casemix adjustment – crude mortality is adjusted for demographics like age and comorbidities in an attempt to level the playing field. And so the Hospital Standardised Mortality Ratio and the Dr Foster controversy was born. Unfortunately, like everything, the HSMR wasn’t perfect. It only looked at inpatient mortality so a Trust that discharged patients who then dropped dead at home wasn’t picked up, and only examined deaths within certain diagnostic groups. It also became possible to improve a trust’s performance on HSMR by making the patients “sicker”, ie with more comorbidities. So potentially a trust’s position in the mortality league tables owed more to the quality of its coders than its clinicians.
This, according to Francis, was part of the problem at mid-Staffs, where the reaction of the trust board to apparently poor mortality figures was to commission research into flaws in the stats, rather than to look at the quality of care. This is not to say that the research they commissioned didn’t raise some very valid points; it’s still worth a read.
Obviously this required a response so the Department of Health commissioned the excellent people at Sheffield Uni (I have to say that, I’m doing my doctorate with them!) to develop an alternative to HSMR, which they did – SHMI. Sadly all methods of casemix adjustment have the same limitations – we want to control for anything, apart from the quality of care, that might affect mortality. The first problem is working out what all these things are, never mind recording them reliably……. The second is assuming that the same thing confers the same risk in all population (the constant risk fallacy). Let’s assume that a diagnosis of asthma is a risk factor for respiratory mortality – does this mean that Mr Smythe in Affluentshire who has an inhaler because occasionally he gets a bit wheezy on the golf course when it’s cold is at the same increased risk as Mr Brown in Deprivedville who in addition to his reactive airways has a 30 pack year history, 5 courses of steroids a year and poor health literacy? Of course not!
One last thought – even if we can sort out all these issues and work out what an institution’s mortality rate “should” be, can we assume that deviation from this reflects quality of care (or lack of it)? Some seriously scary headlines have trumpeted tens of thousands of deaths that “could have been prevented“. Let’s think about that – in terms of avoidable mortality, patients occupy a Goldilocks zone; most will not die irrespective of suboptimal care (assuming the absence of particular acts of commission like inadvertent intravenous potassium administration); many have simply reached the end of their life and care should be aimed at comfort and dignity, not “life-saving”.
So how many patients are in this Goldilocks zone? The NCEPOD report “Time to Intervene” found that over 1/3 hospital cardiac arrest calls (codes for my American friends) were potentially preventable; however they felt 74 of these 156 calls were “preventable” because the patient should already have had a DNAR order. So a preventable crash call, but not preventable mortality.
So that’s why it’s not just as simple as life and death – and that’s before we even consider all the other morbidity-related outcomes that patients care about…….
Update 6th Feb 2014
Just out in Annals of Emergency Medicine, a Canadian Delphi-type study aiming to identify diagnoses where mortality and morbidity may be sensitive to the quality and timeliness of emergency care:
I like the list they came up with (although I’m not sure how much difference we always make to ACS or diverticulitis).