Surgeon data will lead to alerts over high death rates

 
Surgeons performing heart surgery

When data was released last week about the death rates for individual vascular surgeons, the report from the Royal College of Surgeons stressed reassuringly that "all surgeons are performing within the range expected".

Indeed each doctor named in the document was listed with a green triangle symbol to confirm this status.

But in fact, I have discovered, the few surgeons with the worst apparent records are to be flagged as an "alert", their performance will be investigated and their work will be reviewed.

At this stage it is not possible to say how many doctors, or which ones, this applies to. This is because it has been realised that the published statistics contain some errors. The figures are being checked and updated, and a new analysis should be issued soon by the RCS.

Poor results

On the basis of the data as disclosed last Friday, it is probable that eight surgeons would be flagged as an alert because of their mortality rate on one kind of major surgery, the repair of a highly dangerous bulge in the aorta (an abdominal aortic aneurysm).

And three surgeons would probably get an alert flag due to their poor results on another procedure, to correct the narrowing of an artery in the neck (carotid endarterectomy). It is not clear whether these two groups of surgeons would overlap.

This is out of more than 450 consultants in the UK who perform operations of this sort on major blood vessels.

When the initial data was released last week, the Royal College proclaimed that no individual's performance was an "outlier" because none lay outside an outer limit line on graphs known to statisticians as "funnel plots". The report's graphs show this as the "99.8% limits".

Funnel plot for vascular surgery Source: Royal College of Surgeons

This criterion is equivalent to being worse than the average performance by three standard deviations, a statistical measure of distance from the mean or average.

However, when I read the RCS and Vascular Society policy on outliers, I found that it actually states that a surgeon or unit should be flagged as an "alert", prompting further investigation and review, at the threshold of only two, rather than three, standard deviations from the mean. This is broadly equivalent to the 95% limit line drawn on the funnel plots.

'Robust'

It is clear from looking at the funnel plots that several individual surgeons were within this range between the two limit lines, although it is difficult to identify with certainty who they might be due to the way in which the specific data has been provided.

When I put this to David Mitchell, a consultant with North Bristol NHS Trust who chairs the Vascular Society's audit and quality improvement committee, he confirmed that when the data is revised, anyone found to be over two standard deviations worse than the mean will indeed be flagged as an "alert".

The surgeons involved would be notified, and this would result in further scrutiny of their performance data to see if there is any error or justifiable explanation for the outcomes. If the data is confirmed and no explanation is provided, further action would be up to local NHS Trusts and their senior staff responsible for the quality of care.

An elderly man walks past an NHS sign NHS England has moved to open up health service performance information

"If the data is correct, we will inform their clinical governance leads," says Mr Mitchell. "We can't tell them what to do. Some will take a robust view, others won't."

"Beyond the alert line, our advice would be that there should be a review of the Trust's policy on undertaking this service."

'Exceedingly naive'

The funnel plots are based on risk-adjusted data, which reflect the age of the patients and the nature of the operations. However, the figures published for each named surgeon are not risk-adjusted and therefore are less valid as a basis for comparison.

Start Quote

There was much nervousness among some vascular surgeons about releasing outcomes by doctor rather than by hospital”

End Quote

These statistics are being released as part of a broad drive by NHS England towards opening up health service performance information. While similar statistics have been available for several years for individual heart surgeons, figures on clinical outcomes are now being disclosed for several other surgical specialisms, where national audits are also already established.

The vascular consultants have gone further towards transparency than some of their colleagues in other fields. The vascular data lists all surgeons in one document, whereas others such as the bariatric surgeons (who treat obesity) have only issued look-up tables, where you can look up individual doctors one by one and compare their results to national averages.

A patient being taken into surgery Patients now have increasing access to data about their surgeons

Some other specialists are said to regard the Vascular Society as being "exceedingly naive" in this level of openness. Indeed, there was much nervousness among some vascular surgeons about releasing outcomes by doctor rather than by hospital, and six refused to have their individual figures published.

False reassurance

But Mr Mitchell is convinced it is the right approach: "Everything is laid out there in a way you couldn't get from other audits that are reporting. We're allowing people to make a choice. That's the point of data."

Some patients may now feel they were provided with false reassurance over last week's figures when they were told that no surgeon's record was outside expectations, and they may find that disturbing.

But in another view patients may also feel more reassured by the discovery that the relevant professional bodies are not complacently accepting that all the surgeons involved are performing acceptably - and instead they are challenging those who, the evidence suggests, may not be.

 
Martin Rosenbaum Article written by Martin Rosenbaum Martin Rosenbaum Freedom of information specialist

Simon Hughes spells out his pre-election FOI plans

Justice minister Simon Hughes spells out his plans for changes to freedom of information before the general election.

Read full article

More on This Story

Related Stories

The BBC is not responsible for the content of external Internet sites

Comments

This entry is now closed for comments

Jump to comments pagination
 
  • rate this
    0

    Comment number 18.

    The statistics will only cover planned surgery. No surgeon would agreed to be measured on emergency ruptures which invariably are unsuccessful.

    Planned surgery is associted with a mortality of around 5% which is why surgery is only carried out after the aneurysm/bulge reaches a certain size at which the mortality of the operation is exceeded by the risk of the aneurysm bursting.

  • rate this
    0

    Comment number 17.

    As far as aortic aneurysm repairs are concerned, there should be 2 graphs of results - one for planned surgery which has a lower mortality rate, and one for emergency sugary on leaking/burst aneurysm which has a much higher mortality.

  • rate this
    0

    Comment number 16.

    On an abdominal aortic aneurysm - have they separated statistics between those that were found/fixed before they burst vs those that burst so emergency surgery was required. I have heard that once it has burst, there is a higher mortality rate.

  • rate this
    0

    Comment number 15.

    With any set of figures there will be a mean. Hardly any will be bang on this figure, some will be above and, consequently, some will be below. Improving the performance of those below will simply drive up the mean. Using the mean as a basis for flagging up alerts is akin to a dog chasing its tail.

  • rate this
    +1

    Comment number 14.

    2.Rick - "What will actually happen is that surgeons will not do anything without a careful eye on the league tables........"



    Those types of operation will still happen, but just in a few very specialist centres - when such ops are focussed into a few centres the team there get very good at it.

    They'll just start categorising operations by patient risk as well as operation type.

 

Comments 5 of 18

 

Features & Analysis

  • Toronto Mayor Rob FordFord nation

    Toronto mayor's enemies unite with him in new battle


  • Tattooed person using tabletRogue ink

    People who lost their jobs because of their tattoos


  • Climate change protestersBig deal or big dud?

    Success of massive climate march remains in doubt


  • An ant and a humanAnts v humans

    Do all the world's ants really weigh as much as all the humans?


Elsewhere on the BBC

  • GeoguessrWhere in the world...?

    Think you are a geography expert? Test your knowledge with BBC Travel’s interactive game

Programmes

  • TokyoThe Travel Show Watch

    Japan has a reputation for being expensive but can you visit without breaking the bank?

BBC © 2014 The BBC is not responsible for the content of external sites. Read more.

This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.