Is the Commonwealth Fund’s Mirror, Mirror report a distorted lens?

The NHS as a public service accessible to all is often described as one of the greatest humanitarian achievements of the 20th century. Incremental and radical changes have taken place which are putting that in jeopardy, yet a new report from the Commonwealth Fund “Mirror, Mirror 2017:International Comparison Reflects Flaws and Opportunities for Better U.S. Health Carestill rates the NHS at the top of its list of the 11 healthcare systems it uses for comparison. There appears to be a gulf between this report and the concerns of campaigners and health professionals across the UK.

It is difficult at first sight to square this report with reality. The public sector NHS is in crisis with many hospitals regularly on black alert (hospitals at full capacity, closed to further admissions), routine operations being cancelled including cancer surgery, a lack of provision of mental health services, a social care system that is collapsing due to privatisation and de-funding of local authorities, and general practice in meltdown. The Royal College of Nursing has warned that crisis is becoming the new normal. And the BMJ have published the results of an investigation showing that treatment rationing is having an increasing effect.

What the Commonwealth Fund’s report says – and doesn’t say

The Commonwealth Fund is a US private foundation which has the stated charitable aim of improving access to healthcare for America’s poor and excluded groups. It should be noted that this is not the same objective as the NHS which is designed to give universal and equitable care based on health needs not ability to pay. That is to say for rich and poor alike.

The Abstract of the report says, “The United States health care system spends far more than other high-income countries, yet has previously documented gaps in the quality of care”. The purpose of the report is to look at what lessons can be learned from the performance of other countries’ health systems. But should those lessons be taken on face value as supporting the UK government and NHS England in its objectives or the current state of the NHS? We note that Simon Stevens, CEO of NHS England, is also, coincidentally, on the board of the Commonwealth Fund.

According to the Guardian: “An NHS England spokesperson said: “This international research is a welcome reminder of the fundamental strengths of the NHS, and a call to arms in support of the NHS Forward View practical plan to improve cancer, mental health and other outcomes of care.”

But does the report deliver evidence to back NHS England’s position? The limitations of the report state:

“…despite improvements in recent years, the availability of cross-national data on health system performance remains highly variable. The Commonwealth Fund surveys offer unique and detailed data on the experiences of patients and primary care physicians. However, they do not capture important dimensions that might be obtained from medical records or administrative data. Furthermore, patients’ and physicians’ assessments might be affected by their expectations, which could differ by country and culture. In this report, we augment our survey data with other international sources, and include several important indicators of population health and disease-specific outcomes. However, in general, the report relies predominantly on patient experience measures. Moreover, there is little cross-national data available on mental health services and on long-term care services.”

So the bulk of the Commonwealth Fund’s report relies on surveys of patient and clinician experience, not clinical data.

On one count a comparison between the US and the UK turns out to be a comparision between apples and oranges, “the U.S. performs poorly in administrative efficiency mainly because of doctors and patients reporting wasting time on billing and insurance claims”. Clearly the public part of the NHS has no insurance claims, although billing between providers and commissioners is on the rise, but there is no inference drawn that the US can improve its system by eradicating insurance and the market. On the contrary the conclusion is drawn on a sample of the top three in the chart that ‘high performance can be achieved through a variety of payment and organizational approaches’. The transition from a planned system to a market system has increased the NHS’ own transaction costs significantly which is not referred to in the report, nor what if any impact that might have on service delivery. Future reports may not have such a favourable outcome for the NHS, given that this form of ‘efficiency’ is included in its assessment unless its current direction of travel is halted.

The NHS scores highly on access to services. Campaigners across the country are all too aware what loss of access means which is why they are fighting to keep their services local. The US has poor access to services and is at the bottom of the table.

But on what is arguably the most important measure of all, health care outcomes, the NHS is next to the US at the bottom of the table, so either outcomes may not be attributable to good access or other negative factors outweigh the benefits of access. The report does not address that question. Indeed of its top three overall only Australia comes out well on health outcomes, in top position on the chart. Of the other two, the Netherlands is ranked 6th and the NHS 10th out of 11.

According to the report health care outcomes “are intended to reflect outcomes that are attributable to the performance of the countries’ health care delivery systems. The measures fall into three categories: population health outcomes (i.e., those that reflect the chronic disease and mortality of populations, regardless of whether they have received health care), mortality amenable to health care (i.e., deaths under age 75 from specific causes that are considered preventable in the presence of timely and effective health care), and disease-specific health outcomes measures (i.e., mortality rates following stroke or heart attack and the duration of survival after a cancer diagnosis).”

What the Commonwealth Fund doesn’t do is to measure how changes within systems affect their rankings across systems. It also concludes when looking at the three top scorers in its report that organisational structure and funding methods have nothing to do with placings, “The three countries with the best overall health system performance scores have strikingly different health care systems. All three provide universal coverage and access, but do so in different ways, suggesting that high performance can be achieved through a variety of payment and organizational approaches.” If the Commonwealth Fund does not consider health outcomes to carry any more weight than a patient satisfaction survey, how is it defining ‘high peformance’?

Does this give us any real insight into why the NHS is in crisis, as the Royal College of Nurses have said, or explain why treatments are being rationed? Does it offer up a different kind of mirror to campaigners to see that all is not as bad as they feared? The answer is no. A ‘league table’ of the variables used to compare systems is not the same as an impact assessment of those variables on those systems. There is no analysis of the complex interrelation between the variables or any weighting in terms of which variables have most impact on the most important criterion in the report – health care outcomes.

In short the NHS is not doing well, but enough people can still access it and are still hopeful it can be fixed to keep it at the top of the chart.

One particular note of caution in respect of the statistical data drawn from the European Observatory on Health Systems and Policies is that it is from a time range of 2011-2014. For the NHS, for example, the data on Mortality Amenable to Health Care is from 2013. Although the self-reported information on care quality, access, etc, is from 2015 for the clinicians and 2016 for the patients, statistical data on the whole is from 2014 or before. This is not a criticism of the Commonwealth Fund – data collection on this scale is not instantly accessible – but because the NHS’s landscape has changed so rapidly since the Health and Social Care Act came into force in 2013 its current state may not be adequately reflected in the findings.

The positive when the mirror is asked ‘who is the fairest of them all’ is that it is good to see that both clinicians and patients still evaluate the NHS with broad approval. Despite all the claims frequently made for the superiority of other country’s systems, and the dismantling, marketistion, privatisation and de-funding that the NHS has suffered it is still highly regarded.

Even so, if you bite into the apple there is an underlying perception that there is something wrong. 44% of the UK’s survey participants said they thought the NHS system ‘works pretty well and only minor changes are needed to make it work better’. But 46% said ‘there are some good things in our health care system, but fundamental changes are needed to make it work better’. And 7% said ‘our health care system has so much wrong with it that we need to completely rebuild it’.

The changes the NHS needs are not likely to emerge from international comparisons done for the benefit of the US system. We must focus on the real issues brought to the fore by campaigners and health professionals and in so many grim newspaper headlines : de-funding, staff recruitment, training and retention, the closure and sale of hospitals and land, a management consultancy culture and ‘customer focus’ which takes precedence over clinical evidence. And above all focus on the pernicious and pervasive transformation of the NHS into a copy of the US Medicare system that is once again at the bottom of the chart.

The NHS is supposed to benefit by these changes according to its proponents yet nearly twice as many US respondents in the report as UK ones said they had had problems with coordination of services – ironically one of the major selling points touted for the system the NHS is having forced upon it.

Before celebrating the NHS being ‘No 1’, first take a bite into the apple of the survey data. We applaud the staff who are doing their best to hold everything together under extraordinary stress. The patient survey responses tell us that they are and we should welcome any report that boosts staff morale. But the health outcomes show the government and NHS England are letting the staff and everyone else down. The idea of the report as a PR and vindication exercise for them should be rejected. So when you look into the Mirror, Mirror report before you celebrate that ‘No 1’ just remember that mirrors can distort as well as reflect.



One Comment on “Is the Commonwealth Fund’s Mirror, Mirror report a distorted lens?”

  1. Excellent way of explaining, and good piece of writing to get facts regarding my presentation topic, which i am going to convey in school.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.