Last month, the news was flooded (well, mildly soaked at least) with claims that the NHS is paying too much for drugs. My favourite headline was from the Telegraph, which claimed that researchers thought the ‘NHS should stop buying drugs which cost more than £13,000’.

First things first. The claim is not that the NHS drugs bill is too large (it may be, I don’t know). The claim is not that drugs over £13,000 are too expensive. This is a bizarre headline that misses the point entirely. As is often the case, it is a shame that the nuances of the research have been missed because they are important.

The claim is that the NHS in England is prepared to pay more for new drugs than they are worth because we could get more health benefit for the same money by investing elsewhere. The thing that underpins all of this is a hugely important economic principle called opportunity cost.

Who said what?

The Centre for Health Economics at the University of York did the research that caused the media interest, in collaboration with colleagues at the Office of Health Economics and Imperial College London. Karl Claxton is the lead author, so I’ll call it Claxton et al. from now on, just to keep it brief.

The details of the research are published, in full, on their website, and the full report is here. The headline claim is that the threshold used by the National Institute for Health and Care Excellence (NICE) to decide whether a drug should be recommended for the NHS ought to be around £12,936 per QALY. Let’s break that down a little.

The value of £12,936 per QALY the team mention is an Incremental Cost-Effectiveness Ratio (ICER), which is calculated like this:

(1)  

This has some important features:

• £12,936 is not the drug price – it is the incremental cost of providing the new treatment, which includes the drug price as well as the costs of any additional tests needed, treatment of adverse drug reactions, additional nursing care etc. It also includes costs that are avoided. Some new drugs are really expensive, but end up saving some money because they allow us to avoid some health costs
• We measure effects in years of perfect health – QALY stands for ‘quality-adjusted life year’. More detail on this is provided in the blog glossary here, but it is an estimate of the amount of time spent in different health states weighted by the utility associated with the ‘quality of life’ in those health states (as judged by the general population). That’s a bit jargon-heavy so here’s an example:
  • Someone with terminal cancer may survive for an additional 6 months on a new drug, but with a reduced quality of life giving that life a utility of 0.5. Such a person would accrue 0.25 QALYs (0.5 years multiplied by a utility weight of 0.5). Again, it is incremental, so a drug can produce more QALYs by reducing adverse drug reactions compared to existing treatments or by improving survival etc. When we use QALYs, we are doing what is known as a cost-utility analysis. It is possible to use other approaches
• We need an alternative – when you choose to use a new therapy, you are displacing an existing therapy because you are choosing one over the other. This is true even if we are talking about combination therapies. For example, if you choose to use paracodol (a mixture of paracetamol and codeine) for a headache, you are choosing not to use paracetamol alone or codeine alone or aspirin etc. The correct comparator should be the next best treatment. This can be a tricky thing to determine.

I keep using the word incremental. This is important. When we choose to use a drug, we are choosing not to use another drug. That means that, while we get the benefits (and harms) of the new drug, we do not get the benefits (and harms) of the old drug. To stick with the example of paracodol, we know that paracodol is generally effective. However, so is paracetamol alone, and paracodol costs more. To decide whether it is worth choosing paracodol over paracetamol we have to compare the two. An ICER is a way of doing this.

The cost of missed opportunities

Claxton et al. base their estimation of the ‘correct’ NICE ICER threshold on the principle of opportunity cost. In the example above, if we have chosen to take paracodol instead of paracetamol, and so we don’t get the benefits or the costs of paracetamol. That’s all opportunity cost is: the cost of a missed opportunity.

The difficult thing is knowing what you’ve missed.

Health economists compare drugs all the time. What Claxton et al. have done is look at other spending in the health system and see what could have been achieved if money spent on pharmaceuticals had been spent on other items of care. This is a huge undertaking and involves lots of assumptions and tricky estimation. Let’s look at what they did.

What they actually did

This is the bit I really like. The techniques used in this study are very clever, but still elegant.

Massive organisations like the NHS lump expenditure into a hierarchy of budget categories so that they can track it without their heads exploding. So, in the same way you or I might track our own finances by lumping them into car expenses or utilities, the NHS uses programme budgeting to lump expenditure into programme budget categories (PBCs). Handily, these PBCs are based on disease categories. There are 23 of them, and to give you a flavour of what they look like, here are the first 6:

1. Infectious diseases
2. Cancers and tumours
3. Blood disorders
4. Endocrine, nutritional
5. Mental health
6. Learning disability

These PBCs can be aligned with the disease codes that are used in both health records and national statistics, which means that we can link these financial data to actual health events (at least in principle). Hold on to that thought for a moment.

The healthcare providers in the original research were Primary Care Trusts. These don’t exist any more. Nevertheless, whoever holds the purse-strings will always change their budgeting from time to time. Sometimes they’ll invest a little more in their community mental health teams and sometimes they’ll close a sexual health clinic to reduce their costs and keep within budget. Some of these decisions will improve health, and some will result in reduced health. On average, we would expect more expenditure to result in better health up to a certain point.

Let’s return to those linked health data. Claxton et al. could not directly measure health for all of the PBCs, so they had to focus on 11 PBCs that could be cross-referenced with mortality data. They then used that information to estimate the quality-adjusted life years gained or lost for each change in health expenditure. The way they did that is quite complicated, and it’s best to look at the report itself if you are interested in the details. The result, though, is an estimate, for each PBC, or how much health gain or loss (based on mortality) you should get for a given budget change.

In other words, Claxton et al. calculated how much health you miss out on by spending money on drugs (or other health interventions) instead. Clearly, for something to be a worthwhile investment, it should give you a better return than this. Therefore, it is an empirically derived, defensible decision threshold.

It is also much lower than the current threshold NICE uses, and that has inevitably produced some responses.

The criticisms

I’m sure there are lots of criticisms I haven’t captured (let me know any more below), but the responses from the Association of the British Pharmaceutical Industry (ABPI) and the National Institute for Health and Care Excellence (NICE) fall into a number of categories:

• The QALY is a poor measure of health and is not used for most health decisions
• If we used this threshold for non-drug decisions, we wouldn’t have A&E, palliative care for dying patients and maternity services
• The pharmaceutical market is international, so lowering the threshold would not lower prices, it would just remove our access to them

These criticisms haven’t been discussed a great deal in the press because the headline figure is more exciting, and possibly because it’s easier to turn this into a pro-/anti-pharma argument than actually work out how we deal with providing optimal healthcare to a nation.

The claim that the QALY is a poor measure of health does have some foundation. It is genuinely difficult to investigate utility and the quality of life even within an individual over time, let alone for a population with a vast number of different health states. Some of these issues are discussed in an old post of mine here. I would imagine that Claxton’s response would be something along the lines of:

Well, it’s not perfect, but it’s the best measure we have and we would be stupid to make decisions based on anything but the best information we have.

I tend to agree, although there may be situations (end of life care may be one of those) where current utility measures don’t adequately capture the utility of real humans and we could be better off generating utility estimates in some other way. More importantly, though, arguments about QALYs miss the point; NICE does use QALYs in its decisions. That is NICE’s decision, not Claxton’s. It also uses other things. If we want NICE to use a different approach to assessing new treatments, that’s fine. But it’s not the issue Claxton et al. were addressing. It’s also worth pointing out that Claxton et al. concentrated on measuring changes in mortality, so they aren’t making fine-grained judgements about highly subjective health states.

The second argument about palliative care and A&E is something Claxton et al. actually anticipated in their methods. Remember, they looked at the health consequences of changes in health care expenditure. This means that they are talking about the consequences of displacing health expenditure within the current health system. Their approach should not be used to justify removal of A&E, but they do not make any claim like that and their analysis does not deal with that sort of fundamental change in the structure of healthcare.

There is a subtlety to this second criticism. Most new drugs are not so earth shatteringly exciting that they could be considered a fundamental change in health provision. Some, however, are. It may be that some new drugs or medical procedures change the way we treat patients. Examples of this could include highly targeted treatments that provide a cure when none previously existed, or germline gene therapies that allow families to overcome diseases that would otherwise affect future generations. It is reasonable to suggest that these treatments should be handled differently.

The argument that Claxton et al’s approach would result in removal of A&E from the NHS is an attempt to tug the heartstrings rather than a real threat. The research might consider local changes in A&E funding allocations, but not grand national policies that change the shape of the NHS.

The final argument is, again, worth discussing and important, but not really a criticism of Claxton et al’s work. They were looking at what the threshold should be in the UK on the basis of UK buying decisions. What is good value elsewhere may not be good value in the UK. While there is a lot more to getting a drug to market than getting approval in the UK, but this research is about defining value in the UK market.

That does not mean that we should ignore the market. There are a lot of questions that arise here concerning whether the pharmaceuticals market is a free market in the traditional sense, and whether it is distorted by features such as reference pricing that result in UK prices influencing prices in other countries. These all mean that there are situations where we may chose to look at features other than drug value in terms of cost-effectiveness. They are also the reason drug companies and their representatives at the ABPI get a bit nervous.

There is a potential risk that, if the UK market had a negative effect on profits globally, drug companies would choose not to release drugs to the UK. I do not think that is very likely, though. First of all, not all NICE decisions rely entirely on the ICER threshold so higher prices would still be achievable. Secondly, we sometimes see deals that mean that the price the market sees is not the price the NHS pays. More fundamentally, not all drugs are right at the edge of the existing threshold.

My thoughts

There are some analysts in the major consulting firms who look at NICE decisions routinely. One group, for example, looks at what underlies decisions, particularly those that don’t follow the NICE threshold. I contacted this team (declaration of interest: I know these guys and I like them) and asked to take a look at their data. I was actually prompted by seeing that they observed an average ICER in manufacturer’s submissions of about £15000/QALY (I’ll link to this when it’s published).

If we plot out the ICERs that manufacturers submit to NICE, this is what they look like:
[chartboot version= ‘3.0’ code= ‘AD0D’ border= ‘1’ width= ‘800’ height= ‘400’ attribution= ‘1’ jsondesc= ‘{“containerId”:”visualizationAD0D”,”dataTable”:{“cols”:xxx00xxx{“id”:””,”label”:”Random”,”pattern”:””,”type”:”string”,”p”:{}},{“id”:””,”label”:”Manu ICER (no PAS)”,”pattern”:””,”type”:”number”}xxx01xxx,”rows”:xxx00xxx{“c”:xxx00xxx{“v”:”a”},{“v”:14616}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:121367}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:24584}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:19999}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:13189}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:30000}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:23500}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:12931}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:40952}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:47628}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:33065}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:3461}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:17974}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:46192}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:56985}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:51899}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:50286}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:35208}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:15593}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:21285}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:7890}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:10054}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:10465}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:10054}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:9289}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:12381}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:83984}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:38421}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:62829}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:15700}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:36241}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:6200}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:112880}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:11960}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:32898}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:56945}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:49272}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:46700}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:26537}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:70076}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:32036}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:26996}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:15404}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:40792}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:8165}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:11403}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:1764}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:24527}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:31464}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:56683}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:3521}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:17559}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:31982}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:16567}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:21661}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:22509}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:8740}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:8740}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:18691}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:26475}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:11444.5}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:11601}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:55634}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:74938}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:18883}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:63233}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:21874}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:6083}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:14900}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:77318}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:2441}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:7743}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:44085}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:60737}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:51800}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:22655}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:100144}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:40367}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:11008}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:36327}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:13851}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:23073}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:149050}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:144066}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:13252}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:2671}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:73980}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:29000}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:15105}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:16710}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:18481}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:6420}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:32417}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:41544}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:20972}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:3900}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:22700}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:20613}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:5871}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:28423}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:36294}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:8544}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:10076}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:20468}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:47576}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:null}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:56}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:null}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:14}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:47}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:null}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:11}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:19}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:11}xxx01xxx},{“c”:xxx00xxx{“v”:”a”},{“v”:16}xxx01xxx}xxx01xxx},”options”:{“legend”:”none”,”title”:”ICERs reported in manufacturer submissions”,”animation”:{“duration”:500},”booleanRole”:”certainty”,”histogram”:{“hideBucketItems”:true,”bucketSize”:5000,”lastBucketPercentile”:null},”vAxes”:xxx00xxx{“viewWindow”:{“max”:null,”min”:null},”minValue”:null,”maxValue”:null,”useFormatFromData”:true,”title”:”Number of submissions”},{“viewWindow”:{“max”:null,”min”:null},”minValue”:null,”maxValue”:null,”useFormatFromData”:true}xxx01xxx,”hAxis”:{“viewWindow”:{“max”:null,”min”:null},”minValue”:null,”maxValue”:null,”useFormatFromData”:true,”title”:”ICER (£/QALY)”},”height”:371,”width”:600},”state”:{},”isDefaultVisualization”:true,”chartType”:”Histogram”}’ ]

The median value is pretty close to what Claxton et al. recommend. Some of my health economist colleagues reckon that, if we weighted this by the number of patients with the disease of interest, we’d see an even more marked effect.

What does this mean? Well, manufacturers generally produce drugs that they claim are not far off Claxton et al’s estimate of good value. NICE does routinely argue with these estimates, of course, and there is a long tail to the distribution (i.e. a small number of drugs with very poor cost-effectiveness). However, I think overall manufacturers and NICE/the NHS could work together to manage NHS costs within these limits. It might require certain exceptions to be made. For example, maybe genuinely breakthrough drugs need a prize to encourage a race to development (some researchers at Sheffield have a neat take on this).

The threshold itself is highly uncertain, and that uncertainty does need to be reflected in decisions. There are some cases where we are happy to flex the rules a little because the threshold feels unfair in some way. Of course, as the authors note, there does appear to be a law of diminishing returns across health that suggests that investments produce fewer gains the more you put in. That implies that the areas to concentrate on may be the areas we don’t usually think about.

In terms of incentives, it may be that a more rapid NICE decision process is appropriate where manufacturers’ ICERs are well within the probable threshold. I am sure that many manufacturers would welcome this and would understand that more scrutiny is required where the value of a drug is less certain.

Overall, though, I don’t think the threshold determined by Claxton et al. is that surprising and I don’t think research into how we should evaluate drugs has to become a pro- and anti-pharma thing. We all use healthcare and we all pay the cost of overpriced drugs or a lack of innovation.

Beyond the arguments about drug prices, though, is the issue of expenditure on non-drug costs in health. The Claxton et al. team note something that is very close to my heart; some areas of healthcare expenditure, such as mental healthcare, may produce much more health benefit than we generally realise. We underestimate their value because they are not primarily associated with deaths (although, sadly, deaths do occur). In my personal opinion, the research by this team should be used, not just to create a few column inches arguing with pharma companies, but to help us think about where we would like to invest the limited health budget we have.