Quality and Outcomes Framework – summary report of a meeting held by the primary care research groups of the Royal Statistical society (RSS) meeting Monday 21st November, Errol St. London.
Speakers
- Nick Steel
- David Reeves
- Tim Doran
What I report here is my interpretation of what the speakers said combined with the audience discussion afterwards. The speakers are not responsible for anything they disagree with!
Background
The Quality and Outcomes Framework (QOF) is a system of payments to British GPs introduced in April 2004. In brief, GPs have a set of targets, mostly oriented towards certain chronic diseases, and they receive substantial payments for hitting these targets for a high proportion of relevant patients. There is evidence supporting the benefit of hitting each of these targets. The relevant patients are declared by each practice on a series of practice registers, for example one for people with diabetes, one for people with known heart disease, and so on.
For example, a GP practice would register patients at risk of coronary heart disease. If a recent blood pressure (within the last 15 months) is under 150/90 a patient has met that target. A GP where less than 40% of the relevant patients meet the target gets nothing. A GP where 70% of the relevant patients have met the target gets 30 points, and there is a sliding scale between these points.
The QOF contains four main components, known as domains. The four domains are: Clinical Domain, Organisational Domain, Patient Experience Domain and Additional Services Domain. Each domain consists of a set of achievement measures, known as indicators, against which practices score points according to their level of achievement. The 2010/11 QOF measured achievement against 134 indicators; practices scored points on the basis of achievement against each indicator, up to a maximum of 1,000 points.
- clinical care: the domain consists of 86 indicators across 20 clinical areas (e.g. coronary heart disease, heart failure, hypertension) worth up to a maximum of 697 points.
- organisational: the domain consists of 36 indicators (worth up to 167.5 points) across five organisational areas – records and information; information for patients; education and training; practice management and medicines management.
- patient experience: the domain consists of three indicators (worth up to 91.5 points) that relate to length of consultations and to patient experience of access to GPs.
- additional services: the domain consists of nine indicators across four service areas – cervical screening, child health surveillance, maternity service and contraceptive services.
The QOF gives an indication of the overall achievement of a surgery through a points system. Practices aim to deliver high quality care across a range of areas for which they score points. Put simply, the higher the score, the higher the financial reward for the practice. The final payment is adjusted to take account of surgery workload and the prevalence of chronic conditions in the practice’s local area (Source http://www.qof.ic.nhs.uk/). A practice getting maximum points across all targets gets about £ST130,000 extra a year. QOF payments come to GP practices through the Primary Care Trust with which they are affiliated. Overall the QOF costs about £ST1 billion a year.
The targets are measured as reported by the GPs from their IT systems. The PCTs do not have direct access to these, but an abstract is supplied to the PCTs by each practice.
Evaluations
The QOF was introduced in a big bang across the whole of the UK in April 2004. This was at the insistence of the BMA, who opposed a phased roll out. No baseline data were collected beforehand. In particular, no data on the proportion of GP practices meeting the various targets before the QOF started were collected. This was because of political pressure to do something fast about General Practice. The result was that no assessment was done of the QOF. A number of studies have now been done with ambivalent results. These were done using synthetic indicators made from existing GP data gathered for research purposes, for example, the GP Research Database. These show significant improvement in the indicators over time, but little evidence for most indicators, that the rate of improvement sped up when the QOF was introduced. There are indicators which rose faster in the first year of the QOF, but returned to the earlier trend of improvement from the second year on. An extensive systematic review of the QOF will be published in the next few weeks, and these results are from a preview given by one of the authors.
Pay-for-performance, and the cost-effectiveness of the QOF
The OQF is a pay-for-performance system. The research evidence on these systems is that they benefit productivity if the work being done is mechanical and repetitive. If the work involves even rudimentary cognition pay-for-performance does not work. There are case studies from Maine in the USA, where strict targets for diabetes care were set, and primary care doctors got a small sum of money ($10) for hitting these. This was modestly effective. Overall QOF has reduced inequalities in the indicators measured between practices in poor, middle income, and rich areas, but there is little evidence of markedly improved health for a very large investment. Nurses generally like it because it has improved their working patterns, and expanded their practice. Doctors tolerate it, claim not to like it, but feel that it has improved practice.
Issues
There is extensive gaming of the QOF. Most PCTs do not police it with any rigour. This is partly because to do so requires significant skills, and partly because they do not want to rock the boat. The major games relate to minor shading of results, for example the hypertension target is under 150/90, which is not exactly demanding. There seem to be quite a few patients coming ins at 148/88 and similar values.
The targets are not very tough (See Figure 1)! Most practices get maximum marks. There are a few exceptions, for example university student practices, where there are so few people with chronic diseases that it is nearly impossible to get a high score. This is dealt with by local informal special funding arrangements. The targets were originally set with no baseline data, so the maximum and minimum cut-offs were largely made up.
There is a practice called exception reporting. This allows a GP to declare that specific patients, for stated reasons, will not be counted. A good example is the use of beta-blockers after a heart attack. For some patients these are not advised because the patient has a condition which would be worsened by the drug, so these patients are ‘exception reported’ from the heart attack register. However, there is evidence that many practices use exception reporting to game the system, and that practices with high rates of exception reporting are likely to be of poor quality overall.
The targets have been revised somewhat since they were set up, but the targets for maximum payment have hardly been shifted at all, even though most practices are hitting these. A proposal that these targets should be reset to the 75th centile (the cut-off between the top quarter of practices, and the bottom three quarters), was rejected.
There is modest evidence that treatments and patients with diseases which were not incentivised by the QOF scheme have suffered.
Mental health is not covered by QOF at all, partly because it was too hard to come up with indicators.
Advice
The question of whether the speakers would recommend other countries to introduce the QOF was extensively discussed at the meeting. The final answer was ‘Yes-ish’. The recommendations below are what I would recommend before thinking about a QOF for Ireland.
- Establish baseline values for any proposed indicators for a year or two before introducing anything.
- Extract data directly from the GP EHR systems, with no practice-level intervention.
- Monitor patterns of care closely.
- Allow exception reporting, but monitor it closely.
- Prohibit dumping difficult patients (which is common in the USA).
- Set tough targets, based on evidence.
- Possibly give full payment only for 100% compliance with the target, or for a 75th centile performance, revised each year.
- Revise these targets regularly
- Look at the Kaiser system, where practices are evaluated on targets, but do not know till the end of the relevant year what those targets were. This makes gaming pointless, and encourages a wide view of practice improvement