I’m registered with an excellent GP practice, so when I phoned in the afternoon I wasn’t surprised the receptionist allowed me to book an appointment in a month’s time. The GP had suggested I go back to see him if the complaint worsened, but I wanted to record some daily observations before the appointment so that I had some hard facts for the consultation. I suggested to the receptionist an appointment in 3 weeks’ time. “Sorry, he’s away then. What about the following week?” I couldn’t make that, so we agreed on a date just over a month in the future.
Soon after booking the appointment, I saw a BBC News story of 4th April, headlined One in 10 faces three-week wait for GP appointment1. I couldn’t help asking myself ‘Would I be included in the One in 10?’
The source for the BBC story was a 2nd April press release from the Liberal Democrats2.
The headlines in the press release were:
- Vale of York sees 80% increase in four-week long waits for GP appointments in 2023 compared to the previous year
- Liberal Democrat Leader Ed Davey will warn the government has “failed to get a grip on this GP crisis” on a campaign visit to Chippenham
- Almost one in ten patients waited four weeks for a GP appointment last year in worst hit areas including Gloucestershire, Sheffield and Dorset
Some areas of the country saw a shocking 80% rise in the number of four-week waits for a GP appointment last year, new research commissioned by the Liberal Democrats has revealed.
Helpfully it gave the data source: The House of Commons Library analysis looks at NHS data measuring the time between when a GP appointment was booked and when it took place, broken down by local NHS areas (sub-Integrated Care Boards) across England.
I concluded the answer to my question would appear to be ‘Yes, I would be included in the One in 10, because the time between when [my] GP appointment was booked and when it took place was more than a month’. I read further in the press release and came to the ‘Notes to Editors’ section where there is the following sentence:
This data measures the time between booking and the appointment taking place. This includes cases where the patient requested a particular date, so it is not a direct measure of “waiting times”.
There is a link in the press release to an NHS Digital site. From there I got to the Integrated Care Board Appointments Data Dashboard and to the Additional information section3. It says:
Data in the GPAD [GP appointments data] collection is known to contain a number of data quality issues. This is due to variation in working methods and recording between different practices. Practices manage their appointment books in the way they best feel allows them to manage the health and care of their patients and not for the purpose of data analysis and there is a widespread variation in approach to appointment management between practices.
And it’s not just ‘variation in working methods and recording’ that impacts the data quality. The practice’s computer system supplier is also an important factor:
The system supplier used by the practice to manage their appointment book also impacts on the data included in the GPAD collection. Currently data is received from EMIS, TPP, Informatica, Cegedim and Babylon (GP at Hand). Practices using other system suppliers will not have their data included in the GPAD collection. Practices using a combination of suppliers to manage their appointment book may also have only partial data included.
There’s more:
Time elapsed from when the booking was made to when the appointment took place is included as part of this dashboard. There are several factors that influence this, including:
- appointment availability at the practice
- patient availability
- urgency of the appointment
- GP advice
- regular/repeat appointments that may be booked in advance
As such, this field should not be interpreted as a ‘wait time’ as there is no consideration of any of the above factors in the data, nor how many attempts were made by the patient to contact the practice.
And more still:
The use of list or untimed appointments also varies considerably between practices. These appointments are included in the GP system and serve as lists of activities that need to happen at any point in the day and often include telephone triage lists and home visits. These may appear in the appointment book as one appointment whereas other practices may record these as individual appointments throughout the day. Again, this affects the number of appointments included in the GPAD collection.
This Additional information section was the most helpful of all the websites. It reminded me that understanding and communicating the ‘data provenance’ – where the data has come from, and the issues covered above – is an important aspect of the data scientist or statistician’s role. Tim Harford, of BBC Radio 4’s More or Less, summarises this issue well:
Statisticians are sometimes dismissed as bean-counters. The sneering term is misleading as well as unfair. Most of the concepts that matter in policy are not like beans; they are not merely difficult to count, but difficult to define. Once you’re sure what you mean by ‘bean’, the bean-counting itself may come more easily. But if we don’t understand the definition then there is little point in looking at the numbers. We have fooled ourselves before we have begun.
The solution, then: ask what is being counted, what stories lie behind the statistics. It is natural to think that the skills required to evaluate numbers are numerical – understanding how to compute a percentage, or to disentangle your millions from your billions from your zillions. It’s a question of mathematics, is it not?
What I hope you’ve learned over the last few pages is that the truth is more subtle yet in some ways easier: our confusion often lies less in numbers than in words4.
A big difference between NHS secondary care data and primary care data is that there is a much longer history of secondary care data being used for monitoring performance and comparing institutions. When the 18 weeks wait measure was introduced in the early 2000’s, following government consultation with the public over how long they should have to wait for hospital treatment, it was necessary to define what was included in the definition of ‘treatment’, what was a ‘clock start’ and what was a ‘clock stop’. An ’18 weeks rule suite’ was issued so that system suppliers could rewrite their software and hospitals could change their recording practices5.
Hospitals must work out how to manage their appointments so that the patients get the best care, and the appointment data is fit for data analysis. (This is an illustration of how data providers in the NHS must balance working on behalf of the patient and working on behalf of the taxpayer, whose representatives require accurate performance information.) The NHS England 18 weeks flowchart below illustrates the complexity involved in ensuring secondary care data is recorded consistently in the NHS.
It’s easy to extract data that has been recorded for operational purposes (the date of appointment and date it was booked), and easy to derive the wrong message from it. It’s harder to source data that is well-defined and that can be used to compare the performance of different institutions or areas. Developers of analytic software, including AI or Machine Learning applications, must start with an understanding of their data’s provenance and something equivalent to the detailed information covered in the Additional information section referenced above. Xiao-Li Meng goes further and covers the wider subject of data minding, ‘a stringent quality inspection process that scrutinizes data conceptualisation, data pre-processing, data curation and data provenance…’6
The month’s wait to see my GP passed quickly and by the time of the appointment I had recorded over 30 days’ observations. I was happy with the provenance of the data I had collected, and it was crucial in ensuring I got the most out of the appointment.
Note: The views and opinions expressed in this article are not necessarily those of Northumbria Healthcare NHSFT.
References
- https://www.bbc.co.uk/news/articles/c72epm48nwdo
- https://www.libdems.org.uk/press/release/gp-postcode-lottery-80-increase-in-four-week-long-waits-in-some-areas
- https://digital.nhs.uk/dashboards/icb-appointments-data-dashboard#about-the-data
- Tim Harford, How to Make the World Add Up, Rule three: Avoid premature enumeration, p91-92, The Bridge Street Press
- https://www.england.nhs.uk/statistics/statistical-work-areas/rtt-waiting-times/rtt-guidance/
- https://doi.org/10.1111/rssa.12762
The NHS: observations of an Information Specialist 