• Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Home
  • Articles
  • Blog
  • About
  • Contribute
  • Contact

Sceptical Scot

Asking Questions. Seeking Answers.

You are here: Home / Articles / Statistical amnesia

Statistical amnesia

December 6, 2020 by James Urquhart 5 Comments

 

May 21, 2021 Update:  A Scottish government agency breached freedom of information rules by refusing to reveal how many people had died of Covid in each of the country’s care homes. Full BBC report HERE  See also Scottish Information Commission report: Number of Deaths in Care Homes HERE 

In this pandemic there has been an unprecedented obsession with “the data”.

As the pandemic has developed, we have become used to the “big numbers” such as “70,000 deaths in the UK” or £100 billion for the misconceived “Operation Moonshot”. And in the process, little by little, we have become insulated from the true tragedy behind these numbers. Who really stops to think what 500 COVID-19 deaths reported in a single day really means in terms of human distress, the waste of active years of life and long-term detriment to health?

Data has become the stick used to beat the drum about relative success or beat the political opponent over the head.

When I was invited to submit another piece to summarise what the data can tell us now as the year closes, I had to think long and hard whether I really wanted to submerge again in the morass of data that lurks now on the internet. The truth is that although the sheer quantity of data on a single topic is almost certainly unsurpassed in terms of volume, it has become harder and harder to have any faith in its quality. Look at the small print on a data set and you will often find the word “provisional”. Go back a week later and you will find that not only has the data been revised but also that the original data has been obliterated together with any opportunity for an audit trail.

The constant retrospective updating of data could be justified if the only purpose in its collection was to provide a historical record of the pandemic. But we are constantly being told that decision-making is based on the science and hence the data. So, the current quality of the data really matters as does informed analysis untainted by political imperatives.

An historical perspective

The constant updating of the data should mean that the further back we go in time the better the data quality. But there seems to be no time to reflect on what the experience of the last few months might tell us about the future management of the disease. The only data that attracts attention is that for the latest day or week even though by definition this is the data with the least quality.” Thus just in the last week the headlines screamed “Nicola Sturgeon defends handling of Covid as figures show Scottish deaths compare badly to England” but no one it seems had the time to put this into any kind of historical perspective.

The first question that needs to be asked is: is it true that Scottish coronavirus deaths do compare “badly” to England’s. The problem is that the figures quoted in the newspaper and the original television interview relate to week-on-week comparisons between Scotland and England where the weekly period have different start dates in the two countries. What is much more revealing is a comparison between the two countries based on data for the actual date of registration (Figure 1).

 

This shows that from mid-October to mid-November the death rates in Scotland were indeed consistently higher than was the case for England. But what the data also shows is that there appeared to be a dramatic change in the ratio sometime in early October

Questioned about the data, Nicola Sturgeon fell back on the well-worn trope: “We are still in the teeth of this pandemic, so it is premature for any country to be declaring victory or assessing performance”. This was to put it mildly a disappointing response and must have been particularly so for anyone who had been prepared to put aside political differences to praise the Scottish response to the pandemic.

It contrasts strangely with the First Minster’s claims earlier in the year that Scotland’s COVID19 rates were one fifth of those in England – a claim for which she was reprimanded by the Office for Statistical Regulation even though there was some evidence to support at least the general thrust of the claim if not the actual quoted ratio (Figure 2).

Putting it bluntly, this kind of statistical amnesia will seem a little tawdry to anyone who invested belief and political capital in the First Minister’s approach.

The real question

Although it is hard to escape the conclusion that a double standard has been applied in considering data from two different time periods, this is not the real cause for concern about the First Minister’s comments. The appropriate response to any question about the relative death rates in England and Scotland was surely to make a commitment to try to understand why from a relatively low base Scotland’s deaths suddenly increased to a level not seen since May. This is illustrated starkly in Figure 3 which shows the moving average of deaths in Scotland by date of registration. (The dates shown on this graph are those 21 days before the date of registration to give some approximation of the date when the cases which led to the deaths occurred).

What this graph demonstrates is that there was a pivot point that occurred in the middle of September which preceded the surge in COVID 19 deaths. It will not be a coincidence that this pivot point coincides with the time when the Scottish Government switched back from the hopeful objective of “eliminating” the virus to the more familiar territory of virus “suppression”. Looking back it was entirely wrong to offer the hope of eliminating the virus in Scotland when for whatever reason the necessary policies and strategies to achieve this were never in place.

By contrast to Scotland the rise in deaths in England shows a different pattern with no discernible pivot point but rather a sustained rise from late August. The absence of a pivot point is perhaps not surprising given that COVID-19 was already on the rise in several English regions.

However, what is also apparent is that whereas at the end of October in England the peak in cases leading to death had yet to be achieved, the peak in Scotland may have occurred perhaps some two weeks earlier. So, it is entirely possible that similar factors could have been in play in Scotland and England which led to a rise in cases leading to deaths albeit that such a factor might have come into play later in England.

In summary it would appear:

  • That in the middle of September there were some circumstances which appeared to trigger a rapid increase in Scotland in fatal cases
  • Although there was no similar pivot point in England it appears that the peak in deaths in England might have been reached at least two weeks after that in Scotland.
  • Understanding what might have triggered the sudden rise in mid-September is of importance in the future management of the pandemic

There are no data sets which might provide direct answers to the question of what might have triggered the sudden rise in Scotland in September.  But if all that is left is speculation such speculation is by no means idle. If I can think of at least one prime candidate for an event which might have triggered the September rise in Scotland, then I am sure that others will be able to come up with the same, similar or different suggestions. Speculation of this kind is valid if it leads to investigations which at minimum eliminate possible explanations. And speculation is our only recourse if those with the power to do so refuse to engage with the data

The comments page is there. Why not use it to offer your suggestions. If you do so you will not be providing a definitive explanation, but you will be offering a hypothesis and challenging others to prove your hypothesis wrong.

Further reading: Number of Deaths in Care Homes: Scottish Information Commissioner report May 2021 Questions over Sturgeon’s strategy, FT; Sturgeon defends handling, BBC;  Scaling COVID-19 against inequalities, Gerry  McCartney et al, BMJ

Filed Under: Articles, Covid19, Policy, Politics Tagged With: Coronavirus, statistics, Sturgeon

About James Urquhart

James Urquhart is a medical statistician. In 1973 He joined the Research and Intelligence Unit of The Scottish Home and Health Department. When he left he Health Service in 2001 to pursue a different career he was Head of Research and Deputy Director of The Information Services Division of the Scottish Health Service.

Reader Interactions

Comments

  1. Paddy Farrington says

    December 6, 2020 at 6:11 pm

    We cannot be sure that ‘COVID-19 mentioned on the death certificate’ means the same thing in England and Scotland, and that its meaning does not change over time (for example owing to changes in testing practice). Excess deaths, suitably standardised, are a far more reliable indicator for making comparisons, particularly between countries.

    The excess mortality data from EUROMOMO for Scotland and England reveal a rather different picture from that painted by Andrew Marr in his interview with the FM: standardised rates in Scotland were lower than England’s for virtually every week during this Autumn (with the proviso that data for weeks 45+ are subject to reporting delays).

    In the Marr interview, the FM mentioned, in the context of deaths in care homes, the contrast between excess deaths and ‘deaths with Covid-19 on the death certificate’ and how this contrast affects the comparison between death rates in Scotland and England in a very major way.

    I’m not at all sure that Marr got the point.

    Reply
  2. Joseph MELLON says

    December 6, 2020 at 9:52 pm

    > I’m not at all sure that Marr got the point.
    I’m not at all sure that Marr *wanted* to get the point. I think Sir Andrew wanted to gather points towards his knighthood.

    Reply
  3. Julie Wark says

    December 12, 2020 at 11:31 am

    ‘However, what is also apparent is that whereas at the end of October in England the peak in cases leading to death had yet to be achieved, the peak in Scotland may have occurred perhaps some two weeks earlier.’

    School pupils returned in Scotland on Wed 12th August. School pupils returned in England on Tuesday 1st September.

    This is 20 calendar days apart, but 14 school days apart. It could be argued that the weekends provide a buffer to exposure and that it is exposure days which matter most to case levels.

    I cannot think of any other factor which creates a big enough change in behaviour to have such a major impact on growing case numbers and resultant deaths.

    Reply
  4. Ian M Robertson says

    December 12, 2020 at 11:41 am

    Data showing the local geography of cases & deaths would surely strengthen hypotheses for causality. I assume very detailed data of this sort is available to the relevant public service specialists. So strong hypotheses for causality & action should be possible. Whether such work is then made public or acted on takes us right to the heart of issues good policy making & implementation, & of democracy & accountability.

    Reply
  5. Andrew says

    January 10, 2021 at 10:46 am

    Good observations about the difficulty of interpretation and the importance of excess deaths. Another point: we know that in general BAME mortality rates have been much higher than in the white population, even allowing for known risk factors. Given the much lower percentage of BAME people in Scotland (there are many fewer people in Scotland in total than BAME people in England), it seems quite likely that the death rate for whites has been as high or higher north of the border than on its other side. The current rapid increase in cases and deaths seems to be particularly bad in London, 41% of whose population is BAME, according to trustforlondon.org.uk.

    Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Primary Sidebar

About Sceptical Scot

Welcome to Sceptical Scot, Scotland’s premier non-tribal forum for passionate, informed debate. Sceptical Scot is for all who care about Scotland’s future, regardless of how they vote: for party, independence or union, EU or Brexit. We aim to provide an arena that is both broader and deeper than current online/print offers with a rich diet of well-researched, polemical, thought-provoking writing. Read more » about About Sceptical Scot

What’s new on Sceptical Scot

  • Another struggle to deliver independence March 28, 2023
  • Humza faces a hard landing March 27, 2023
  • Eradicating poverty in Scotland: a long way to go March 27, 2023
  • Scotland does not need nuclear energy March 27, 2023
  • The rules of the road in the Highlands March 22, 2023
  • How sustainable are Scottish tax and spend policies? March 22, 2023
  • Scotland needs a new politics – and fresh policy options March 17, 2023
  • What does Hunt’s first Budget mean for Scotland? (Update) March 16, 2023
  • ChatGPT, you’re fired! March 14, 2023
  • Impartiality and public service media March 13, 2023

The Sceptical Newsletter

The Sceptical Scot cartoon

Categories

  • Articles (664)
  • Blog (543)
  • Books & Poetry (26)
  • Brexit (206)
  • climate crisis (5)
  • climate crisis (28)
  • Covid19 (65)
  • Criminal justice (17)
  • Culture (306)
  • Devo20 (1)
  • Economics (191)
  • Economy (110)
  • Education (75)
  • Elections (187)
  • Energy (1)
  • Environment (67)
  • European Union (259)
  • Featured (41)
  • Federalism (19)
  • federalism (13)
  • Health (63)
  • History (69)
  • Housing (23)
  • Humour (11)
  • identity (14)
  • Independence (279)
  • Inequality (77)
  • International (36)
  • Ireland (6)
  • Ireland (8)
  • Local government (82)
  • Longer reads (72)
  • Media (11)
  • Podcast (3)
  • Poetry (72)
  • Policy (218)
  • Politics (342)
  • Polls and quizzes (1)
  • Reviews (24)
  • Social democracy (84)
  • Trump (10)
  • UK (344)
  • Uncategorized (6)

Sceptical Scot elsewhere

Facebook
Twitter

Footer

About Sceptical Scot

Since 2014 Sceptical Scot has offered a non-tribal forum for passionate, informed debate for all who care about Scotland’s future

Recommended

  • Bella Caledonia
  • Centre on Constitutional Change
  • The UK in a Changing Europe
  • Common Space
  • Gerry Hassan
  • Scottish Review
  • Social Europe
  • Think Scotland

Archives

Copyright © 2023 · Magazine Pro on Genesis Framework · WordPress · Log in

 

Loading Comments...