 |
| Dr Robert Worden |
According to Mr Green (in a definition I can't find anywhere else on the internet), User Error Bias is an institutional or individual inclination to blame the user for a problem which may or may not originate with a machine or computer.
Having introduced the idea of user error bias on day six of the Horizon trial, he successfully persuaded a Post Office director, Angela van den Bogerd, that not only did "UEB" exist, but that the Post Office suffered from it.
Over the course of the last two weeks I have been wondering about IEB, a bias which appears to afflict independent experts.
Today was Doctor Robert Worden's final day of cross-examination in the Horizon trial. Dr Worden is the independent IT expert contracted to the Post Office to report on the Horizon issues. Apart from a few occasions over the last three days he presented himself as a chuckling, enthusiastic, genial old cove. But as I listened to his evidence, I did start thinking about bias. Not because his reports' conclusions neatly match the Post Office's party line on Horizon (client bias), but because the assumptions he built into his methodologies seemed to match his own expectations of human behaviour and endeavour.
Dr Worden is the sort of person who tries to do the right thing and do it well. I wonder if he might believe this attitude is a default attribute of well-meaning corporations and computer engineers? And if that informs his investigative approach?
After all, why drill down into the nitty-gritty if a summary document tells you everything's fine? Why believe a malcontent if the company who employed him reckon he's wrong? Why check what actually happened when you can listen to credible people who tell you what would have happened? Why decide you don't need to see a document when your client informs you it's out of scope?
The best of all possible worlds
To give just one example, in Dr Worden's dismissal of a 2010 Horizon bug he writes:
"The KEL [Known Error Log] implies that PO and Fujitsu were able to identify all occurrences of the problem, without being notified by any Subpostmaster I would therefore expect them to have corrected any impact on branch accounts as part of normal error correction processes."
He read an implication into a document, and formed an expectation which possibly then fell into his statistical model.
This sort of blithe belief that everything is for the best in the best of all possible worlds seems to underpin Dr Worden's working assumptions: if Horizon works well for most people most of the time and the experts who give me information are both honest and too clever to make mistakes, then all I need to do is... just... run... a... feeeew... numbers... and bingo! Statistical proof Horizon's processes are spot on. The claimants must be mistaken, or making things up.
As Mr Green ramped up his cross-examination, large chunks of Dr Worden's evidence appeared to unravel. On Tuesday Dr Worden seemed surprised when internal Post Office documents described Horizon as "old", "slow", "clumsy", "user unfriendly" and the entire Post Office IT set up as "not fit for purpose". Today he was being shown documents which revealed the depth and scale of security breaches, bugs and errors.
He was also tested on his assumption that when bugs were identified, techy types at Fujitsu would apply a solid fix and alert the good people at the Post Office. This would lead to poor benighted Subpostmasters having their branch discrepancies universally and retrospectively made good by swift dispatch of a trusty Transaction Correction. The evidence on the ground, of which Mr Green had plenty, told a different story.
Not vanishingly small, then
The most important moment today was when Dr Worden appeared to directly contradict his (and the Post Office's) conclusion that the chances of any claimant Subpostmasters being materially affected by a bug (or error or defect) in Horizon was "vanishingly small".
Mr Green asked Dr Worden to do some sums based on some agreed numbers. Dr Worden had estimated that over 20 years Horizon had been hit by 145 bugs (central esitmate) or 672 (conservative estimate).
Mr Green said there were four known bugs which affected an average of 48 branches each. Dr Worden didn't consider one of these known bugs a bug, but agreed to accept it for the purposes of the example.
So 145 bugs x 48 branches = 6,960 bug incidents (central estimate)
or 672 bugs x 48 branches = 32,256 (conservative estimate).
 |
| Patrick Green QC and colleague Kathleen Donnelly |
If the bugs on average only affected 40 branches, then the number of incidences would be:
145 x 40 = 5800 incidences of bugs affecting branches
or if you went for the "conservative" estimate:
672 x 40 = 26,880 incidences of bugs affecting branches.
Mr Green pointed out there were only 550 claimants "and even with few multiple branches and a few people having been affected by more than one instance, on your definition of consistent, the figures we have just been looking at are consistent with the complaints made by the claimants in this case."
Dr Worden wasn't sure, so he did some sums of his own.
He took the 672 bugs affecting 48 branches (conservative estimate) = 32,256 and rounded it down for convenience to 32,000.
Then he added some new figures - that of the 3,000,000 monthly branch accounts across the network in 20 years, and the total 52,000 monthly branch accounts the 550 claimants who had Post Office branches were in charge for (which he then rounded down to 50,000).
Dr Worden took 32,000 bugs, 50,000 branch accounts, divided them by 3,000,000 and (after taking out all the zeroes) finished with 32 x 50/3 = 500.
"So" he announced "it is consistent with one occurrence of a bug to each claimant branch during their tenure."
To which Mr Green said: "thank you, Dr Worden."
Though, as Dr Worden pointed out, the central estimate would give a smaller number.
I wouldn't touch it with yours
We are nearly at the end of the Horizon trial. This is what I have learned over four weeks of evidence:
- we are no closer to knowing whether Horizon is "robust" or "relatively robust" because the experts are both agreed the true extent of Horizon itself is unknowable.
- the answer as to whether bugs had a lasting impact on claimant branch accounts is "possibly" and "possibly not".
- Fujitsu and the Post Office have and always have had the power to go into branch accounts and do whatever they want with or without Subpostmasters' knowledge or permission.
- neither independent expert looked at the specific circumstances of each claimant's branch, despite this being the most obvious way to see if Horizon errors were the source of their discrepancies.
- statistical conclusions built on too many incomplete or assumed variables are more or less meaningless.
- investigative conclusions reached via incomplete data are more or less meaningless.
- this trial has cost a hell of a lot of money, taken up a lot of time and moved forward our understanding of the key issues in this litigation a tiny, tiny amount.
- if I could be ever held liable for a single penny, I would not go near Horizon with a bargepole.
Still. It was fun to report. The Horizon trial resumes for summing up in on 1 July. We can expect a judgment in the autumn.
*******************
If you can, please help keep this crowdfunded public-interest journalism project going by chucking a few quid in the tip jar below. Contributors who give £20 or more will start receiving regular "secret" emails which have all the info and gossip about this litigation as it makes its way through the courts.
If you want to find out a little bit more about the underlying story, click here.