Thursday, June 13, 2019

Horizon trial day 19: No One Is Getting In A DeLorean

Dr Emmett Brown
There were two big issues raised today by Patrick Green (QC for the claimants) in his cross-examination of Dr Robert Worden, the independent IT expert contracted by the Post Office to look into the Horizon issues raised in this trial.

The first was the information Dr Worden might have factored into his statistical model which he either chose not to consider, mistakenly failed to consider, or simply wasn't told about.

This was striking when Patrick Green was pressing him on transaction corrections. Transaction corrections are issued by the Post Office to individual branches, groups of branches or the entire network of Post Office branches in order to correct something which has gone wrong.

If a bug or user error causes a branch to be £500 out of pocket (or in surplus), once the bug or error has been flagged and examined, a Transaction Correction or TC is manually issued to the branch via Horizon, which the Subpostmaster accepts into her or his branch accounts by pressing a button on ;her or his screen and all is made good. Or at least, that's the theory. There are hundreds of thousands of transaction corrections issued every year.

"We have got to do it"

Mr Green wanted to know on what basis Dr Worden assumed that errors, having either been picked up in branch, automatically or by some manual process would be made good via a transaction correction. The answer was revealing:

DW: "The procedural side of how someone was compensated for
 a bug and how this team pushed numbers around and so on, that procedural side is outside the PEAKs and generally not something I'm... you know, how the suspense account team worked, what their policy was, what you could do,
 and so on, that sort of stuff I don't know."
PG: "But your expectation that it won't amount to a lasting discrepancy."
DW: "Yes, is that this procedure will somehow work out and
 these teams will decide eventually: you do it, we do it, we have got to do it."

Go Post Office!

and...

PG: "You inferred from a KEL [Fujitsu Known Error Log] that didn't say it that
 transaction corrections would be issued?
"
DW: "I mean very often one has to infer from KELs things that
 are not said. They are written by people who know things that we don't know and they are not sort of written for us."
PG: "You have spread that across other KELs which don't
 mention transaction corrections and reached the same inference, haven't you?
DW: "I have made that same inference in several cases, yes.
"

You might think this is a perfectly innocent expectation, but it assumes motivation and competency. Which really does boil down to the whole point of this litigation. The claimants contend that when there was a problem with Horizon, they didn't get a transaction correction (or much other help) to solve it. Dr Worden appears to have assumed they did and used it to inform his conclusion that the claimants are wrong.

Mr Green made the point that the Post Office didn't have a huge incentive to do issue transaction corrections which favour a Subpostmaster, given:

a) it was much easier and much more cost-effective to "pressure" Subpostmasters to cover the cost of bug, error or defect by making it good themselves.

b) there was a possibility that any branch discrepancy could end up in the Post Office's own suspense account, which after three years is then funnelled into its overall profit and loss account.

Giant admin cost

It led to this interesting exchange:

PG: "If Post Office was gaining money at the expense of Subpostmasters... that in itself would not be an incentive to try and
 give the money back, would it?

DW: "Well, I was asked to consider Post Office's motivations,
 and in that context I think that one of Post Office's
 motivations is to keep down the administrative problems
 of things recurring and not being sorted, and if you
 don't sort the discrepancy and the postmaster keeps
 coming back and so on, your admin costs will soon rise.
 So it is a complicated... there is a lot of motivations
 at play there.
"
PG: "There are a lot of motivations at play. If
 Subpostmasters are, let's say hypothetically, pressured to pay regardless of who is at fault and they just pay
 up because of the way they are asked, hypothetically,
 that reduces your admin costs a lot, and if they have
 suffered a loss you don't have to give the money back,
 so that meets two objectives?"
DW: "That is part of the trade off. There will be the ones,
 as you say, that just knuckle under and take it, and the
 ones that keep banging the table will cost you a lot of 
money.
"

It made me wonder if this trial is the giant admin cost caused by a failure to fix problems when claimant Subpostmasters were having them.

The second big issue was about the assumptions and estimates Dr Worden relied on when he came to his conclusion that the chances of any claimant being affected by any meaningful bug to be vanishingly small, something which weirdly coincides with the Post Office's argument.

Mr Green explored a few of them this morning - in particular a calculation Dr Worden had made about the chances of a Transaction Correction being issued in error. It's this that led to everyone suddenly thinking about Dr Emmett Brown.

Dr Worden accepted that TCs could be issued in error, and on the basis of figures provided to him by Post Office (which in cross-examination transpired to be a guess) concluded that for the average erroneously issued TC costs each branch in the network £6 per month, averaged down to £2 per month for claimant branches (based on the the size and number of transactions they processed).

Twitter gets involved

Dr Worden used this calculation in his second report to inform his conclusion that erroneous transaction corrections could notbe the source of claimants' problems.

Mr Green picked up Dr Worden's logic and asked him about Naushad Abdulla, one of the lead claimants in the Common Issues trial. Now we know that Mr Abdulla had erroneous transaction corrections of £1092 issued to him twice in two months.

Mr Green and Dr Worden agreed that if the two transaction corrections had no correlation (ie they weren't issued for the same or related reasons, which could affect their statistical probability) there was a 1 in 500,000 chance of this happening.

Amazingly, it happened to Mr Abdulla. Yet this was not used by Dr Worden as a way of evaluating his calculations. Dr Worden didn't look at what had actually happened to claimants.

"Why?" asked Mr Green asking Dr Worden to think about his Penny Black analogy from Tuesday.

Mr Green did not take Dr Worden through the analogy again, but chose instead to use one posited on twitter by Professor Steven Murdoch, a Royal Society Research Fellow who has been following this trial:



Dr Worden was not impressed. He accepted that claimants being special in some way was important, but said it was "not a material factor
 in whether Horizon during your tenure caused bugs.
"

Mr Green pushed back: "Dr Worden, just because you join [the claimant group, it] doesn't mean that
 we go Back to the Future with the professor in the
 DeLorean and Horizon causes problems in the past."
πŸ‘†πŸ‘†πŸ‘†πŸ‘†πŸ‘†πŸ‘†πŸ‘†πŸ‘†πŸ‘†πŸ‘†πŸ‘†πŸ‘†πŸ‘†πŸ‘†πŸ‘†πŸ‘†πŸ‘†πŸ‘†πŸ‘†πŸ‘†POPULAR CULTURE HIGH COURT REFERENCE KLAXONπŸ‘ŠπŸ‘ŠπŸ‘ŠπŸ‘ŠπŸ‘ŠπŸ‘ŠπŸ‘Š

Dr Worden stood his ground: "They have self-selected.
... the causation is back to front, and causation
 is what's at issue here.
"

Dr Worden has taken a view of Horizon as a whole, which he is entitled to do. After all, the Horizon issues don't require him to look at specific users of the system.

My analogy is best

But I can also appreciate how the claimants have problems with his approach. If some people are being murdered in the corner of a 1,000,000 acre swamp, there is nothing bad happening in 99.999% of the swamp. The mangoes are growing, the sun is shining, the air is humid. The whole eco-system is working as it should. But there are still people being killed in the corner of the swamp.

Someone who has a vested interest in growing and selling mangoes may want to focus on the success of the mangoes across 99.999% of the swamp. Those more concerned with people being killed will take a different view.

Dr Worden's approach has value. He's a super-bright particle physicist and very experienced expert witness. The judge will consider his approach thoroughly, as he should.

I am not even that surprised at Dr Worden's incuriosity as to what actually happened to claimants. If you look at Professor MacLachlan's approach to Horizon in the Seema Misra trial, the Post Office barrister ridiculed him in court for not even picking up the phone to Mrs Misra to get an idea of what she thought was going on. He looked at the system as a whole. It didn't seem to serve her very well.

Horizon issue 4 specifically asked the IT experts to deal with possible errors in data entry (ie Subpostmasters pressing the wrong buttons). Dr Worden didn't want to go near that - so there is an element of selection bias. His prerogative, I guess.

Dr Worden's final day of being cross-examined begins at 10.30am tomorrow, Friday 14 June. I'll be live tweeting here. I hope you can join me.

PS Just before lunch, the judge noted that the transcribers, in their realtime transcription of the day's proceedings, had spelt DeLorean "De Lorey Yum". He asked that counsel ensure the correct spelling be supplied for the transcript. They did.

*******************
If you can, please help keep this crowdfunded public-interest journalism project going by chucking a few quid in the tip jar below. Contributors who give £20 or more will start receiving regular "secret" emails which have all the info and gossip about this litigation as it makes its way through the courts.

If you want to find out a little bit more about the underlying story, click here.
Choose an amount