Wednesday, June 5, 2019

Horizon trial day 15: slugging it out

Jason Coyne
There were some short sharp wins and big hits against the claimants' independent IT expert Jason Coyne during his second day of cross-examination.

The Post Office QC, Anthony de Garr Robinson, built a picture of a picture of a man who had latched onto real, but largely inconsequential problems with Horizon, and used them to wildly extrapolate into unsubstantiated theories which were not supported by credible evidence.

You can click on the transcript to read it as it happened, or go to my collated live tweets for a blow-by-blow experience. Or you can read my summary report below.

Straight in, no mucking around

In the first session of the day Mr de Garr Robinson landed some clean punches, first quoting a section of Mr Coyne's report which referred to an internal Post Office document regarding cash management systems:

"whilst the Post Office was looking at ways to improve cash management, it is also indicative that the system was generally far from perfect and there existed a real risk of bugs/errors/defects adversely impacting on branch accounts despite the processes in place at the time to prevent this."

Mr de Garr Robinson called up the Post Office document and took Mr Coyne through it. Where, he wondered, was there any evidence of his claim there "existed a real risk of bugs/errors/defects adversely impacting on branch accounts"?

There was a long pause whilst Mr Coyne looked through every page on the document. Then he answered:

"It is incorrect to find that from that document."

Mr de Garr Robinson took Mr Coyne to another statement he made in his report and read from it:

""It is clear that in some instances it is not always apparent whether recurring discrepancies were as a result of system bugs or the Subpostmaster's own actions, or other things beyond the control of the subpostmaster." Then you have two footnotes. Then: "However the fact that the SSC support team were unable to assist or identify the root cause does undermine the credibility of Horizon itself.""

Mr de Garr Robinson brought up the source document referred to in one of the footnotes. Interestingly it is a Fujitsu log of problems happening at Seema Misra's branch in West Byfleet.

(Seema Misra is the Subpostmaster who was thrown in prison whilst pregnant after being convicted of theft. It was a phone call from her husband in 2010 - whilst Seema was still in prison - which provoked my interest in this story.)

This document is dated 23 Feb 2006 and was written by Fujitsu engineer Ann Chambers. It says:

"I have checked very carefully and can see no indication that the continuing discrepancies are due to a system problem. I have not been able to pin down discrepancies to individual days or stock units because the branch does not seem to be operating in a particularly organised manner... I recommend that this call is passed back to NBSC tier 2 [Post Office Horizon support] for further investigation, since there is no evidence that the discrepancies are being caused by a system problem."

"Why," asked Mr de Garr Robinson, "do you say that this log undermines the credibility of Horizon itself?
"

Mr Coyne accepted it didn't.

Then Mr de Garr Robinson went on to cast doubt on Mr Coyne's frames of reference and attention to detail. He contended Mr Coyne was looking for problems with Horizon from the available data in order to support a claimant-friendly predetermined thesis that Horizon is riddled with errors, bugs and defects which could have collectively cost the claimants £19,000,000.

Mr de Garr Robinson highlighted an example from the Helen Rose report, which Mr Coyne uses to suggest that a Horizon error stopped an interrupted session receipt being generated when a terminal crashed in a particular branch.

I've written about the Helen Rose report in the past and posted it up on this blog. It was used in court earlier in the Horizon trial by the claimants QC to make a point to a Post Office witness.

Today Mr de Garr Robinson used it to note that Ms Rose had erroneously suggested an interrupted session receipt was not generated in the case she was investigating. In fact her report even includes an exchange of emails which clearly states the receipt was generated.

Mr de Garr Robinson put it to Mr Coyne that: "in your anxiety to write a bad thing... in
 your report about Horizon, you recorded what was said on the first page of the report, but you didn't look at the
 second page of the report which would have shown that
 that bad thing wasn't in fact correct?"

Mr Coyne demurred from the wider point, but accepted he was wrong on the receipt printing issue. Later Mr de Garr Robinson pushed his thesis:

DG: You are perfectly happy when you see an example of
 a handful of things happening to say things often happen when they favour a case -- that they help build a case that Horizon is bad. But when I ask you a simple question, "You have seen lots of PEAKs [error notes logged by Fujitsu] in which these exceptions are investigated?" you are unwilling even to concede that it happens a lot of times. I'm quite interested in why you should have a different attitude depending on whether or not something is a criticism of Horizon or in praise of Horizon?
JC: I'm attempting to be as precise as possible with my answers to you.
DG: When it comes to saying something positive about Horizon you are very precise indeed. Can I suggest to you, Mr Coyne, that you are rather less precise when it comes to criticising it.
JC: That's certainly not my intention.

The big boo-boo

The Post Office legal team did their homework last night - diligently reviewing Mr Coyne's first day of evidence. It allowed Mr de Garr Robinson to bring up something Mr Coyne had saidabout Post Office decisions to let bugs lie on the system because they would be too expensive to fix.

This is how the exchange when in yesterday's transcript:

DG: "[quoting JC's report] "there is also evidence to indicate that a cost/benefit analysis was applied to the fixing of bugs/errors/defects and that the possibility of error was not reduced as far as possible." So you're saying, well, countermeasures are applied in are likely to apply most of the time, but there is this problem. And you are suggesting there is a problem in the support process because the support process - that's Fujitsu, the SSC [third tier support] - appears only to have fixed bugs on the basis of a cost benefit analysis... You repeat this point in the third joint statement and you refer to it half a dozen times in your second report, don't you?
JC: I do, but it is important because the question that was asked was: "was it reduced as far as possible?"
DG: I'm sorry, where is that asked?
JC: It is one of the Horizon Issues.
DG: I see.
JC: So when you consider "as far as possible", cost benefit shouldn't really be involved in that consideration. It should be whatever is possible.

Mr de Garr Robinson raised it today:

DG: In your evidence yesterday we discussed your approach, remember, to whether and to what extent Post Office and Fujitsu did things on a cost benefit basis?
JC: Yes.
DG: In the course of that evidence I recall you indicating that you regarded it as important to ascertain whether the possibility of error was reduced as far as possible. Do you remember that exchange that we had?
JC: Yes.
DG: Was it your objective in your reports to address that question?
JC: Was it my objective at the outset to address that question?
DG: To consider not whether the risk was reduced as far as reasonable or to consider whether the risk was reduced as far as practicable, but to consider whether the risk was reduced as far as possible, which is a much more
 exacting standard?
JC: I believe that was the word that was used in the Horizon Issues.
DG: So would the answer to my question be yes, that when you produced both your reports you did so with the objective of applying that test when determining whether something
 constituted a problem in the system or not, whether it satisfied the test of reducing a risk as far as
 possible?
JC: Yes.

Mr de Garr Robinson, all casual-like, then asked Mr Coyne to take him to exactly where in the Horizon issues it asks whether risk was reduced as far as possible.

There was a long pause as Mr Coyne looked through the Horizon issues. And then he said:

"The reference at [Horizon] Issue 6... is reduced to an extremely low level, the risk."

Mr de Garr Robinson replied: "Yes. So it is a factual question as to how low level the risk was. It is not a question whether Post Office had reduced the risk to the lowest possible level, is it?  I'm just wondering, Mr Coyne, whether you may have applied in your entire approach to your reports the wrong test for the purposes of these proceedings...."

Mr Coyne didn't think he had, but the damage had been done.

The fightback

Mr Coyne was on stronger ground when Mr de Garr Robinson challenged him on his report's opinion that audit logs (held by Fujitsu) should be used for investigations, not the Management Information System software used by Post Office.

Mr de Garr Robinson was at pains to point out how raw audit data, in this case known as ARQ logs were slow, difficult and costly to access, to the extent that checking them every time there was a discrepancy would cost the Post Office more than £50m a year in fees to Fujitsu. Mr Coyne (to paraphrase) said, well, that's because you've got a stupid system:

"Audit systems are often very easily accessible to be able to be read by certain users.  There's nothing
 inherently difficult about that... That's the purpose of having an audit store. There is no other reason for it other than looking back at what actually happened...  If you are making decisions based purely on a cut-down version of the data in a Management Information System you have to
 decide what data is going to be cut out of that.  So if everything is in the audit store and just a portion of that data is in your Management Information Systems, then you are going to necessarily make a decision based on a subset of the data and not the whole of the data."

Reality vs Likelihood

The afternoon session was taken up with discussions around the massive methodological battle between the two expert witnesses.

Mr Coyne has spent his time investigating real bugs (and in Mr de Garr's opinion, wildly extrapolating from there).

Dr Worden (the Post Office's IT expert) has also looked at real bugs and then done a statistical analysis of the likely impact of all bugs on the claimants based on his interpretation of the effect of the real ones.

The general point from Mr Coyne is that we only know the known bugs (and even then we're not precisely sure of their impact). The unknown bugs could have wreaked havoc.

I only got this from the exchanges in court today - I haven't yet seen his reports - but Dr Worden seems to take the view that if you compare the bugs, the way they were dealt with and the branches they affected with branches which had error-free operation (or at least branches which were apparently free from IT errors which impact their finances), there is a tiny, tiny likelihood that anyone has suffered much from a Horizon bug, error or defect. The inference being most discrepancies are therefore due to some kind of mistake in branch or dishonesty.

Because of Mr Coyne's refusal to usefully quantify anything - Mr de Garr Robinson tried to nail him into a ballpark, to assess the "many", "often", "thousands", "likely", "potentially", "possible" pronouns, adverbs and vague numbers which come up in his expert report.

It was knuckle-chewing stuff and grindingly slow. But like a dentist tasked with extracting a full set of wisdom teeth, over two and a half hours, Mr de Garr Robinson made a very decent fist of it.

You can read the full transcript here - search for "lunch" and read from there. The reasons Mr Coyne might have had for not coming up with at least ballpark guesses on the impact of Horizon errors, or better still, informed estimates are relentlessly stripped away.

Using the information in both Mr Coyne and Dr Worden's agreed statements the Post Office QC suggested that if Mr Coyne had wanted to quantify the likelihood of widespread errors in Horizon which had a multimillion pound impact on branch accounts, he would have agreed with Dr Worden's conclusion that they were staggeringly unlikely to exist.

Mr Coyne's defence was that Dr Worden's methodology was all wrong.

We'll see. There are two more days of this cross-examination to go. I wonder what delights Thursday will bring.

*******************
If you can, please help keep this crowdfunded public-interest journalism project going by chucking a few quid in the tip jar below. Contributors who give £20 or more will start receiving regular "secret" emails which have all the info and gossip about this litigation as it makes its way through the courts.

If you want to see how much I have raised to date and how I've spent it, click here.
If you want to find out more about the underlying story, click here.
Choose an amount