1 Thursday, 23 February 2023 2 (10.00 am) 3 MR BLAKE: Good morning, sir. Can you see and hear 4 me? 5 SIR WYN WILLIAMS: I can indeed, yes. 6 MR BLAKE: Thank you very much. I'm going to call 7 Mr D'Alvarez, please. 8 ALAN D'ALVAREZ (sworn) 9 Questioned by MR BLAKE 10 MR BLAKE: Thank you very much. Can you give your 11 full name please. 12 A. Alan George D'Alvarez. 13 Q. Mr D'Alvarez, you have previously given evidence 14 to this Inquiry. Your witness statement is 15 WITN04800100 and that's already gone into 16 evidence. Do you have a copy of that in front 17 of you? It should be behind tab A. 18 A. Yes, I do. 19 Q. You've confirmed the truth of that statement 20 before but just to confirm once again, does that 21 remain true to the best of your knowledge and 22 belief? 23 A. It does. 24 Q. Thank you. As I say, because you've given 25 evidence, I'm not going to go into detail about 1 1 your background, we already know about that. 2 Just one question about your background is 3 really when you first became involved in what we 4 know as Horizon Online or HNG-X? 5 A. Horizon Online, I got involved in 2009. 6 Q. The reason I ask is I just want to bring one 7 document to your attention, it's FUJ00116732. 8 The document itself isn't really important, it's 9 a PowerPoint presentation but it's page 7 of 10 that document. It's the fourth entry on page 7. 11 Thank you. If we could zoom in, it has your 12 name there. These are documents that are 13 reviewed and it says there was a previous 14 independent review by A D'Alvarez, July 2007; do 15 you remember that involvement at all? 16 A. Yes, I do. 17 Q. What was that? 18 A. I was, at that time, on assignment in the USA, 19 and on my -- when I was back in the UK for some 20 meetings that we had, because of my previous 21 engagement with Post Office, I was asked could 22 I provide an overview because there were some 23 issues, could I just talk to some people and 24 just do a review in my two weeks that I was in 25 the UK, take two of the four weeks I was in the 2 1 UK, just to discuss the situation. 2 Q. Broadly, can you tell us what you concluded in 3 that report? 4 A. I concluded -- so what I concluded in that 5 report broadly was the solution that had been 6 signed up to was different to what I recall when 7 I was -- that was being discussed with Post 8 Office when I was still on the account back in 9 2005. That the approaches being used with 10 regard to development were not appropriate, and 11 they should look to do more of a kind of 12 a classic approach, as opposed to an agile 13 approach to development, and just gave some 14 explanations as to what I believe needed to 15 happen to bring the project under greater 16 control. 17 Q. Do you know who asked you to do that report? 18 A. I believe it was Lester Young, I believe. 19 Q. Were there concerns about HNG-X at that stage 20 within Horizon, withing Fujitsu? 21 A. Can I correct myself, I think it was Peter 22 Jeram. 23 Q. Were there concerns within Fujitsu about how the 24 project was working? 25 A. It was late. It was already signalling that it 3 1 was missing its key milestones. 2 Q. Thank you. I'm going to now take you to some 3 documents that I took Mr Burley to yesterday. 4 I don't think you've seen all of Mr Burley's 5 evidence from yesterday; is that right? 6 A. I only saw part of it. 7 Q. It may be that I'm repeating matters I went 8 through yesterday but that's for your benefit 9 rather than for everybody else's. Can we look 10 at FUJ00092754, please. These are the "Notes of 11 the Horizon Next Generation Joint 12 Progress/Release Board" of 28 January 2010. 13 Mr Burley was the chair and you attended that 14 meeting; is this something you remember in broad 15 terms? 16 A. These were regular meetings that we had, yes. 17 Q. It's page 3 of that document that I asked 18 Mr Burley about yesterday, and it's the second 19 entry on page 3. It says there: 20 "The delay in the commencement of Volume 21 testing means that we will not be able to 22 perform a significant amount of testing before 23 commencing the Medium Volume Pilot. Hence we 24 will need a significant amount of data to be 25 collected from the Live Branches and Data 4 1 Centre. The data will also require careful and 2 thorough analysis. 3 "AD to confirm how this will be achieved." 4 Is it your recollection that there was 5 a reduction in the testing before commencing the 6 medium volume pilot, howsoever small it may have 7 been? 8 A. So that's specific to the volume testing. So we 9 had a volume test schedule and with the volume 10 test schedule it went in stages in different 11 parts of the system but it also blocked out to 12 where we would go to 25 per cent volumes, 13 50 per cent volumes. The intention of all the 14 testing, what we call laboratory testing, would 15 be complete before we go to pilot and pilot's 16 live testing. So the intention was to complete 17 all the laboratory testing before. We hadn't 18 completed all the volume testing. 19 We had completed an amount -- I can't recall 20 where we got to, either 25 per cent or 21 50 per cent loads. It was assessed that to go 22 into a pilot and to go to medium volume pilot, 23 which I believe was around 250 post offices, 24 that we stressed the system sufficiently. 25 However, what we proposed was we would 5 1 monitor -- we were already monitoring the 2 performance at the branches. What we would do 3 we'd collate those because when you do 4 laboratory testing it's in sterile conditions. 5 You don't have the real world rounds and 6 networks and that, so it's kind of model 7 testing. So to give us greater assurance, 8 because we hadn't completed up to full load 9 testing, we're going to take the performance 10 statistics and compare them to our testing 11 statistics or our test results to see whether 12 what happening live correlates to what we see in 13 testing, to give us that greater confidence. 14 Q. Would it be fair for me to say that that kind of 15 testing that was carried out, or the analysis of 16 the data, is less than was originally proposed? 17 A. More. When I say the analysis of the data was 18 more, so we hadn't completed the testing. We 19 did complete the test within the next four 20 weeks, four or five weeks. We hadn't completed 21 the testing at that time. So going into medium 22 volume pilot, it was -- we hadn't completed the 23 testing that we had on the schedule. Was it of 24 material impact? We didn't believe so. So we 25 did complete all the testing. We completed 6 1 testing up to 50 per cent -- 25 or 50 per cent 2 load to equivalent to 3,000 to 6,000 post 3 offices we've tested at scale, in our 4 laboratories and the medium volume pilot was, 5 I think, 250 or thereabouts. 6 Q. The reference in this particular entry about not 7 being able to complete a significant amount of 8 testing before commencing the pilot suggests 9 that there was less testing. 10 A. In the volume. 11 Q. In the volume. 12 A. Only in the volume, not in the functional 13 testing of the business application. 14 Q. Are there risks involved in reducing the amount 15 of testing in the volume; howsoever small, are 16 there risks in reducing that amount of testing? 17 A. Our assessment from Fujitsu was no because we'd 18 done sufficient testing for the amount of stores 19 that we were going to bounce off -- or post 20 offices -- were deploying in medium. However, 21 as I've explained, because it was laboratory 22 testing, we wanted to put in some additional 23 controls just to make sure that what we've 24 tested, the results in testing, correlate to 25 what we were seeing in live, so give us greater 7 1 confidence. 2 Q. Why would it be proposed originally as an idea? 3 A. Proposed? 4 Q. Why was it proposed originally that there was 5 more testing and that amount of testing was 6 reduced or are you saying it simply wasn't 7 reduced? 8 A. It wasn't reduced. It was completed. We 9 completed all the testing. It was the 10 timescales in which we completed the testing. 11 Q. So they were over a longer period? 12 A. Yes. So we continued testing the performance 13 whilst we was in pilot. The original plan was 14 to complete all our laboratory testing before 15 going into pilot. Only on the performance, we 16 continued to test whilst we was in pilot. 17 Q. Is there an advantage in completing that testing 18 before the pilot? 19 A. It allows the team to focus more on the pilot 20 because, obviously, we wanted to clear 21 everything that we had so we can focus all our 22 attention on supporting the pilot. But I don't 23 believe it had any material impact on risk or 24 anything going into the pilot. 25 Q. Thank you. I'd like to take you into the next 8 1 document, that's FUJ00097159. Again, it's 2 a document I took Mr Burley to yesterday. It's 3 a meeting of the same day, this time of the 4 release authorisation joint board. Again, is 5 that something you recall, those meetings? 6 A. Yes, they were the meetings that we used to 7 track our position against the various 8 acceptance case. 9 Q. If we go over the page, please, and it's the 10 shaded section that I'm going to start with, it 11 says: 12 "DC confirmed that there are no outstanding 13 High Severity Acceptance Incidents and that all 14 other thresholds are within tolerance for 15 Acceptance Gateway 3." 16 The third paragraph says: 17 "However it was agreed that the high 18 priority fixes in 'Reset 4' ..." 19 Do you remember what Reset 4 was? 20 A. I believe that's a release that we had, so we 21 had a number of releases that we called, and 22 that was -- basically will be delivered as 23 part -- so we had maintenance releases that went 24 out not just for the programme deliverables but 25 also just for general maintenance patching and 9 1 things, and then the Reset 4 would be the 2 programme release that we were to put into that 3 maintenance release. 4 Q. Thank you. So the high priority fixes: 5 "... to be delivered as part of [the] 6 Maintenance Release ... could constitute a High 7 Severity [Acceptance Incident] if not delivered 8 in time for the High Volume Pilot ..." 9 As you said, it's just over 200 branches. 10 If we scroll down, it says there that 11 Mr Burley: 12 "MB offered an option to remove items from 13 Reset 04 which are not regarded as High 14 priority -- if they are at risk of missing the 15 High Volume Pilot deadline, or affecting the 16 delivery of items which are High priority." 17 Is it your recollection that Mr Burley and 18 the Post Office were trying to make it easier 19 and quicker to get the pilot up and running and 20 speed things up a bit? 21 A. It's an option that's discussed. So with all 22 programmes you have to balance risk against 23 progress. So one of the options that was 24 discussed and was -- we, I think we agreed to 25 actually take away and understand what that 10 1 actually meant with regard to risk and that, but 2 one of the options were -- there were some fixes 3 that needed to go in and we couldn't progress. 4 Others were of less impact, some could be 5 cosmetic. Some may be -- and the bigger you 6 make a release, the more risk you put on the 7 timescales that that release will be complete, 8 fully tested and ready to go. 9 So it was to -- I think the discussion at 10 the meeting was very specific to if we wanted to 11 achieve the milestones, was there anything of 12 less significance that could wait to a later 13 release and would that -- if we did remove that 14 from that particular earlier release -- would 15 that derisk making a timescale? 16 So it was, you know, we'd -- delivering 17 a programme, you know, you're always looking at 18 the balance of risk and timescales. 19 Q. Absolutely. So you describe risk against 20 progress. Was there, coming from the Post 21 Office, a push -- we've spoken about the delay 22 in the programme, for example. Were they keen 23 on making progress and accepting a greater 24 degree of risk? 25 A. It depends what part of the Post Office, really. 11 1 So within the programme team, we were pretty 2 much joined up at the hip, myself and Mark and 3 our team and his team, in as much as we had 4 a focus on -- there are certain items of 5 delivery that you had to get right before we 6 progress. Others were up for risk assessment. 7 I had pressure from my company. I don't know 8 what pressure he got but I'm sure he would have 9 got pressure outside because people want to see 10 progress. 11 Q. Was he making clear to you that he was under 12 pressure to make progress? 13 A. We had an approach where, when we had to 14 potentially signal delays, we would work 15 together to what is the messaging to our joint 16 -- you know, do we agree firstly amongst 17 ourselves that a delay is the right thing? 18 Let's have look at the whole position, look at 19 the risk. If we do agree amongst ourselves that 20 a delay is the appropriate course, then we would 21 work on the messaging to our respective 22 organisations because with that comes quite 23 a lot of disruption in the organisation, quite 24 a lot of disappointment, and what -- we wanted 25 to make sure that the messaging was clear. 12 1 So in discussions, it's the normal pressures 2 of a large programme. People want to see it 3 succeed. People want to see it go out, and it's 4 our job to navigate to make sure that we're 5 protecting -- you know, that the risk is being 6 managed appropriately. 7 Q. If we stick with this document and move to the 8 penultimate page, page 9, there are a couple of 9 issues that are mentioned at the bottom of 10 page 9 and into page 10. These are new actions, 11 28 January 2010. The first is: 12 "Branch Trading Statements at Coton House 13 and Warwick. It was confirmed that on screen 14 error messages had occurred at both offices 15 which meant that their Branch Trading Statements 16 could not be completed. 17 "The root cause position at Coton House was 18 understood, but the Warwick office had 19 additional complexities." 20 Just pausing there, do you remember this 21 particular issue? 22 A. I remember there were three issues. We refer to 23 them as the CWD, I think it was Coton, Warwick 24 and Derby, and when we went into pilot and it's 25 the -- 10 or 12 offices, I forget how many, but 13 1 there were a small number of offices, we now 2 moved into a live situation. So what we put in 3 was a huge amount of support, huge amount of 4 monitoring, because it's still test, the pilot 5 is still test, and there were three different 6 incidents that come up and they are two of the 7 three. 8 Q. Thank you. If we scroll down, "Double 9 settlement at Derby". 10 A. Yes, that's the other one. That's the third. 11 Q. Can you tell us about that particular issue? 12 A. That was of greater concern because that one had 13 potential integrity implications. So I remember 14 at the time -- so I'm much closer to that one, 15 so I took ownership of that because of the 16 potential severity of that. The other two were, 17 on the initial assessment, more to do with 18 operational rather than integrity. 19 Q. Then looking below, it says: 20 "Decision -- Postponement of the next 10 21 branches. Based on the lack of a known root 22 cause for AG3.70 [the first of those two] and 23 AG3.71 [the Derby issue] it was agreed that the 24 next 10 branches should be postponed until the 25 impact and way forward is fully understood." 14 1 A. Correct. 2 Q. We've seen that only a few days passed and that 3 postponement was reversed. Do you remember 4 that? 5 A. I recollect that we quickly got to 6 an understanding of the root cause of the Derby, 7 and I recollect that we had a fix and that fix 8 had to go in before we could progress because it 9 was significant, because it was a data integrity 10 issue. Geoff Butts led on the other two because 11 he is my deployment migration manager. 12 If I remember -- I might get these in the 13 wrong order. I believe Coton was to do with the 14 migration itself, ie the migration tooling not 15 completing its data downloads and we had 16 a workaround, ie we would do the download 17 through the migration tooling from Horizon to 18 HNG-X, and I believe we would do a -- we -- 19 someone had written a script to check is it 20 complete as a compare. If it's not, we will 21 repeat the downloads to complete it. 22 So we still went through the migration 23 tooling, so where we were satisfied that whilst 24 we had to fix the migration tooling, that we 25 could avoid that happening again. 15 1 The Warwick one, I believe that was to do 2 with reporting, in as much as the underlying 3 data was correct but there was a report that was 4 created and it wasn't the report -- figures in 5 the report were incorrect. 6 Q. We'll come to the report issue. I think 7 ultimately actually the Warwick one involved 8 a node as well and it's a bit more complicated. 9 A. Okay. 10 Q. We don't need to go into the detail of that 11 right now. 12 A. However, but I think by the time -- within -- 13 you know, we worked over the weekend on this. 14 I do remember that everyone was in over the 15 weekend and we satisfied ourselves that we had 16 fix for the Derby one, which was the critical 17 one, and we had, I believe, appropriate way 18 forwards or we had proposed, because we can only 19 propose potential workarounds, it's for Post 20 Office to accept whether they're acceptable or 21 not. So we had a proposal, which I believe was 22 accepted. 23 I don't know how quickly we went into, but 24 it wasn't long, two weeks later we went. 25 Q. So the postponement was largely or significantly 16 1 because you recognised the importance of matters 2 that impacted on data integrity? 3 A. All three we need to assure ourselves that, you 4 know, but certainly data -- that there was a way 5 forward that would -- that we could mitigate any 6 impact. But certainly data integrity was the 7 postponement, yes. 8 Q. Can we look at FUJ00092875, please. It's page 3 9 of that document. It's an email from yourself, 10 which addresses the decision to deploy HNG-X to 11 a further ten branches. This is 3 February now 12 so it's very shortly after you explained there 13 was work over the weekend. It says there: 14 "The decision has been taken to deploy HNG-X 15 to a further 10 branches with the migration 16 button being pressed tomorrow for migration to 17 complete Friday." 18 There were, however, still two outstanding 19 issues, the first is the branch trading 20 statement issue, and it says there: 21 "This is where the in day migration process 22 that happens once a branch hits the migration 23 button is not correctly migrating across the 24 summary data. This data is used to produce the 25 branch trading statement", et cetera. 17 1 The second issue being counter pauses in 2 live: 3 "Yesterday and today a number of branches 4 reported 'screen freezes' whilst operating 5 HNG-X." 6 A. Mm. 7 Q. So is it fair to say that you were very much 8 aware that there were still outstanding issues 9 that you needed to get to the bottom of but the 10 decision was taken to deploy HNG-X to a further 11 ten branches because you were sufficiently 12 satisfied with the work you had done over that 13 weekend? 14 A. We proposed that -- we make -- well, we jointly 15 looked at the position with the joint programme 16 team and our proposal was to -- we satisfied 17 ourselves the risk was manageable. 18 Q. Thank you. Your evidence is very much about 19 joint decisions and joint working. Was there 20 a difference in opinion at all on these kinds of 21 issues between Fujitsu and the Post Office? 22 A. Sometimes. On occasion there would be -- we 23 would propose something but Post Office would 24 have the final say. 25 Q. In what sense? 18 1 A. In a sense that if we felt that the risk was 2 manageable to move forward, and that was our 3 proposal, we had a workaround to an issue or 4 something, Post Office, if they say they're not 5 comfortable, they're the customer. It's their 6 business. They have the final say on 7 programmatic decisions. 8 Q. It's paragraph 4 of this email that I took 9 Mr Burley to yesterday and I will read that for 10 you. It says: 11 "We had a meeting with Post Office this 12 evening which Mark Burley led from the Post 13 Office side. Post Office are desperate for 14 a date to start planning/rescheduling medium 15 volume pilot. They accepted our position that 16 we were not able to give this today. I expect 17 that Mark will be keeping Dave Smith briefed and 18 my reading is that if we are not in a position 19 to give a target date by [close of play] 20 tomorrow it's likely to result in an escalation 21 to Mike Young." 22 Certainly the impression that's given there 23 is that pressure is being put on Fujitsu to get 24 on with things and that, if they don't, then it 25 will be escalated. Am I wrong to form that 19 1 impression? 2 A. Get on with things. So the situation is 3 specific here, is -- so Post Office were 4 responsible for the business change activities 5 associated with the programme and we're now into 6 live pilot and, therefore, there's a lot of 7 communications, planning, you know, lining up of 8 post offices. So "desperate" is my word, you 9 know, that's my word, in as much as, you know, 10 being a qualified programme manager, and 11 understanding the pressures that they'll be 12 under, is that what we had is a situation where 13 we halted the pilot, we're going cautiously into 14 the continuance of the low volume pilot, which 15 was always intended to be around 20 offices. 16 They would have had all the communications, 17 all the countdowns, people lined up for the 18 medium volume pilot because you do that many 19 days before, like 45 days before there's a big 20 chain. Now we're into a set of uncertainty 21 where we're signalling in to both organisations 22 and they're having to signal in to their 23 business that they've paused and they cannot 24 confirm dates of when we can line up so the Post 25 Office would have been communicated to some of 20 1 those that they had dates. They now had to be 2 communicated to, to say they're paused, but we 3 can't tell them when because we haven't yet got 4 to the root causes of those problems. So it 5 wasn't a case of we just need to get on to do 6 the pilot; we needed to understand how long 7 would it take us to resolve those problems. 8 So once we get to the root cause we can then 9 assess is it a quick fix? Is it a long, complex 10 fix that's going to take several weeks? And we 11 need to give -- you know, Fujitsu needs to give 12 Mark and his team an indication as to what are 13 their planning assumptions they can make so that 14 when they communicate to post offices, as 15 opposed to leaving them, we're going to delay 16 the deployment but we don't know when to, they 17 can be more certain as to kind of provide that. 18 It kind of -- it's better when you're 19 communicating out that you can give someone 20 alternative dates or a clear understanding of 21 expectations. At this point we couldn't give 22 Post Office an expectation of how long it will 23 be before we can go into medium pilot and that 24 was the key. So for me, my take, he needs to 25 know that because if he doesn't, the business 21 1 will start to ask questions, it goes up for 2 their management, it goes to their management, 3 and it creates a lot of distraction. 4 Q. Mr D'Alvarez, did you hear Mr Burley's evidence 5 on this or have you been told about Mr Burley's 6 evidence -- 7 A. I only saw about half hour of his evidence, and 8 it was really around Horizon, which surprised me 9 because I didn't recognise him working on 10 Horizon. 11 Q. You were quite careful today to distinguish the 12 word "desperate" as being your word rather than 13 his. Is that something you've given some 14 thought to? 15 A. It's just me, you know, just saying that he's 16 desperate because you asked me because there 17 pressure. 18 Q. Yes. 19 A. That's my, you know -- 20 Q. The impression that you're giving today is that 21 it was quite a calm atmosphere, and -- 22 A. (The witness laughed) 23 Q. -- there wasn't pressure being brought, 24 despite -- 25 A. No, there was a lot of pressure. A lot of 22 1 pressure. 2 Q. Where was that pressure coming from? 3 A. Well, it comes from within because we have 4 committed to a plan, so -- and then we have to 5 explain when we, we're not making key milestones 6 that impact both organisations, and also, now 7 because we're in pilot, we're not -- you know, 8 it's not just impacting the people working on 9 the programme. Delays and issues impact the 10 business. So that there's the pressure to get 11 it right, there's a pressure to be clear in 12 communications and there's a pressure to, you 13 know, successfully deliver the programme. 14 Q. So although "desperate" itself is your 15 language -- 16 A. Yes -- 17 Q. -- you would accept that there was pressure 18 coming from the Post Office to get on with the 19 pilot? 20 A. And from Fujitsu, yes. 21 Q. In terms of an escalation to Mike Young, what 22 did that involve? 23 A. It's my assessment that I needed to brief my 24 senior executives because it could very well, 25 that if, by telling post offices that the 23 1 schedule times were being delayed, not giving 2 them further information as to alternatives and 3 that, things typically go up the chain, up their 4 management. They would go up, it would go to 5 my -- it would go to someone who is sitting over 6 the programme and then they would call my 7 executive. 8 So it's a briefing to say "Be prepared, this 9 is the position, you may well get a call from 10 Mike Young because he may well get 11 an escalation, either from within his programme 12 or from outside his programme". 13 So for them to be aware of the situation and 14 not be seen not to understand what's happened. 15 And also I needed to brief Fujitsu as well, 16 because it's a delay which causes us pain. 17 Q. Pain in what sense? 18 A. Because we got teams lined up, time is cost. 19 Q. Can we look at FUJ00093056, please. This 20 a "Horizon Next Generation Progress Joint Board 21 Meeting" of 11 February 2010. Again, Mr Burley 22 is the chair and you're listed there as 23 attending. 24 Can we look at page 4. About halfway down 25 the page on page 4, we get to the BTS issues at 24 1 Warwick again. They're mentioned again and it 2 says they're new BTS issues in Warwick: 3 "BE to scan for the latest BTS reports from 4 Warwick", et cetera, et cetera. 5 So that's just an update there in relation 6 to the Warwick issue that we spoke, the branch 7 trading statement issue. 8 Over the page, the final substantive entry 9 in that table, there's a separate issue. It 10 says: 11 "Error message is seen at branches but not 12 flagged up by FS [that's Fujitsu] monitoring 13 systems. The following message [appears]", 14 et cetera. 15 Was it typical, common, or do you recall 16 instances where error messages may be seen at 17 branches, but aren't flagged up to Fujitsu's own 18 monitoring systems? 19 A. So this was a new system. We're going into 20 pilot. The -- not only -- okay, so if we can 21 just step back a bit. So the whole premise of 22 HNG-X had three different elements to it, and 23 I think the whole premise was the business 24 processes didn't change, so it wasn't a business 25 transformation; it was a technical 25 1 transformation. 2 And it was based on -- I think there were 3 two key drivers. But there was a data centre 4 element where we were migrating into Fujitsu 5 secure data centres. There was a technical 6 refresh of the back end and there was a refresh 7 of the application. 8 So with all those we had to -- you know, we 9 didn't -- it's not a case we're putting a new 10 application in and we're tuning the system, or 11 we have a system that's already working; we have 12 a whole new infrastructure and everything else. 13 So we had to put in the monitoring and that and 14 with monitoring, if you put too much -- if you 15 monitor everything, you can't see the wood for 16 the trees because, you know, you don't know 17 what's important. 18 So it's getting the tuning and the 19 monitoring right. So in the early pilots, part 20 of the pilot is if we're -- if there are issues 21 that are being seen in the Post Office that 22 impact the branches and we didn't pick it up in 23 our monitoring, we had to make sure our 24 monitoring was extended for that. But what that 25 meant was we didn't get advanced sight before 26 1 the postmaster got impacted or the clerk got 2 impacted. So that would result in to helpdesk. 3 It may not -- if we monitored that, we may 4 not have been able to intercept it before the 5 postmaster, I don't know that particular issue 6 but that was specific to making sure that our 7 monitoring was properly tuned through the pilot. 8 Q. Irrespective of that particular issue, do you 9 remember instances where error messages or other 10 things might be seen by the subpostmaster that 11 wouldn't be flagged by Fujitsu's own monitoring 12 systems? 13 A. That document reminded me of an instance. 14 I can't recall one -- others, but there may well 15 be. 16 SIR WYN WILLIAMS: Mr Blake, I'm sorry to interrupt 17 you but my screen is just showing "Post Office 18 Horizon IT Inquiry", and I'm not seeing either 19 of you at the moment. 20 Now, I am. That's fine. 21 MR BLAKE: Let's move on to FUJ00094192, we're now 22 at 18 February. It's another meeting of the 23 board. 24 Can we look at page 3. Towards the bottom 25 of page 3, it's again the Warwick issue. We can 27 1 see there it's shaded, and I mentioned earlier 2 the issue of, I think -- arranged for a base 3 unit swapout at Warwick, so it seems as though 4 the Warwick solution wasn't just a software 5 solution, it was a hardware solution; is that 6 a fair interpretation? Node 6? Or is that -- 7 have I misunderstood that? 8 A. I don't know the reasons for that. That could, 9 as I say, Geoff Butts was the lead on the 10 Warwick one. Whether that was felt as an added 11 precaution or whether that was felt as 12 necessary, I couldn't say. 13 Q. Are you able to assist us at all with the words 14 there around not setting a precedent? 15 A. We're not setting a precedent that if you have 16 issues, I'd expect that we don't want to -- 17 every time there's an issue in the Post Office, 18 that we would go and swap out a counter. 19 Because that would be not appropriate. 20 Q. It would be expensive, certainly? 21 A. Not -- and again, that leads me -- those words 22 lead me to conclude, but without any facts 23 behind it, that that was just a secondary 24 precaution, as opposed to a necessary step. 25 Q. Can we go over the page, please, to page 4, 28 1 "Volume Testing", so I think you've said volume 2 testing was occurring during the pilot. 3 A. Mm-hm. 4 Q. "LF reported that problem encountered to date 5 with the first phase of volume testing have put 6 the target end date of 20th February at risk. 7 "GA to reassure that correct resources are 8 in place to resolve any problems as soon as 9 possible." 10 Then you have there LF -- I think that's Lee 11 Farman of the Post Office -- was confident that 12 the correct focus is now in place. Is this 13 something you remember at all? 14 A. It's just that we took the decision to do volume 15 testing and volume testing is quite complex. So 16 it's very much stop/start. 17 Q. It seems as though it was closed because Mr -- 18 do you remember Lee Farman at all? 19 A. Yeah, he was in the joint testing. He was one 20 of their leads. I believe there was two, 21 I can't remember the other person's name. If 22 you told me I would. I believe he was on the 23 non-functional side of testing. 24 Q. Was reliance placed on him and the Post Office 25 to assure you that you could get on with things 29 1 at that stage? So it seems as though it was 2 closed because he was confident that the correct 3 focus was in place. You've described a lot of 4 your working relationship to be a joint one. 5 A. Mm-hm. 6 Q. But certainly it seems to have been closed 7 because he had confidence. 8 A. Yeah. So he would work closely with my test 9 lead, Debbie Richardson. They shared the same 10 environment. They were both based in Bracknell, 11 on the same floor, in the same area. 12 Q. But she's not mentioned there. Is it because 13 ultimately the decision to progress matters lay 14 with the Post Office, or -- 15 A. Correct. 16 Q. Do you recall what the problems were that you 17 encountered at that particular stage? It may be 18 too specific a question because it's 18 February 19 2010. 20 A. No, I don't recall. I don't recall. 21 Q. If we scroll down that page, there is the 22 reference to the "Trial Report/Final Balance 23 Issue". So this is an issue we addressed 24 yesterday where the balance sheet prints the 25 correct report in the trial report but not the 30 1 final report, I think, and it says: 2 "PN to check if the proposed workaround is 3 acceptable to the business. 4 "Permanent fix targeted for R01.08. 5 However; this is dependent on the acceptability 6 of the workaround, it may need to be a Hot Fix." 7 Is this something you recall at all? 8 A. Isn't that the Warwick issue? It's -- it looks 9 similar to the Warwick issue to me. 10 Q. I think it's fair to bring to your attention the 11 entry that is below that, which says: 12 "NOTE: for the avoidance of doubt, any 13 workarounds have to be agreed by POL before 14 implementation." 15 Is that something you recall? 16 A. Yes. 17 Q. Was that always enforced? 18 A. Yes. 19 Q. If we go over the page to page 5, and the bottom 20 three entries in that table, we have: 21 "MB [I think Mr Burley] thanked GA, DR and 22 their teams for all their efforts in ensuring 23 that the manual BTS fixes were successful 24 applied as required." 25 Do you remember at all what the manual BTS 31 1 fixes were? 2 A. No. 3 Q. Then the "Warwick Issue due to BAL node failure" 4 is mentioned below that. Does that assist you 5 at all with the Warwick issue? 6 A. I can't recall it. 7 Q. Can we move on then to FUJ00094268. This was 8 an email chain I took Mr Burley to yesterday and 9 it relates to the balance trading statement 10 issue. It's slightly difficult to read. 11 I think if we start at page 10. Was this 12 something that you saw in yesterday's evidence 13 or -- 14 A. No. 15 Q. -- do we need to spend a bit of time on the 16 actual content? 17 A. No, I didn't. 18 Q. Okay, so let's look at page 10. 19 A. This is in my pack though, I recognise this from 20 my pack. 21 Q. There's an email from Geoff Butts to Will 22 Russell, Mark Burley, Barry Evans, et cetera. 23 If we go over the page, we can see the issue 24 there. You were copied into this email? 25 A. Yes. 32 1 Q. Can I just ask you to read to yourself that 2 paragraph and see if that refreshes your memory 3 about what this particular issue is? 4 A. So: 5 "This is to confirm the position on a fix 6 for the BTS issue whereby printing the Trial 7 Balance Report for BTS results in incorrect data 8 being displayed in the Final Balance Report. 9 A fix can be delivered and tested for inclusion 10 within the 01.08 Maintenance Release as 11 a counter fix. In the meantime, branches need 12 to use the BTS Trial Balance Report, which is 13 correct and discard the BTS Final Balance 14 Report, which is incorrect." 15 Q. Yes. Does that jog your memory about this 16 particular issue? 17 A. It does. This is what -- Geoff was leading on 18 the Warwick issue and I did keep abreast as to 19 the progress on that. 20 Q. So you say keep abreast, some of these are sent 21 to you, you're copied into some. 22 A. Mm-hm. 23 Q. What kind of a role did you play in this 24 particular issue? 25 A. Well, Geoff reported in to me, and so when we 33 1 had -- as I say, earlier in the pilot there are 2 three significant issues that were raised: one 3 was at Coton, one was at Warwick, one was at 4 Derby, ie I said to Geoff I will take ownership 5 of the Derby one that had -- in the initial 6 review of those incidents it had data integrity 7 implications, and he took leadership because 8 that was his role. 9 He was the person that was heading up 10 deployments and heading up what we call 11 Hypercare, ie the additional support given to 12 pilot sites. He took the leadership or the lead 13 role in resolving these issues. 14 Q. Can we look at page 9, please, which is 15 a response to Mr Butts from Mark Burley. If we 16 scroll down to page 9, he says there: 17 "Geoff 18 "Thanks but can I ask who you have agreed 19 this workaround with? This is a legal document 20 and there is a difference between a trial 21 balance and a Final Balance. I would always 22 expect CS to have a KEL for this -- if agreed -- 23 to be able to explain the position to any 24 subpostmaster who calls in." 25 Do you remember this? 34 1 A. I remember Geoff highlighting to me that the 2 issue was wider and I gave Geoff the advice to 3 make sure that our legal counsel was now engaged 4 in making sure. So I believe this was Geoff 5 looking at the incident as a technical incident, 6 proposing a workaround, and then the response is 7 "That workaround, are you sure because it's 8 a legal document?" 9 So I do remember Geoff raising it, I do 10 remember specifically me saying to Geoff saying 11 "Would you make sure that Jean-Pierre is now 12 engaged on this?" 13 Q. Do you remember whether Geoff was concerned by 14 the fact that it was a legal document? 15 A. I wouldn't say "concerned", we knew that we now 16 had to make sure that anything we proposed and 17 go forward with, our legal people are happy with 18 our proposal. But, again, ultimately I'd expect 19 also Post Office to make sure their legal people 20 were happy with any, if we were to go forward 21 without a fix. So I don't know what -- I can't 22 recall how we actually went forward with this 23 one but if we went forward with a workaround and 24 not a fix, I would expect everyone to be 25 consulting their legal counsel as this is 35 1 a legal document. 2 Q. Would it be typical where fixes or workarounds 3 affected what are described as legal documents, 4 or documents used in legal proceedings, to be 5 escalated within the company, as you say, to 6 general counsel and others. 7 A. We had a legal department so if there's anything 8 which we felt provided an issue with regards to 9 either integrity or any other legal aspect to 10 our system, we would engage -- we should engage 11 and my expectation is we did. I always engaged 12 the legal and I had advised Geoff in this 13 instance to engage with our legal team. 14 Q. Was there anybody else within management outside 15 of legal who you would typically consult with in 16 relation to those kinds of issues? 17 A. I would certainly inform -- at this stage, 18 I believe, Gavin Bounds was on the account. He 19 was the account business unit director. So I'd 20 make him aware. 21 Q. Can we look at page 4 and scrolling into page 5, 22 please. This is the -- an email to Phil Norton 23 from -- I believe it may be from Gareth Jenkins, 24 and it says: 25 "Phil, 36 1 "Alan D'Alvarez has asked me to respond to 2 your concerns below. 3 "I'll try and explain the issue and what has 4 caused it. 5 "I assume you've seen the attached write-up 6 of the issue which was sent to Barry Evans (and 7 others) ..." 8 Then there's an explanation of the problem. 9 Do you remember asking anybody to assist 10 with this particular issue? 11 A. So I believe Geoff was on leave. Geoff wasn't 12 around because Geoff was leading on this. So 13 Geoff was on leave. So I asked who was 14 supporting Geoff in the investigations into that 15 and was told that Gareth was. So I spoke with 16 Gareth and asked him, "I got a response from 17 Post Office, Geoff is on leave, could he please 18 look at this and provide a response to Post 19 Office?" 20 Q. What did you understand Gareth Jenkins' role at 21 this stage to be? 22 A. So Gareth Jenkins' role, he was always on the 23 Horizon side and when we was migrating over to 24 HNG-X, because I believe when we went live with 25 Horizon, he -- we'd become part of a customer 37 1 services and, I believe, an architects group. 2 There was an architects group, general outside 3 of the programme. And his role was basically as 4 a senior architect, he was acknowledged on the 5 account as an expert on the counter 6 applications. 7 So I'd expect that Geoff, because this is 8 about balance sheets and things, to go to our 9 expert on the account as to "Could you now look 10 at this as an issue" because, as I said, what -- 11 in Horizon, what we did not do was change the 12 business processes, or it was not a business 13 transformation it was a refresh of the 14 technology and how we actually supported their 15 stuff. 16 So Gareth would be very knowledgeable as to 17 what the consequence of balance trading 18 statements are in respect of the business. 19 Q. He identifies there in paragraph 4 that there's 20 a bug in the way that the report is produced 21 such that some of the in-memory copy of the data 22 is overwritten when the trial BTS is produced. 23 A. Yes. 24 Q. Do you recall discussing with Mr Jenkins the 25 concerns within this email chain about the trial 38 1 report being a legal document? 2 A. I just recall discussing with Geoff that that 3 position, because Geoff discussed it with me. 4 Q. Do you recall Mr Jenkins as being someone who 5 was aware of the significance of a report such 6 as that for legal proceedings? 7 A. I'm not aware, but I -- he's our expert so 8 I would have an expectation that he would be 9 aware. 10 Q. Were you familiar at this time with his 11 involvement in any criminal or civil 12 proceedings? 13 A. I understood from my original tenure on Horizon 14 that he and others had given evidence. 15 Q. If we look at page 1, there is the email from 16 Phil Norton at the Post Office to yourself. He 17 has met with the Finance team to discuss the 18 issue and he sets out deliverables that he would 19 like, including: 20 "A complete and comprehensive list of all 21 products where the volumes on the Final Balance 22 ... differ ... 23 "A definitive statement detailing: 24 "How this defect has arisen", et cetera, 25 et cetera, including: 39 1 "A commitment to support POL in proving the 2 integrity of the system in any subsequent legal 3 action (specifically where the difference in the 4 two reports is used as a means to challenge the 5 integrity of the system)." 6 Is this something you remember being 7 requested by the Post Office? 8 A. I remember -- well, I obviously received that 9 e-mail. I don't remember specifically at the -- 10 you know, from memory, but from my pack 11 I remember I can see I received the email and 12 I forwarded it to Geoff because, as I said, 13 Geoff was leading on this particular one. And 14 then also, in a previous conversation with Geoff 15 I advised him to make sure that our legal team 16 was fully engaged with any responses we gave on 17 this subject. 18 Q. Why would Phil have sent it to you rather than 19 to Geoff? 20 A. Perhaps -- well, I was the programme lead. 21 I was the head person. So I assume Phil felt it 22 significant enough to send it to the person that 23 was leading the -- you'd have to ask Phil. 24 Q. Was it typical to you to be asked by the Post 25 Office to provide a commitment to support POL in 40 1 proving the integrity of the system? 2 A. No. 3 Q. Was this something novel? 4 A. I wouldn't say novel. It wasn't something that 5 was typical. When I saw that -- as soon as 6 I saw that, I recognised that we needed to 7 engage our -- you know, make sure that, as it is 8 a legal document, anything that we propose, we 9 ourselves assure ourselves that we're not 10 compromising Post Office. 11 Q. Did it concern you at all? 12 A. On concern me, it concerned that with all issues 13 that impact the end customer is a concern. This 14 was a serious issue and that's why we had, you 15 know, appointed one of my senior people to take 16 ownership to getting it resolved. There will be 17 a number of bugs that will be raised during 18 a pilot. Many of those just will be managed by 19 the resolver groups, ie it goes through the 20 process. 21 The serious of the Coton, Warwick and Derby 22 was recognised and we put senior people on to 23 make sure that we understood fully what was 24 going on and our -- and we had a proper 25 resolution and that we were satisfied that the 41 1 resolution was that the right resolution. 2 Q. Having seen in the previous months that there 3 were these issues cropping up -- bugs, as you 4 described them -- do you recall your reaction to 5 being asked to support the Post Office in 6 proving the integrity of the system in any 7 subsequent legal action? 8 A. My -- I can't recall my reaction but it would be 9 make sure that our legal counsel understands 10 that we had this request and anything we provide 11 back, they need to assure. 12 Q. Would you have thought that you could prove the 13 integrity of the system in any subsequent legal 14 action? 15 A. The -- that's an interesting question. Well, 16 that's a question which, because of the Derby 17 incident, there was an incident in Derby in 18 which there was a double entry, and that clearly 19 had integrity issues. Because of that incident, 20 I immediately went to my senior in the 21 organisation and said, "We've received it" -- 22 and that was Maz Kostuch, so I worked in the 23 programme project management organisation -- 24 Q. Sorry that was who, sorry? 25 A. Someone called Maz Kostuch. Maz Kostuch was the 42 1 head of programme and project management for the 2 public -- private services division which Post 3 Office was part of. 4 SIR WYN WILLIAMS: (Unclear) -- so that I make 5 sure -- I know there's a transcript but I'd like 6 to get that name accurate, if I could, please. 7 A. Kostuch, K-O-U -- I believe K-O-U-T-U-C-H, (sic) 8 I believe that's the spelling. 9 SIR WYN WILLIAMS: Thank you very much. 10 A. He was the person that was the head of programme 11 project management and I said, we have 12 an incident here, and it's quite significant. 13 He then engaged the lead technical person in 14 the -- who was -- the name will come to me, I'm 15 sure -- and we agreed that we will get 16 an independent review of the decision because 17 there was two aspects which I was concerned 18 about: one, there was that defect which was 19 an integrity defect; and secondly, we didn't 20 pick it up in our testing, in our laboratory 21 testing, we picked it up in the pilot, which is 22 a test phase. And we needed to understand, 23 "Well, hold on, if that happens, we can't just 24 say we fix that bug and just move on. Is there 25 something inherent?" 43 1 So we arranged for some experts, application 2 experts, there was two of them, I forget their 3 names, immediately assigned them to this, 4 dropped all their other work, and they undertook 5 a review of the solution of the testing and gave 6 a report. 7 MR BLAKE: We'll come to that report shortly -- 8 A. So it's that report that led me to have 9 confidence that the integrity of the system was 10 good. 11 Q. We'll come to some documents but do you recall 12 the end result of this request from Phil Norton? 13 A. It's in my pack. I kind of briefly skimmed over 14 it but it was a response that Geoff prepared, 15 went to counsel, and I believe that was sent to 16 Phil. 17 Q. Thank you. We'll go through that now. Can we 18 look at FUJ00094472. This isn't the response 19 itself, but this is -- shows the sequence. 20 These are further board minutes of the 21 11 March 2010. If we could turn to page 3 22 please. Thank you. 23 At the bottom there it refers to the "Trial 24 Report/Final Balance issue": 25 "PN to check if the proposed workaround is 44 1 acceptable to the business", et cetera. 2 But it's the right-hand column that sets out 3 the sequence, so we have 25 February: 4 "POL have requested that this be a Hot Fix 5 as it is required before we migrate any further 6 branches. 7 "Fujitsu to ensure deliverables listed in PN 8 email ... for current live branches are 9 included. 10 "[4 March] GB has received feedback on 11 integrity statement from Fujitsu legal. Info 12 will be forwarded to PN." 13 Then 11 March: 14 "PN has passed statement to P&BA who are 15 reviewing with POL Legal team. PN will feed 16 back to GB." 17 This kind of collaborative approach on 18 an integrity statement, is this something that 19 you recall, is this something that was typical? 20 Am I right or wrong to say that it was 21 a collaborative approach? 22 A. Yes, it was a collaborative approach. We worked 23 as a joint team. 24 Q. Do you recall other collaborative approaches of 25 this nature with regards to the drafting of 45 1 an integrity statement? 2 A. No. 3 Q. In terms of the resolution of this particular 4 issue, irrespective of the ultimate issue that 5 this branch trading statement may not ultimately 6 matter for the sake of this Inquiry, but in 7 cases of a bug of this nature, would you expect 8 the Post Office to have told all branches, 9 cascaded the information down or do you think 10 that, a bug of this nature, it would be 11 sufficient for it to be on a Known Error Log for 12 subpostmasters to call in and, if they have 13 a problem, that would be known on the Known 14 Error Log? 15 A. So, there's two aspects to that. Firstly, we 16 had to fix it. So if a Post Office hadn't 17 received a release, there was no reason to 18 inform them. I don't know how Post Office 19 themselves communicated this. Would it be 20 reasonable to advise those Post Office that were 21 participating in the pilot test phase? That's 22 an advisable approach -- I don't know, we didn't 23 have that discussion -- that they were 24 responsible for communications to their 25 business. 46 1 Q. What would your recommendation be, though, for 2 something that affects the trading statement? 3 How -- looking at a recommendation for the 4 future or something along those lines, if you 5 come across an issue that has been phrased as 6 affecting a legal document, the trading 7 statement itself, do you think it is sufficient 8 for it to be placed on the Known Error Log or do 9 you think more action should be taken to draw 10 that to subpostmasters' attention? 11 A. My understanding was that the workaround was to 12 use the trial balance report. 13 Q. Yes. 14 A. If that's a workaround, that workaround will get 15 communicated to postmasters because it's 16 a workaround, ie a workaround is there's a known 17 issue, to work round this issue, this is what 18 we're asking you to do. We'd also have it on 19 the known error list because a postmaster may 20 have forgotten about it or not read the advisory 21 notice and, therefore, phoned up the service 22 desk and it will be on a known error so they 23 could then give that same advice. 24 But a workaround is very much something 25 which, if it means asking the postmaster or 47 1 their staff to do something different, that must 2 be communicated to them for that workaround to 3 be effective. 4 Q. Is your recollection that workarounds were 5 routinely communicated to subpostmasters or only 6 communicated to those who phoned the helpdesk 7 with a problem? 8 A. It depends on the workaround. So if the 9 workaround is as part of your -- what you do for 10 your daily business, you have to apply this 11 because the solution is not quite performing as 12 we expect to it, that should be proactively 13 communicated. If there's an issue where there 14 is perhaps a failure -- you gave an example 15 before an error message. You know, sometimes 16 error messages could come up. 17 We wouldn't necessarily -- it's not 18 necessarily appropriate just to say to everyone, 19 "Here's 101 workarounds" or there's ten 20 workarounds or whatever the number is but that 21 might be sporadic, comes up once or twice, 22 therefore it's appropriate for the service desk 23 to have that as a known error, that if they do 24 get a call they can say "Ah, it's a known error, 25 this is how you workaround that error. So it 48 1 depends on the workaround. 2 Q. Still on this document, another topic -- I'll 3 return to this topic but just while we're on 4 this document -- can we just look at page 5, 5 please. There's reference there to the 6 excessive number of recoveries and screen 7 freezes. 8 If we go to the third entry, the third row 9 down, it says, "Excessive number of 10 Recoveries/Screen Freezes" and on 11 March 11 that's closed, and it says: 12 "Superseded by introduction of r108 at data 13 centre which has significantly reduced the 14 screen freezes. The data on recoveries is being 15 separately progressed by IT and Gareth Jenkins 16 ..." 17 Do you recall this at all? 18 A. I recall there were screen freezes that impacted 19 the Post Office, and I can see that -- I don't 20 recall specifically this. I was aware of screen 21 freezes as an issue. 22 Q. How about recoveries? The recovery issue, that 23 was a particular issue that was identified. 24 A. I don't recall that. 25 Q. Is it fair to say that it's been closed but it 49 1 hasn't been eliminated because it says that it 2 significantly reduced the screen freezes but it 3 doesn't say there are no longer any screen 4 freezes and, in terms of the recoveries issue, 5 that's being progressed by Gareth Jenkins; it 6 hasn't been closed because that issue has been 7 resolved? 8 A. But it says "as per 146" so is 146 open? In 9 which case, it's been -- that data recovery is 10 if you go up -- 11 Q. That's been closed. That's slightly further up 12 on the same page. 13 A. Now, we've got to go to 148.01. 14 Q. If we go over the page, that's page 6, it's 15 about halfway down, it's a new item there, 16 11 March. 17 A. Okay. I don't recall it specifically but it 18 looks as though they've created an item, so 19 there was a number of issues all in one, part of 20 those issues are being addressed, others still 21 remaining. So it looks to me, I don't recall 22 this specifically, but it looks to me they've 23 opened the new action so not to confuse it 24 with -- the screen freezes may have led to other 25 consequences and they want to track the other 50 1 consequences. 2 Q. The screen freeze issue itself had been 3 significantly reduced but not eliminated? 4 A. I -- again, I don't recall -- 5 Q. Can we go to the bottom of page 5. It says: 6 "As per Action 147.01 it was agreed that 7 rollout would not be [recommended] until 8 a period of stability had been achieved. GB/WR 9 to agree what is meant by 'Stability' and what 10 would be an adequate period." 11 Then if we look in the column next to it on 12 the page above, 11 March: 13 "A set of criteria has been provided by POL 14 and this was reviewed for understanding in the 15 meeting. Fujitsu will provide much supporting 16 data/evidence by ... 12th March to enable 17 a decision to be made if pilot can restart", 18 et cetera. 19 "It was agreed that the data would not be 20 perfect or complete, but needs to provide 21 sufficient relevant information if a restart 22 decision can be made." 23 Again, in terms of the working relationship 24 between POL and Fujitsu, it seems there that the 25 criteria provided by the Post Office, in terms 51 1 of resuming the rollout -- were Post Office, at 2 this stage -- so we are 11 March 2010 now -- the 3 ones who were driving forward the rollout? 4 A. Sorry, so when I read that, it's -- I think as 5 I've explained before, when we take a decision 6 that the solution is not -- or the risks of 7 moving forward further with our pilots and/or 8 deployments is such that we're going to freeze 9 or have a halt or pause, the -- you know, we 10 would work jointly but the ultimate decision is 11 with Post Office. 12 So here, what I read from that is that we've 13 received a set of restart criteria. So with 14 this issue, we needed make sure we understood 15 what would be the basis of us understanding that 16 it's resolved and to give ourselves confidence 17 because at the date -- and we're talking about 18 stability, we're talking about screen freezes, 19 I can't be sure, but this may be related to 20 a significant issue we had with Oracle, 21 an extremely significant issue we had with 22 Oracle that did impact the offices 23 intermittently that had migrated to HNG-X in as 24 much as it loss -- it didn't lose connectivity, 25 the connectivity hung and they could no 52 1 longer -- so the screen freezes -- no longer 2 trade until the branch database come up again. 3 And that was a very significant issue. So 4 what we had to assure ourselves before we 5 deployed further, once we've deployed fixes and 6 things, that we understood what is the criteria 7 for restarting, and make sure -- and Post Office 8 had the final say. So is it a period of "N" 9 number of days, "N" weeks, whatever, is it 10 acceptable to have? Because if you have 11 a screen freeze, we had -- in the Oracle issue 12 that we had, there were regular, you know, 13 throughout the week, four or five times, where 14 all the post offices operating in the pilot 15 would be impacted for a period of 15 minutes to 16 30 minutes, and that's significant when they're 17 serving customers. 18 But, again, there could be a screen freeze 19 for reasons not related to that. So you're not 20 going to say you had no screen freezes 21 whatsoever, it could have been an intermittent 22 one-off screen freeze. So the criteria was very 23 important, that once you stop a deployment, to 24 actually understand -- do you understand what 25 the issue is and, for us to make sure we make 53 1 the right decision, do we have a clear set of 2 criteria that we're going to apply to the 3 decision? Have we met that criteria to go 4 forward? 5 Again, that criteria was also, dare I say 6 it, to protect the joint programme from people 7 wanting us to move forward. Come on, you've got 8 to keep going. So we haven't met this criteria, 9 or we have met the criteria so we can 10 demonstrate to ourselves and to our respective 11 organisations the appropriate time when -- why 12 is it appropriate to move forward or why is it 13 not appropriate to move forward? 14 Q. You mentioned earlier in your evidence about the 15 risk versus progress balance. 16 A. Yes. 17 Q. Were Fujitsu and the Post Office both very much 18 aware that that is the balance that was in play 19 at this time when there were those evident 20 issues still cropping up? 21 A. We made them aware. Because that's our job. 22 I mean our job as programme management is -- 23 that's what we do. So it's my job to make my 24 organisation aware and Mark's job to make his 25 organisation aware of that. 54 1 Q. I'm going to return to the balance trading 2 statement issue. Can we look at -- do you need 3 a break at all? 4 A. I'm fine, thank you. 5 Q. Thank you. 6 Sir, just for your information -- I'll deal 7 with the balance trading statement issue and 8 then, in about -- well, around 11.30 we may be 9 able to take a break. I'm hoping that we will 10 be able to take one longer break this morning, 11 and not need to take a lunch break. That's my 12 intention. But my intention yesterday didn't 13 prove correct, so -- 14 SIR WYN WILLIAMS: I have every faith in you 15 bringing home your intentions, Mr Blake. 16 MR BLAKE: Thank you, sir. 17 Can we look at FUJ00094265, please. This is 18 an email, you spoke earlier about raising the 19 issue with Fujitsu's general counsel and that's 20 Jean-Pierre Prevost; is that right? 21 A. He is the legal representative assigned to Post 22 Office. 23 Q. We have there an email from Geoff Butts saying: 24 "J-P 25 "Can you review this draft response [in] an 55 1 email from Post Office ... about Fujitsu's 2 approach to resolution and interim management of 3 a software defect relating to the Final Balance 4 Report for Branch Trading Statements, and let me 5 know if any changes are required before it is 6 sent. I've copied the HNG-X Leadership Team for 7 information. This issue has been flagged as 8 critical to fix before the start of rollout." 9 In terms of those names at the top, are they 10 all Fujitsu names? 11 A. Yes. 12 Q. Then if we scroll down, these are the answers to 13 the questions that have been requested by the 14 Post Office, or proposed answers that are being 15 run by general counsel. Were you involved in 16 drafting this in any way? 17 A. I would likely have reviewed it. But not 18 specifically to -- 19 Q. Do you remember reviewing it? 20 A. I don't recall reviewing it but it would be my 21 normal operation to review it. Although, having 22 said that, I may have reviewed it as part of 23 being a CC list. 24 Q. If we scroll through it, it explains the cause, 25 addresses the key questions, and it's the final 56 1 entry, really, that I want to read out now. 2 We've seen this document before. It says: 3 "Can Fujitsu provide a commitment to support 4 POL in proving the integrity of the system in 5 any subsequent legal action (specifically where 6 the difference in the two reports is used as 7 a means to challenge the integrity of the 8 system)?" 9 The proposed answer is: 10 "Yes, Fujitsu is willing to positive 11 commitment to prove the integrity of the system 12 in any subsequent legal action." 13 We do, then, have the final version that was 14 sent. Can we look at FUJ00142190, please. It 15 seems to have been sent to Mark Burley by 16 yourself on 8 April 2010. The email chain 17 I just took you to was February, we're now 18 moving, so some time has passed. We're in 19 April. This says: 20 "Dear Mark, 21 "Please find enclosed a statement detailing 22 Fujitsu's position with respect to the Final 23 Balance Report issue from the HNG-X pilot. This 24 statement has now been confirmed by our 25 Commercial, Legal and Programme Leads." 57 1 It's sent by you. Do you remember sending 2 this letter? 3 A. I don't recall specifically sending it. 4 Q. Yesterday, Mr Burley couldn't remember what 5 happened to this issue. He couldn't remember 6 whether a letter was sent or not. We have here 7 the letter. Do you remember confirming it with 8 Commercial, Legal and Programme Leads at all? 9 A. So it is likely that the advice from our counsel 10 is it should come from me because I was the head 11 of the programme or head of the 12 transformation -- lead of transformation. As 13 I said, we would have -- I would have reviewed 14 that but we would be taking legal counsel 15 specific to this statement. It's out of my 16 sphere of expertise. So I'm -- I will be solely 17 reliant on legal counsel's view on that. 18 I haven't seen -- you can -- I don't believe 19 this was in my pack, this letter. 20 Q. It was in your pack. 21 A. It was? Okay. Sorry. 22 Q. It certainly has been provided to you or it may 23 have -- forgive me, if I'm wrong on that, but in 24 any event, it's got your name on it. 25 A. These kind of -- I'm stretching my memory. I do 58 1 remember the -- having reviewed the content 2 because it wasn't just the legal -- I don't know 3 whether this letter contains just the legal 4 statement or whether it also has an explanation 5 of all the questions or response to all the 6 questions that was posed by Phil. 7 Q. Let's turn over the page, then. 8 A. It does. So it's, basically -- yes. 9 Q. What's interesting in this letter is if we turn 10 over the page, the final entry here is 11 different: only slightly different, but 12 significantly different, in its content. Can we 13 just blow up that final section and slightly 14 above as well, from "Can Fujitsu". 15 It says there: 16 "Can Fujitsu provide a commitment to support 17 POL in proving the integrity of the system in 18 any subsequent legal action (specifically where 19 the two reports is used as a means to challenge 20 the integrity of the system)?" 21 That's the question posed and the answer is 22 now: 23 "Fujitsu is willing to provide a commitment 24 to assist in trying to prove the integrity of 25 the system in any subsequent legal action in 59 1 accordance with the existing contractual 2 arrangements." 3 Do you remember that form of words being 4 inserted? 5 A. I don't recall it, you know, but I can read that 6 and that would have been on the recommendation 7 of legal counsel. 8 Q. The words, before they were changed here, are 9 both interesting because they say "Fujitsu is 10 willing to provide a commitment to assist in 11 trying to prove the integrity of the system". 12 So rather than supporting POL improving the 13 integrity of the system, they are providing 14 a commitment to assist in trying to prove the 15 integrity of the system in any subsequent legal 16 action. Was Fujitsu at this stage concerned 17 that they couldn't actually prove the integrity 18 of the system in any legal action or they might 19 not be able to prove the integrity of the system 20 in any subsequent legal action? 21 A. Not to my knowledge. 22 Q. Would you have been happy at this stage to have 23 put your name to a document that said that you 24 are willing to prove the integrity of the system 25 in any subsequent legal action? 60 1 A. So, firstly, if you go back to the original 2 draft -- 3 Q. That's FUJ00094265. It's the same wording as 4 the beginning of that sentence in that original 5 draft. It's page 2. 6 A. Okay, and it's specific -- 7 Q. You can -- perhaps we can bring them -- 8 A. No, that's fine. 9 Q. Can we bring those side by side? So it's page 2 10 of this document and page 2 of the document that 11 ends 190, and perhaps we can highlight the "Can 12 Fujitsu" section the final section on that page, 13 and the final section on the other page. 14 A. It's actually the question, so if the question 15 is the same I should have read that. So the 16 question is specific to the two reports. When 17 I was reviewing and the briefings that I got 18 with regards to this specific issue, it was very 19 clear that the underlying data that was held on 20 the system was correct and it was how we -- how 21 we presented the reports or how the reports 22 were -- got their data was where the issue was. 23 So it wasn't the data on the system, that had 24 integrity issues, it was how the report was 25 created, it was an issue where it was going from 61 1 a step in the process where it could have had 2 data overwritten and therefore, you know, not 3 had the correct data from the system. 4 So the integrity of the solution on this 5 issue was not in question. It was -- the report 6 was taking its information from an incorrect 7 area of the system, which has volatile memory 8 and therefore open to change, and the fix was to 9 make sure it went straight back to the source 10 information on the branch database. 11 So the integrity of the system specific to 12 this, there was no -- from my perspective, there 13 was no issue with the integrity of the system. 14 I was satisfied when I -- David John's was my 15 technical lead and I had extreme confidence and 16 he was satisfied and he explained it to me and 17 I was satisfied. 18 The question below seems to be a wider -- it 19 appears to me just a wider statement of any 20 evidence we might give at any time for any 21 reason. 22 Q. Absolutely. Actually, I mean, forgive me, 23 I made a mistake. The wording is actually 24 different between those two versions if we look 25 at them now side by side. It's even more 62 1 caveatted in that one on the right-hand side 2 and, as you say, it's a question about can you 3 prove the integrity of the system. So we can 4 put aside the branch trading statement issue, 5 and this addresses really the system as a whole. 6 The final wording -- the initial wording is 7 we're willing to provide commitment to prove the 8 integrity of the system, "a commitment to 9 prove", and now it's not only changed at the 10 beginning -- so it's "a commitment to assist in 11 trying to prove the integrity of the system" -- 12 but then also there's that bit about existing 13 contractual arrangements. 14 Now, forgive me if this wasn't a document 15 that was provided to you in your pack. That was 16 would be my fault but looking at it now and 17 considering it, does it bring back any memories 18 of this particular issue? 19 A. Not particularly but that first one would be 20 from our investigations, from our understanding, 21 our proposal. That would be the programme's 22 proposal of how confident we were and 23 comfortable with regard to this. But we take 24 legal counsel advice. So what comes back I'm 25 not going to dispute because they are the 63 1 experts in this matter. 2 Q. Does it suggest that, at this point in time, 3 Fujitsu didn't have sufficient confidence itself 4 that it could prove the text of the system? 5 A. I don't think it suggests -- the way I read that 6 is any computer system may develop a fault which 7 may, at any time, have some impact on integrity, 8 as we had the one at Derby. And when we had 9 that issue at Derby, I -- under -- you know, 10 I initiated through my line management a review 11 of the solution. 12 So -- because we were concerned that there 13 were potential -- and they gave us confidence 14 what happened at Derby and why it happened we 15 didn't pick up in testing, we understood that, 16 and they also looked at the whole system and the 17 protections we put in. 18 Now, there's -- there may always be a change 19 that's made on the system or peculiar 20 circumstances of, you know, events that happen 21 on a particular machine, which may create 22 a condition. So you can never absolutely say 23 all the time but, again, what we should be doing 24 is supporting by saying this is what we've done 25 to demonstrate the integrity of the system and 64 1 at the point of whatever investigation you are 2 on, were there any known incidents raised that 3 may or may not have an impact on that. 4 But that's, you know, what you'd expect to 5 happen in any investigation. They wouldn't just 6 say at a point in time, back in when -- I think 7 it was February we done that report, because 8 Derby was at the end of January, wasn't it, and 9 it was very quickly turned around because it was 10 so urgent. 11 So, at that point in time, but it doesn't 12 mean it wouldn't change in the future if 13 a release goes out potentially as an intended 14 consequence. 15 Q. I'm going to take you to two documents before we 16 take that break. The first is FUJ00142193. 17 Forgive me, this may again be one of the 18 documents that wasn't in your pack. I think 19 that's the last of the documents that wasn't in 20 your pack, and these are ones I think have only 21 gone through our system relatively recently. 22 Please do say if you need more time and 23 I can provide this to you over the break if you 24 want to re-read it as well. 25 If we could look at page 3. We'll start by 65 1 looking at page 3. We're now at 29 March, and 2 your letter was 8 April, so just before -- so 3 this is before your letter was sent. 4 One of the requests from Phil Norton, 5 looking down that page, the final paragraph 6 there is they are proposing a form of words: 7 "Can Fujitsu provide a commitment to support 8 POL in proving the integrity of the system in 9 any subsequent legal action (specifically where 10 the difference in the two reports is used as 11 a means to challenge the integrity of the 12 system)? 13 "Yes, Fujitsu is willing to provide 14 commitment to assist in trying to prove the 15 integrity of the system in any subsequent legal 16 action." 17 POL Legal are proposing there a different 18 form of words: 19 "Fujitsu is convinced of the integrity of 20 the HNG-X system and as such will, at its own 21 expense, provide a commitment to POL to assist 22 in trying to prove the integrity of the system 23 in any subsequent legal action." 24 So it seems there POL Legal is proposing 25 a statement that says that Fujitsu is convinced 66 1 of the integrity of the system. Is that 2 something that you remember? 3 A. Jogged my memory in getting a response. I do 4 remember immediately -- now I've seen this, I do 5 recall receiving this and discussing this with 6 Gavin Bounds, who is the Business Unit Director, 7 and agreeing that this is now both a legal and 8 a commercial question being given to Fujitsu. 9 So again, I sent it to legal counsel, but we 10 also looked at commercially what was our 11 commercial position. But again, I would take 12 their advice from this point onwards. 13 Q. Would it be typical for the Post Office to 14 provide, and Post Office Legal to propose, forms 15 of words to be adopted by Fujitsu? 16 A. This will be the first time I've seen it. 17 Q. If we go to the first page of this email. From 18 Phil Norton, 8 April. So this is a day -- the 19 same day as your letter is dated, and it's sent 20 to you. I'll just read that out. It says: 21 "Alan 22 "As discussed at the interim JSB, please be 23 advised that POL do not accept the latest 24 Fujitsu response. The area that Fujitsu need to 25 reconsider concerns the proposal from POL Legal 67 1 that Fujitsu will, at its own expense, provide 2 a commitment to POL to assist in trying to prove 3 the integrity of the system in any subsequent 4 legal action. This was not evident in the 5 proposed final version of the statement." 6 So there was a version that was proposed 7 where that form of words was not included. 8 "In the statement from Fujitsu they seek to 9 limit the cost of their involvement in any 10 subsequent legal proceedings to be in accordance 11 with the existing contractual arrangements. 12 "As the error is entirely of Fujitsu's 13 making they should not seek to limit their 14 assistance as per the contract. Would you 15 please arrange for the statement to be revisited 16 and a more agreeable form of words to be 17 proposed." 18 Do you remember this? 19 A. Not directly, but I do remember that, you know, 20 there was further conversations with regard to 21 a proposed legal statement and, as I said, that 22 then passed it to my direct report on the 23 account, and we agreed that it needs to go to 24 Legal and Commercial for a response. But 25 I cannot recall what that response was. 68 1 Q. Do you recall if the letter was changed in any 2 way, or the statement? 3 A. I cannot recall. 4 Q. They're quite strong words from Phil Norton 5 there. 6 A. Yes. 7 Q. Did you get the feeling that you were being 8 dictated to a bit by POL in relation to this 9 issue? 10 A. I -- I mean, looking at this, it's a thing that 11 we often have with customers, you know, who is 12 going to bear the liability of any costs going 13 forward and, again, that becomes a contractual 14 position. Again, you know, it's -- when I saw 15 the previous response, it occurs to me that -- 16 you know, it comes to cross to me that there was 17 a specific question asked and the response was 18 for a more general thing. 19 So whether the discussion -- I do remember 20 briefing our legal department as to we had 21 a review of the integrity of the solution, we 22 got some additional reactions just to kind of 23 assure ourselves with regard to strengthening 24 our testing and things going forward. 25 But in any discussion I've have on any 69 1 system, I say that at a point in time, you can 2 review and it's fine but it could always be 3 a work where something, you know, causes 4 a change which could compromise that and we've 5 always got to be alert to that and that's why we 6 have support. 7 Q. Thank you, if we just scroll up on this email 8 and this is my last question. Are you able to 9 assist us with the recipients of this email? 10 I see Gareth Jenkins is a recipient. Are you 11 aware of him being aware of this particular 12 issue? 13 A. Well, clearly because he's -- I took a -- when 14 Geoff was off, I was pointed in his direction 15 that Geoff was working with him and others with 16 regard to the issues around the branch trading 17 statement. So, clearly, I know that Gareth was 18 aware and Gareth did respond to Phil Norton. So 19 Phil would know that he was engaged so it was 20 right for him to be on this. 21 Q. Would Gareth have been aware that Fujitsu didn't 22 want to provide that blanket assurance? 23 A. He's on this email. So he has got the 24 opportunity to be. I didn't discuss it with him 25 directly but he's on this email. 70 1 Q. Can you give us an indication of who these other 2 recipients are? 3 A. Okay, so Debbie Richardson is the head of 4 testing, Geoff Butts was the deployment lead, 5 Graham Allen was my head of development, Graham 6 Welsh was the lead customer service 7 representative, which we had engaged in our 8 Hypercare. Jean-Philippe Prenovost was our 9 legal counsel. John Wheeler, Mark Andrews and 10 Kevin Talbot I believe -- I can't say -- 11 I believe they're Post Office. I don't believe 12 they're Fujitsu. Mark Burley is Post Office, 13 Will Russell was Geoff Butts' opposite number in 14 deployment, and Mark Burley was my opposite 15 number. 16 MR BLAKE: Thank you very much I think that's 17 an appropriate time to take our break. 18 Sir, I will conclude -- if we take 19 a 20-minute break now, I imagine I will be done 20 within an hour or so. 21 SIR WYN WILLIAMS: All right. Just to let you know 22 that I may literally have to take a three or 23 four-minute break at around 1.00, as it happens, 24 Mr Blake, just to take a phone call. But I'm 25 sure we will manage it some way or another, 71 1 which is reasonable in all the circumstances, 2 all right? 3 MR BLAKE: Absolutely. Unless people get 4 particularly hungry we could take two breaks 5 this morning and achieve it that way. 6 SIR WYN WILLIAMS: Well, whatever. As we get closer 7 to 1.00, we will review precisely what we are 8 going to do. All right. 9 MR BLAKE: Thank you very much. 10 SIR WYN WILLIAMS: Thank you, Mr Blake. So 11 20 minutes from now. 12 MR BLAKE: Thank you. 13 Mr D'Alvarez remains obviously on oath and 14 won't be speaking to anybody about his evidence. 15 SIR WYN WILLIAMS: Fine, thank you. 16 (11.35 am) 17 (A short break) 18 (11.57 am) 19 MR BLAKE: Thank you, sir. 20 Mr D'Alvarez, can I ask you to look at 21 FUJ00094296, please. This is a document from 22 around the same time, 3 March 2010, and you'll 23 see in the top right-hand corner your name as 24 a recipient. There is reference in this email 25 chain to a fix to get data for a court case, and 72 1 I just wanted to know if this was a similar 2 issue, the same issue or something totally 3 different. 4 Let's look at page 3, the bottom of page 2 5 and top of page 3. We that have an email there 6 from Pat Lywood, who is the service 7 implementation manager, and she says: 8 "I will cross-check these with prayers 9 [that's something relating to the below]. 10 However I suspect, for today at least, that the 11 fix to get data for the court case may be more 12 important than these fixes." 13 If we go to page 2, so above that, you say: 14 "Pat, 15 "What is the fix for the court case?" 16 Then if we go above that one and Graham 17 Allen says: 18 "A fix to Audit -- was developed yesterday 19 and passed through Integration last night." 20 Do you recall what this exchange is about at 21 all? 22 A. I recall there was an incident raised where 23 they -- with regard to the audit server having 24 duplicate -- so, basically, having duplicate 25 records stored to it and when reports were -- 73 1 the previous system would extract any duplicates 2 and just have the single version of what 3 happened, and the new version of what we 4 implemented did not have that duplicate spotting 5 extraction purpose -- functionality. 6 So I assume that that is with regard to -- 7 something with regard to the information stored 8 on the audit server. 9 Q. So this is a totally separate issue to the 10 branch trading statement? 11 A. I believe so, yes. 12 Q. If we look at page 3, that email, the first 13 email that I showed you, it refers to a fix to 14 get data for the court case. Can you assist us 15 with what that means? It's the top of page 3, 16 thank you. 17 A. No, I can't. That's why I asked Pat, question 18 mark, "What do you mean, fix for a court case?" 19 Q. Did you ultimately understand, having asked that 20 question, what that meant? 21 A. She replied saying it's the audit fix, so 22 I probably didn't pursue it more other than made 23 an assumption that they were looking to retrieve 24 audit records and they wanted that fix in there, 25 so that the audit records that they retrieved 74 1 was accurate. 2 Q. So does it suggest there that there was an issue 3 with the accuracy of audit data at that time? 4 A. Not accuracy. It's just that it would -- there 5 would be potentially duplicate records which you 6 would have to filter out. 7 Q. How do you know that it's a duplicate issue 8 rather than a reliability issue? 9 A. I'm only -- I'm presupposing that we had 10 an issue with audit records and duplicate audit 11 records, and we had to provide the fix for it. 12 Q. You remember there was a specific issue with 13 duplication? 14 A. I remember there was a specific issue with the 15 audit and I'm just making an assumption that 16 that is to do with that. 17 Q. Similar period but a different issue and I think 18 it was one you were talking about earlier, can 19 we look at FUJ00093030 and this the "HNG-X 20 Counter Review" issue. If we look at that 21 bottom email, it's an email from Stuart Rye, who 22 was Stuart Rye? 23 A. He's, I believe he's a business consultant, and 24 he was brought in as part of, I think he was 25 brought in by -- let's have look at the names, 75 1 David Leask. David Leask, he's the chief 2 technical person alongside Maz who is the chief 3 programme person for private sector and they 4 bought in two people to undertake an independent 5 review. 6 Q. This is exactly that issue you were talking 7 about earlier in your evidence? 8 A. Yes. 9 Q. Is that other person Paul Roberts, who was 10 copied in there? 11 A. Yes. 12 Q. So they were two interpreter individuals who 13 were asked to conduct a review -- 14 A. Correct. 15 Q. -- by Fujitsu -- 16 A. Correct. 17 Q. -- because of a concern that Fujitsu had about, 18 it says there, "the incident of a duplicated 19 basket at the end of January"; do you remember 20 that issue? 21 A. That is the Derby -- that is the Derby incident 22 which I immediately engaged with Maz that 23 I wanted to assure, because it is a serious 24 incident and we need to understand (i) the 25 integrity of the solution and (ii) how we missed 76 1 it in testing. 2 Q. Thank you. If we scroll up, that email and that 3 report, the review -- it's volume 1 there, 4 9 February 2010 -- is sent to Gareth Jenkins by 5 Graham Allen. Do you know by Gareth Jenkins 6 would have been asked to comment on that report? 7 A. I believe Gareth was the one that was assisting 8 Geoff in the analysis of -- oh no, sorry this 9 one? I assume because he is the recognised 10 counter SME, the subject matter expert, for 11 Horizon? 12 Q. So issues -- a concern that you had internally 13 in Fujitsu with regards to the integrity of data 14 because of a duplicated basket, that report was 15 being sent to Gareth Jenkins because he was the 16 lead on what issue? 17 A. No, he's the SME. He's the subject matter 18 expert that -- whether he was still working in 19 the customer services area or whether he was 20 part of the architecture group, we have subject 21 matter experts, so he's the recognised a counter 22 application subject matter expert. 23 Q. I'd like to look at that attachment and that can 24 be found at FUJ00093031, please. This is 25 version 1 of the report. I know you've been 77 1 handed a version 2 and we'll get to that and 2 I'll give you time to look at that. But this is 3 version 1., and that's, it seems, the version 4 that was attached to that email. 5 Now, the circulation, is that an internal 6 circulation list; is that right? 7 A. At this stage, yes. 8 Q. Then you have there the two authors, the 9 independent authors. I'm going to read to you. 10 If we scroll down, it explains what the problem 11 is there. It says: 12 "On 28th January 2010, the Data 13 Reconciliation Service ... process detected 14 an error in a banking transaction. Subsequent 15 investigations revealed that the Branch database 16 had two transactions with different JSNs but the 17 same SSN for a specific Counter on that day but 18 the 3rd Party banking system only had one 19 transaction. The clerk did not know that 20 a duplicate transaction had been created." 21 So as you've said, that is self-evidently 22 a serious issue? 23 A. Yes. 24 Q. It then goes on to say: 25 "An analysis of the database has revealed 78 1 one other occurrence, again at Derby but on 2 a different day and involving a different 3 clerk." 4 So is that the same Post Office but it's 5 a different individual? 6 A. That's what it says, yes. 7 Q. "The net effect would be that the Post Office 8 and Branch records would not match. Where this 9 happens, the Post Office investigates the branch 10 and postmaster, with a view to retraining or 11 even uncovering fraud. It would seriously 12 undermine Post Office credibility and possibly 13 historic cases if it could be shown that 14 a discrepancy could be caused by a system error 15 rather than a postmaster/clerk action. Most 16 importantly, the central database as the system 17 of recorded would be called into question." 18 Now, at this stage, whose words was that? 19 Was that the two independent authors who had 20 written that? 21 A. Yes. 22 Q. Can we go over the page, please. The second 23 paragraph there. It says: 24 "The development team concluded the failure 25 was caused by a bug and a resolution has been 79 1 identified which includes further measures to 2 remove the possibility of this occurring in 3 future." 4 So there's no dispute here, this was very 5 much caused by a bug in Horizon Online. 6 A. Yes. 7 Q. If we go over to page 5, there are the 8 conclusions. Thank you. It says there in bold: 9 "Overall, the actions taken to redress the 10 Derby issue are appropriate. We believe the 11 Counter Application fully supports the need to 12 protect the integrity of financial 13 transactions." 14 What do you understand that statement to 15 mean? 16 A. It means that on the request I didn't want them 17 to just look at this specific incident because 18 we've already diagnosed the specific incident 19 and got a conclusion. I wanted them to look at 20 how we've implemented the counter application to 21 see whether there's any flaws in its design or 22 its implementation. 23 Q. My reading of that is that it isn't saying that 24 it guarantees the integrity of all financial 25 transactions or anything like that, it's using 80 1 quite careful language that the counter 2 application fully supports the need to protect 3 the integrity. So it's identifying the need to 4 protect the integrity of financial transactions 5 rather than signing off the integrity? 6 A. I took that as, from what we've designed and 7 implemented, it fulfils a brief that we have -- 8 should have high levels of confidence in the 9 integrity of the solution. I think it gives 10 examples as well as to how we put protections 11 in. 12 Q. Sorry, you've used the word "solution". Did you 13 intentionally mean "solution" rather than 14 "system"? 15 A. The -- okay, the system. But the counter 16 application solution is what I was looking at. 17 So the solution -- okay. I would, in this 18 context, one and the same system/solution, the 19 solution being we have a counter application. 20 Looking at the counter application itself, is it 21 defensive against system failures? So the 22 system can fail at many different ways, power 23 cuts, cables being, you know, intermittent, kind 24 of -- all sorts of things could cause a system 25 failure. Is the solution robust enough to 81 1 manage those error conditions in a way which 2 doesn't impact integrity? So I do mean the 3 solution, as in the application. Is it 4 defensive? 5 Q. I'm happy for you to take a little bit of time 6 just reading that to yourself, that passage in 7 bold, just to be absolutely sure that that is 8 a fair reading of what is said there. 9 A. Yes. 10 Q. So it's your reading of that that actually that 11 is saying that the counter application has 12 integrity -- 13 A. Yes. 14 Q. -- not that it supports the need to protect the 15 integrity or -- did you not see a distinction 16 between those two? 17 A. I haven't read it that way, personally. 18 Q. We saw earlier that there was a request from the 19 Post Office to provide a commitment to support 20 POL in proving the integrity of the system, 21 of -- yes, of the system. Was this, do you 22 think, meeting that earlier request in any way? 23 A. I think it was a later request. 24 Q. Sorry, the later request. Do you think -- you 25 were asked for a statement to sign off the 82 1 integrity of the system. Looking at this, does 2 this look like a statement to the effect that it 3 is signing off the integrity of the system? 4 A. In the previous example, which I believe is the 5 Warwick example, the question posed was, could 6 we sign off the integrity of our solution with 7 regard to the branch trading statement? That 8 specific question. And, absolutely, this is 9 looking at the application and is it defensive? 10 I would like to refer to -- I've said a number 11 of times now that, from our position, we've 12 satisfied ourselves that what we've implemented 13 will demonstrate and actually fulfils 14 an integrity brief but things can happen on any 15 system which could change that, so that's 16 a static point in time. 17 Q. So it's a point in time that addresses 18 a specific problem that this report is 19 addressing? 20 A. And also a request for me to have a design and 21 code review of what was implemented to say 22 that -- is that appropriate and robust with 23 regards to normal system standards for integrity 24 of financial data. 25 Q. Because when we looked at the earlier example, 83 1 you drew a very clear distinction between 2 signing off the integrity vis à vis the specific 3 issue and signing off the integrity of the 4 system. Here, you're saying that the -- your 5 reading of this report is that, at this point in 6 time, it was signing off the integrity of the 7 system? 8 A. Of the application. 9 Q. The application being the counter application? 10 A. The counter application, the solution for the -- 11 what we provided to Post Office for the 12 postmasters and their staff to actually 13 interoperate with Horizon. So this is the 14 integrity of the counter application. 15 Q. But the counter application must include the 16 cash account, for example? 17 A. Yes, it does. 18 Q. So it would be signing off the integrity of the 19 cash account? 20 A. Yes. 21 Q. Do you think that was the author's intention in 22 this passage? 23 A. I've concluded -- my -- I believe so, yes. 24 Q. Can we turn over the page, please. We're still 25 on conclusions. 3.20, there is a conclusion 84 1 that reads as follows: 2 "Stock check is not a reliable method for 3 catching issues due to the erratic nature of the 4 stock check occurring." 5 Is that something that you recall or 6 understand? 7 A. Not specifically but stock check, from my 8 understanding, is just a check of what the 9 system says it's -- of stock. 10 Q. Why might it not be liable? 11 A. I can't recall. 12 Q. Is that something that occurs on the Horizon 13 System, the stock check? 14 A. I believe a postmaster can do a stock check at 15 any time. 16 Q. It's not a physical stock check of how many 17 stamps are held in branch; it's something that 18 is using the Horizon system to compute? 19 A. Yes. 20 Q. If we turn over the page to "Recommendations", 21 we have recommendation -- I'm going to take you 22 to 4.5 and 4.6. 4.5 is: 23 "Consider advising the Post Office of the 24 benefit of more effective stock control as 25 an indicator of clerk errors or Fraud." 85 1 What do you understand that to mean? 2 A. I don't at this stage recall what that refers 3 to. 4 Q. I mean, reading it now, what does it seem to you 5 to mean? 6 A. It's given -- it's just highlighting that the 7 system may allow variances to be input into the 8 system. 9 Q. Does it suggest that there should be another 10 method in order to assure the Post Office or 11 assure yourselves that there is, in fact, either 12 a clerk error or fraud? 13 A. That's what it says but I don't recall 14 specifically back to the intent of that, but 15 that's what it suggests. 16 Q. Would you advise the Post Office of ways in 17 which to improve the reliability of their 18 prosecutions for fraud, for example? 19 A. I would not but if it felt that Fujitsu felt it 20 was appropriate to at least have that 21 conversation then I would pass that advice on. 22 Q. So it's not that that recommendation itself is 23 problematic because it's suggesting that you 24 advise the Post Office of something? 25 A. I don't believe so, no. 86 1 Q. 4.6: 2 "Review and strengthen negative testing, if 3 appropriate. The recent problems reflect the 4 asynchronous nature of the new application and 5 traditional or historic test cases may not 6 reflect this." 7 Can you tell us, what is negative testing? 8 A. So negative testing, so typically when you test 9 a system you run a set of tests which 10 demonstrate that if you go through the right 11 processes and that and everything is set up 12 properly the system performs as it should. 13 Negative testing will include things like 14 what happens if you put in the wrong inputs? 15 What happens if a cable is disconnected? What 16 happens if it loses connection with the data 17 centre? So we would deliberately create 18 scenarios which is not normal operating 19 scenarios, to see whether the system recovers 20 gracefully from those situations. 21 Q. So you might have some testing, is it positive 22 testing; is that the opposite of negative 23 testing? 24 A. Yes. 25 Q. Yes, positive testing, where you're testing 87 1 a case where a subpostmaster puts in exactly the 2 right information, presses exactly the right 3 keys that you're meant to do? 4 A. Following the processes, yes. 5 Q. That's positive testing. Negative testing is 6 what if the subpostmaster presses the wrong key 7 at a particular time? 8 A. Correct -- or what if it goes offline? What if 9 a cable gets pulled out? 10 Q. Absolutely. One of the recommendations here is 11 there should be more negative testing? 12 A. It's to review. Have we created all the 13 scenarios? Because I believe one of the trigger 14 points for this was a capacity constraint that 15 was caused by, I believe -- because we were 16 monitoring the system all the time during pilot, 17 we were picture putting diagnostics and 18 performance monitoring things to make sure 19 everything was going right and it was believed 20 that that created a capacity issue, which then 21 created the issue which led to the incident. 22 And, in our testing, we was looking at 23 normal as if it was rolled out. We did not take 24 account of, during the pilot, which is a test 25 phase, we're going to put additional monitoring 88 1 tools into the thing which created more usage, 2 so we didn't stress the system the way. 3 So that's something we did miss specifically 4 and that's why we missed this in our testing, 5 because we assumed it was -- those tools that we 6 put in to support the pilot were only there 7 temporarily, and that wasn't taken account in 8 the testing. 9 Q. Thinking back to this morning, the very early 10 questions that I was asking you about, and 11 I took you to that document that I discussed 12 with Mr Burley yesterday, where there is 13 a suggestion of reduction in the testing that 14 occurred before the pilot took place, and 15 I think your evidence was "We didn't cut down on 16 the testing, we just did it while the pilot was 17 taking place". 18 I mean, looking at that combined with this 19 4.6, reflecting on it, do you think that there 20 was insufficient negative testing before pilot? 21 A. Performance testing isn't negative testing. 22 That's non-functional testing. 23 Q. Yes. 24 A. Negative testing is creating a failure 25 condition. So the volume testing is specific to 89 1 the data centre and how much loads can the data 2 centre take? So it's completely different from 3 this scenario. 4 Q. Putting aside volume testing, on reflection, and 5 looking at this recommendation, do you think 6 there was sufficient or insufficient negative 7 testing before the pilot took place? 8 A. Clearly, this showed that consideration wasn't 9 taken to some of the additional tools that we 10 were using to monitor the solution during the 11 pilot. So it found a weakness in our testing 12 that we had to review and we filled that. 13 Q. In plain English, do you think there should have 14 been more negative testing before the pilot? 15 A. There was a miss in the scoping of our negative 16 testing. When you say more, we should have -- 17 there should have been consideration of the 18 additional tools that was temporarily in place 19 in the pilot, and that was a miss in the test 20 analysis. 21 Q. Forgive me, "a miss in the test analysis", for 22 me, maybe just me, is not plain English. In 23 plain English, should there have been negative 24 testing before the pilot? 25 A. There was negative testing before the pilot, but 90 1 it didn't capture this scenario. 2 Q. So was there sufficient negative testing before 3 the pilot? 4 A. When the analysis was done we believe so but 5 this in hindsight shows that there was a miss. 6 So in hindsight, on the negative set of 7 testing, you could -- it starts to get 8 exponential as to how many different failure 9 conditions you could create, and it comes to -- 10 I've seen negative testing where you actually 11 question how could you have that scenario where 12 this happens at that time and that time? So 13 it's a case of you take -- you know, a take 14 a balance as to what are the likely scenarios? 15 Then you also have some good test resource, and 16 we had some very good test resource, that would 17 just play about with the system and just do 18 what-ifs. But at the time the analysis was done 19 and it was believed and, you know, it was 20 jointly reviewed with us in the Post Office, 21 that the testing in the negative side was 22 sufficient but this showed that it could have 23 been more robust. But that's in the benefit of 24 hindsight. 25 Q. We know what subsequently happened, and events 91 1 that have affected people's lives. Looking at 2 what ultimately happened, do you consider that 3 there was sufficient or insufficient negative 4 testing? 5 A. So I considered at the time our testing was 6 sufficient. The pilot phase is also a test 7 phase. So this was picked up in a test phase. 8 It's a live test. So the pilot phase picked up 9 a scenario that we didn't have in our laboratory 10 testing and picked up this error before it went 11 to general, and it picked it up very, very 12 quickly, with just the 10 or 12 post offices 13 that was part of the pilot. 14 Q. 4.6 isn't just talking about this specific 15 scenario. This is talking about negative 16 testing more broadly, isn't it? 17 A. So they said review -- so once -- so it 18 identified that in our laboratory testing, we 19 did not cater for this. So the recommendation 20 is, "Could you review everything you've done to 21 see if you can think of other things you may 22 wish to test?" And that action was given to 23 Debbie Richardson to do so. 24 Q. After receiving the ultimate recommendations, 25 and we'll look at the final report, was there 92 1 a significant increase or improvement in, and 2 strengthening of, negative testing? 3 A. I'm aware that there was areas of improvement, 4 the size and number I would have to refer back 5 to Debbie Richardson, and Lee Farman would know 6 as well because they would have reviewed it 7 jointly. 8 Q. So you're not aware of whether there was or 9 wasn't a significant improvement in negative 10 testing? 11 A. I'm aware that they reviewed and they 12 strengthened testing. How much I can't recall. 13 Q. Can we go to FUJ00094290, please. This is 14 an email from yourself to the general counsel, 15 Jean-Philippe Prenovost, and it is the 2 March 16 2010. You've attached the "HNG-X Counter Review 17 volume 2" that's dated 24 February 2010, and you 18 say as follows -- it's entitled "Integrity 19 review of HNG-X -- request for review": 20 "Jean-Philippe, 21 "You may be aware that an incident was 22 raised during the HNG-X pilot that there have 23 been two instances where a single transaction 24 was recorded twice on the system. As you can 25 appreciate from your workaround PCI Compliance, 93 1 the integrity of the system is critical to be 2 able to support Post Office in any litigation 3 they choose to take against Postmasters should 4 they uncover any financial irregularities. As 5 such, we took this issue very seriously and part 6 of our rectification plan was sponsor 7 an independent review. The report from this 8 review is attached and it is my intention to 9 share this with Post Office. Prior to sending, 10 could I ask you to assure that there are no 11 legal issues or consequences associated with the 12 content of the review that need to be 13 considered." 14 What did you have in mind when you were 15 asking Jean-Philippe Prenovost to review that 16 report? Were you looking for comments on the 17 content of the report and track changes and 18 things like that? 19 A. I was -- it's hard to recall my thoughts at the 20 time but I would expect to be asking does this 21 -- you know, we have an obligation, we have 22 a contractual obligation to support this. 23 There's a major issue, a major incident that 24 occurred, which obviously has a material impact 25 on that obligation. Here's a report to show 94 1 what we got. So I'm asking him to -- from 2 a legal standpoint, are we satisfied with this 3 report that we can still fulfil our obligations? 4 Q. So are you satisfied with -- were you expecting 5 legal advice in relation to the overall report, 6 rather than comments on individual entries and 7 passages? 8 A. Yeah. I wouldn't, you know -- and also, is 9 anything in there something which we need to 10 strength then or it's incorrect from a legal 11 perspective? So I'd expect Legal to assure 12 themselves and to be aware themselves of this 13 issue, and how we've addressed this issue. 14 Q. It certainly seems, at this time, you were very 15 live to issues about the integrity of the system 16 being critical to be able to support the Post 17 Office in litigation. I mean, you're raising 18 that issue with him. Was that very much live in 19 your mind at the time? 20 A. Absolutely. It was the number 1 priority that 21 was given to me by Mark Burley. 22 Q. Do you recall, say, in 2009, the Computer Weekly 23 article about Horizon and concerns that were 24 being generated around this period about the 25 Horizon System? 95 1 A. I don't recall, no. I -- sorry. My attention 2 has been drawn to it now. I don't recall 3 knowing of it at that time. 4 Q. Was there anything in particular that 5 highlighted this particular issue to your mind 6 to raise it as such a significant issue? 7 A. Reading that Computer Weekly, which I done 8 probably before Christmas because it was part of 9 the Inquiry discussions, that referred to 10 Horizon, not HNG-X. 11 So when I took on the position to lead the 12 programme, the first thing I done, as I would 13 with any programme, was to sit with the customer 14 and say "What your business priorities? What 15 are the priorities? What are your business 16 drivers? What are your drivers?" And Mark made 17 it very clear to me, number 1, integrity of the 18 solution, whatever we put out, we must ensure 19 integrity. And, for me, that was sacrosanct. 20 Q. Can we look at FUJ00094392. It's the bottom 21 email of the first page that I'd like to look 22 at. Thank you. This is 10 March now. You have 23 copied below this email the feedback that you 24 have received. So there's been a review of the 25 version that you sent to the legal team, and 96 1 this is the feedback. I'm just going to read 2 that feedback, if you could scroll down 3 slightly -- thank you very much: 4 First paragraph, page 2 -- The following 5 section is potentially problematic ..." 6 That's the section that said: 7 "'It would seriously undermine the Post 8 Office credibility and possibly historic cases 9 if it could be shown that a discrepancy could be 10 caused by a system error rather than 11 a postmaster/clerk [error]. Most importantly, 12 the central database as the system of record 13 would be called into question'. As discussed, 14 there is no need to paint this in the worst 15 possible light. I would suggest the following 16 as being accurate without being unduly alarmist: 17 'If it could be shown that a discrepancy could 18 be caused by a system error rather than 19 a postmaster/clerk action, it could potentially 20 call into question the effectiveness of the 21 central database as a system of record'." 22 It says there "further to our conversation", 23 can you recall the conversation or "further to 24 our discussions" on Friday; do you recall the 25 discussions about that particular paragraph? 97 1 A. I recall a meeting with Legal, just to -- 2 because they wanted to understand better the 3 scenarios and the context and everything else 4 around it, not just "You're giving me one 5 problem". So I remember a meeting. It was, 6 I believe, about an hour long, and lots of 7 things were discussed at that meeting. I also, 8 I do recall them now, now seeing that, I do 9 recall them asking -- I think, as you did -- who 10 wrote that, and I said that was written -- that 11 wasn't any words from the programme, that was 12 written by the reviewers. 13 Q. So it was the words of independent reviewers. 14 It was also very much highlighted in your 15 covering email to the general counsel when you 16 sent the original draft about the importance of 17 the integrity of the system for legal 18 proceedings. So it was something that was very 19 much in your mind as well, and that was your 20 evidence a few moments ago. 21 A. Yes. 22 Q. Were you surprised to have received that 23 feedback from general counsel? 24 A. When you say surprised, that's what our legal 25 representative wished us to do. 98 1 Q. I mean, painting something in the worst possible 2 light, that's not necessarily legal advice, is 3 it? That's more public relations, isn't it? 4 A. All I remember is, in the discussions, being 5 asked where did that come from? And I said it 6 just come from -- and it's why Legal felt that 7 we should be factual, that the -- that it would 8 be, you know, this issue, if not resolved 9 properly, and we reviewed the rest of the 10 system, the record of accounts would be done, is 11 a better one. You'd need to ask Legal why their 12 advice was that. But I would not go against our 13 legal advice. 14 Q. You wouldn't go against their legal advice but 15 there's nothing in that italicised section 16 that's untrue, is there? 17 A. So when we say, "possibly historic cases", I'm 18 not aware of historic cases. I don't know 19 whether or how -- whether that is just general 20 discussion points that they've picked up during 21 the review or whether they had evidence of that. 22 I don't know. 23 Q. The concern about undermining the integrity of 24 the system, the central database as a system of 25 record would be called into question, that was 99 1 very much in line with your concern and the very 2 purpose of requesting a report, wasn't it? 3 A. So my concern is the integrity of the solution, 4 and does it (1) protect Post Office and their 5 employees and (2) does it fulfil our contractual 6 requirements? Historic cases has nothing to do 7 with HNG-X so why that's in there, I don't know. 8 What relevance that has to HNG-X, I don't know. 9 Q. So we can take out the words "possibly historic 10 cases" although it says "possibly" but let's 11 take that out. The rest of it, it's accurate, 12 isn't it? 13 A. It could be called into question if not 14 resolved. 15 Q. Let's look at what ultimately happened and the 16 prosecution of subpostmasters and the 17 overturning of those convictions by the Court of 18 Appeal because of the reliability of the data 19 that was being used in those prosecutions. 20 I mean that was spot on, wasn't it, that section 21 in italics? 22 A. I -- I don't have an opinion on that. It's 23 a case, this was a report, looking at HNG-X. 24 Q. But you could reflect on what's ultimately 25 happened. Look at those words in italics and 100 1 agree or disagree whether that actually reflects 2 the very reason why we're here? 3 A. As you put the question to me, yes. 4 Q. Let's look at FUJ00142175 -- sorry, can we stay 5 with that for one second. Point 2: 6 "Section 3.19 on page 7 -- You'll recall 7 that we cannot really make sense of what was 8 being referred to here." 9 I've looked back and it locks as though, and 10 you can -- we'll go through it -- but it looks 11 as though that was actually 3.20 and that was 12 one of the conclusions about the stock check 13 issue that we just discussed. 14 A. Mm. 15 Q. We can have a look at that and see that that 16 also comes out. Thank you. We can go back to 17 175. 18 Can I ask, 175, so FUJ00142175, this is what 19 we're told is the final version. It has version 20 2.0 at the bottom right-hand corner but we are 21 told it is the final version, version 3. Can we 22 put up on the screen, I'm going to ask for a bit 23 of work on this display once again, I'm afraid. 24 It's FUJ00093031. This is the earlier version 25 I took you to, just so that we can look at some 101 1 of the changes that happened after that feedback 2 had been received. 3 Thank you very much. So on the left-hand 4 side we have version 1, on the right-hand side 5 we have version 2 -- or version 3, actually, is 6 the one on the right-hand side. Now, an initial 7 difference we see there is that on the -- it's 8 initially you're one of the circulation list and 9 that's changed to one of the reviewers. Do you 10 know why that change was made? 11 A. Because they're asking for a review, or they, 12 sorry, they -- 13 Q. It's ultimately the same document. 14 A. Yes, sorry. So it is a final report to say that 15 those people that was on the circulation list 16 had been given the opportunity to review and 17 feed back comments. 18 Q. Because this is the version that's going to go 19 to the Post Office, you're named as a reviewer 20 rather than part of the circulation list; is 21 that right? 22 A. Correct. 23 Q. Yes, thank you. 24 Then if we look at the third substantive 25 paragraph, that has been added on the right-hand 102 1 side. It says: 2 "This report reflects the findings from 3 a visit by Paul Roberts and Stuart Rye on 4 4th February 2010 and a follow-up review with 5 David Johns on 24th February." 6 So there was a subsequent follow-up review, 7 do you recall that? 8 A. I don't -- no, I don't recall it. 9 Q. Thank you. If we look, staying with the 10 right-hand side, there is an update that is 11 provided at the bottom of that page, that isn't 12 present in the first version, and if we could 13 zoom in on that update, it says: 14 "22nd February: Searches of the database 15 since pilot launch and being run daily revealed 16 one further incident of a dual settlement (this 17 time without a banking transaction). There are 18 over 100 branches live as of this date." 19 So an update on 22 February, it's happened 20 again, this time without a banking transaction, 21 so it's a different type of transaction, 22 suggests that the issues actually got worse by 23 that stage. 24 A. It shows that another incident was detected. 25 Q. Does it not show that the incidents -- the 103 1 matters got worse? I mean, it's a new incident, 2 a different type of transaction. 3 A. Another issue -- well, a similar symptom but 4 a different trigger point. 5 Q. There are only 100 branches live at that stage. 6 Is that what, that's three out of 100 this issue 7 is occurring in or known to be occurring in? 8 A. So there's three known incidents across two post 9 offices out of 100 post offices that may have 10 had, at that time, several hundred thousand 11 transactions, yes. 12 Q. Well, in terms of branch numbers, what do we 13 have? We have three out of 100? 14 A. Well, two, because two was at Derby. 15 Q. Was that with two different counter clerks? 16 A. Sorry, you was just saying in branches. 17 Q. Yes. 18 A. Okay, so three incidents. 19 Q. Three incidents. Some to do -- two of them or 20 three of them to do with the banking 21 transactions, one not to do with banking 22 transactions. It seems to have got worse. 23 A. They found another incident, yes. 24 Q. Is there a reason why you can't agree that it 25 seems to have got worse? Is there a technical 104 1 reason for that? 2 A. So if we picked up after two or three days two 3 incidents after a week of going live of a pilot, 4 two incidents, and then in another month you 5 pick up another incident, yes, three is worse 6 than two. It depends how you say has it got 7 worse, ie is it just snowballing, is it just 8 getting continually getting worse? I wouldn't 9 say so but it was another incident detected. 10 Q. Involving a different type of transaction? 11 A. That's what it says here, yes. 12 Q. An additional type of transaction? 13 A. A transaction without a banking element. 14 Q. Can we go to the second page, please, on both of 15 them. Is that possible? Am I asking too much? 16 Excellent. 17 In fact, we can just look at the new one so 18 42175, if that can just be expanded and we move 19 over to the second page. There is now no 20 mention of that passage that general counsel had 21 suggested should be removed. So that appeared 22 in the first version and it now doesn't appear 23 in the background section. Do you remember that 24 passage being removed? 25 A. I don't recall it, no. 105 1 Q. You received the email from general counsel 2 saying, "Can this be taken out?" 3 A. And that would have gone back to the authors. 4 I would not -- so I'm not the author of this 5 document so I would not personally take it out. 6 I would submit it back to Stuart Rye and Paul 7 Roberts to deal with the feedback comments. 8 Q. Do you recall general counsel's advice being 9 taken? 10 A. It appears to, if they've taken it out, yes. 11 Q. Can we look at page 5, please, of the right-hand 12 side. We can take the left-hand side one off, 13 I think. We're just going to try and get it up 14 on the left-hand side so we can compare the 15 original. 16 We'll just take a moment because it will 17 help to see them side by side, I think. Just to 18 clarify also, I've been asked to clarify, 19 Mr Prenovost or Prevost, he was part of a team 20 to lawyers, he wasn't general counsel. I don't 21 know if that's -- 22 A. We just call them Legal. 23 Q. Was he particularly senior in Legal, from your 24 recollection? 25 A. He's the person that I was put in contact with 106 1 so he was the interface I had for Legal. 2 I don't know his status within Legal. 3 Q. Thank you. There we have the two different 4 conclusions. The original conclusion said: 5 "Overall, the actions taken to redress the 6 Derby issue are appropriate. We believe the 7 Counter Application fully supports the need to 8 protect the integrity of financial 9 transactions." 10 Then the new version says: 11 "We believe the ... Application with the 12 identified fix fully supports the need to 13 protect the integrity of financial 14 transactions." 15 Can you assist us with that change? 16 A. Unless you applied a fix to the issue that we 17 had, the integrity would not be maintained. 18 Q. Would a fix be applied to all counters or just 19 those who raised the issue or just -- 20 A. All counters. It was part of the release and it 21 also was mandated to be applied prior to going 22 to any other post offices. 23 Q. Is it saying here now that the fact that we have 24 applied the fix supports the need to protect the 25 integrity of financial transactions -- 107 1 A. That's what it's saying, yes. 2 Q. -- ie the fact that we found the problem and 3 found a solution supports the need to protect 4 the integrity of the financial transactions? 5 A. Yes. 6 Q. So it's quite specific, is it not, to this 7 particular problem but we discussed earlier 8 about whether this is signing off the system 9 itself or the problem. Doesn't it now seem to 10 be even more specific to the problem rather than 11 the system? 12 A. If I -- and I'm looking across at the 13 conclusions and I just looked at 3.6: 14 "The business control at the Counter 15 recognises that dual settlement risk is inherent 16 by having two buttons that can initiate 17 settlement. The business requirement is that 18 the buttons operate on an exclusive basis -- 19 ie the use of one ..." 20 So that's not specific to this issue. That 21 is demonstrating how, because you can have two 22 ways of operating into the system, you cannot do 23 them concurrently, because that could cause -- 24 So the conclusions are wider, are showing 25 protections that have been put as built in and 108 1 designed in as part of the solution, which is 2 not to do with that specific issue. 3 Q. So you still believe in this later report, later 4 version, that it is signing off the integrity of 5 the system itself? 6 A. It's validating our design and our 7 implementation of that design that we should 8 have confidence of the integrity of the system, 9 yes. 10 Q. Can we go over the page, it's page 7 on the 11 right-hand side. It's still part of the 12 conclusions, 3.19 and 3.20. If it's possible to 13 do a similar thing on the left-hand side. 14 So you'll recall I took you to 3.20, which 15 is: 16 "Stock check is not a reliable method for 17 catching issues due to the erratic nature of the 18 stock check occurring." 19 That doesn't seem to exist in the later 20 version, and also the recommendation that 21 related to stock checking. If we could have 22 a scroll down on the left-hand side if possible, 23 to the recommendations. You'll recall there was 24 that recommendation: 25 "Consider advising the Post Office of the 109 1 benefit of more effective stock control as an 2 indicator of clerk errors or fraud." 3 That seems to have been removed in the final 4 version, as well. Are you able to assist us 5 with why those were removed? 6 A. Because I think, when you asked me that first 7 question about stock check, I couldn't 8 understand, on that first one, why that's 9 relating to the integrity of the system. And 10 I believe you highlighted that. You know, from 11 our discussions, we couldn't understand why this 12 was part of this report, because it's not to do 13 with how system behaves; it's how postmasters 14 interacting with the system behaves. And it's 15 not to do with the subject matter in point. 16 So the advice from Legal was that if it's 17 not part of anything to do with integrity of the 18 system, remove it. 19 Q. Do you think, looking back, it would have been 20 helpful for Fujitsu to have taken that 21 independent recommendation, which was to 22 consider advising the Post Office of the benefit 23 of more effective stock control as an indicator 24 of clerk errors or fraud? Some other system, as 25 an indicator of clerk errors or fraud, to back 110 1 up whatever they were carrying out at that 2 stage? 3 A. I don't even understand that advice. I don't 4 understand what that advice means. 5 MR BLAKE: Thank you. 6 Sir, that might be an appropriate moment. 7 If you're content to take, say, a 15, 20-minute 8 break now, we could go on after that and I'll be 9 15 minutes. 10 SIR WYN WILLIAMS: Yes, that's fine, Mr Blake. 11 Shall we say we'll start again at 1.10, and then 12 you will have completed approximately by 1.30. 13 Is that what you're telling me? 14 MR BLAKE: I'll definitely -- I'll conclude my own 15 questioning by 1.30. There may be some 16 questions. I may have covered everything. So, 17 with goodwill, we will -- 18 SIR WYN WILLIAMS: All right. Anyway, with 19 a limited time thereafter. Fine. All right. 20 MR BLAKE: Thank you very much. 21 (12.53 pm) 22 (A short break) 23 (1.10 pm) 24 MR BLAKE: Thank you very much. We're now going to 25 move to March 2010. Can we look at FUJ00094958, 111 1 please. This is an email exchange of 2 26 March 2010. You are copied in, it's 3 forwarded to you by David Keeling. Who was 4 David Keeling? 5 A. David Keeling at that time was the customer 6 services director. 7 Q. He says: 8 "Alan, Peter 9 "Please see Gavin's note below and Dave 10 Smith's mail." 11 Now, this is forwarding, at the bottom of 12 that page, an email from David Smith. Do you 13 remember who David Smith was? 14 A. David Smith was Mark Burley's -- Mark Burley 15 directly reported in to David Smith. I think he 16 was the head of technology in Post Office. 17 Q. This email was in your bundle, have you read it 18 recently? 19 A. I was given it today. 20 Q. No, not this email. 21 A. Oh, sorry, this one? 22 Q. This email you've definitely had for some time. 23 We can go through it that's fine. It starts 24 off: 25 "Gavin 112 1 "I want to follow up our earlier telecon 2 rather more formally. 3 "Whilst we don't yet have a root cause of 4 today's issue given recent events it is 5 difficult not to suspect that it might be 6 related to the introduction of a change. Quite 7 simply there have been too many incidents where 8 poor execution of change has caused a problem in 9 live. The situation demands that Fujitsu take 10 action that is game changing whether that be 11 increased rigour, an injection of [I think 12 'different'] skills or a change in mindset. 13 "I also have to be concerned that we seem to 14 be ahead of you and finding out for ourselves 15 that there has been an incident in live rather 16 than hearing from you. We have been here before 17 and I will take a lot of convincing that this is 18 not symptomatic of a reactive mindset. Again we 19 need to see action that is game changing to 20 a proactive style of management. 21 "The wider POL business and major 22 stakeholders have been incredibly patient thus 23 are. I believe we are now on the cusp of losing 24 them and if we do then experience tells us that 25 we could well end up on the front page of the 113 1 Daily Mail. That will do damage to the 2 reputation of both our businesses." 3 This is quite an angry email, I think it's 4 fair to say. Do you remember it? 5 A. I remember with David Keeling drafting 6 a response on behalf of Gavin, yes. 7 Q. That's the document that you were provided with. 8 A. Yes. 9 Q. But do you remember the actual -- seeing this 10 email that was forwarded to you? 11 A. Yes. 12 Q. Yes? Do you remember what "today's issue" was? 13 A. The Oracle. The Oracle bug that caused 14 an intermittent failure to all the pilot post 15 offices. 16 Q. So a significant issue? 17 A. Yes. 18 Q. Do you think that a letter of this nature was 19 justified? 20 A. I think, as I -- so I'd imagine that David Smith 21 is under a lot of pressure from the business 22 because having service outages at the pilot post 23 offices for a period of time, which is losing 24 business for those subpostmasters and Crown 25 Offices, plus requiring compensation, David 114 1 Smith will be getting a lot of pressure from his 2 business and he wants answers from Gavin as to 3 what are we doing about it. 4 Q. Before the break, we were discussing the 5 independent report that was written and the 6 conclusions of that report and it was your 7 belief that that concluded, essentially, that 8 the counter application had integrity or 9 something along those lines. Do these kinds of 10 incidents cause you to doubt such a conclusion, 11 if that was their conclusion? 12 A. This incident was preventing Post Office from 13 trading. Nothing to do with the integrity of 14 what was traded. The incident that triggered 15 this email was post offices could no longer 16 trade, so it's nothing to do with integrity; 17 it's to do with the ability of the business, the 18 business impact of not being able to trade. 19 Q. "Quite simply there have been too many incidents 20 where poor execution of change has caused 21 a problem in live." 22 Isn't that a broader complaint coming from 23 the Post Office, that there had been too many 24 incidents. Rather than just focusing on the one 25 Oracle issue, this is a broader complaint, isn't 115 1 it? 2 A. I believe that the Oracle issue triggered this. 3 My understanding is that the previous issues 4 that have been referred to, have been to do with 5 maintenance releases that's gone out to the 6 Horizon estate, was my understanding at the 7 time. 8 Q. So does nothing in this email cause you concern 9 or to question a statement that the counter 10 system had integrity? 11 A. Had issues with integrity? No. 12 Q. We'll look at the response. It's FUJ00094958. 13 This is a draft response. So in the email we've 14 just seen, if we scroll up, or we may have 15 passed it already but in the email we've just 16 seen it says: 17 "Please see Gavin's note below and Dave 18 Smith's mail. I've attached a draft note." 19 So that is David Keeling has sent you this 20 note in response. Do you recall this? 21 A. He sent Gavin Bounds this note in response and 22 I, with David Keeling, drafted this response. 23 Q. So you drafted the response? 24 A. With David, yes. 25 Q. It starts: 116 1 "In response to your note of Friday, I do 2 understand your concern regarding outages being 3 contributed to by operational changes in the 4 live estate and had already enforced the message 5 of rigorous change control [access] across our 6 technical and service management teams." 7 I'll move to the final paragraph, it says: 8 "The conflicting needs for deployment 9 progression and stabilisation [that's the issue 10 we were talking about earlier] is always 11 a challenge; which we recognise. However we are 12 striving to demonstrate stability and continuing 13 improvements to the existing Horizon Online 14 branches." 15 So that was a draft response. Are you aware 16 of whether that response was sent or not? 17 A. I assume it was, but I don't know -- if it was, 18 I was not copied on it, or I don't recall being 19 copied on it. 20 Q. So nothing in the David Smith email caused you 21 concern with regards to the integrity of the 22 system. Can we look at FUJ00094958 -- actually, 23 sorry, FUJ00095107. Did you not think in that 24 email that Mr Smith himself was questioning the 25 integrity of the system? 117 1 Sorry, putting aside this that's document on 2 screen currently, the earlier communication from 3 David Smith from the Post Office, did you not 4 think that he himself was questioning the 5 integrity of the system? 6 A. I read that document that he was raising 7 concerns that when changes are made in to the 8 production system, there's impact against the 9 business, such as the Oracle issue which has 10 caused a, you know, lack of trading. It 11 didn't -- I didn't read it as "I have concerns 12 on the integrity of the system". He said he is 13 concerned that, as part of our process of 14 releasing change into the production, we appear 15 to disrupt the business. 16 Q. Because changes affect other parts of system, 17 which then can cause errors in the system? 18 A. I'm not going to come to that conclusion. 19 Q. We've heard in this Inquiry about things like 20 code regression, when you change code it may 21 have a negative impact somewhere else, or 22 something along those lines. Was his complaint 23 not that -- 24 A. Code regression? 25 Q. I'm taking that as an example. His email was 118 1 expressing a frustration that the changes that 2 were being made were causing other problems, 3 wasn't he? 4 A. Impact to the business, yes, whether -- 5 Q. Well, impact to the system, as well as the 6 business. 7 A. Could you put the email up again, please? 8 Q. Absolutely. If we go back to FUJ00094958. It's 9 the first paragraph of the bottom email on that 10 page. He refers to the recent events. He says: 11 "Quite simply there have been too many 12 incidents where poor execution of change has 13 caused a problem in live." 14 He's not confining that to some sort of 15 business problem. I mean, does it not read as 16 though it's concerned impact on the Horizon 17 System and problems with the Horizon System, 18 or -- 19 A. Okay, so that's very wide. Okay, I don't know 20 what he meant by that. 21 Q. Let's look at FUJ00095107. I'm only going to 22 deal with this email briefly. Before it comes 23 on screen, I can tell you it's an email from 24 Geoff Butts. You're included in the copy list. 25 It's 6 April, so we're now moving to April, and 119 1 he says: 2 "I was asked to provide some criteria for 3 restarting the High Volume Pilot." 4 He sets out various matters there. You have 5 added in the issues in square brackets, and by 6 way of example, we can look at five. This is 7 the Oracle issue, it says: 8 "A significant reduction in the number of 9 Oracle 3136 errors seen in live." 10 I think that's your comment: 11 "What is the point of this? The main issue 12 around Oracle 3136 errors was that it sent the 13 BRDB into a non-recoverable 'spin' which led to 14 catastrophic failures." 15 Can you expand? Can you tell us what you 16 were saying there? 17 A. What I'm saying is you can't have a reduction in 18 the number of Oracle 3136 error messages. If 19 that error message occurs, you have 20 a catastrophic failure. Therefore, we have to 21 fix it before we go forward. We can't have 22 a reduction. 23 Q. We have there nine individual numbered examples 24 or issues. The number of issues raised here and 25 your comment on, for example, that Oracle issue, 120 1 again, it doesn't sit well with a commitment to 2 the integrity of the system at that time, does 3 it? 4 A. I don't recognise the BAL failures. I don't 5 recognise what branch database failures mean in 6 the context of (unclear). I clearly didn't 7 understand what number 3 was. Outages are not 8 integrity related. Number 5 is not integrity 9 related. Number 6 is not integrity related. 10 Q. So you were satisfied at this stage still -- 11 A. Number 7 is a wide -- so, from my recollection, 12 those that had a material impact or potential 13 impact on integrity of the solution we took very 14 seriously and we took to resolution. 15 Q. Were you open and honest and upfront with the 16 Post Office about that? 17 A. Yes. 18 Q. Always? 19 A. Yes. 20 Q. Let's look at POL00033100. This is the 21 acceptance report for HNG-X. It's dated in the 22 bottom right-hand corner, 7 July 2010. We have, 23 in this document, if we turn to page 9, we can 24 see acceptance criteria. I only need to look at 25 those very briefly. But by this stage, the 121 1 stage of this report, 9 July, it says: 2 "there are no outstanding HNG-X High 3 Severity Acceptance Incidents." 4 That's point (b) under 1.1. I'm going to 5 look at page 31 in a little more detail. We 6 have on page 31 medium Acceptance Incidents and 7 there are two of those at that time. Thank you. 8 If we look at the table, it says, Low, 146, 9 so there are 146 Acceptance Incidents there; 10 medium, 2; high, 0. If we go over the page we 11 can look at those medium ones at the bottom of 12 that page. The first concerns patch management, 13 and the second concerns cash declarations. Do 14 you remember the cash declarations issue? 15 A. I don't recall it, no. 16 Q. If we turn to page 37, it there summarises that 17 Acceptance Incident, the cash declarations 18 issue. So TS058. Could I ask you to have 19 a quick look at that and see if it brings 20 back -- 21 A. Oh, okay, the cash holdings, overnight cash 22 holdings. I recognise that one, yes. 23 Q. Can you tell us what that one was, in simple 24 terms? 25 A. So in simple terms -- goodness. As I recall it 122 1 in simple terms, that the process that -- so the 2 overnight cash handling process was a process 3 that Post Office had to ensure that there was 4 adequate cash levels in each of the branches or 5 ATMs. They don't want too much cash at a branch 6 they don't want too little cash in the branch 7 because if they have too little they run out, 8 they have to have emergency releases and things, 9 that cost them money. 10 So part of the Horizon, and in the HNG-X 11 system, will provide a statement called the 12 ONCH, the overnight cash handling statement back 13 to the centre to say this is the cash position 14 at this branch and that will determine whether 15 or not the Post Office deliver cash or pick up 16 cash from that Post Office to make sure it has 17 the right, adequate levels of cash for its 18 business. 19 The issue here was, introducing HNG-X, the 20 postmaster could do something out of the 21 sequence to the process and the information 22 going back to the centre about the cash position 23 was incorrect. Now, the information of the cash 24 position in the statement of accounts in the 25 branch was correct and, therefore, it could 123 1 create a situation where the cash position or 2 the delivery or collection of cash to that Post 3 Office was incorrect, and Post Office was very 4 concerned that if a post office or an ATM ran 5 out of cash it causes them extra costs because 6 they have to go and do that. 7 So this was about a process, in as much as 8 to -- needed to be reworked so that the accurate 9 cash position is put to the centre, so that they 10 can do their cash replenishment. 11 Q. This was a medium severity incident and one 12 that, as you said, the Post Office were not 13 happy at all about. I'll take you to the 14 contemporaneous email on that it's FUJ00097001. 15 Can we look at page 6 and 7 of that. You 16 have been copied in to this email chain higher 17 up. But if we can look at page 6 and 7. Phil 18 Norton sets out, if we could scroll down and 19 over to the next page, a whole load of reasons 20 why he says it needs to be fixed, and I'll just 21 highlight for the sake of time the final 22 paragraph. So it says: 23 "For all these reasons, this defect needs to 24 be fixed as a matter of urgency and, contrary to 25 the proposal by Fujitsu, without the raising of 124 1 a CR. The Use Case 'obtain daily cash on hand 2 statements for SAP ADS' was a retained 3 functionality UC, and therefore HNG should 4 operate in the same way as Horizon. It doesn't 5 and therefore this is a defect -- not a Change. 6 The paper from Gareth at the foot of this mail 7 gives more detail of the issue and the proposed 8 fix." 9 This is still the same issue, isn't it? 10 A. This overnight cash handling -- obtain daily 11 cash on hand statements for SAPADS. 12 Q. I don't want to rush you, you can have a little 13 look above, if you like. 14 A. Could you please. 15 Q. It says, "This [relates to] the feed to SAPADS 16 having NO cash information ... 17 "The operation implications of this are 18 significant -- both to the Network and the Cash 19 Inventory Team." 20 Or is this a different issue? 21 A. I think that might be a different issue. 22 Q. So there's another issue -- 23 A. It might be the same. We need to go back to the 24 technical experts that resolved this issue. 25 Q. We can scroll on, if you like, to see if it 125 1 assists you. Perhaps the final page, page 10, 2 is the email from Gareth Jenkins. Do you recall 3 the Post Office in June 2010, getting angry 4 about what they called a defect, whether it's 5 the same issue as mentioned in that Acceptance 6 Incident or not? 7 A. It's probably the same issue. 8 Q. Probably is the same issue. Do you remember the 9 Post Office -- this angry email from Phil 10 Norton? 11 A. And can you show me the email, please? 12 Q. Page 7. Page 6 and 7, bottom of page 6, top of 13 page 7. You'll see, if we scroll up to page 5, 14 you've been copied in. 15 A. This email here from Graham Allen? 16 Q. Yeah, you're copied in there. 17 A. So this email from Graham Allen you're asking me 18 about? 19 Q. Yes. What I've referred to as the angry email, 20 it is simply the one I read to you before from 21 Phil Norton which says: 22 "For all of these reasons, this defect needs 23 to be fixed as a matter of urgency and, contrary 24 to the proposal by Fujitsu, without the raising 25 of a CR, the use [et cetera, et cetera]. It 126 1 doesn't and therefore this is a defect -- not 2 a Change." 3 He's specifically referring to this as 4 defect. 5 A. Okay. 6 Q. Do you remember the Post Office in this period, 7 so June 2010, being concerned and raising those 8 kinds of concerns? 9 A. My -- I don't have a direct recollection, but 10 Phil Norton is just debating, so I'm making 11 an assumption that a response went back to Post 12 Office saying this is a change, not a defect, 13 and they come back saying they believe it is 14 a defect and, on further investigation, it was 15 determined to be a defect. 16 Q. Were they right to be angry about it? 17 A. I don't know if Phil was angry. I mean, can you 18 show me the email again? 19 Q. Absolutely. It's page 7. "Angry" is my word. 20 It may not be angry but certainly insistent that 21 it is a defect not a change, because of the 22 significant implications it would have. 23 A. So this email here, was I copied on this email? 24 Q. You were copied in the chain, not copied in the 25 email. 127 1 A. So he's -- I think he's being forceful, that it 2 needed to be fixed, as a matter of urgency, as 3 a priority. 4 Q. Yes, especially as how close it is to ultimate 5 acceptance of the system? 6 A. Yes. 7 Q. Can we look at FUJ00097058, still in June 2010. 8 Is this a document that you recall? It's 9 a report by somebody called Penny Thomas. 10 A. I don't recall it from the time, but I recall it 11 from the pack that was presented to me. 12 Q. What was the issue with the ARQ return, in very 13 simple terms, and what's said here to be 14 duplicate transaction records? 15 A. So this is -- I think I may have referred to 16 this earlier, and mistook another potential 17 issue, where we have an audit record that we 18 keep, and there are times, just for -- in any 19 system, records can be sent twice and previous 20 system that was dealing with audit records on 21 Horizon had a facility to detect duplicate 22 records and then to reconcile those. 23 My understanding of this is that the HNG-X 24 did not do a similar thing and we had to kind of 25 put in a facility to identify duplicate records. 128 1 Q. If we scroll down it says: 2 "A discussion took place between Alan 3 Holmes, Gareth Jenkins and Penny Thomas." 4 What were their respective positions? 5 A. Well, Gareth Jenkins, again, SME. Penny Thomas, 6 I am not sure but Alan Holmes was very much 7 working in the requirement space and in also 8 the -- I think he was a business analyst. 9 Q. Point 3, one of the tasks/actions is: 10 "Gareth agreed to draft a statement for 11 management review detailing the issue for onward 12 transition to [Post Office]." 13 Then it says: 14 "A separate issue was also identified where 15 a seemingly duplicated transaction had 16 a different NUM and Gareth agreed to review of 17 the detail." 18 Do you remember that at all? 19 A. No. 20 Q. No. If we turn over the page: 21 "We need to identify which cases provided 22 with ARQ returns since the HNG-X application has 23 been live have progressed to prosecution and 24 identify whether duplicate records were 25 included. We will need POL involvement to 129 1 ensure all instances are covered. A very quick 2 review identifies that both West Byfleet and 3 Porters Avenue are included here, to what extent 4 is not yet known." 5 Are you aware of the relevance of 6 prosecutions to this particular issue? 7 A. No. 8 Q. Can we look at FUJ00097046, please. There is 9 subsequent discussion of this issue, and let's 10 look at page 4 -- page 3 scrolling down to 11 page 4. So at the bottom of page 3, there's 12 an email from Penny Thomas. You're not at this 13 stage copied in. The subject is "Duplication of 14 Transaction Records on ARQ Returns". We have 15 here number of ARQs affected, 112; 12 ARQs where 16 court action is known, that's the fourth 17 entry -- 12 ARQs where court action is known; 18 number of cases, two; 8 ARQs where witness 19 statement requested but has not been provided; 20 number of cases, three. 21 Are you aware of the relevance of this ARQ 22 information to prosecutions? 23 A. No. 24 Q. Are you aware of concerns within Fujitsu copied 25 in there, Gareth Jenkins from Penny Thomas about 130 1 the relationship between this duplication of 2 transaction records and its potential impact on 3 prosecutions? 4 A. I do not recall it. 5 Q. This is June 2010. Do you know how long this 6 particular issue had been going on for? 7 A. No. I've read this email trail for my pack, and 8 whilst not an excuse, I always take the last two 9 weeks of June off on holiday, so that's my -- 10 I always do that. I was copied. Geoff 11 responded, Geoff being my deputy. I do not 12 recall this incident, and it may have been 13 resolved because, in reading the email trails, 14 there was a fix available. It may have been 15 fixed by the time I returned and I wasn't 16 briefed about it. I just do not recall it. 17 Q. Okay, let's look at page 3. The bottom email 18 from Guy Wilkerson. Who was Guy Wilkerson? 19 A. Guy Wilkerson was part of our -- legal and 20 commercial person that was assigned to the 21 account. 22 Q. So he's -- 23 A. Guy was more the commercial, and Jean-Pierre was 24 the legal. 25 Q. Thank you. Guy says: 131 1 "Penny, would the additional transactions 2 make any difference to the charges for 3 a subpostmaster?" 4 If we scroll up, Penny Thomas says: 5 "These are original records which have been 6 duplicated when copying to Audit Server. We are 7 not suggesting that original records have been 8 duplicated. 9 "If analysis was undertaken on the audit 10 data some transactions would be duplicated; both 11 plus and minus (we hope!). Analysis on stock 12 units could be out as TI would show duplicated 13 transfers and equally would TO. Cash on hand 14 analysis would also be out." 15 Are you able to assist us with what that 16 means? 17 A. That means specifically that if you don't spot 18 there's a duplicate transaction in the audit 19 record, not in the record of accounts in the 20 Post Office -- so that is -- there's no 21 integrity question on the record of accounts. 22 However, if you wanted to take a copy from the 23 audit server of what happened, in that account, 24 and you don't spot there's a duplicate 25 transaction in there, then you may double count 132 1 that and you may come to the wrong conclusion. 2 Q. Is that significant for the prosecution of 3 subpostmasters? 4 A. I don't know. 5 Q. Is it potentially significant? 6 A. I don't know how this information was used. 7 Q. Is it significant to the reliability of the 8 data? 9 A. I don't know how this information was used. 10 Q. If we scroll up to page 2, we have Gareth 11 Jenkins responding. He says: 12 "Penny, 13 "No, that pretty well covers it. NB there 14 is no guarantee that the duplicates are even 15 complete sessions in which case the sum of all 16 transactions may even be out. 17 "In summary, any detailed analysis of the 18 finances of a Branch which is done with 19 duplicate transactions without realising that 20 there are duplicates (and so removing them) will 21 give incorrect results." 22 Does that strike you as a particularly 23 serious problem, given that the Post Office was 24 involved in the prosecution of subpostmasters 25 and others? 133 1 A. If this evidence -- if this, sorry, information, 2 this data, was used as part of that evidence, 3 yes, it would. 4 Q. Scrolling up, from Guy Wilkerson: 5 "Gareth/Penny, 6 "I think we need Alan D'Alvarez or Geoff 7 Butts to look at this -- I'd hate to have POL 8 raise this as an issue with an HNG-X Acceptance 9 Board on Tuesday." 10 Do you know why your name is being mentioned 11 there. 12 A. Because there's an acceptance board for HNG-X 13 and this is an HNG-X issue. 14 Q. Do you recall that that was, in fact, the 15 acceptance board meeting where there was 16 acceptance of the HNG-X system? 17 A. I -- it was a meeting that was scheduled for 18 when I was on leave, but I don't believe it took 19 place until later. 20 Q. If we scroll up, were you back in early July? 21 A. Yes. 22 Q. Penny Thomas says to Guy Wilkerson and Gareth 23 Jenkins, copied in to you -- would you have 24 received emails? I mean, 2010, did you have 25 probably a BlackBerry or something at that time? 134 1 A. So I don't know if I had a BlackBerry, certainly 2 didn't have an iPhone. So I would receive 3 hundreds of emails a day. When I'm on leave, 4 I would probably -- I would have taken my laptop 5 and I would have scanned just those emails that 6 were directly sent to me, not cc'd me. Because 7 if it's sent directly to me, I may need to 8 forward that email for someone to action on my 9 behalf. 10 Q. So you made a specific decision to only review 11 emails that were sent to you rather than copied 12 to you? 13 A. When I'm on leave, yes. 14 Q. How did you know they were copied to you until 15 you've opened them? 16 A. Because you've got "to", and "cc", and my email 17 system is set up for those emails that are sent 18 where I am on a "to" list, and the "cc" list 19 goes into another folder. 20 Q. Even on your phone or whatever, you may have 21 been taking on holiday? 22 A. I didn't have access to emails by my phone. As 23 I said, I probably took my laptop on holiday. 24 Q. Your laptop. From Penny Thomas, she says: 25 "Guy, should I hold off advising my 135 1 counterpart on this issue?" 2 So Penny Thomas was at Fujitsu. Presumably 3 "counterpart" there is a reference to 4 a counterpart at the Post Office? 5 A. I don't recall what Penny Thomas's name is -- 6 sorry, role was. 7 Q. It says there "Security Analyst, Customer 8 Services". 9 A. Okay. 10 Q. Can we scroll up. From Geoff Butts: 11 "Please do not make any communication on 12 this issue with POL for the moment. We have 13 been looking at this today and are waiting to 14 determine if a proposed workaround is adequate. 15 I will come back to you either tomorrow or 16 Monday on progress." 17 Then, above, Guy says: 18 "My nose never fails!" 19 We've seen in this email chain -- yes, we've 20 seen in this email chain reference to criminal 21 prosecutions, a number of court cases, a number 22 of cases where witness statements have been 23 requested, et cetera. Given that the Post 24 Office were prosecuting subpostmasters, 25 assistants and others during this period, do you 136 1 think it was right for Fujitsu to hold off 2 telling the Post Office until they had a fix or 3 a workaround? 4 A. Typically what we would do was make sure we had 5 the right information, because if you go to -- 6 if we go to Post Office and say "We have an 7 issue", you get a barrage of questions. It 8 appears -- only appears, I've got no facts from 9 this email -- that there's a workaround. So to 10 go to the Post Office and say "We have this 11 issue, here's a workaround", is better than "We 12 have an issue", and then get 20 emails in the 13 next two or three days of "What are you going to 14 do? What can we do?" Et cetera. 15 I don't know the scenario of what Geoff was 16 managing at the time. It appears that he wanted 17 to review the workaround so that we don't go to 18 Post Office and propose something that's 19 inappropriate, or it's -- and it's robust. 20 Q. Was it a culture within Fujitsu at the time to 21 do as you say: to hold off telling the Post 22 Office until you found the workaround? 23 A. No, until we have a clear definition. In Derby, 24 we didn't hold off telling the Post Office but 25 we had a clear understanding there was an issue 137 1 and what the issue was. 2 Q. There seems to be a pretty clear identification 3 of what the issue is here, just not the 4 workaround? 5 A. It says, "To determine if a proposed workaround 6 is adequate." 7 So I think Geoff -- I'm making an 8 assumption, you had need to ask Geoff Butts 9 this -- it appears to me from this email he's 10 reviewing a workaround, and before he discusses 11 it with Post Office, he doesn't want to propose 12 something that then we have to withdraw. 13 Q. Given that it might impact on whether 14 information that's being provided in a criminal 15 court is correct or not, do you think it was 16 right to hold off telling POL at that stage? 17 A. If it was going to be within a day to go back 18 with them, that could have been an appropriate 19 action. 20 Q. Is there any indication there, it says "We've 21 been looking at this today", but is there any 22 indication that it was going to happen then and 23 there? The reason I ask is -- 24 A. You need to ask Geoff. 25 Q. -- is it significant that there was the meeting 138 1 to discuss the acceptance of HNG-X shortly 2 after? Do you know if this issue was raised 3 with the Post Office before acceptance? 4 A. I would expect it would have done, with any 5 either fix or workaround, yeah. 6 Q. You would have expected it to have. Did it? 7 A. I don't recall. 8 Q. Would it have been wrong if it hadn't been 9 raised with the Post Office before acceptance? 10 A. Yes. 11 Q. Do you think Gareth Jenkins understood the 12 significance in relation to the provision of 13 that kind of information to a criminal court? 14 A. I can only go by what he's written there, but he 15 appears to have understood that. 16 Q. From your discussions with him, and knowledge of 17 him as a person, do you believe that he would 18 have known? 19 A. Technically he's very able. I don't know his 20 knowledge of proceedings and evidence and that. 21 Q. And how about Penny Thomas? 22 A. As I say, I don't recall Penny. I can see, as 23 you've pointed out what her role is, but she was 24 in a different department within the account. 25 Q. What about those assurances about the integrity 139 1 of the system that we've been discussing all day 2 today? Do you think they were undermined by 3 this issue? 4 A. So the integrity of the cash account and 5 integrity of the statement of record of what 6 happened at a branch, this issue does not 7 impact. 8 Q. But it impacts on information that's purporting 9 to show the integrity of the system, or a system 10 that has integrity? 11 A. It shows a management information system has 12 duplicate records, yes. 13 Q. Quickly, can we go to FUJ00097080. This is the 14 final email in that chain, 24 June, or the final 15 email we have in that chain, 24 June. In fact 16 that's the same one as I've just been showing 17 you. 18 Can we go back, please, to POL00033100. 19 This is the acceptance report we've already 20 looked at, 7 July. Shortly after this email 21 chain, you're presumably back in the country by 22 this stage? 23 A. Yes. 24 Q. Can we look at page 4. So version 1 was issued 25 to the Acceptance Board on 23 June. That's the 140 1 bottom of that page. If we scroll down to 2 7 July: 3 "This has been updated to reflect changes 4 presented to the board arising from new 5 Acceptance Incidents", et cetera. 6 I believe the meeting actually may have 7 taken place on 29 June. Were you at the 8 meeting? 9 A. If it was on 29 June, I would not have been. 10 Q. Are you sure that you weren't at this meeting? 11 Would you have been invited to this meeting? 12 A. I would have been invited, yes. 13 Q. Do you recall the final meeting where HNG-X was 14 accepted? I may be wrong on the date of the 15 29th. 16 A. I remember attending a meeting in Old Street for 17 the acceptance, yes. 18 Q. At that meeting, before this was published and 19 finalised, did you, or anybody at Fujitsu, to 20 the best of your knowledge, raise that very 21 issue that we've just been discussing? 22 A. I don't recall. 23 Q. Because Fujitsu were holding off telling POL 24 until they may have a workaround. Do you recall 25 anyone informing the Post Office either that 141 1 there was a problem with the ARQ returns, or 2 that that had been resolved? 3 A. I don't recall. It may have been fixed by that 4 point. 5 Q. Do you think it would have been right to have 6 told the Post Office, before HNG-X was accepted, 7 about that issue? 8 A. Yes, but if it was fixed it wouldn't be part of 9 the acceptance report. 10 Q. Well, it may be a concern, though. I mean we've 11 seen that email from the Post Office about 12 another defect, and I can't remember whether -- 13 I used "angry", you used "forceful" as the word. 14 Wouldn't raising this issue with the Post Office 15 have received a forceful response of some sort 16 that may call into question acceptance? 17 Something of such significance? 18 A. I don't recall. As I say, Geoff obviously led 19 this. I don't know whether it was fixed 20 immediately, how quickly it was fixed. The 21 email trail which I read -- which, when I saw 22 this, seemed to suggest there was a fix. But -- 23 Q. To use -- I think I'll borrow words from 24 Mr Stein -- that email chain we saw, it looks as 25 though it's a bit of a cover-up at that time. 142 1 Would I be wrong to say that? 2 A. I think you would. 3 Q. Not telling POL about a significant issue at 4 this period, during this period, sounds quite 5 significant. Am I wrong on that? 6 A. I don't know whether POL was told or not. 7 Q. But there's an email chain that said "Hold off 8 telling them"? 9 A. For a day or two. 10 Q. Does it say "a day or two"? 11 A. It says "until tomorrow or Monday." That's what 12 I read, sorry. I may have misread that. 13 Q. It says, "I'll come back to you either today or 14 Monday on progress." That's Guy Wilkerson to 15 Geoff. 16 A. Or Geoff to Guy Wilkerson, I would have thought. 17 Q. "Guy/Penny, please don't make any communication 18 on this issue for the moment. We've been 19 looking at this today and are waiting to 20 determine if a proposed workaround is adequate. 21 I'll come back to you either tomorrow or Monday 22 on progress." 23 A. Okay. 24 Q. So it's not saying that they will come back to 25 POL on that issue. 143 1 A. No. 2 Q. "Tomorrow or Monday". 3 Very finally, the very final document I'll 4 take you to -- 5 SIR WYN WILLIAMS: Before you do that, Mr Blake, is 6 there any evidence that you have seen, written 7 evidence, which shows whether or not POL was 8 told? 9 MR BLAKE: Are you asking Mr D'Alvarez? 10 SIR WYN WILLIAMS: I'm asking both of you, in truth. 11 You as Counsel to the Inquiry, in case there was 12 a document one way or the other, but I rather 13 doubt, given the nature of your questions. 14 So I'll ask you, Mr D'Alvarez: have you ever 15 seen a document which demonstrates that POL was 16 told of this issue? 17 A. I don't recall seeing one. 18 SIR WYN WILLIAMS: Right, fine. So if there is such 19 a document, no doubt either POL or Fujitsu will 20 disclose it. 21 MR BLAKE: Yes. Ms Page actually has direct 22 involvement in this matter to some extent, and 23 she may be able to briefly address you on that. 24 SIR WYN WILLIAMS: Well, I don't need to be 25 addressed. If there is a document, no doubt -- 144 1 which exists now -- the relevant party will 2 disclose it. And I'm just asking them to do it, 3 basically. 4 MR BLAKE: Thank you. 5 Final document is FUJ00097140. This is 6 an email from Graham Welsh, the Migration 7 Governance Manager. He says: 8 "For the avoidance of any doubt, please 9 ensure this is cascaded to all the teams. All 10 the hard work and long hours of the past months 11 has achieved a position whereby we've entered 12 into rollout." 13 So 30 June 2010 is confirming that they had 14 entered into rollout. So shortly after this 15 email chain. Is that your recollection? 16 A. What's the date of this? 17 Q. 30 June 2010. 18 A. I mean, I don't recall the dates but I do recall 19 we entered into -- 20 Q. Are you aware of anything occurring between 21 24 June and 29 June in regards to that issue, in 22 communication with the Post Office? 23 A. I don't recall anything, no. 24 MR BLAKE: Thank you, sir. Those are all of my 25 questions. I think Ms Page -- no, she doesn't. 145 1 Mr Whittam has a question or two. 2 Questioned by MR WHITTAM 3 MR WHITTAM: Very shortly, please, sir. 4 SIR WYN WILLIAMS: Of course. 5 MR WHITTAM: Richard Whittam on behalf of Fujitsu. 6 I'd be grateful if we could have FUJ00094959 on 7 the screen, please. 8 Just returning to this, the response you 9 drafted to what Mr Blake described as the "angry 10 email" from Dave Smith. When it comes up, if we 11 look at the fifth paragraph down. 12 "It's already been agreed and acknowledged 13 that our prime objective is to minimise 14 disruption and the ability of the branch network 15 to trade. I am acutely aware of the support and 16 tolerance of the postmasters and postmistresses 17 in the HNG-X pilot, and recognise their 18 patience. To that end, for the last four weeks 19 we have been proactively calling the PMs in the 20 pilot to gauge their experiences and to provide 21 feedback on any issues they had logged to date. 22 We have also attended the recent NFSP meeting in 23 conjunction with POL programme." 24 You drafted that. Was that accurate? 25 A. I drafted that with Dave Keeling, yes. 146 1 Q. Were you aware of the contact with postmasters? 2 A. So contact with postmasters, I know there was 3 a programme led by Graham Welsh, who was 4 leading, with Geoff Butts, the Hypercare Team to 5 get feedback. Myself and Mark Burley visited 6 post offices and spent a morning with post 7 offices that were having particular 8 difficulties, to get their feedback and to give 9 explanations as to what we're doing, and moving 10 forward. So I personally visited post offices 11 with Mr Burley. 12 Q. In relation to attending the NFSP meeting, in 13 conjunction with POL programme, did POL have any 14 involvement in that connection? 15 A. When you say "the POL programme", I believe that 16 reference is to Mark Burley. So Mark had 17 a standing -- I don't know what he called it, 18 but he had a committee where he had 19 representatives from I think both Crown Offices 20 and postmasters. And we attended those 21 meetings, as well, as -- to give explanations on 22 some of the issue that they are seeing what our 23 approach is to resolution of those issues, so 24 yes. 25 MR WHITTAM: Thank you. 147 1 Thank you, sir. That's all I ask. 2 MR BLAKE: Sir, Mr Moloney has a very brief question 3 as well. 4 SIR WYN WILLIAMS: Yes. 5 Questioned by MR MOLONEY 6 MR MOLONEY: Thank you, sir. 7 Mr D'Alvarez, I represent subpostmaster Core 8 Participants. I've just got one technical 9 matter to ask you about, and that's the swapping 10 out of a base unit in a branch. You may 11 remember that this morning Mr Blake asked you 12 about issues in the Warwick branch in 13 February 2010. At a meeting on 11 February it 14 was noted that there were new BTS issues in 15 Warwick and the base unit was to be swapped out, 16 but that was noted as not setting a precedent, 17 yes? Why were you concerned that it shouldn't 18 set a precedent? 19 A. I wasn't -- when you say -- it's a case of if 20 you have issues as an office, and because it was 21 in the pilot site, we wanted to make sure, belt 22 and braces, that once we've done that, we'd got 23 a clean view as to what we'd done. I don't 24 recall the exact nature of why they chose to 25 swap out the base unit. But if we state what we 148 1 should do every time there's an issue, we should 2 make sure we swap out the base unit, that would 3 be very disruptive and also costly, and it will 4 serve no real purpose. But I do not recall the 5 decision making to do that, other than it looks, 6 in what I read this morning, like a -- and as 7 a belt and braces, let's make sure they have 8 a clean start so we eradicate every possibility 9 of what could have caused the fault. 10 Q. So, in essence, it would be important, in order 11 to be able to establish what may have gone 12 wrong? 13 A. Yeah, and it was in the pilot. So in the pilot 14 we put so much additional support, so much 15 additional more analysis. We want to know 16 everything that's going on to assure ourselves 17 that when we roll out to a large number of 18 offices, that it's as robust as we can make the 19 solution and the system. 20 Q. Would you, as a matter of principle, make 21 a distinction between the pilot and, as it were, 22 ordinary operations, post-rollout, in that 23 regard? 24 A. Yes. A pilot is a test phase. 25 Q. But would, if there were problems experienced 149 1 within an office, would substituting the process 2 of swapping it out be a solution that one would 3 try? 4 A. It depends what the issue is. 5 Q. So in some circumstances it might be right and 6 in others it might not? 7 A. Correct. 8 Q. In some circumstances you might want to 9 investigate whether or not there was an issue 10 with the process before -- well, after you'd 11 swapped it out? 12 A. Correct. 13 MR MOLONEY: Thank you, Mr D'Alvarez. 14 SIR WYN WILLIAMS: Is that it, Mr Blake? 15 MR BLAKE: That is it, but I have an answer, and 16 a document will be going up on CP View to Core 17 Participants in due course. But I've certainly 18 just seen correspondence from 30 June, so that 19 is the day the email was sent about having 20 completed rollout, or entered into rollout, from 21 Fujitsu to the Post Office. So there was a -- 22 so we have the email from Graham Welsh that we 23 saw of 30 June at 9.30 in the morning about 24 having entered a position whereby they have 25 entered rollout. There is an email that will be 150 1 coming in due course -- it's unlikely to be an 2 email for this witness, it will likely be an 3 email for other witnesses -- of the same day in 4 which the Post Office are told by Fujitsu of the 5 ARQ problem. 6 SIR WYN WILLIAMS: Right. Fine. Well, no doubt 7 that will be put in evidence when appropriate. 8 So thank you, Mr D'Alvarez, for giving 9 evidence for the second time to the Inquiry. 10 Thank you, too, for being flexible enough to 11 give evidence today without having a proper 12 lunch break, and I extend that thanks to 13 everyone else in the room. That was primarily 14 for my convenience, so I should own up to that. 15 I have inconvenienced you all, but I'm grateful 16 to you all. 17 So we'll see you again at 10.00 tomorrow, 18 yes, Mr Blake? 19 MR BLAKE: Yes. Thank you very much. 20 SIR WYN WILLIAMS: Thank you very much. 21 (2.08 pm) 22 (The hearing adjourned until 10.00 am 23 the following day) 24 25 151 I N D E X ALAN D'ALVAREZ (sworn) ...............................1 Questioned by MR BLAKE ........................1 Questioned by MR WHITTAM ....................146 Questioned by MR MOLONEY ....................148 152