The Scottish Fire and Rescue Service Command and Control Mobilising System (CCMS)

ICT and Supplier Support Arrangements

37. All three OCs have a process for raising faults and issues relating to their CCMS, which involves submitting a request for assistance with SFRS ICT department as a helpdesk request. There is an OC/ICT Service Level Agreement, and fault response is graded by impact. The prioritising and addressing of faults helps to ensure business continuity at each OC.

38. SFRS ICT department staff rectify faults for issues relating to the SFRS supplied hardware or software. For faults relating to supplied CCMS software or hardware, there are two support contracts in place (Motorola for JOC, and SSS Public Safety for EOC and DOC). Staff from the ICT department liaise with suppliers directly to try and facilitate a remedy for the reported fault. Both SFRS ICT and the support contractors provide support outside normal business hours.

39. The support contracts in place run until July 2023 for JOC, and March 2023 for EOC and DOC.

40. As all three CCMSs use old technology, the Service has some concerns around whether the level of expertise to provide the required support will be available in the long term.

41. The SFRS ICT department is aware of the fragility of the systems and as a method of managing system capacity and resilience system restarts and updates are managed and controlled. System updates are completed with the relevant system support contractor’s advice and involvement. In some cases, data additions and server updates have been held back due to concerns over capacity and fragility of systems.

42. The SFRS has concern over one of the support suppliers’ ability to provide continued service for an ageing product beyond the calendar year of 2023 due to the age of the system. To this end, SSS Public Safety is providing its current support on a ‘reasonable-endeavours only’ basis, reducing assurance and highlighting concerns regarding the fragility of the system in use at EOC and DOC.

Business Continuity

43. All three OCs have local buddy arrangements in place. In the event that one of the OCs cannot take a 999 call, due to spate calls or system failure for example, the call will be diverted by BT to another OC. DOC and EOC are ‘buddies’ as are JOC and Northern Ireland Fire and Rescue Service Control. This arrangement provides assurance that 999 calls can still be received by control staff even in the event of issues referred to above.

44. There is also a UK system in place - Operation Willow Beck1. This is supported by the National Fire Chiefs Council (NFCC) and has been designed in collaboration with the Home Office and key partners to mitigate increased emergency call volumes during spate call conditions. This call distribution process allows BT to route calls to control rooms with capacity, without over-burdening a single buddy control room. SFRS has not experienced outwardly routed calls due to spate conditions, but has handled calls for other FRSs when Operation Willow Beck has been implemented. We were advised by staff that this arrangement also offers resilience if one of the SFRS OCs cannot handle calls due to a system outage.

45. As each of the OCs are operating on different systems, they are not able to mobilise resources to incidents within another SDA at short notice. Typically, a ‘buddy’ OC would log calls and ultimately pass these back to the OC to mobilise resources to. If the system outage was to last for a longer period, all OCs have the ability to implement ‘Crash Laptops’ to be able to mobilise resources in another SDA.

46. The provision of backup servers provide resilience that if one server fails, the back-up option, although not necessarily instantaneously, would enable the functionality of CCMS after a period of down-time.

47. Each OC has a number of laptops to use as back up if their own system fails, or as business continuity if another OC has a total system outage. These fallback arrangements require a level of manual intervention by operators.

48. Each OC has a standby facility at a physically separate location to allow staff to relocate to if the OC site is unusable due to building access and use issues, for example. These locations involve staff travelling to the alternative site, and arrangements are in place to facilitate travel. However, this arrangement would still rely on the use of crash laptops in the event of a total system outage. Equipment available at Standby locations are tested regularly, though due to staffing having been at consistently low levels the weekly testing regime has not always been consistently maintained.

49. The provision of stand-by locations would not be necessary if there was a system of full interoperability between the OCs, but at present the stand-by arrangements provide a degree of resilience for continuity of functions.

50. Each OC has a Business Continuity Plan and Evacuation Plans are in place which are tested regularly. These plans have been tested but there has not been an exercise to incorporate the scenario of a prolonged downtime of any CCMS.

51. The SFRS has procured a disaster recovery option of CCMS, ‘Vision 5’ for use in DOC and EOC, due to the Systel system not being implemented. This was a condition of the ongoing maintenance contract. We were told, “This is to see us through this period until a new system is in place.' However, due to system difficulties, this disaster recovery version is not yet fully installed.

52. The fact that the three OCs use a different CCMS with no interoperability means that in the event of total failure of a single system, the other two systems will not be directly affected by the other CCMS’ fault. Consequently, at any one time, only one OC will be directly impacted by the total failure of a CCMS.