How safe is safe enough?

Safety is key. It is generally accepted that safety for MASS must be as good as, or better than, human driven systems. So, a key element of assuring MASS is providing evidence it can navigate safely and in compliance with the International Regulations on Prevention of Collisions at Sea (COLREGS) without a human onboard.

Rapid developments in robotics, AI/ML and communications technology are making Maritime Autonomy ever more capable. Ship owners and operators are keen to exploit the safety, efficiency and sustainability benefits that autonomy can bring. However, the regulatory environment is complex, evolving and varies depending on area of operation. With this apparent dichotomy between technology and regulation how can we bridge the gap to provide assurance that Authorities require to certify MASS and enable operators to get on and utilise the technology?

The process for certifying human mariners involves, to varying degrees, assessment in simulators (synthetic environments). But when it’s a machine making the decisions using sensor feeds and AI how do we assess its conformance with the COLREGS?

The answer is surprisingly similar to the human case if the right technology and expertise is applied. As trusted, independent and impartial maritime experts, BMT has developed a rigorous Synthetic Environment Assurance Service (SEAS) to test how machines perform in a range of representative shipping scenarios. SEAS combines unrivalled expertise in navigation and COLREGS, with a highly immersive market leading synthetic environment REMBRANDT to provide regulators, classification societies, P&I clubs and owners with independent assurance on the COLREG compliance of MASS.  

Figure 1. Certification process for mariners.

Simulation is used successfully to test autonomy in land and air domains; it is equally applicable to the maritime domain. Indeed, this approach is widely advocated by leading research into safety assurance of automated systems such as the University of Warwick WMG Cross Domain Safety Assurance Framework for Automated Transport Systems, University of York Assuring Autonomy International Programme (AAIP) and the National Physical Laboratory report into the requirements for a virtual test environments for autonomous vessels.

However, to provide credible evidence to support the safety argument the virtual test environment itself needs to reflect the real world as closely as possible. This is where BMT’s REMBRANDT is the perfect tool for the job. The system is a fully scalable Synthetic Environment (SE) supporting DNV Type-Approved Full Mission Bridge Simulation. It is fully federated and easily networked with an open architecture. It can be integrated with tidal and bathymetric data sets and next generation S-100 layers. When combined with BMT’s TUFLOW, a world-leading environmental hydraulic modelling tool, it provides very high-resolution vessel hydrodynamics making the REMBRANDT system the most versatile and capable SE commercially available.

To take realism to the ultimate level REMBRANDT can be run in real time with live real-world contacts, it can operate in concert with remote autonomous or remote-controlled platforms, live conventional assets and simulated participants in the same SE, or it may be used as a stand-alone SE supporting a wide range of activities. REMBRANDT provides a sufficiently realistic and accurate model of the real world to give confidence in results. 

As an example, the defence sector is one of the user groups understandably very keen to operationalise MASS to exploit the advantages it brings. BMT is working with the UK MoD to help solve the complex issues around assurance and certification of MASS. With our iterative test and evaluation Synthetic Environment Assurance Service we are the only MoD recognised organisation for COLREG assurance testing of MASS.

Figure 2. BMT Synthetic Environment Assurance Service iterative process.

The iterative SEAS process involves five key stages:

• Tailor assessment for vessel type & task;
• Test machine’s decision making & COLREG compliance in synthetic environment;
• Analyse results, highlight areas of strong & weak performance;
• Report provides compelling body of evidence to enable appropriate authority to define safe operating envelope;
• Certify based on a defined operating envelope.

Central to the SEAS process is the formulation test scenarios that are appropriate to the intended use case. There is no point testing a system in congested inshore waters when its intended use is offshore. Once the system limitations are known a safe operating envelope can be defined. The safety argument can then be formulated around known tolerable events ie technical or operational events for which there is a designed response that keeps the system within its operational envelope[1].

So, to circle back to the original question ‘how safe is safe enough’, this is bounded by the operating envelope. Understanding your vessel and system performance is critical to building the evidence to support the definition of the safe operating envelope.

This allows you to manage the risks and put appropriate mitigations in place. Being able to tailor the operating envelope based on performance and adapt it for specific circumstances such as mode of operation, level of autonomy, task, geographic areas, vessel size etc allows you to prepare for live trials and real-world operations.

Figure 3. SEAS test and development cycle to expanding the operating envelope.

With respect to the autonomous system as a whole and within the context of what is described as a Tolerable event: the system can be considered safe enough if a failure to make a decision with due regard to the observance of good seamanship is made apparent to the human operator in ample time for them to intervene.

For this to be the case you need:

• A clear understanding of system performance
• A defined safe operating envelope which allows system failure to be tolerable

In summary, synthetic environment assurance allows the COLREG performance of the system to be understood in a way which allows an appropriate operating envelope to be defined. This means immature technology can be tested, developed certified and operated safely with the minimum of constraint. 

Want to know more?  Talk to one of our experts today.

[1] ISO TS 23860: Vocabulary related to autonomous ship systems