A Navy official in charge of managing prototype unmanned sea systems this month was optimistic in how large prototype unmanned surface vessels (USVs) testing is going and how the systems can and will be improved.
Capt. Scot Searles, program manager of Unmanned Maritime Systems (PMS 406), said the Overlord USVs being tested in the Pacific were disabled 13 times by on-board safety personnel during the Integrated Battle Problem 23.2 [IBP 23.2] in 2023,” but less than half were related to the autonomy systems.
The Overlord USVs operate with a small crew that can interrupt the autonomy operations by hitting a red button on the bridge to take over if they determine they need to take control. They have been operating in the Pacific Ocean for the past several months in IBP 32.2.
The IBP tested the Ranger and Mariner Overlord USVs as well as the Seahawk and Sea Hunter medium displacement USVs in conjunction with manned vessels.
“The other interventions were due to things like rough seas, or concerns with some other system on the vessel. So that drops the mean time between human interventions down to once every 411 hours, or about once every 17 days. So out of continuous operation, we’re having to actually have a human step in and correct the autonomy less than one time per month,” Searles said during a briefing at the Surface Navy Association symposium on Jan. 10.
Searles said the prototyping work aims to put the vessels in as many diverse situations as they can to explore the autonomy systems’ boundaries and put stress on it.
“We’ve been successful there on this deployment with over 183 COLREGS interactions across the four vessels. Over 80% of the time, the autonomy is handed those encounters exactly as everyone expected. Of the remainder, about half of those were considered questionable – meaning it did it okay, it was an absolutely appropriate response to a COLREGS interaction. It’s just not what we would have expected to do,” he said.
COLREGS are the International Regulations for Prevention of Collision at Sea, the international navigation rules to prevent sea collisions. Searles said it is important the USVs not merely observe COLREGS rules, but operate more as a human would be expected to do since the future fleet is envisioned as a manned-unmanned team.
“So even though it behaved in a way that is compliant with the COLREGS, we want to continue to investigate that and make sure that the vessels were blending in with the manned ships around them.”
He said eight percent, or 14 interactions, were “felt to be inappropriate.”
Searles emphasized his program is particularly interested in those 14 interactions.
“That’s our real learning. That’s what we’re out there for. And that’s why we’re on deployment, is to find those 14 interactions, where we still need to go correct the software. So for us, it’s been an extremely valuable thing…these ships have been operating for years, 10,000 hours, 80,000 miles, but we still managed to find 14 times where we need to go fix the software. So that’s a great learning opportunity for us.”
When pressed to elaborate on those 14 inappropriate interactions, Searles said six were COLREGS interactions with issues like an antenna being blocked and contacts being in the ship’s blind spot. Another type of issue was weather related, where rough seas made a ship see false contacts and then reacted to those contacts in ways that led to increasingly complex unnecessary maneuvering.
One other issue was one vessel had an appropriate interaction with a contact but “occasionally, the things we wanted it to do, like stay in this box or stay relative bearing and distance it struggled to return to that.”
“So that tells us okay, there’s something in that feedback loop that we need to tweak. So that’s kind of a flavor of what those 14 were.”
Searles also noted the Navy is learning how the autonomy perception systems need to be improved to deal with mistakes that make a wave or rain cloud look like a contact the ship has to go around.
He said the Navy expects the perception-contact issues to be fixed in the near term because they are developing two optical systems to help with perception and use artificial intelligence to help the system distinguish false from real contacts and obstacles.
“We hope to have fielded both of those prototype systems on the ships for testing by the end of this year.”
Searles also said during IBP 23.2 the Navy identified 157 human interventions of the autonomy software over thousands of operating hours, leading to one intervention about every 28 hours.
He said 109 of the interventions were related to off-ship command, control, communications, computers, and intelligence (C4I) systems that required on-board personnel to troubleshoot or “matters of human convenience.”
He argued this means there were only 48 human interventions on the USVs during IBP 23.2 that the program needs to act on to improve the system, or once every 42 hours.
Searles said 11 of the 48 interventions were related to hull, mechanical and electrical issues; three due to weather; and three odd issues not easily classified, like a vibration knocking a handset loose to land on a switch.
17 human interventions were the aforementioned weather-related sensing and perception issues, where the sensing system tells the autonomy system what it needs to maneuver around something needlessly.
Searles also noted nine of the 48 human interventions were on-board personnel concerns of the autonomy’s maneuvering decisions.
“So when you combine those two, perception and autonomy, that puts the meantime between human interventions for autonomy at about once every four days. But under good weather conditions, when perception’s not dealing with rain or false contacts, the meantime between human interventions for autonomy, there’s more like once every 12 days. And, by comparison, you go back to those 11 interventions for H,M&E, that’s about once every 17 days.”
He argued this means under current USV maturity the ships can go unmanned for about 11-17 days without an intervention related to autonomy, the C4I or HM&E.
“So we’re really happy with where we’re at a level of maturity and the pace we’re on…we’re not fielding a ship tomorrow, we’re not buying a ship tomorrow, we’re not deploying a ship tomorrow,” Searles said.
He added they “feel great about where the data point is, and the fix we’ve taken today. And we’re right on track for where we need to be in support of those. The good news is, we’re still learning.”
Searles noted IBP 23.2 also allowed the USV program to gather data on ship to ship encounters outside of the autonomy design’s parameters, seeing how the system handles something they did not previously plan for yet.