TY - JOUR
T1 - Continuous Autonomous Ship Learning Framework for Human Policies on Simulation
AU - Kim, Junoh
AU - Park, Jisun
AU - Cho, Kyungeun
N1 - Publisher Copyright:
© 2022 by the authors. Licensee MDPI, Basel, Switzerland.
PY - 2022/2/1
Y1 - 2022/2/1
N2 - Considering autonomous navigation in busy marine traffic environments (including harbors and coasts), major study issues to be solved for autonomous ships are avoidance of static and dynamic obstacles, surface vehicle control in consideration of the environment, and compliance with human-defined navigation rules. The reinforcement learning (RL) algorithm, which demonstrates high potential in autonomous cars, has been presented as an alternative to mathematical algorithms and has advanced in studies on autonomous ships. However, the RL algorithm, through interactions with the environment, receives relatively fewer data from the marine environment. Moreover, the open marine environment causes difficulties for autonomous ships in learning human-defined navigation rules because of excessive degrees of freedom. This study proposes a sustainable, intelligent learning framework for autonomous ships (ILFAS), which helps solve these difficulties and learns navigation rules specified by human beings through neighboring ships. The application of case-based RL enables the participation of humans in the RL learning process through neighboring ships and the learning of human-defined rules. Cases built as curriculums can achieve high learning effects with fewer data along with the RL of layered autonomous ships. The experiment aims at autonomous navigation from a harbor, where marine traffic occurs on a neighboring coast. The learning results using ILFAS and those in an environment where random marine traffic occurs are compared. Based on the experiment, the learning time was reduced by a tenth. Moreover, the success rate of arrival at a destination was higher with fewer controls than the random method in the new marine traffic scenario. ILFAS can continuously respond to advances in ship manufacturing technology and changes in the marine environment.
AB - Considering autonomous navigation in busy marine traffic environments (including harbors and coasts), major study issues to be solved for autonomous ships are avoidance of static and dynamic obstacles, surface vehicle control in consideration of the environment, and compliance with human-defined navigation rules. The reinforcement learning (RL) algorithm, which demonstrates high potential in autonomous cars, has been presented as an alternative to mathematical algorithms and has advanced in studies on autonomous ships. However, the RL algorithm, through interactions with the environment, receives relatively fewer data from the marine environment. Moreover, the open marine environment causes difficulties for autonomous ships in learning human-defined navigation rules because of excessive degrees of freedom. This study proposes a sustainable, intelligent learning framework for autonomous ships (ILFAS), which helps solve these difficulties and learns navigation rules specified by human beings through neighboring ships. The application of case-based RL enables the participation of humans in the RL learning process through neighboring ships and the learning of human-defined rules. Cases built as curriculums can achieve high learning effects with fewer data along with the RL of layered autonomous ships. The experiment aims at autonomous navigation from a harbor, where marine traffic occurs on a neighboring coast. The learning results using ILFAS and those in an environment where random marine traffic occurs are compared. Based on the experiment, the learning time was reduced by a tenth. Moreover, the success rate of arrival at a destination was higher with fewer controls than the random method in the new marine traffic scenario. ILFAS can continuously respond to advances in ship manufacturing technology and changes in the marine environment.
KW - Autonomous ship
KW - Case-based curriculum learning
KW - Learning framework
UR - http://www.scopus.com/inward/record.url?scp=85124180751&partnerID=8YFLogxK
U2 - 10.3390/app12031631
DO - 10.3390/app12031631
M3 - Article
AN - SCOPUS:85124180751
SN - 2076-3417
VL - 12
JO - Applied Sciences (Switzerland)
JF - Applied Sciences (Switzerland)
IS - 3
M1 - 1631
ER -