Martin Holen

• Holen, Martin; Rogovchenko, Svitlana; Noorsumar, Gulshan & Goodwin, Morten (2025). Predicting Vehicle Impact Severity With Deep Neural Network for Reinforcement Learning Based Autonomous Vehicle Simulators. IEEE Conference on Emerging Technologies and Factory Automation. ISSN 1946-0740. p. 1553–1558. doi: 10.1109/ICMLA61862.2024.00239. Full text in Research Archive Show summary

  Road accidents leads to a significant number of deaths in the world, with human error being attributed as one of the roots of the problem. One of the possible alternatives to address the issue is the development of autonomous vehi-cles. Of the many technological advancements in the field of autonomous vehicles, Reinforcement Learning (RL) has shown progress because its methodology is not reliant on human data. However, one of its challenges is the definition of the reward function, which is critical to RL algorithms. In this context, determining the severity of a crash allows for an improvement in the reward function, especially in simulators. In this study, we present a Deep Neural Network (DNN) based model that predicts the crash impact severity in terms of vehicle deformation during the event, along with the de-formation we create a reward strategy for RL- based vehicle simulators. The model is developed based on data generated from finite element simulations replicating a full-frontal vehicle impact against a rigid barrier. We compare the trained model with a variety of NN methods, and different input data. The Neural Network model performs well, supporting the development of a reward system to improve RL-based autonomous vehicle simulators.

• Holen, Martin; Knausgård, Kristian Muri & Goodwin, Morten (2022). Development of a Simulator for Prototyping Reinforcement Learning-Based Autonomous Cars. Informatics. 9(2). doi: 10.3390/informatics9020033. Full text in Research Archive Show summary

  Autonomous driving is a research field that has received attention in recent years, with increasing applications of reinforcement learning (RL) algorithms. It is impractical to train an autonomous vehicle thoroughly in the physical space, i.e., the so-called ’real world’; therefore, simulators are used in almost all training of autonomous driving algorithms. There are numerous autonomous driving simulators, very few of which are specifically targeted at RL. RL-based cars are challenging due to the variety of reward functions available. There is a lack of simulators addressing many central RL research tasks within autonomous driving, such as scene understanding, localization and mapping, planning and driving policies, and control, which have diverse requirements and goals. It is, therefore, challenging to prototype new RL projects with different simulators, especially when there is a need to examine several reward functions at once. This paper introduces a modified simulator based on the Udacity simulator, made for autonomous cars using RL. It creates reward functions, along with sensors to create a baseline implementation for RL-based vehicles. The modified simulator also resets the vehicle when it gets stuck or is in a non-terminating loop, making it more reliable. Overall, the paper seeks to make the prototyping of new systems simple, with the testing of different RL-based systems.

• Holen, Martin; Knausgård, Kristian Muri & Goodwin, Morten (2022). An Evaluation of Autonomous Car Simulators and Their Applicability for Supervised and Reinforcement Learning. In Sanfilippo, Filippo; Granmo, Ole-Christoffer; Yayilgan, Sule Yildirim & Bajwa, Imran Sarwar (Ed.), Intelligent Technologies and Applications: 4th International Conference, INTAP 2021, Grimstad, Norway, October 11–13, 2021, Revised Selected Papers. Springer. ISSN 9783031105241. p. 367–379. doi: 10.1007/978-3-031-10525-8_29. Full text in Research Archive Show summary

  Recent advancements in the field of Machine Learning have sprouted a renewed interest in the area of autonomous cars. Companies use different techniques to develop autonomous cars, including buying several vehicles to develop Advanced Driver Assistance Systems (ADAS), while others use car simulators. Simulators for autonomous cars include a wide variety of different sensors. Some simulators come free or even open-source, while others require to pay for the simulator itself, server time, or each sensor. The quality and focus of each of these vary, with some having LIDAR scanned roads for highly realistic roads, while others have entirely natural and realistic vehicle dynamics.

• Holen, Martin; Saha, Rupsa; Goodwin, Morten; Omlin, Christian Walter Peter & Sandsmark, Knut Eivind (2020). Road Detection for Reinforcement Learning Based Autonomous Car, ICISS 2020: Proceedings of the 2020 The 3rd International Conference on Information Science and System. ACM Publications. ISSN 9781450377256. p. 67–71. doi: 10.1145/3388176.3388199. Full text in Research Archive Show summary

  Human mistakes in traffic often have terrible consequences. The long-awaited introduction of self-driving vehicles may solve many of the problems with traffic, but much research is still needed before cars are fully autonomous. In this paper, we propose a new Road Detection algorithm using online supervised learning based on a Neural Network architecture. This algorithm is designed to support a Reinforcement Learning algorithm (for example, the standard Proximal Policy Optimization or PPO) by detecting when the car is in an adverse condition. Specifically, the PPO gets a penalty whenever the virtual automobile gets stuck or drives off the road with any of its four wheels. Initial experiments show significantly improved results for PPO when using our Road Detection algorithm, as compared to not using any form of Road Detection. In fact, without this detection algorithm, the vehicle often gets into non-terminating loops (for example, driving into the dividers, getting stuck, or driving into a pit).

View all works in NVA

• Holen, Martin (2024). Norwegian AI Cloud User survey results. Full text in Research Archive
• Holen, Martin; Ruud, Else-Line Malene; Warakagoda, Narada Dilp; Granmo, Ole-Christoffer; Engelstad, Paal E. & Knausgård, Kristian Muri (2022). Towards Using Reinforcement Learning for Autonomous Docking of Unmanned Surface Vehicles. doi: 10.1007/978-3-031-08223-8_38. Full text in Research Archive Show summary

  Providing full autonomy to Unmanned Surface Vehicles (USV) is a challenging goal to achieve. Autonomous docking is a subtask that is particularly difficult. The vessel has to distinguish between obstacles and the dock, and the obstacles can be either static or moving. This paper developed a simulator using Reinforcement Learning (RL) to approach the problem. We studied several scenarios for the task of docking a USV in a simulator environment. The scenarios were defined with different sensor inputs and start-stop procedures but a simple shared reward function. The results show that the system solved the task when the IMU (Inertial Measurement Unit) and GNSS (Global Navigation Satellite System) sensors were used to estimate the state, despite the simplicity of the reward function.

View all works in NVA