Per-Arne Andersen

• Halenka, Vojtech; Amini, Mohammadreza; Andersen, Per-Arne; Granmo, Ole-Christoffer; Jiao, Lei & Kantarci, Burak (2026). Analyzing Backdoor Vulnerabilities in Interpretable Machine Learning for Wireless Jamming Detection Using Tsetlin Machines. IEEE Transactions on Network Science and Engineering (IEEE TNSE). 13, p. 11869–11886. doi: 10.1109/TNSE.2026.3727545. Full text in Research Archive Show summary

  Interpretable machine learning is increasingly sought in next-generation cellular networks, where trust, accountability, and robustness are essential and jamming remains a persistent threat. Federated learning enables collaborative jamming detection without sharing raw in-phase and quadrature data, while Tsetlin Machines provide a logic-based option for transparent, resource-constrained deployments. However, the security implications of interpretability in federated wireless environments remain poorly understood. We introduce a backdoor threat model for interpretable federated jamming detectors, showing how Tsetlin Machine clause structure can be exploited by a compromised client to induce targeted detection failures. In a federated framework with weighted Coalesced Tsetlin Machines and Krum aggregation, we analyze clause-based vulnerability to spectrogram-domain trigger perturbations and formulate a multi-objective problem balancing trigger effectiveness, model deviation, and aggregation selection. We further show that clause-informed trigger initialization lowers adversarial optimization effort, revealing a tension between transparency and robustness. Evaluation on 60,000 spectrograms from a 5G NR synchronization-signal testbed shows that optimized triggers can exceed 70% trial-level triggered false-negative rates in optimizer benchmarks, reaching approximately 90% in best-case configurations; supplemental multi-round federated evaluation shows that trial-level and global triggered false-negative rate (TFR) can diverge substantially (gaps from -31.4 to +3.0 percentage points across evaluated partitions), with no sustained global TFR≥70% after the last malicious selection round. These results highlight the need for defense-aware design of explainable models in next-generation networks.

• Grundetjern, Morten; Andersen, Per-Arne & Goodwin, Morten (2025). Synthetic Personas: Enhancing Demographic Response Simulation through Large Language Models and Genetic Algorithms. International Journal on Cybernetics & Informatics. ISSN 2320-8430. 14(2), p. 21–41. doi: 10.5121/ijci.2025.140202. Full text in Research Archive Show summary

  Understanding diverse demographic groups presents a significant challenge in market research. In this paper, we introduce a novel system that integrates large language models with genetic algorithms to create synthetic personas capable of generating feedback that approximates real-world human responses. Our experimental evaluation demonstrates that synthetic personas not only exhibit age-differentiated technology usage patterns consistent with documented trends but also benefit from genetic algorithm optimization, which improves response accuracy from 60.4% to 78.5% on training questions and from 62.6% to 68.8% on hidden questions—outperforming human estimators. Moreover, the optimized personas achieve a 51.1% better correspondence with actual income distributions compared to random profiles. This approach makes it possible to rapidly generate feedback without requiring participants, facilitates iterative follow-ups, and systematically enhances demographic representativeness.

• Voigt, Jakob Michael; Grundetjern, Morten; Andersen, Per-Arne & Goodwin, Morten (2025). DESS: Dimensional-PDFs for Embedding Space Sampling. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 16301, p. 3–17. doi: 10.1007/978-3-032-11402-0_1. Full text in Research Archive Show summary

  Probabilistic sampling in machine learning models is a technique for introducing beneficial randomness and variability during output generation. It enhances output diversity and creativity, particularly in tasks where slight variations can lead to novel and interesting results. Large Language Models (LLMs), for instance, use this technique to generate varied, contextually appropriate responses that mirror the flexibility of human language. This paper presents a novel method for training a neural network to output the defining parameters of one Probability Density Function (PDF) for each dimension in an embedding space. Our approach, Dimensional-PDFs for Embedding Space Sampling (DESS), leverages these PDFs at inference time to sample from an embedding space, offering a new approach to probabilistic sampling in model outputs. Our method employs simple loss functions and a multi-step loss computation to learn both mean and variance parameters without requiring sampling during training, making it computationally efficient and easy to integrate with existing architectures. We validate DESS through controlled experiments on synthetic datasets, demonstrating accurate approximations of ground-truth distribution parameters across varying Gaussian target distributions. The framework has potential applications in large-vocabulary models where it could enable significant parameter reductions while maintaining calibrated uncertainty estimates. We present DESS using normal distributions, though the approach can be extended to other distribution types for specific sampling requirements.

• Brådland, Henrik; Goodwin, Morten; Andersen, Per-Arne; Nossum, Alexander & Gupta, Aditya (2025). A New HOPE: Domain-agnostic Automatic Evaluation of Text Chunking. SIGIR '25: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. doi: 10.1145/3726302.3729882. Full text in Research Archive Show summary

  Document chunking fundamentally impacts Retrieval-Augmented Generation (RAG) by determining how source materials are segmented before indexing. Despite evidence that Large Language Models (LLMs) are sensitive to the layout and structure of retrieved data, there is currently no framework to analyze the impact of different chunking methods. In this paper, we introduce a novel methodology that defines essential characteristics of the chunking process at three levels: intrinsic passage properties, extrinsic passage properties, and passages-document coherence. We propose HOPE (Holistic Passage Evaluation), a domain-agnostic, automatic evaluation metric that quantifies and aggregates these characteristics. Our empirical evaluations across seven domains demonstrate that the HOPE metric correlates significantly (𝜌 > 0.13) with various RAG performance indicators, revealing contrasts between the importance of extrinsic and intrinsic properties of passages. Semantic independence between passages proves essential for system performance with a performance gain of up to 56.2% in factual correctness and 21.1% in answer correctness. On the contrary, traditional assumptions about maintaining concept unity within passages show minimal impact. These findings provide actionable insights for optimizing chunking strategies, thus improving RAG system design to produce more factually correct responses.

• Jeeru, Sindhusha; Omslandseter, Rebekka Olsson; Andersen, Per-Arne; Morrison, Aiden James; Sokolova, Nadezda & Granmo, Ole-Christoffer [Show all 7 contributors for this article] (2025). MixCTME: A Mixture of Convolutional Tsetlin Machine Experts Using Diverse Spectrogram Visualizations for Jamming Signal Classification. IEEE Internet of Things Journal. ISSN 2327-4662. 12(22), p. 47667–47678. doi: 10.1109/jiot.2025.3606366. Full text in Research Archive Show summary

  global navigation satellite systems (GNSSs) are vulnerable to jamming, which can degrade or even disable their services by interfering with signal reception. Such interference may interrupt GNSS positioning, navigation, timing, and communication functions. Therefore, robust jamming detection strategies are essential to ensure continuous and reliable services. Many existing jamming detection approaches use closed box machine learning (ML) models, which lack transparency and accuracy. In this article, we propose a novel mixture of convolutional Tsetlin machine experts (MixCTME) approach, using diverse spectrogram visualizations to enhance the accuracy and interpretability of jamming signal classification. To facilitate this, we collected raw in-phase and quadrature (IQ) data and named it the RealRFI dataset, which corresponds to ten different jamming classes. The data were collected through multiple radio-frequency interference (RFI) monitoring stations deployed at SINTEF companies across Europe and Scandinavia. Next, diverse visualizations were generated using short time Fourier transform (STFT), dominant frequency, fast Fourier transform (FFT), power spectral density, and standard deviation. These visualizations were binarized using the Otsu thresholding technique, and the binarized spectrograms were fed to the MixCTME model. Our method utilizes a combination of distinct convolutional Tsetlin machine (CTM) experts and employs a novel nonlinear technique, referred to as confidence-based gating, for weighting the experts’ inputs to make the final decision. We compared our method against state-of-the-art approaches via the self-collected and labeled dataset. Through experiments, our model achieved an accuracy of 90.70% on the collected data and 99.46% on a benchmark dataset, outperforming the state-of-the-art approaches. Additionally, we highlight the trustworthiness, scalability, and interpretability of the proposed approach, presenting a promising solution for accurate and interpretable jamming classification.

• Belaid, Mohamed-Bachir; Sharma, Jivitesh; Lei, Jiao; Granmo, Ole-Christoffer; Andersen, Per-Arne & Yazidi, Anis (2025). Generalized Convergence Analysis of Tsetlin Automaton Based Algorithms: A Probabilistic Approach to Concept Learning. In Walsh, Toby; Shah, Julie & Kolter, Zico (Ed.), Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence. AAAI Press. ISSN 9781577358978. p. 15524–15531. doi: 10.1609/aaai.v39i15.33704. Full text in Research Archive
• Jeeru, Sindhusha; Lei, Jiao; Andersen, Per-Arne & Granmo, Ole-Christoffer (2025). Interpretable Rule-based Architecture for GNSS Jamming Signal Classification. IEEE Sensors Journal. ISSN 1530-437X. 25(10), p. 17942–17959. doi: 10.1109/JSEN.2025.3558966. Full text in Research Archive Show summary

  Jamming is a fatal threat to a global navigation satellite system (GNSS), and an efficient anti-jamming system relies on successful classification and identification of jamming types to respond effectively. The existing solutions suffer either from poor accuracy or lack of interpretability, and they are prone to learning simple statistical correlations rather than more fundamental and general relationships. In this study, we propose a novel approach to classify GNSS jamming signals as intentional or unintentional. The approach introduces a new standard deviation-based denoising method, which makes it possible to use the logical rule-based architecture of the convolutional Tsetlin machine (CTM) for interpretable jamming signal analysis. CTM is a recently developed algorithm that solves complex classification problems using conjunctive propositional formulas through a team of Tsetlin automata (TA). Unlike traditional opaque models based on deep learning, our approach goes beyond classification and provides a human-level interpretation of features. This interpretation capability allows a deeper comprehension of the characteristics and underlying patterns of the jamming signals, significantly easing the decision-making process. Furthermore, the CTM approach is also evaluated with different Booleanization techniques. Through experiments, we show that the proposed approach with CTM achieves an F1-score of 98.7%, on the collected dataset, which is superior to the state-of-the-art.

• Grundetjern, Morten; Andersen, Per-Arne; Goodwin, Morten & Borgersen, Karl Audun (2024). NER Explainability Framework: Utilizing LIME to Enhance Clarity and Robustness in Named Entity Recognition. In Bramer, Max & Stahl, Frederic (Ed.), Artificial Intelligence XLI: 44th SGAI International Conference on Artificial Intelligence, AI 2024. Springer. ISSN 9783031779145. p. 3–15. doi: 10.1007/978-3-031-77915-2_1. Full text in Research Archive Show summary

  Abstract. Named Entity Recognition (NER) is crucial for many Nat- ural Language Processing (NLP) applications, yet current models of- ten lack explainability and robustness, particularly against lexical varia- tions. While methods like Local Interpretable Model-agnostic Explana- tions (LIME) have enhanced explainability in text classification, their application to sequence-based tasks like NER remains an open chal- lenge. This paper introduces the NER Explainability Framework (NEF), a novel approach that transforms the sequence-based NER task into a multi-classification problem. This transformation enables the application of LIME to any NER architecture. NEF provides precise, interpretable insights into model decisions by highlighting which parts of the input text most influence tagging decisions, significantly advancing explainability in NER. To address the critical gap in assessing model resilience, we also present the Misspelling Robustness Score (MRS), a new metric designed to quantify NER model resilience against lexical variations, particularly misspellings. MRS offers a comprehensive evaluation of model perfor- mance in the face of spelling variations and errors. We apply NEF to four widely used NER models (SpaCy, Flair, Stanza, and RoBERTa) using the CoNLL-2003 dataset, revealing critical patterns and weaknesses in model behavior. Our method enables targeted improvements, demonstrated by fine-tuning the SpaCy model to achieve a significant 17.5% increase in robustness against misspellings. These findings lay the groundwork for developing more robust, accurate, and interpretable NER systems, pro- viding practical strategies to address existing challenges in NLP. The universal applicability of NEF across NER architectures and the insights provided by MRS offers valuable tools for researchers and practitioners in advancing the field of NER.

• Brådland, Henrik; Goodwin, Morten; Andersen, Per-Arne & Nossum, Alexander (2024). PlanBERT: From Messy Zonal Plans to Informative Vector Embeddings. In Bramer, Max & Stahl, Frederic (Ed.), Artificial Intelligence XLI: 44th SGAI International Conference on Artificial Intelligence, AI 2024. Springer. ISSN 9783031779145. p. 175–188. doi: 10.1007/978-3-031-77915-2_13. Full text in Research Archive Show summary

  Text embedding models trained on vast web-scraped corpus generalize well to daily language. However, they often fall short when applied in specialized domains that require precise language and foreign terms, like law and medicine. This gap highlights the necessity for data-efficient methodologies to fine-tune these models for narrow-domain applications. This paper introduces PlanBERT, a new approach for enabling data-efficient domain adaptation and fine-tuning of embedding models. The approach builds on self-supervised contrastive pre-training, synthetic training data generated by large language models (LLMs), and decorrelation of embedding features. The paper also introduces the term “informative vector embeddings” to adjust the training objectives to incentivise more analytics-friendly embeddings and demonstrate that PlanBERT can learn the domain language of zonal plans and outperform larger and more complex state-of-the-art models in challenging real-world zonal plan tasks.

• Andersen, Per-Arne; Goodwin, Morten & Granmo, Ole-Christoffer (2024). Towards safe and sustainable reinforcement learning for real-time strategy games. Information Sciences. ISSN 0020-0255. 679. doi: 10.1016/j.ins.2024.120980. Full text in Research Archive
• Halenka, Vojtech; Kadhim, Ahmed Khalid; Paul, Clarke; Bhattarai, Bimal; Saha, Rupsa & Granmo, Ole-Christoffer [Show all 8 contributors for this article] (2024). Exploring Effects of Hyperdimensional Vectors for Tsetlin Machines. In NN, NN (Eds.), 2024 International Symposium on the Tsetlin Machine (ISTM). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798331504984. doi: 10.1109/ISTM62799.2024.10931546. Full text in Research Archive Show summary

  Tsetlin machines (TMs) have been successful in several application domains, operating with high efficiency on Boolean representations of the input data. However, Booleanizing complex data structures such as sequences, graphs, images, signal spectra, chemical compounds, and natural language is non-trivial. In this paper, we propose a hypervector (HV) based method for expressing arbitrarily large sets of concepts associated with any input data. Using a hyperdimensional space to build vectors drastically expands the capacity and flexibility of the TM. We demonstrate how images, chemical compounds, and natural language text are encoded according to the proposed method, and how the resulting HV -powered TM can achieve significantly higher accuracy and faster learning on well-known benchmarks. Our results thus open up a new research direction for TMs, namely how to expand and exploit the benefits of operating in hyperspace, including new booleanization strategies, optimization of TM inference and learning, as well as new TM applications.

• Thantharate, Aayushi Vinod; Goodwin, Morten; Andersen, Per-Arne & Gupta, Aditya (2024). Towards Robust Road Quality Detection Using Different Detection Models. In Maglogiannis, Ilias; Iliadis, Lazaros; MacIntyre, J.; Avlonitis, M & Papaleonidas, A (Ed.), Artificial Intelligence Applications and Innovations: 20th IFIP WG 12.5 International Conference, AIAI 2024, Corfu, Greece, June 27–30, 2024, Proceedings, Part III. Springer. ISSN 9783031632181. p. 125–138. doi: 10.1007/978-3-031-63219-8_10. Full text in Research Archive
• Laursen, Rune Alexander; Alo, Peshang; Goodwin, Morten & Andersen, Per-Arne (2023). Distinct Sequential Models for Inference Boosting. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 14381, p. 198–203. doi: 10.1007/978-3-031-47994-6_15. Full text in Research Archive Show summary

  Type 1 Diabetes (T1D) is a chronic disease where the body is unable to regulate the Blood Glucose Level (BGL), leading to severe health consequences if not regulated. Accurate BGL predictions can enable better disease management and improve treatment decisions. However, predicting future BGLs is a complex problem due to the inherent complexity and variability of the human body. This paper investigates using a new technique to outperform a State-of-the-Art (SotA) Convolutional Recurrent Neural Network (CRNN) model by forecasting BGLs on the same dataset. The problem is structured, and the data is preprocessed as a multivariate multi-step time series. The Distinct Sequential Models for Inference Boosting (DSMIB) technique is used, which manages missing data and counters potential issues from other techniques by employing both a Long Short-Term Memory (LSTM) model and a Transformer-based model together. The experimental results show that this technique reduces the Root Mean Squared Error (RMSE) by approximately 14.28% when predicting the BGL 30 min in the future compared to the SotA model. This improvement highlights the potential of this approach to assist diabetes patients with effective disease management.

• Evensen, Vetle Nesland; Henriksen, Gabriel Bergman; Melhus, Sondre; Olsen, Ole Steine; Haugen, Kristina & Dolmen, Dag [Show all 11 contributors for this article] (2023). ReFrogID: Pattern Recognition for Pool Frog Identification Using Deep Learning and Feature Matching. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 14381, p. 365–376. doi: 10.1007/978-3-031-47994-6_33. Full text in Research Archive Show summary

  The global decline in amphibian populations is a pressing issue, with numerous species facing the threat of extinction. One such species is the pool frog, Pelophylax lessonae, whose Norwegian population has experienced a significant long-term decline since monitoring began in 1996. This decline has pushed the species to the verge of local extinction. A substantial knowledge gap in the species’ biology hinders the proposal and evaluation of effective management actions. Consequently, there is a pressing need for efficient techniques to gather data on population size and composition. Recent advancements in Machine Learning (ML) and Deep Learning (DL) have shown promising results in various domains, including ecology and evolution. Current research in these fields primarily focuses on species modeling, behavior detection, and identity recognition. The progress in mobile technology, ML, and DL has equipped researchers across numerous disciplines, including ecology, with innovative data collection methods for building knowledge bases on species and ecosystems. This study addresses the need for systematic field data collection for monitoring endangered species like the pool frog by employing deep learning and image processing techniques. In this research, a multidisciplinary team developed a technique, termed ReFrogID, to identify individual frogs using their unique abdominal patterns. Utilizing RGB images, the system operates on two main principles: (1) a DL algorithm for automatic segmentation achieving AP@89.147, AP50@99.123, and AP75@98.942, and (2) pattern matching via local feature detection and matching methods. A new dataset, pelophylax_lessonae, addresses the identity recognition problem in pool frogs. The effectiveness of ReFrogIDis validated by its ability to identify frogs even when human experts fail. Source code is available at here.

• Jyhne, Sander; Andersen, Per-Arne; Goodwin, Morten & Oveland, Ivar (2023). A Contrastive Learning Scheme with Transformer Innate Patches. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 14381, p. 103–114. doi: 10.1007/978-3-031-47994-6_8. Full text in Research Archive Show summary

  This paper presents Contrastive Transformer (CT), a contrastive learning scheme using the innate transformer patches. CT enables existing contrastive learning techniques, often used for image classification, to benefit dense downstream prediction tasks such as semantic segmentation. The scheme performs supervised patch-level contrastive learning, selecting the patches based on the ground truth mask, subsequently used for hard-negative and hard-positive sampling. The scheme applies to all patch-based vision-transformer architectures, is easy to implement, and introduces minimal additional memory footprint. Additionally, the scheme removes the need for huge batch sizes, as each patch is treated as an image. We apply and test CT for the case of aerial image segmentation, known for low-resolution data, large class imbalance, and similar semantic classes. We perform extensive experiments to show the efficacy of the CT scheme on the ISPRS Potsdam aerial image segmentation dataset. Additionally, we show the generalizability of our scheme by applying it to multiple inherently different transformer architectures. Ultimately, the results show a consistent increase in mean Intersection-over-Union (IoU) across all classes.

• Granmo, Ole-Christoffer; Andersen, Per-Arne; Lei, Jiao; Zhang, Xuan; Blakely, Christian Dallas & Berge, Geir Thore [Show all 7 contributors for this article] (2023). Learning Minimalistic Tsetlin Machine Clauses with Markov Boundary-Guided Pruning, 2023 International Symposium on the Tsetlin Machine (ISTM). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798350344776. doi: 10.1109/ISTM58889.2023.10454914. Full text in Research Archive
• Jyhne, Sander; Jacobsen, Jørgen Åsbu; Goodwin, Morten & Andersen, Per-Arne (2023). DeNISE: Deep Networks for Improved Segmentation Edges. IFIP Advances in Information and Communication Technology. ISSN 1868-4238. 675, p. 81–89. doi: 10.1007/978-3-031-34111-3_8. Full text in Research Archive Show summary

  This paper presents Deep Networks for Improved Segmentation Edges (DeNISE), a novel data enhancement technique using edge detection and segmentation models to improve the boundary quality of segmentation masks. DeNISE utilizes the inherent differences in two sequential deep neural architectures to improve the accuracy of the predicted segmentation edge. DeNISE applies to all types of neural networks and is not trained end-to-end, allowing rapid experiments to discover which models complement each other. We test and apply DeNISE for building segmentation in aerial images. Aerial images are known for difficult conditions as they have a low resolution with optical noise, such as reflections, shadows, and visual obstructions. Overall the paper demonstrates the potential for DeNISE. Using the technique, we improve the baseline results with a building IoU of 78.9%.

• Bhattarai, Bimal; Granmo, Ole-Christoffer; Lei, Jiao; Andersen, Per-Arne; Tunheim, Svein Anders & Shafik, Rishad Ahmed [Show all 7 contributors for this article] (2023). Contracting Tsetlin Machine with Absorbing Automata, 2023 International Symposium on the Tsetlin Machine (ISTM). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798350344776. doi: 10.1109/ISTM58889.2023.10455040. Full text in Research Archive
• Drøsdal, Didrik Kallhovd; Grimsmo, Andreas; Andersen, Per-Arne; Granmo, Ole-Christoffer & Goodwin, Morten (2023). Exploring the Potential of Model-Free Reinforcement Learning using Tsetlin Machines, 2023 International Symposium on the Tsetlin Machine (ISTM). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798350344776. doi: 10.1109/ISTM58889.2023.10455080. Full text in Research Archive
• Gunvaldsen, Ole; Thorsen, Henning Blomfeld; Andersen, Per-Arne; Granmo, Ole-Christoffer & Goodwin, Morten (2023). Towards IoT Anomaly Detection with Tsetlin Machines, 2023 International Symposium on the Tsetlin Machine (ISTM). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798350344776. doi: 10.1109/ISTM58889.2023.10455063. Full text in Research Archive
• Anderson, Abraham; Olafarson, Einar Julius; Andersen, Per-Arne & Noori, Nadia Saad (2022). The ODeLIndA Dataset for Field-of-View Obstruction Detection Using Transfer Learning for Real-Time Industrial Applications. In Bramer, Max & Stahl, Frederic (Ed.), Artificial Intelligence XXXIX 42nd SGAI International Conference on Artificial Intelligence, AI 2022. Springer. ISSN 9783031214417. p. 197–210. doi: 10.1007/978-3-031-21441-7_14. Full text in Research Archive Show summary

  Several challenges arise in industrial applications when using machine vision in safety-critical operations, such as field of view (FoV) obstruction, limited visibility of the equipment’s operational perimeter, difficulty obtaining datasets, and high accuracy requirement. This research shows how machine learning methods such as convolutional neural networks (CNN) and transfer learning can tackle these challenges. Transfer learning works well when the source and target domain images are similar, which is the case for identifying the operational conditions and FoV status in a fixed industrial environment where heavy equipment is operating with the aid of a machine vision module. In this work, eight categories describing FoV status and environmental conditions were selected as follows: clean lens/clear view, cracked lens, dirty lens, foggy, frost, dark view, flares, and rainy images. We used 21 616 images for the training set and 9450 for the test and validation. We benchmark the performance of several transfer learning models and networks (i.e., MobileNet, VGG16, VGG19, DenseNet, Inceptionv3, ResNet50, and Xception) against a CNN model/network trained from scratch. The experiments show that MobileNet combined with transfer learning performs best, with 98% accuracy compared to 93% for the traditional CNN model. Furthermore, the model can detect obstructions in real-time using commodity hardware when tested with a video stream. Finally, we provide a novel dataset for multi-class obstruction detection, which can be found here.

• Rasmussen, Ingeborg; Kvalsvik, Sigurd; Andersen, Per-Arne; Aune, Teodor N. & Hagen, Daniel (2022). Development of a Novel Object Detection System Based on Synthetic Data Generated from Unreal Game Engine. Applied Sciences. 12(17). doi: 10.3390/app12178534. Full text in Research Archive Show summary

  This paper presents a novel approach to training a real-world object detection system based on synthetic data utilizing state-of-the-art technologies. Training an object detection system can be challenging and time-consuming as machine learning requires substantial volumes of training data with associated metadata. Synthetic data can solve this by providing unlimited desired training data with automatic generation. However, the main challenge is creating a balanced dataset that closes the reality gap and generalizes well when deployed in the real world. A state-of-the-art game engine, Unreal Engine 4, was used to approach the challenge of generating a photorealistic dataset for deep learning model training. In addition, a comprehensive domain randomized environment was implemented to create a robust dataset that generalizes the training data well. The randomized environment was reinforced by adding high-dynamic-range image scenes. Finally, a modern neural network was used to train the object detection system, providing a robust framework for an adaptive and self-learning model. The final models were deployed in simulation and in the real world to evaluate the training. The results of this study show that it is possible to train a real-world object detection system on synthetic data. However, the models showcase a lot of potential for improvements regarding the stability and confidence of the inference results. In addition, the paper provides valuable insight into how the number of assets and training data influence the resulting model.

• Andersen, Per-Arne; Goodwin, Morten & Granmo, Ole-Christoffer (2022). CaiRL: A High-Performance Reinforcement Learning Environment Toolkit, 2022 IEEE Conference on Games (CoG). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781665459891. p. 361–368. doi: 10.1109/CoG51982.2022.9893661. Full text in Research Archive Show summary

  This paper addresses the dire need for a platform that efficiently provides a framework for running reinforcement learning (RL) experiments. We propose the CaiRL Environment Toolkit as an efficient, compatible, and more sustainable alternative for training learning agents and propose methods to develop more efficient environment simulations. There is an increasing focus on developing sustainable artificial intelligence. However, little effort has been made to improve the efficiency of running environment simulations. The most popular development toolkit for reinforcement learning, OpenAI Gym, is built using Python, a powerful but slow programming language. We propose a toolkit written in C++ with the same flexibility level but works orders of magnitude faster to make up for Python's inefficiency. This would drastically cut climate emissions. CaiRL also presents the first reinforcement learning toolkit with a built-in JVM and Flash support for running legacy flash games for reinforcement learning research. We demonstrate the effectiveness of CaiRL in the classic control benchmark, comparing the execution speed to OpenAI Gym. Furthermore, we illustrate that CaiRL can act as a drop-in replacement for OpenAI Gym to leverage significantly faster training speeds because of the reduced environment computation time.

• Jyhne, Sander; Goodwin, Morten; Andersen, Per-Arne; Oveland, Ivar; Nossum, Alexander Salveson & Ormseth, Karianne Øydegard [Show all 8 contributors for this article] (2022). MapAI: Precision in BuildingSegmentation. Nordic Machine Intelligence (NMI). 2, p. 1–3. doi: 10.5617/nmi.9849. Full text in Research Archive Show summary

  MapAI: Precision in Building Segmentation is a competition arranged with the Norwegian Artificial Intelligence Research Consortium (NORA) 1 in collaboration with Centre for Artificial Intelligence Research at the University of Agder (CAIR)2 , the Norwegian Mapping Authority3 , AI:Hub4 , Norkart5 , and the Danish Agency for Data Supply and Infrastructure6 . The competition will be held in the fall of 2022. It will be concluded at the Northern Lights Deep Learning conference focusing on the segmentation of buildings using aerial images and laser data. We propose two different tasks to segment buildings, where the first task can only utilize aerial images, while the second must use laser data (LiDAR) with or without aerial images. Furthermore, we use IoU and Boundary IoU [1] to properly evaluate the precision of the models, with the latter being an IoU measure that evaluates the results’ boundaries. We provide the participants with a training dataset and keep a test dataset for evaluation.

• Andersen, Per-Arne; Goodwin, Morten & Granmo, Ole-Christoffer (2021). ORACLE: End-to-End Model Based Reinforcement Learning. In Bramer, Max & Ellis, Richard (Ed.), Artificial Intelligence XXXVIII, 41st SGAI International Conference on Artificial Intelligence. Springer. ISSN 9783030910990. doi: 10.1007/978-3-030-91100-3_4. Full text in Research Archive
• Sharma, Jivitesh; Andersen, Per-Arne; Granmo, Ole-Christoffer & Goodwin, Morten (2021). Deep Q-Learning with Q-Matrix Transfer Learning for Novel Fire Evacuation Environment. IEEE Transactions on Systems, Man & Cybernetics. Systems. ISSN 2168-2216. 51(12), p. 7363–7381. doi: 10.1109/TSMC.2020.2967936. Full text in Research Archive
• Andersen, Per-Arne; Goodwin, Morten & Granmo, Ole-Christoffer (2020). Increasing sample efficiency in deep reinforcement learning using generative environment modelling. Expert Systems. ISSN 0266-4720. 38(7). doi: 10.1111/exsy.12537. Full text in Research Archive Show summary

  Reinforcement learning is a broad scheme of learning algorithms that, in recent times, has shown astonishing performance in controlling agents in environments presented as Markov decision processes. There are several unsolved problems in current state-of-the-art that causes algorithms to learn suboptimal policies, or even diverge and collapse completely. Parts of the solution to address these issues may be related to short- and long-term planning, memory management and exploration for reinforcement learning algorithms. Games are frequently used to benchmark reinforcement learning algorithms as they provide a flexible, reproducible and easy to control environments. Regardless, few games feature the ability to perceive how the algorithm performs exploration, memorization and planning. This article presents The Dreaming Variational Autoencoder with Stochastic Weight Averaging and Generative Adversarial Networks (DVAE-SWAGAN), a neural network-based generative modelling architecture for exploration in environments with sparse feedback. We present deep maze, a novel and flexible maze game-engine that challenges DVAE-SWAGAN in partial and fully observable state-spaces, long-horizon tasks and deterministic and stochastic problems. We show results between different variants of the algorithm and encourage future study in reinforcement learning driven by generative exploration.

• Andersen, Per-Arne; Goodwin, Morten & Granmo, Ole-Christoffer (2020). Towards safe reinforcement-learning in industrial grid-warehousing. Information Sciences. ISSN 0020-0255. 537, p. 467–484. doi: 10.1016/j.ins.2020.06.010. Full text in Research Archive Show summary

  Reinforcement learning has shown to be profoundly successful at learning optimal policies for simulated environments using distributed training with extensive compute capacity. Model-free reinforcement learning uses the notion of trial and error, where the error is a vital part of learning the agent to behave optimally. In mission-critical, real-world environments, there is little tolerance for failure and can cause damaging effects on humans and equipment. In these environments, current state-of-the-art reinforcement learning approaches are not sufficient to learn optimal control policies safely. On the other hand, model-based reinforcement learning tries to encode environment transition dynamics into a predictive model. The transition dynamics describes the mapping from one state to another, conditioned on an action. If this model is accurate enough, the predictive model is sufficient to train agents for optimal behavior in real environments. This paper presents the Dreaming Variational Autoencoder (DVAE) for safely learning good policies with a significantly lower risk of catastrophes occurring during training. The algorithm combines variational autoencoders, risk-directed exploration, and curiosity to train deep-q networks inside ”dream” states. We introduce a novel environment, ASRS-Lab, for research in the safe learning of autonomous vehicles in grid-based warehousing. The work shows that the proposed algorithm has better sample efficiency with similar performance to novel model-free deep reinforcement learning algorithms while maintaining safety during training.

• Andersen, Per-Arne; Goodwin, Morten & Granmo, Ole-Christoffer (2020). Safer Reinforcement Learning for Agents in Industrial Grid-Warehousing, Machine Learning, Optimization, and Data Science: 6th International Conference, LOD 2020. Springer Nature. ISSN 9783030645793. p. 169–180. doi: 10.1007/978-3-030-64580-9_14. Full text in Research Archive
• Andersen, Per-Arne; Goodwin, Morten & Granmo, Ole-Christoffer (2020). Increasing sample efficiency in deep reinforcement learningusing generative environment modelling. Expert Systems. ISSN 0266-4720. doi: 10.1111/exsy.12537. Full text in Research Archive
• Andersen, Per-Arne; Goodwin, Morten & Granmo, Ole-Christoffer (2019). Towards Model-Based Reinforcement Learning for Industry-Near Environments. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 11927 LNAI, p. 36–49. doi: 10.1007/978-3-030-34885-4_3. Full text in Research Archive Show summary

  Deep reinforcement learning has over the past few years shown great potential in learning near-optimal control in complex simulated environments with little visible information. Rainbow (Q-Learning) and PPO (Policy Optimisation) have shown outstanding performance in a variety of tasks, including Atari 2600, MuJoCo, and Roboschool test suite. Although these algorithms are fundamentally different, both suffer from high variance, low sample efficiency, and hyperparameter sensitivity that, in practice, make these algorithms a no-go for critical operations in the industry. On the other hand, model-based reinforcement learning focuses on learning the transition dynamics between states in an environment. If the environment dynamics are adequately learned, a model-based approach is perhaps the most sample efficient method for learning agents to act in an environment optimally. The traits of model-based reinforcement are ideal for real-world environments where sampling is slow and in mission-critical operations. In the warehouse industry, there is an increasing motivation to minimise time and to maximise production. In many of these environments, the literature suggests that the autonomous agents in these environments act suboptimally using handcrafted policies for a significant portion of the state-space. In this paper, we present The Dreaming Variational Autoencoder v2 (DVAE-2), a model-based reinforcement learning algorithm that increases sample efficiency, hence enable algorithms with low sample efficiency function better in real-world environments. We introduce the Deep Warehouse environment for industry-near testing of autonomous agents in logistic warehouses. We illustrate that the DVAE-2 algorithm improves the sample efficiency for the Deep Warehouse compared to model-free methods.

• Andersen, Per-Arne; Goodwin, Morten & Granmo, Ole-Christoffer (2019). Towards Model-based Reinforcement Learning for Industry-near Environments. arXiv.org. ISSN 2331-8422. doi: 10.1007/978-3-030-34885-4_3. Full text in Research Archive
• Sharma, Jivitesh; Andersen, Per-Arne; Granmo, Ole-Christoffer & Goodwin, Morten (2019). Deep Q-Learning with Q-Matrix Transfer Learning for Novel Fire Evacuation Environment. arXiv.org. ISSN 2331-8422. doi: 10.1109/tsmc.2020.2967936. Full text in Research Archive Show summary

  We focus on the important problem of emergency evacuation, which clearly could benefit from reinforcement learning that has been largely unaddressed. Emergency evacuation is a complex task which is difficult to solve with reinforcement learning, since an emergency situation is highly dynamic, with a lot of changing variables and complex constraints that makes it difficult to train on. In this paper, we propose the first fire evacuation environment to train reinforcement learning agents for evacuation planning. The environment is modelled as a graph capturing the building structure. It consists of realistic features like fire spread, uncertainty and bottlenecks. We have implemented the environment in the OpenAI gym format, to facilitate future research. We also propose a new reinforcement learning approach that entails pretraining the network weights of a DQN based agents to incorporate information on the shortest path to the exit. We achieved this by using tabular Q-learning to learn the shortest path on the building model's graph. This information is transferred to the network by deliberately overfitting it on the Q-matrix. Then, the pretrained DQN model is trained on the fire evacuation environment to generate the optimal evacuation path under time varying conditions. We perform comparisons of the proposed approach with state-of-the-art reinforcement learning algorithms like PPO, VPG, SARSA, A2C and ACKTR. The results show that our method is able to outperform state-of-the-art models by a huge margin including the original DQN based models. Finally, we test our model on a large and complex real building consisting of 91 rooms, with the possibility to move to any other room, hence giving 8281 actions. We use an attention based mechanism to deal with large action spaces. Our model achieves near optimal performance on the real world emergency environment.

• Andersen, Per-Arne; Goodwin, Morten & Granmo, Ole-Christoffer (2018). Deep RTS: A Game Environment for Deep Reinforcement Learning in Real-Time Strategy Games, 2018 IEEE Conference on Computational Intelligence and Games (CIG). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781538643594. doi: 10.1109/CIG.2018.8490409. Full text in Research Archive Show summary

  Reinforcement learning (RL) is an area of research that has blossomed tremendously in recent years and has shown remarkable potential for artificial intelligence based opponents in computer games. This success is primarily due to the vast capabilities of convolutional neural networks, that can extract useful features from noisy and complex data. Games are excellent tools to test and push the boundaries of novel RL algorithms because they give valuable insight into how well an algorithm can perform in isolated environments without the real-life consequences. Real-time strategy games (RTS) is a genre that has tremendous complexity and challenges the player in short and long-term planning. There is much research that focuses on applied RL in RTS games, and novel advances are therefore anticipated in the not too distant future. However, there are to date few environments for testing RTS AIs. Environments in the literature are often either overly simplistic, such as microRTS, or complex and without the possibility for accelerated learning on consumer hardware like StarCraft II. This paper introduces the Deep RTS game environment for testing cutting-edge artificial intelligence algorithms for RTS games. Deep RTS is a high-performance RTS game made specifically for artificial intelligence research. It supports accelerated learning, meaning that it can learn at a magnitude of 50 000 times faster compared to existing RTS games. Deep RTS has a flexible configuration, enabling research in several different RTS scenarios, including partially observable state-spaces and map complexity. We show that Deep RTS lives up to our promises by comparing its performance with microRTS, ELF, and StarCraft II on high-end consumer hardware. Using Deep RTS, we show that a Deep Q-Network agent beats random-play agents over 70% of the time. Deep RTS is publicly available at https://github.com/cair/DeepRTS.

• Andersen, Per-Arne; Goodwin, Morten & Granmo, Ole-Christoffer (2018). The dreaming variational autoencoder for reinforcement learning environments. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 11311 LNAI, p. 143–155. doi: 10.1007/978-3-030-04191-5_11. Full text in Research Archive Show summary

  Reinforcement learning has shown great potential in generalizing over raw sensory data using only a single neural network for value optimization. There are several challenges in the current state-of-the-art reinforcement learning algorithms that prevent them from converging towards the global optima. It is likely that the solution to these problems lies in short- and long-term planning, exploration and memory management for reinforcement learning algorithms. Games are often used to benchmark reinforcement learning algorithms as they provide a flexible, reproducible, and easy to control environment. Regardless, few games feature a state-space where results in exploration, memory, and planning are easily perceived. This paper presents The Dreaming Variational Autoencoder (DVAE), a neural network based generative modeling architecture for exploration in environments with sparse feedback. We further present Deep Maze, a novel and flexible maze engine that challenges DVAE in partial and fully-observable state-spaces, long-horizon tasks, and deterministic and stochastic problems. We show initial findings and encourage further work in reinforcement learning driven by generative exploration.

• Andersen, Per-Arne; Goodwin, Morten & Granmo, Ole-Christoffer (2017). FlashRL: A Reinforcement Learning Platform for Flash Games. NIKT: Norsk IKT-konferanse for forskning og utdanning. ISSN 1892-0713. Full text in Research Archive Show summary

  Reinforcement Learning (RL) is a research area that has blossomed tremendously in recent years and has shown remarkable potential in among others successfully playing computer games. However, there only exists a few game platforms that provide diversity in tasks and state- space needed to advance RL algorithms. The existing platforms offer RL access to Atari- and a few web-based games, but no platform fully expose access to Flash games. This is unfortunate because applying RL to Flash games have potential to push the research of RL algorithms. This paper introduces the Flash Reinforcement Learning platform (FlashRL) which attempts to fill this gap by providing an environment for thousands of Flash games on a novel platform for Flash automation. It opens up easy experimentation with RL algorithms for Flash games, which has previously been challenging. The platform shows excellent performance with as little as 5% CPU utilization on consumer hardware. It shows promising results for novel reinforcement learning algorithms.

• Andersen, Per-Arne; Goodwin, Morten & Granmo, Ole-Christoffer (2017). Towards a deep reinforcement learning approach for tower line wars. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 10630 LNAI, p. 101–114. doi: 10.1007/978-3-319-71078-5_8. Full text in Research Archive Show summary

  There have been numerous breakthroughs with reinforcement learning in the recent years, perhaps most notably on Deep Reinforcement Learning successfully playing and winning relatively advanced computer games. There is undoubtedly an anticipation that Deep Reinforcement Learning will play a major role when the first AI masters the complicated game plays needed to beat a professional Real-Time Strategy game player. For this to be possible, there needs to be a game environment that targets and fosters AI research, and specifically Deep Reinforcement Learning. Some game environments already exist, however, these are either overly simplistic such as Atari 2600 or complex such as Starcraft II from Blizzard Entertainment. We propose a game environment in between Atari 2600 and Starcraft II, particularly targeting Deep Reinforcement Learning algorithm research. The environment is a variant of Tower Line Wars from Warcraft III, Blizzard Entertainment. Further, as a proof of concept that the environment can harbor Deep Reinforcement algorithms, we propose and apply a Deep Q-Reinforcement architecture. The architecture simplifies the state space so that it is applicable to Q-learning, and in turn improves performance compared to current state-of-the-art methods. Our experiments show that the proposed architecture can learn to play the environment well, and score 33% better than standard Deep Q-learning—which in turn proves the usefulness of the game environment.

• Andersen, Per-Arne; Haraldseid, Christian Kråkevik; Goodwin, Morten & Yazidi, Anis (2016). Adaptive Task Assignment in Online Learning Environments. In NN, NN (Eds.), Proceedings of the 6th International Conference on Web Intelligence, Mining and Semantics, WIMS 2016. Association for Computing Machinery (ACM). ISSN 9781450340564. p. 1–10. doi: 10.1145/2912845.2912854. Full text in Research Archive

View all works in NVA

• Andersen, Per-Arne & Jiao, Lei (2024). Exploring Efficient Job Shop Scheduling Using Deep Reinforcement Learning. Springer. ISBN 9783031779183. doi: 10.1007/978-3-031-77918-3_20. Full text in Research Archive

View all works in NVA

• Andersen, Per-Arne; Goodwin, Morten & Lyngroth, Maren Thormodsæter (2024). Vi spår kunstig intelligens-året 2024 og fem år frem i tid. [Internet]. Podcast Addict. Full text in Research Archive
• Andersen, Per-Arne & Bjørkeng, Per Kristian (2023). KI lurte menneske trill rundt. Nå krever over 1000 eksperter forbud. [Journal]. Aftenposten. Full text in Research Archive
• Andersen, Per-Arne & Haugen, Martin (2023). Vil bruke kunstig intelligens til å håndtere kriser. [Journal]. Grimstad Adressetidene. Full text in Research Archive
• Andersen, Per-Arne (2023). Uforberedt og ubeskyttet: Norges manglende forsvar mot cyberangrep. Full text in Research Archive
• Lanestedt, Gjermund; Goodwin, Morten & Andersen, Per-Arne (2023). Tid for en (mer) intelligent statsforvaltning? Stat & Styring. ISSN 0803-0103. 33(3), p. 7–14. doi: 10.18261/stat.33.3.2. Full text in Research Archive
• Andersen, Per-Arne (2023). Norge i skyggen av AI-revolusjonen: Cybersikkerhet krever en opptrapping av innsatsen. Full text in Research Archive
• Andersen, Per-Arne (2023). Teknologipositivisme for verdier og demokrati. [Journal]. Stat & Styring. Full text in Research Archive
• Goodwin, Morten & Andersen, Per-Arne (2022). Næringen henger ikke med i kunstig intelligens-kappløpet. Full text in Research Archive
• Andersen, Per-Arne; Goodwin, Morten & Granmo, Ole-Christoffer (2020). CostNet: An End-to-End Framework for Goal-Directed Reinforcement Learning, SGAI 2020: Artificial Intelligence XXXVII. Springer. ISSN 9783030637989. doi: 10.1007/978-3-030-63799-6_7. Full text in Research Archive
• Finnøy, Isak; Liland, Kristian Hovde; Indahl, Ulf Geir & Andersen, Per-Arne (2023). Can Tabular Generative Models generate realistic synthetic Near Infrared spectroscopic data? Norges miljø- og biovitenskapelige universitet (NMBU). Full text in Research Archive Show summary

  In this thesis, we evaluated the performance of two generative models, Conditional Tabular Generative Adversarial Network (CTGAN) and Tabular Variational Autoencoder (TVAE), from the open-source library Synthetic Data Vault (SDV), for generating synthetic Near Infrared (NIR) spectral data. The aim was to assess the viability of these models in synthetic data generation for predicting Dry Matter Content (DMC) in the field of NIR spectroscopy. The fidelity and utility of the synthetic data were examined through a series of benchmarks, including statistical comparisons, dimensionality reduction, and machine learning tasks. The results showed that while both CTGAN and TVAE could generate synthetic data with statistical properties similar to real data, TVAE outperformed CTGAN in terms of preserving the correlation structure of the data and the relationship between the features and the target variable, DMC. However, the synthetic data fell short in fooling machine learning classifiers, indicating a persisting challenge in synthetic data generation. With respect to utility, neither synthetic dataset produced by CTGAN or TVAE could serve as a satisfactory substitute for real data in training machine learning models for predicting DMC. Although TVAE-generated synthetic data showed some potential when used with Random Forest (RF) and K-Nearest Neighbors (KNN) classifiers, the performance was still inadequate for practical use. This study offers valuable insights into the use of generative models for synthetic NIR spectral data generation, highlighting their current limitations and potential areas for future research.

• Andersen, Per-Arne; Goodwin, Morten & Granmo, Ole-Christoffer (2022). Advancements in Safe Deep Reinforcement Learning for Real-Time Strategy Games and Industry Applications. Universitetet i Agder. ISSN 9788284270708. Full text in Research Archive Show summary

  Deep reinforcement learning has attracted considerable attention from industry and academia, among others, because of its success in solving intricate video games and industrial applications. Recent advancements in hardware and computing exponentially increase computational power availability, facilitating deep neural networks training. These networks can learn the RL behavior policy from high-dimensional data and perform significantly better than exact tabular solutions, albeit requiring considerably more computer resources. Games are among the most used applications to assess reinforcement learning (RL) algorithms’ behavioral properties and planning efficiency. They can provide the data structure and volume required to train deep learning models. Specially crafted games can express real-world industry applications to reduce setup costs while drastically increasing reproducibility. RL can improve efficiency in industrial applications where expert systems dominate the scene, reduce manual and potentially dangerous labor. The problem with applied industrial reinforcement learning is that traditional methods learn by trial and error. Because of this, RL agents risk encountering catastrophic events during learning, which can cause damage to humans or equipment. Therefore, using games to train and study safe RL agents is appealing. Real-Time Strategy (RTS) games are especially captivating because of their high dimensional state and action spaces. Furthermore, RTS games share many attributes with industrial and real-world applications, such as simultaneous actions, imperfect information, and system stochasticity. Recent advancements show that model-free RL algorithms can learn superhuman performance in games such as StarCraft II, again using substantial computational power. Therefore, the downside is that these algorithms are expensive and hard to train, making it challenging to use the same methods for industrial applications. There are also substantial state-space complexity gaps in open-source environments. This restricts algorithm evaluation to only a subset of tasks required for operating sufficiently in industry applications.

• Balcik, Burcu; Akca, B; Yucesoy, E; Andersen, Per-Arne; Boey, Lise & Baharmand, Hossein (2021). Project report #2: Mathematical Modelling and Performance Measurement. Universitetet i Agder. ISSN 9788284270364. Full text in Research Archive Show summary

  The vaccine distribution network within a country starts from the main entry point of the country, in which COVID-19 vaccine supplies can arrive in multiple supply waves. In each supply wave, the vaccines, which arrive at a national storage point must be allocated among the smaller administrative units (municipalities or districts) of the country. This is called central vaccine allocation problem (CVAP). In CVAP, the decision makers must consider a variety of factors simultaneously in making allocation decisions such as the type of the vaccine, the size and distribution of the priority groups in the country, and the current status of infection. Moreover, multiple objectives must be considered such as total coverage achieved at national level as well as equity achieved across the municipalities. Cost of logistics can also be a concern, although at a lower criticality, due to implications of decisions on saving human lives. We propose an integer programming model to solve the CVAP. The mathematical model can incorporate different features depending on the decision making needs. The trade-offs among different objectives result in various allocation decisions. In CVAP, decision-makers can prefer whether to achieve a more efficient allocation or more equitable allocation at the beginning of the model. We elaborate on such issues through several illustrative examples.

• Andersen, Per-Arne; Goodwin, Morten & Granmo, Ole-Christoffer (2017). Deep Reinforcement Learning using Capsules in Advanced Game Environments. Universitetet i Agder. Full text in Research Archive Show summary

  Reinforcement Learning (RL) is a research area that has blossomed tremendously in recent years and has shown remarkable potential for artificial intelligence based opponents in computer games. This success is primarily due to vast capabilities of Convolutional Neural Networks (ConvNet), enabling algorithms to extract useful information from noisy environments. Capsule Network (CapsNet) is a recent introduction to the Deep Learning algorithm group and has only barely begun to be explored. The network is an architecture for image classification, with superior performance for classification of the MNIST dataset. CapsNets have not been explored beyond image classification. This thesis introduces the use of CapsNet for Q-Learning based game algorithms. To successfully apply CapsNet in advanced game play, three main contributions follow. First, the introduction of four new game environments as frameworks for RL research with increasing complexity, namely Flash RL, Deep Line Wars, Deep RTS, and Deep Maze. These environments fill the gap between relatively simple and more complex game environments available for RL research and are in the thesis used to test and explore the CapsNet behavior. Second, the thesis introduces a generative modeling approach to produce artificial training data for use in Deep Learning models including CapsNets. We empirically show that conditional generative modeling can successfully generate game data of sufficient quality to train a Deep Q-Network well. Third, we show that CapsNet is a reliable architecture for Deep Q-Learning based algorithms for game AI. A capsule is a group of neurons that determine the presence of objects in the data and is in the literature shown to increase the robustness of training and predictions while lowering the amount training data needed. It should, therefore, be ideally suited for game plays.

View all works in NVA