Publications
-
Cardenas, Juan Diego; Falzari, Massimiliano; Zullich, Marco & Sabatelli, Matthia
(2025).
Upside-Down Reinforcement Learning for More Interpretable Optimal Control.
Proceedings of the International Conference on Agents and Artificial Intelligence (ICAART).
ISSN 2184-3589.
1,
p. 859–869.
doi:
10.5220/0013299400003890.
Full text in Research Archive
Show summary
Model-Free Reinforcement Learning (RL) algorithms either learn how to map states to expected rewards or search for policies that can maximize a certain performance function. Model-Based algorithms instead, aim to learn an approximation of the underlying model of the RL environment and then use it in combination with planning algorithms. Upside-Down Reinforcement Learning (UDRL) is a novel learning paradigm that aims to learn how to predict actions from states and desired commands. This task is formulated as a Supervised Learning problem and has successfully been tackled by Neural Networks (NNs). In this paper, we investigate whether function approximation algorithms other than NNs can also be used within a UDRL framework. Our experiments, performed over several popular optimal control benchmarks, show that tree-based methods like Random Forests and Extremely Randomized Trees can perform just as well as NNs with the significant benefit of resulting in policies that are inherently more interpretable than NNs, therefore paving the way for more transparent, safe, and robust RL.
-
Hosamo, Haidar; Imran, Aksa; Cardenas, Juan Diego; Svennevig, Paul Ragnar; svidt, kjeld & Nielsen, Henrik Kofoed
(2022).
A Review of the Digital Twin Technology in the AEC-FM Industry.
Advances in Civil Engineering.
ISSN 1687-8086.
2022.
doi:
10.1155/2022/2185170.
Full text in Research Archive
Show summary
The Architecture, Engineering, Construction, and Facility Management (AEC-FM) industry is increasingly affected by digital technologies that monitor sensor network data and control automation systems. Advances in digital technologies like Digital Twin offer a high-level representation of buildings and their assets by integrating the physical and digital world. This paper examines patterns, gaps, and trends in the AEC-FM sector and contributes to digitalization and automation solutions for building management. This work covers a broad range of research topics, from intelligent information management of complex models to building information management and the interaction of building systems, where researchers are increasingly interested in using the Digital Twin to manage their information and in developing new research lines focused on data interchange and the interoperability of building information modeling (BIM) and facility management (FM). After a complete bibliometric search of several databases and following selection criteria, 77 academic publications about the Digital Twin application in the AEC-FM industry were labeled and clustered accordingly. This study analyzed in detail the concept of key technologies, including “Digital Twin in Facility Lifecycle Management,” “Digital Twin-Information Integration Standards,” “Digital Twin-Based Occupants Centric Building Design,” “Digital Twin-Based Predictive Maintenance,” “Semantic Digital Twin for Facility Maintenance,” and “Digital Twin-Based Human Knowledge.” The findings show that information standardization is the first major hurdle that must be overcome before the actual use of Digital Twin can be realized in the AEC-FM industry. Based on that, this paper provides a conceptual framework of Digital Twin for building management as a starting point for future research.
-
Cardenas, Juan Diego; Elnourani, Mohamed Gafar Ahmed & Beferull-Lozano, Baltasar
(2022).
Forecasting Aquaponic Systems Behaviour With Recurrent Neural Networks Models.
Proceedings of the Northern Lights Deep Learning Workshop.
3.
doi:
10.7557/18.6236.
Full text in Research Archive
Show summary
Aquaponic systems provide a reliable solution to grow vegetables while cultivating fish (or other aquatic organisms) in a controlled environment. The main advantage of these systems compared with traditional soil-based agriculture and aquaculture installations is the ability to produce fish and vegetables with low water consumption. Aquaponics requires a robust control system capable of optimizing fish and plant growth while ensuring a safe operation. To support the control system, this work explores the design process of Deep Learning models based on Recurrent Neural Networks to forecast one hour of pH values in small-scale industrial Aquaponics. This implementation guides us through the machine learning life-cycle with industrial time-series data, i.e. data acquisition, pre-processing, feature engineering, architecture selection, training, and model verification.
-
Cardenas, Juan Diego; Beferull-Lozano, Baltasar; Elnourani, Mohamed & Romero, Daniel
(2022).
RISK-AWARE PARTICLE FILTERING FOR STATE ESTIMATION IN RECIRCULATING AQUACULTURE SYSTEMS,
2022 56th Asilomar Conference on Signals, Systems, and Computers.
IEEE (Institute of Electrical and Electronics Engineers).
ISSN 9781665459068.
p. 829–833.
doi:
10.1109/IEEECONF56349.2022.10051993.
Full text in Research Archive
Show summary
We design a sequential non-linear risk-aware estimator based on particle filtering to compute estimates and approximate the system state posterior distribution. For this purpose, we consider the risk given by the expected variance of the squared error between the system state and the estimate conditioned on the observations. We compare the proposed estimator with existing risk-neutral estimators in terms of error variance, mean squared error, and execution time per iteration. The comparison is carried out using a simulation of a Recirculating Aquaculture System, a case study for non-linear critical systems, where the performance in estimation for low probability events becomes an important aspect. Our simulation results demonstrate a competitive estimation performance while ensuring a lower risk. 1 1 The reader can find the code for the proposed estimators in github.com/uia-wisenet/Risk_Aware_MMSE_PF.
View all works in NVA
Published
Sep. 11, 2025 5:53 AM
- Last modified
Sep. 11, 2025 5:53 AM