Research
- Deep Reinforcement Learning
- Multi-Agent Reinforcement Learning
- Multi-Agent Collaboration
Publications
-
Brekke, Vegard; Berge, Erlend Odd; Dybdahl, Eirik; Singh, Jayant & Tyapin, Ilya
(2026).
ROS 2-Driven Navigation and Sensor Platform for Quadruped Robots.
Robotics.
15(4).
doi:
10.3390/robotics15040070.
Full text in Research Archive
Show summary
This paper presents an open-source ROS 2 navigation and sensor platform for quadruped robots, demonstrated on Boston Dynamics Spot in a laboratory environment. The platform integrates SLAM Toolbox for mapping and localisation, Navigation2 with MPPI and Smac Hybrid-A* for global path planning, and a frontier-based autonomous exploration module with practical handling of unreachable frontiers. The paper validates and verifies current, open-source algorithms deployed on off-the-shelf hardware. A greedy wavefront-based frontier selection method is presented that prioritizes Time-to-Closest-Viable-Frontier (TCVF) by terminating the search as soon as a feasible frontier is identified. On a real robot dataset replayed across five goal scenarios, the method reduces median selection latency from 94.31 ms to 51.08 ms (95th percentile: 109.54 ms to 56.99 ms), corresponding to a 1.85-times improvement in compute time compared to a standard implementation. The system also employs Zenoh middleware and Foxglove for remote monitoring and control, enabling flexible, high-bandwidth operation. The platform, including configuration files and launch scripts, is released openly to support future research and deployment on quadruped robots.
-
Singh, Jayant; Zhou, Jing; Beferull-Lozano, Baltasar; Yan, Shijun & Khan, Shalman
(2024).
Haptic Teleoperation Framework for Learning Task Space Fine Manipulation Skills.
IEEE Conference on Industrial Electronics and Applications.
doi:
10.1109/ICIEA61579.2024.10665270.
Full text in Research Archive
Show summary
This paper introduces a framework for teaching robot fine-manipulation skills by combining haptic teleoperation with Probabilistic Movement Primitives (ProMPs). Addressing the challenges of executing precise tasks such as snap-joint manipulation, our framework utilises ROS2 alongside a cartesian compliance controller for task space operation. This setup facilitates the intuitive guidance of robots by human operators, capturing the detailed movements critical for fine-manipulation tasks. Through experiments with the UR5e 6-DOF collaborative robot arm and a 3DSystems Touch haptic device, we illustrate the accurate representation and replication of the human demonstrated tasks.
-
Singh, Jayant; Zhou, Jing & Beferull-Lozano, Baltasar
(2023).
Enhancing Multi-Agent Reinforcement Learning: Set Function Approximation and Dynamic Policy.
Proceedings of the Annual Conference of the IEEE Industrial Electronics Society (IECON).
ISSN 1553-572X.
doi:
10.1109/IECON51785.2023.10312543.
Full text in Research Archive
Show summary
While Deep Learning based methods can solve complex problems by employing Neural Networks to act as powerful
function approximators, they often suffer from inflexibility in
terms of deployment beyond the training scenario and also
include irrelevant data priors in the form of an ordered array of
input values. This problem is quite evident in the field of MultiAgent Reinforcement Learning (MARL), where most research
covers methods that are trained on a fixed number of agents,
restricted by the fixed size of the input vector. In this paper,
we argue that this is not a reasonable assumption, both in
terms of the inflexible amount of environmental information
and the restrictive nature of the structure of the information.
We explore DeepSets and Set Transformers as two powerful set
function approximators to address the problem of cardinality
invariance and permutation invariance in the observation space of
a reinforcement learning agent. We explain Set-Input Reinforcement Learning (SIRL) in detail and evaluate the performance of
the DeepSets and Set Transformer methods through simulated
experiments on a challenging multi-agent environment, that
otherwise yields sub-optimal policies through traditional function
approximation approaches. We demonstrate that both DeepSets
and Set Transformer based encoders scale well to increasing the
number of agents from training to evaluation.
-
Kumar, Ravi; Dale, Jørgen; Singh, Jayant & Zhou, Jing
(2023).
Design and Development of an Anthropomorphic Gripper for Service Robotics and Prosthetic Applications.
International Conference on Control, Mechatronics and Automation.
ISSN 2837-5114.
p. 233–238.
doi:
10.1109/ICCMA59762.2023.10374949.
Full text in Research Archive
-
Singh, Jayant; Zhou, Jing; Beferull-Lozano, Baltasar & Tyapin, Ilya
(2022).
Learning Cooperative Multi-Agent Policies with Multi-Channel Reward Curriculum Based Q-Learning.
Proceedings of the Annual Conference of the IEEE Industrial Electronics Society (IECON).
ISSN 1553-572X.
doi:
10.1109/IECON49645.2022.9969046.
Full text in Research Archive
Show summary
Multi-Agent Reinforcement Learning (MARL) algorithms based on the Centralised Training Decentralised Execution (CTDE) approach have seen a great deal of interest in recent years. Most of the recent works focus on a specific class of environments built on the StartCraft Multi-Agent Challenge (SMAC) environment suite. However, experiments with the PettingZoo multi-particle environments show poor performance in an interesting subset of tasks. In this paper, the nature of these environments and the reward structures are analyzed. It shows that poor performance in these tasks is not due to a lack of representation power in the individual Q function or mixing functions, but rather a result of convergence to a suboptimal equilibrium of dual channel rewards and the issue of agent-reward decoupling that can be a common problem for many MARL environments. We present reward curriculum-based versions of QMIX and VDN as a solution to these problems and compare their results with the standard algorithms. The results show a clear performance gain in terms of a common cumulative reward metric.
View all works in NVA
Published
Apr. 16, 2024 10:50 AM
- Last modified
Sep. 17, 2024 6:05 PM
