-
Discovering Decision Manifolds to Assure Trusted Autonomous Systems
Authors:
Matthew Litton,
Doron Drusinsky,
James Bret Michael
Abstract:
Developing and fielding complex systems requires proof that they are reliably correct with respect to their design and operating requirements. Especially for autonomous systems which exhibit unanticipated emergent behavior, fully enumerating the range of possible correct and incorrect behaviors is intractable. Therefore, we propose an optimization-based search technique for generating high-quality…
▽ More
Developing and fielding complex systems requires proof that they are reliably correct with respect to their design and operating requirements. Especially for autonomous systems which exhibit unanticipated emergent behavior, fully enumerating the range of possible correct and incorrect behaviors is intractable. Therefore, we propose an optimization-based search technique for generating high-quality, high-variance, and non-trivial data which captures the range of correct and incorrect responses a system could exhibit. This manifold between desired and undesired behavior provides a more detailed understanding of system reliability than traditional testing or Monte Carlo simulations. After discovering data points along the manifold, we apply machine learning techniques to quantify the decision manifold's underlying mathematical function. Such models serve as correctness properties which can be utilized to enable both verification during development and testing, as well as continuous assurance during operation, even amidst system adaptations and dynamic operating environments. This method can be applied in combination with a simulator in order to provide evidence of dependability to system designers and users, with the ultimate aim of establishing trust in the deployment of complex systems. In this proof-of-concept, we apply our method to a software-in-the-loop evaluation of an autonomous vehicle.
△ Less
Submitted 26 February, 2024; v1 submitted 12 February, 2024;
originally announced February 2024.
-
Bayesian Disturbance Injection: Robust Imitation Learning of Flexible Policies for Robot Manipulation
Authors:
Hanbit Oh,
Hikaru Sasaki,
Brendan Michael,
Takamitsu Matsubara
Abstract:
Humans demonstrate a variety of interesting behavioral characteristics when performing tasks, such as selecting between seemingly equivalent optimal actions, performing recovery actions when deviating from the optimal trajectory, or moderating actions in response to sensed risks. However, imitation learning, which attempts to teach robots to perform these same tasks from observations of human demo…
▽ More
Humans demonstrate a variety of interesting behavioral characteristics when performing tasks, such as selecting between seemingly equivalent optimal actions, performing recovery actions when deviating from the optimal trajectory, or moderating actions in response to sensed risks. However, imitation learning, which attempts to teach robots to perform these same tasks from observations of human demonstrations, often fails to capture such behavior. Specifically, commonly used learning algorithms embody inherent contradictions between the learning assumptions (e.g., single optimal action) and actual human behavior (e.g., multiple optimal actions), thereby limiting robot generalizability, applicability, and demonstration feasibility. To address this, this paper proposes designing imitation learning algorithms with a focus on utilizing human behavioral characteristics, thereby embodying principles for capturing and exploiting actual demonstrator behavioral characteristics. This paper presents the first imitation learning framework, Bayesian Disturbance Injection (BDI), that typifies human behavioral characteristics by incorporating model flexibility, robustification, and risk sensitivity. Bayesian inference is used to learn flexible non-parametric multi-action policies, while simultaneously robustifying policies by injecting risk-sensitive disturbances to induce human recovery action and ensuring demonstration feasibility. Our method is evaluated through risk-sensitive simulations and real-robot experiments (e.g., table-sweep task, shaft-reach task and shaft-insertion task) using the UR5e 6-DOF robotic arm, to demonstrate the improved characterisation of behavior. Results show significant improvement in task performance, through improved flexibility, robustness as well as demonstration feasibility.
△ Less
Submitted 7 November, 2022;
originally announced November 2022.
-
Deep Koopman with Control: Spectral Analysis of Soft Robot Dynamics
Authors:
Naoto Komeno,
Brendan Michael,
Katharina Küchler,
Edgar Anarossi,
Takamitsu Matsubara
Abstract:
Soft robots are challenging to model and control as inherent non-linearities (e.g., elasticity and deformation), often requires complex explicit physics-based analytical modeling (e.g., a priori geometric definitions). While machine learning can be used to learn non-linear control models in a data-driven approach, these models often lack an intuitive internal physical interpretation and representa…
▽ More
Soft robots are challenging to model and control as inherent non-linearities (e.g., elasticity and deformation), often requires complex explicit physics-based analytical modeling (e.g., a priori geometric definitions). While machine learning can be used to learn non-linear control models in a data-driven approach, these models often lack an intuitive internal physical interpretation and representation, limiting dynamical analysis. To address this, this paper presents an approach using Koopman operator theory and deep neural networks to provide a global linear description of the non-linear control systems. Specifically, by globally linearising dynamics, the Koopman operator is analyzed using spectral decomposition to characterises important physics-based interpretations, such as functional growths and oscillations. Experiments in this paper demonstrate this approach for controlling non-linear soft robotics, and shows model outputs are interpretable in the context of spectral analysis.
△ Less
Submitted 14 October, 2022;
originally announced October 2022.
-
Disturbance-Injected Robust Imitation Learning with Task Achievement
Authors:
Hirotaka Tahara,
Hikaru Sasaki,
Hanbit Oh,
Brendan Michael,
Takamitsu Matsubara
Abstract:
Robust imitation learning using disturbance injections overcomes issues of limited variation in demonstrations. However, these methods assume demonstrations are optimal, and that policy stabilization can be learned via simple augmentations. In real-world scenarios, demonstrations are often of diverse-quality, and disturbance injection instead learns sub-optimal policies that fail to replicate desi…
▽ More
Robust imitation learning using disturbance injections overcomes issues of limited variation in demonstrations. However, these methods assume demonstrations are optimal, and that policy stabilization can be learned via simple augmentations. In real-world scenarios, demonstrations are often of diverse-quality, and disturbance injection instead learns sub-optimal policies that fail to replicate desired behavior. To address this issue, this paper proposes a novel imitation learning framework that combines both policy robustification and optimal demonstration learning. Specifically, this combinatorial approach forces policy learning and disturbance injection optimization to focus on mainly learning from high task achievement demonstrations, while utilizing low achievement ones to decrease the number of samples needed. The effectiveness of the proposed method is verified through experiments using an excavation task in both simulations and a real robot, resulting in high-achieving policies that are more stable and robust to diverse-quality demonstrations. In addition, this method utilizes all of the weighted sub-optimal demonstrations without eliminating them, resulting in practical data efficiency benefits.
△ Less
Submitted 9 May, 2022;
originally announced May 2022.
-
PSA-GAN: Progressive Self Attention GANs for Synthetic Time Series
Authors:
Jeha Paul,
Bohlke-Schneider Michael,
Mercado Pedro,
Kapoor Shubham,
Singh Nirwan Rajbir,
Flunkert Valentin,
Gasthaus Jan,
Januschowski Tim
Abstract:
Realistic synthetic time series data of sufficient length enables practical applications in time series modeling tasks, such as forecasting, but remains a challenge. In this paper we present PSA-GAN, a generative adversarial network (GAN) that generates long time series samples of high quality using progressive growing of GANs and self-attention. We show that PSA-GAN can be used to reduce the erro…
▽ More
Realistic synthetic time series data of sufficient length enables practical applications in time series modeling tasks, such as forecasting, but remains a challenge. In this paper we present PSA-GAN, a generative adversarial network (GAN) that generates long time series samples of high quality using progressive growing of GANs and self-attention. We show that PSA-GAN can be used to reduce the error in two downstream forecasting tasks over baselines that only use real data. We also introduce a Frechet-Inception Distance-like score, Context-FID, assessing the quality of synthetic time series samples. In our downstream tasks, we find that the lowest scoring models correspond to the best-performing ones. Therefore, Context-FID could be a useful tool to develop time series GAN models.
△ Less
Submitted 28 March, 2022; v1 submitted 2 August, 2021;
originally announced August 2021.
-
Bayesian Disturbance Injection: Robust Imitation Learning of Flexible Policies
Authors:
Hanbit Oh,
Hikaru Sasaki,
Brendan Michael,
Takamitsu Matsubara
Abstract:
Scenarios requiring humans to choose from multiple seemingly optimal actions are commonplace, however standard imitation learning often fails to capture this behavior. Instead, an over-reliance on replicating expert actions induces inflexible and unstable policies, leading to poor generalizability in an application. To address the problem, this paper presents the first imitation learning framework…
▽ More
Scenarios requiring humans to choose from multiple seemingly optimal actions are commonplace, however standard imitation learning often fails to capture this behavior. Instead, an over-reliance on replicating expert actions induces inflexible and unstable policies, leading to poor generalizability in an application. To address the problem, this paper presents the first imitation learning framework that incorporates Bayesian variational inference for learning flexible non-parametric multi-action policies, while simultaneously robustifying the policies against sources of error, by introducing and optimizing disturbances to create a richer demonstration dataset. This combinatorial approach forces the policy to adapt to challenging situations, enabling stable multi-action policies to be learned efficiently. The effectiveness of our proposed method is evaluated through simulations and real-robot experiments for a table-sweep task using the UR3 6-DOF robotic arm. Results show that, through improved flexibility and robustness, the learning performance and control safety are better than comparison methods.
△ Less
Submitted 7 November, 2022; v1 submitted 25 March, 2021;
originally announced March 2021.
-
Improving Task-Parameterised Movement Learning Generalisation with Frame-Weighted Trajectory Generation
Authors:
Aran Sena,
Brendan Michael,
Matthew Howard
Abstract:
Learning from Demonstration depends on a robot learner generalising its learned model to unseen conditions, as it is not feasible for a person to provide a demonstration set that accounts for all possible variations in non-trivial tasks. While there are many learning methods that can handle interpolation of observed data effectively, extrapolation from observed data offers a much greater challenge…
▽ More
Learning from Demonstration depends on a robot learner generalising its learned model to unseen conditions, as it is not feasible for a person to provide a demonstration set that accounts for all possible variations in non-trivial tasks. While there are many learning methods that can handle interpolation of observed data effectively, extrapolation from observed data offers a much greater challenge. To address this problem of generalisation, this paper proposes a modified Task-Parameterised Gaussian Mixture Regression method that considers the relevance of task parameters during trajectory generation, as determined by variance in the data. The benefits of the proposed method are first explored using a simulated reaching task data set. Here it is shown that the proposed method offers far-reaching, low-error extrapolation abilities that are different in nature to existing learning methods. Data collected from novice users for a real-world manipulation task is then considered, where it is shown that the proposed method is able to effectively reduce grasping performance errors by ${\sim30\%}$ and extrapolate to unseen grasp targets under real-world conditions. These results indicate the proposed method serves to benefit novice users by placing less reliance on the user to provide high quality demonstration data sets.
△ Less
Submitted 4 March, 2019;
originally announced March 2019.
-
Wearable Embroidered Muscle Activity Sensing Device for the Human Upper Leg
Authors:
R. B. Ribas Manero,
J. Grewal,
B. Michael,
A. Shafti,
K. Althoefer,
J. Ll. Ribas Fernandez,
M. J. Howard
Abstract:
Within the last decade, running has become one of the most popular physical activities in the world. Although the benefits of running are numerous, there is a risk of Running Related Injuries (RRI) of the lower extremities. Electromyography (EMG) techniques have previously been used to study causes of RRIs, but the complexity of this technology limits its use to a laboratory setting. As running is…
▽ More
Within the last decade, running has become one of the most popular physical activities in the world. Although the benefits of running are numerous, there is a risk of Running Related Injuries (RRI) of the lower extremities. Electromyography (EMG) techniques have previously been used to study causes of RRIs, but the complexity of this technology limits its use to a laboratory setting. As running is primarily an outdoors activity, this lack of technology acts as a barrier to the study of RRIs in natural environments. This study presents a minimally invasive wearable muscle sensing device consisting of jogging leggings with embroidered surface EMG (sEMG) electrodes capable of recording muscle activity data of the quadriceps group. To test the use of the device, a proof of concept study consisting of $N=2$ runners performing a set of $5km$ running trials is presented in which the effect of running surfaces on muscle fatigue, a potential cause of RRIs, is evaluated. Results show that muscle fatigue can be analysed from the sEMG data obtained through the wearable device, and that running on soft surfaces (such as sand) may increase the likelihood of suffering from RRIs.
△ Less
Submitted 15 February, 2016;
originally announced February 2016.
