Skip to content

Adaptive Risk Tendency Implicit Quantile Network for Drone Navigation under Partial Observability.

Notifications You must be signed in to change notification settings

tudelft/risk-sensitive-rl

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

15 Commits
 
 
 
 
 
 

Repository files navigation

Risk-sensitive-RL

Code base for paper: Adaptive Risk Tendency: Nano Drone Navigation in Cluttered Environments with Distributional Reinforcement Learning

Alt Text

Dependencies

  • python 3.6+
  • gym 0.10.5
  • numpy 1.19.5
  • pytorch 1.9.0

Installation of Crazyflie Training Environment

git clone https://github.com/tudelft/risk-sensitive-rl.git
cd risk-sensitive-rl/crazyflie-env
python3 -m pip install -e '.[crazyflie-env]'

Import Crazyflie Environment

import gym, crazyflie_env
env = gym.make('CrazyflieEnv-v0') # env id

Note that the interface for this environment is slightly different than OpenAI gym. By calling env.step(<action>), the returned state is a object that need to be transferred to numpy arrays. This is implemented by a transfer function series in art-iqn/utils/util.py.

For the details of using this environment, refer to art-iqn/train.py or art-iqn/tactical.py.

Reproduction

To reproduce our real-world experiments, one needs a crazyflie drone system equiped with multiranger deck and an global position system like OptiTrack to locate the drone's absolute position. Onboard position estimation system can also be used but there maybe exist large drift.

IQN policy with fixed CVaR values

python3 art-iqn/run_cyber_zoo.py --cvar=<your_cvar_value> --exp_id='<your_experiment_id>'

ART-IQN

python3 art-iqn/run_cyber_zoo_art.py 

Video

Supplementary video for our real-world experiment can be found in art-iqn/experiments/videos.

About

Adaptive Risk Tendency Implicit Quantile Network for Drone Navigation under Partial Observability.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages