Try Reinforcement Learning with Donkey Car

1 create virtual env

I'm using pyenv

$ python -m virtualenv py37 --python=python3.7

zsh
Activate py37 and install packages

$ pip install python-socketio flask eventlet pygame numpy pillow h5py scikit-image opencv-python gym

I created dCar folder for the following repos

2 Clone donkeycar

$ mkdir dCar
$ cd dCar
$ git clone https://github.com/wroscoe/donkey donkeycar
$ pip install -e donkeycar

3 Clone self-driving sandbox

$ git clone https://github.com/tawnkramer/sdsandbox.git
$ cd sdsandbox
$ pip install -r requirements.txt

4 Clone donkey_gym

$ git clone https://github.com/tawnkramer/donkey_gym
$ pip install -e donkey_gym

Almost there. We need one more thing for running the simulator.
Download binary https://github.com/tawnkramer/donkey_gym/releases
Then, store it into Applications

5 Run simulator

In this case, we will run simulator from dCar directory

$ python donkey_gym/examples/reinforcement_learning/ddqn.py --sim=/Applications/donkey_sim.app/Contents/MacOS/donkey_sim

Then you will see the simulator like below

Hit Play! to start learning.

/Users/koji.kanao/Documents/py37/lib/python3.7/site-packages/skimage/viewer/__init__.py:6: UserWarning: Viewer requires Qt
  warn('Viewer requires Qt')
WARNING: Logging before flag parsing goes to stderr.
W0709 21:17:37.700056 4583351744 deprecation_wrapper.py:119] From ddqn.py:26: The name tf.keras.initializers.normal is deprecated. Please use tf.compat.v1.keras.initializers.normal instead.

W0709 21:17:37.701951 4583351744 deprecation_wrapper.py:119] From ddqn.py:218: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

W0709 21:17:37.702128 4583351744 deprecation_wrapper.py:119] From ddqn.py:220: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2019-07-09 21:17:37.712400: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
W0709 21:17:37.712966 4583351744 deprecation_wrapper.py:119] From ddqn.py:221: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead.

starting DonkeyGym env
donkey subprocess started
binding to ('0.0.0.0', 9091)
waiting for sim to start..
2019-07-09 21:17:37.883 donkey_sim[69097:904450] Could not find image named 'ScreenSelector'.
waiting for sim to start..
waiting for sim to start..
waiting for sim to start..
waiting for sim to start..
waiting for sim to start..
2019-07-09 21:17:54.684 donkey_sim[69097:904450] Color LCD preferred device: AMD Radeon Pro 560 (high power)
2019-07-09 21:17:54.684 donkey_sim[69097:904450] Metal devices available: 2
2019-07-09 21:17:54.684 donkey_sim[69097:904450] 0: Intel(R) HD Graphics 630 (low power)
2019-07-09 21:17:54.684 donkey_sim[69097:904450] 1: AMD Radeon Pro 560 (high power)
2019-07-09 21:17:54.684 donkey_sim[69097:904450] Using device AMD Radeon Pro 560 (high power)
waiting for sim to start..
got a new client ('127.0.0.1', 58233)
SceneSelectionReady
connection dropped
waiting for sim to start..
got a new client ('127.0.0.1', 58234)
W0709 21:18:01.760221 4583351744 deprecation.py:506] From /Users/koji.kanao/Documents/py37/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
Episode:  0
EPISODE 0 TIMESTEP 30 / ACTION [0.70797646, 0.3] / REWARD 0.8670342954954999 / EPISODE LENGTH 30 / Q_MAX  0
fps 14.5105030403251
EPISODE 0 TIMESTEP 60 / ACTION [-0.5889623, 0.3] / REWARD -0.4946022760490001 / EPISODE LENGTH 60 / Q_MAX  0
EPISODE 0 TIMESTEP 90 / ACTION [0.40704942, 0.3] / REWARD -1.9541489999891999 / EPISODE LENGTH 90 / Q_MAX  0
fps 20.018557585621934
episode: 0   memory length: 107   epsilon: 0.9895139999999955  episode length: 107
Episode:  1
episode: 1   memory length: 108   epsilon: 0.9894159999999954  episode length: 1
Episode:  2
EPISODE 2 TIMESTEP 120 / ACTION [0.884501, 0.3] / REWARD 0.772067625056 / EPISODE LENGTH 12 / Q_MAX  26.828781
episode: 2   memory length: 147   epsilon: 0.9855939999999938  episode length: 39
Episode:  3
episode: 3   memory length: 148   epsilon: 0.9854959999999937  episode length: 1
Episode:  4
EPISODE 4 TIMESTEP 150 / ACTION [0.49025267, 0.3] / REWARD 0.86992308065572 / EPISODE LENGTH 2 / Q_MAX  26.773907
EPISODE 4 TIMESTEP 180 / ACTION [0.06281003, 0.3] / REWARD -1.0 / EPISODE LENGTH 32 / Q_MAX  24.477978
episode: 4   memory length: 180   epsilon: 0.9823599999999924  episode length: 32
Episode:  5
EPISODE 5 TIMESTEP 210 / ACTION [-0.56007874, 0.3] / REWARD 4.7462194558588 / EPISODE LENGTH 30 / Q_MAX  26.608269
episode: 5   memory length: 211   epsilon: 0.979321999999991  episode length: 31
Episode:  6
episode: 6   memory length: 212   epsilon: 0.979223999999991  episode length: 1
Episode:  7
EPISODE 7 TIMESTEP 240 / ACTION [0.34383777, 0.3] / REWARD 1.068961473664648 / EPISODE LENGTH 28 / Q_MAX  26.291294
episode: 7   memory length: 251   epsilon: 0.9754019999999893  episode length: 39
Episode:  8
episode: 8   memory length: 252   epsilon: 0.9753039999999893  episode length: 1
Episode:  9
EPISODE 9 TIMESTEP 270 / ACTION [0.60506356, 0.3] / REWARD -0.38350943820499994 / EPISODE LENGTH 18 / Q_MAX  27.504139
episode: 9   memory length: 294   epsilon: 0.9711879999999875  episode length: 42
Episode:  10
episode: 10   memory length: 295   epsilon: 0.9710899999999875  episode length: 1
Episode:  11
EPISODE 11 TIMESTEP 300 / ACTION [-0.45778775, 0.3] / REWARD 1.26087653748968 / EPISODE LENGTH 5 / Q_MAX  34.701992