Why does the robot's perform poor during testing compare to training phase?

[am-amani]

Dear @reiniscimurs

I have been using Turtlebot Waffel Pi and Burger Autorace with 2D lidar. During training, the robot performed very well and reached the target goal. However, I have noticed a strange behavior during the testing phase. The robot sometimes gets stuck in the following behaviors:

• The robot is stuck at one point, neither moving forward nor turning left/right
• The robot is stuck at one point, just turning left or right but not moving forward

Please check the following two videos for more clarity:
https://github.com/reiniscimurs/DRL-robot-navigation/assets/33548412/4c6ea1a5-bc39-4847-ac93-71ad2b461159
https://github.com/reiniscimurs/DRL-robot-navigation/assets/33548412/e8941948-0a44-4de3-8ecc-b32e258f59f7

I am using the simple laser scan on the turtles (default one) with the following descriptions:

<gazebo reference="base_scan">
    <material>Gazebo/FlatBlack</material>
    <sensor type="ray" name="lds_lfcd_sensor">
      <pose>0 0 0 0 0 0</pose>
      <visualize>$(arg laser_visual)</visualize>
      <update_rate>100</update_rate>
      <ray>
        <scan>
          <horizontal>
        <samples>360</samples>
            <resolution>1</resolution>
            <min_angle>-1.5708</min_angle>
            <max_angle>1.5708</max_angle>les>
          </horizontal>
        </scan>
        <range>
          <min>0.19</min>
          <max>12.0</max>
          <resolution>0.01</resolution>
        </range>
        <noise>
          <type>gaussian</type>
          <mean>0.0</mean>
          <stddev>0.01</stddev>
        </noise>
      </ray>
      <plugin name="gazebo_ros_lds_lfcd_controller" filename="libgazebo_ros_laser.so">
        <topicName>scan</topicName>
        <frameName>base_scan</frameName>
      </plugin>
    </sensor>
  </gazebo>

Also, I noticed that the laser scan callback could be faster when calculating the min range scan. The laser scan suddenly jumps when the robot moves, and an obstacle is in front of it. For example, the value of minimum scan data is 0.88, 0.83, 0.80, 0.45! Is this because of the GAZEBO or laser scan description?

[reiniscimurs]

Most likely your model has encountered a local optimum. Since there is no history information in the states, the robot gets stuck in a cycle where left action is followed by right action that leads again back to the left action. See more on the discussion about that here: #89

For the sudden laser jumps, that would depend entirely on the situation at hand, if it is a problem or not. Without more information it is difficult to say what is happening there