Borja-videos

Modular Multiagent Reinforcement Learning approach to L-MCRS systems

Results from our simulations with 6 physicaly-linked robots using a Modular Reinforcement Learning system.

Local goals

From an initial position of the hose, the agents must reach a final configuration (green). Each of the robots has its own local goal.

Succesful episodes

• Episode #10,000: media:Episode10000.avi
• Episode #10,001: media:Episode10001.avi
• Episode #10,002: media:Episode10002.avi

Failed episodes

• Episode #10,003: media:Episode10003.avi

Team goal

The robot more distant from the source of the hose (center of the grid) is desired to reach the goal, which is represented as a green dot, and they are all attached to a hose which is represented as blue segments.

Succesful episodes

• Episode #80,001: media:Episode80001.avi
• Episode #80,005: media:Episode80005.avi
• Episode #80,006: media:Episode80006.avi
• Episode #80,008: media:Episode80008.avi
• Episode #80,009: media:Episode80009.avi
• Episode #80,010: media:Episode80010.avi

Failed episodes

• Episode #80,004: media:Episode80004.avi
• Episode #80,050: media:Episode80050.avi

Consensus-based approach to L-MCRS systems

These are some examples of real life experiences on the hose transportation problem. Robot detection and control software is run on a PC. Red dots represent the references (where robots "should be") and green dots the posture given by the camera (where "they are"). Commands are sent to robots using radio transceivers:

A) Non-Linked Robots

• A.1 Tangential speeds for all robots were limited to 0.02 m/s.(media:2010.5.run1.avi)

No physical links are used and robots perform relatively well. Due to communication errors, delays, servo inaccuracies and nature of PI controllers, robots oscillate around the path.

B) Linked Robots

• B.1 Tangential speeds for all robots were limited to 0.02 m/s. (media:2010.5.run3.avi)

Steering behaves worse as the physical link introduces some traction effects on the system. For the same reason, it takes longer for the robots to catch the references.

• B.2 Max. tangential speed for last robot was limited (50%). References move full-speed. (media:2010.5.run5.avi)

The last robot is forced to move slower than the rest and, because of this, the robots aren't capable of catching the references. Error spreads among the system.

• B.3 Max. tangential speed for last robot was limited (50%). References move at 75% speed.(media:2010.5.run6.avi)

The last robot is forced again to move at half-speed and references move at 75% speed, yet the robots aren't able to follow the path in an acceptable way.

• B.4 Max. tangential speed for last robot was limited (50%). References move at 50% speed.(media:2010.5.run7.avi)

The last robot is running at half-speed and the references move at half-speed too, showing that if all the robots move faster or equally fast as the references, the overall system behavior is better, no matter the maximum speed differences between the robots. Near the end of the path, traction forces between robots are higher than the forces applied by the robots and they are not capable of steering correctly.

• B.5 Last robot is switched off. References move full-speed. (media:2010.5.run8.avi)

One interesting application of physically-linked multicomponent robotic systems is the fail-tolerance. In this run, last robot remains switched-off and the robots still follow the path acceptably good. The robot switched off makes following the path harder to the rest.

• B.6 Last robot is switched off. References move at 50% speed. (media:2010.5.run9.avi)

This time the references move slower allowing the robots to catch them faster. The last robot is switched off and that makes the rest behave worse.