Service robots are becoming more common in various daily tasks, such as delivering packages, assisting the visually impaired, serving customers at airports, or inspecting construction sites. These robots are capable of moving in different ways, whether it’s through legs, wheels, or by flying, and they can find the shortest or easiest path to their destination.
Despite their abilities, robots still struggle with one basic task: navigating through crowded areas. They rely on cameras and other sensors to observe their surroundings, but their movements can be jerky, with constant changes of direction and frequent stops. As a result, robots are often not permitted to travel independently in crowded places.
Now, researchers at the University of Eastern Finland have developed an innovative algorithm that could enable intelligent and efficient robot navigation among the crowd.
The challenges faced by the latest robots do not lie in identifying their destination or observing the surrounding environment. Instead, the real challenge is their ability to react to various crowds and situations in real-time. The current methods require a lot of computing resources, making them unsuitable for real-time applications where quick reactions are essential.
Chengmin Zhou, MSc, tackled this issue in their dissertation by using reinforcement learning algorithms (RL) for service robot navigation. These algorithms address navigation tasks, even in cases where there are several moving obstacles, such as when a robot moves through a crowd of people and has limited time to react.
The solution that proved to be the best involved a model-free RL algorithm, which allows robots to learn from their past experiences. With this algorithm, robots can now survive even in difficult situations. However, the model-free RL algorithm faces numerous challenges, including slow learning efficiency. In this dissertation, two methods have been employed to improve the learning efficiency:
- The utilization of data collected during robot operation for training. As the robot operates, new real-time data is collected and can be combined with previous training data to enhance the robot’s training.
- Translating environmental information. The sensor information collected from the robot’s operating environment cannot be learned efficiently and accurately. Thus, it needs to be interpreted or translated to enable easy learning by the robot. The learned knowledge (trained model) can then be used for navigation in other similar situations.
Robotic navigation has been improved from three technical aspects. Firstly, robots are given limited action choices to choose the next action. Secondly, real-time data and historical data are mixed to enhance navigation. Lastly, robots are trained by exploiting relational data, which involves utilizing the relationship between the robot and obstacles.
The algorithms developed to implement these improvements have been tested both with computer simulations and in a laboratory environment at Shenzhen Technology University, China.
