Tuesday, May 21, 2024

MIT’s robotic hand can accurately identify objects with only one grasp

Taking inspiration from the human finger, MIT researchers have developed a robotic hand with high-tough resolution sensing that can accurately identify an object after just one grasp. The experimental device is known as the GelSight EndoFlex.

Many robotic hands place all of their powerful sensors into their fingertips, so an object must be in full contact with those fingertips to be identified, which can necessitate multiple grasps. Other designs use lower-resolution sensors spread across the entire length of the finger, but these don’t capture as much detail, so multiple grips are often required as well.

Now, the MIT team has created a robotic finger with a rigid 3D-printed endoskeleton encased in a soft outer shell that has multiple high-resolution sensors embedded under its transparent silicone skin.

The endoskeleton of each finger contains a pair of GelSight touch sensors – embedded into the top and middle sections underneath the transparent skin. Using a camera and colored LEDs – to gather visual information about an object’s shape, the sensors provide continuous detection along the entire length of the finger. Each finger captures rich data on many parts of an object simultaneously.

A special algorithm performs backward calculations to map the contours on the grasped object’s surface and turn them into 3D images. Those images are then sent to a machine-learning algorithm that uses them as inputs to identify the object. Thanks to tactile sensing covering all its fingers, the hand can gather rich tactile data from a single grasp.

Using this design, the researchers built a three-fingered robotic hand using two fingers arranged in a Y pattern with a third finger as an opposing thumb. The hand captures six images when it grasps an object (two from each finger) and sends those images to a machine-learning algorithm that uses them as inputs to identify the object with about 85% accuracy. The number should improve as the technology is developed further.

The rigid skeleton makes the fingers strong enough to pick up a heavy item, such as a drill, while the soft skin enables them to securely grasp a pliable item without crushing.

“Having both soft and rigid elements is very important in any hand, but so is being able to perform great sensing over a really large area, especially if we want to consider doing very complicated manipulation tasks like what our own hands can do. Our goal with this work was to combine all the things that make our human hands so good into a robotic finger that can do tasks other robotic fingers can’t currently do,” says mechanical engineering graduate student Sandra Liu, co-lead author of a research paper on the robotic finger.

In future work, the researchers also want to improve the hardware to reduce the amount of wear and tear in the silicone over time. They also plan to add more actuation to the thumb so it can perform a wider variety of tasks.