New MIT system: Robots learn self-control via video, without complex sensors
Scientists at the Massachusetts Institute of Technology (MIT CSAIL) have developed a novel system called "Neural Jacobian Fields" (NJF). This innovation enables robots to learn to control their own bodies using only video input from a camera, thereby eliminating the need for complex sensors or extensive pre-programming.
The NJF system represents a significant departure from traditional robot development. Instead of rigid structures equipped with numerous sensors, the robot independently constructs a model of its own body based on observations. This approach is particularly crucial for soft and deformable robots. As lead researcher Sijie Lester Li notes, "This work is a transition from programming robots to teaching robots."
The learning process involves the robot performing random movements and observing the outcomes through cameras. NJF utilizes a neural network to determine the relationship between the robot's geometry and its response to control signals. This learning is entirely autonomous, requiring no human intervention or prior knowledge about the robot. Once trained, only a single camera is needed for control.
The system has been successfully tested on various robot types, including both soft and rigid manipulators. NJF holds substantial promise for practical applications: robots will be able to perform precise tasks in agriculture, construction, and dynamic environments without the need for expensive sensors. Future development plans include enabling training even through smartphone video, which would significantly simplify the implementation of this technology.
ORIENT
