A vision based robotics system capable of following a person executing a mission has many applications such as search, rescue or combat in previously unvisited locations. Its main advantage over other modes of person following is its passivity in the sense that it does not transmit or receive detectable signals. However, vision based robotic person following in outdoor environments is a very challenging problem due to extreme variations in the lighting conditions and its effect of the color of person’s outfit, as well as the effect of the reflected colored lights from buildings, trees, etc.
We have developed three generations of person following robots. The first generation used a fixed camera on the robot, monochrome images (Robotic Person Following Using Fuzzy Control and Image Segmentation). The second generation used a 4 wheel all terrain rover for traversal over rough terrain (left figure below). The camera pan and tilt as well as robot steering and speed were also controlled to keep the rover at a reasonably constant distance from the person.
The third generation, sponsored by Center for Commercialization of advanced Technology and SPAWAR, uses a Segway platform (right figure above) to enable maneuvering in tight locations. The developed system addresses the difficulties mentioned above by controlling various camera and robot parameters. The research and development effort has resulted in a robotic system that is capable of following a person wearing a single color shirt in several environments both indoors and outdoors. The robot has been equipped with an onboard computer, vision system and various other components. Novel algorithms and software have been developed as part of this project for various image processing and control functions. The video on top of the page demonstrates the capabilities of the system. Currently improvements are planned to make the system more robust and to cope with multi-colored person’s outfit.