We propose to develop an "Intelligent Observer" (IO), a robotic system which can automatically detect, track, and follow targets moving through complex and cluttered environments. The system consists of a number of moving cameras, each mounted on a mobile platform equipped with multiple range sensors. Either autonomously or under the high-level control of a human user, the IO will use real-time vision algorithms to acquire and track a target. It will integrate on-line motion planning and control algorithms which will allow it to automatically move its cameras to maintain a good view of the target, taking into account not only obstacles to the motion of the robots, but also occlusions of the cameras' views. The software will use a geometric model of the environment in which it must plan and execute camera motions. Also, if a geometric model of the target is available, the IO will use it to estimate the 3D location and pose of the target by fusing information from multiple sensors. The project will result in a prototype system where a camera mounted on a NOMAD 200 robot tracks an object while moving to keep the object in the field of view of its cameras and simultaneously avoiding obstacles in the environment. Anticipated
Benefits: Potential applications of such a system are numerous, both in the Federal Government and in the private sector. In space exploration, an IO can be used to observe experiments or other "interesting" activities. In military applications, an IO can be deployed to detect and track enemy targets. A security system whose cameras could automatically detect and track intruders would be more reliable and perhaps more cost effective than human surveillance. Other possible applications are in the areas of hazardous material handling, tele-conferencing, and remote surgery.
