3D Object Recognition and Retrieval Using Navigating Robots with Active Vision

Abstract

This work describes the development of a new type of "service robot" called "MROLR" capable of searching for, locating (pose determining), recognizing, and retrieving 3D objects in an indoor environment. Key related contributions include the development of 3D object-model creation and editing facilities, and the application of a vision enhanced navigation algorithm. MROLR comprises a mobile robot transporting a four degrees of freedom (DOF) stereo head, equipped with two calibrated charge-coupled device (CCD) video cameras, and a docking robot arm. The principal navigation mode utilises Simultaneous Localisation and Map-building (SLAM) to update and maintain a map of the robot's location and multiple navigational-feature positions as it steers towards specified coordinates. The algorithm is Kalman Filter based. Navigation-features comprise either of known (previously stored) or newly acquired, visible landmarks. During navigation, automatic navigation-feature selection and measurement is performed and the results of measurements used to substantiate or correct odometry readings. Objects to be retrieved from a scene have an associated (previously created and stored) objectmodel Object-models are created by the integration of a sequence of 3D models constructed from stereo image pairs, each representative of the object in varying angular positions. The model of an object sought is retrieved from a database and matched against scene-models until recognition and pose is established. Scene-models are formed during the search while panning the scene. Both object-models and scene-models comprise 3D straight line and conic segments. Recognition is based on verifying the existence of mutual groups of 3D line and or conic edge features in both the scene and model object. Where the object has sufficient distinctive features, recognition is view independent and tolerant to both scale variations and occlusions. On finding the object, a six DOF robot arm, attached to a caster platform, is manually docked with the mobile robot. Using the object's pose transform the arm is able to grasp and place the object on the mobile base. The arm is manually de-coupled from the mobile robot and the object transported back to the home position. While it would be possible to mount the head and arm on one mobile base, the intention here is to ultimately have a light weight fast moving "scout robot" to seek and find using vision, and a slower heavy-duty transport robot to lift and carry. Finally a "3D Virtual Environment" for simulating MROLR, that could be useful for evaluating alternative map-building strategies, object grasping points, as well as for demonstration and educational purposes, has been implemented, although is not yet complete.

Cite this paper

@inproceedings{Wingate20103DOR, title={3D Object Recognition and Retrieval Using Navigating Robots with Active Vision}, author={Michael T Wingate and Patrick Leung and Hao Shi}, year={2010} }