High-speed visual servoing of a 6-d.o.f. manipulator using multivariable predictive control

  title={High-speed visual servoing of a 6-d.o.f. manipulator using multivariable predictive control},
  author={Jacques Gangloff and Michel de Mathelin},
  journal={Advanced Robotics},
  pages={1021 - 993}
This paper presents a new approach to model and control high-speed 6-d.o.f. visual servo loops. The modeling and control strategy take into account the dynamics of the velocity-controlled 6-d.o.f. manipulator as well as a simplified model of the camera and acquisition system in order to significantly increase the bandwidth of the servo loop. Multi-input multi-output generalized predictive control (GPC) is used to optimally control the visual loop with respect to the proposed dynamical model… 

High Speed Visual Servoing with Ultrasonic Motors

The main objective of this work is to propose a new dynamic model for the vision sensor that can be modeled by a simple transfer function, thus simplifying the decoupling between vision and actuation.

Reference Trajectory-Based Visual Predictive Control

An image predictive controller for an eye-in-hand-type servoing architecture, composed of a 6-d.o.f. robot and a camera mounted on the gripper, using a novel architecture for integrating reference trajectory and image prediction for use in predictive control of visual servoing systems.

Visual servoing of moving targets through fractional control: A feasibility study

  • C. Copot
  • Engineering
    2017 IEEE 21st International Conference on Intelligent Engineering Systems (INES)
  • 2017
Simulation results demonstrate that the proposed approach based on a fractional-order controller has a stable and convergent behavior when dealing with visual servoing applications.


In this paper an image controller using model predictive control (MPC) for visual servoing systems is presented and its performances are analyzed. An eye-inhand type servoing architecture, composed


In this paper, a review of predictive control algorithms developed by the authors for visual servoing of robots in manipulation applications is presented. Using these algorithms, a control predictive

Visual Servoing of Robot Manipulators Using Model-based Predictive Control

  • C. LazarA. Burlacu
  • Engineering, Computer Science
    2009 7th IEEE International Conference on Industrial Informatics
  • 2009
A model predictive control strategy is presented to visual servoing a robot manipulator with eye-in-hand configuration through nonlinear constraint optimization, which takes into consideration the visibility loss of features due to camera motion.

Real-time visual predictive control of manipulation systems

The results obtained by visual predictive control laws in real-time visual servoing architectures are presented and a comparison with classical image based control laws is conducted.

Modeling of visual servo open-loop for robot manipulators

  • C. LazarA. Burlacu
  • Engineering, Computer Science
    2008 IEEE International Symposium on Industrial Electronics
  • 2008
This paper presents a multivariable open-loop model for visual servoing of robot manipulators using image based strategy. The modeling considers the dynamics of the velocity controlled robot as well

Nonlinear Model Based Predictive Control of Visual Servoing Systems Using Image Moments

A model predictive controller based on image moments derived from point features is proposed in this paper that uses a local model based predictor for image moments and a reference trajectory in order to ensure a robust behavior.



High speed visual servoing of a 6 DOF manipulator using MIMO predictive control

  • J. GangloffM. Mathelin
  • Engineering
    Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065)
  • 2000
Multi-input multi-output (MIMO) generalized predictive control (GPC) is used to optimally control the visual loop with respect to the proposed dynamic model and Experimental results on a 6 DOF industrial robot are presented that validate the proposed model.

Visual servoing of a 6-DOF manipulator for unknown 3-d profile following

The visual servoing of a six degrees of freedom (6-DOF) manipulator for unknown three-dimensional profile following exhibits better accuracy with the GPC in simulations and in real experiments on a 6- DOF manipulator due to the predictive property of the algorithm.

Visual servoing with hand-eye manipulator-optimal control approach

A control theoretic formulation and a controller design method for the feature-based visual servoing with redundant features that can deal with the redundant features and gives flexibility for performance improvement instead of the very limited design parameters provided by the generalized inverse and task function controllers.

Dynamic effects in visual closed-loop systems

The concept of visual dynamic control is introduced, which is concerned with dynamic effects due to the manipulator and machine vision sensor which limit the performance, and combined with axis velocity control, is shown to result in robust and high performance tracking.

Visual tracking of a moving target by a camera mounted on a robot: a combination of control and vision

The authors present algorithms for robotic (eye-in-hand configuration) real-time visual tracking of arbitrary 3D objects traveling at unknown velocities in a 2D space (depth is given as known).

Robust asymptotically stable visual servoing of planar robots

  • R. Kelly
  • Engineering
    IEEE Trans. Robotics Autom.
  • 1996
It is formally proved that the overall closed-loop system composed by the full nonlinear robot dynamics and the controller is Lyapunov stable and it is demonstrated that the Controller is capable to yield an asymptotically stable system which is robust against radial lens distortions and uncertainty in the camera orientation.

Controlled active vision

The framework of "controlled active vision" is presented which states that a controlled motion of a vision sensor can maximize the performance of any robotic algorithm that involves a moving vision sensor.

Robust Visual-Servo Control of Robot Manipulators in the Presence of Uncertainty

A robust controller is designed that compensates for uncertainty throughout the whole robot-camera system and ensures global uniformly Ultimately bounded position tracking for the fixed-camera configuration and a setpoint controller for the camera-in-hand configuration that achieves global uniformly ultimately bounded regulation.

Parallel Vision System and It ' s Application of High Speed Target Tracking

A 1ms vision system, in which the image feature value is extracted in 1ms cycle-time for visual servoing, and a high speed Active Vision System (A VS)-II, which makes a gaze of vision system to move at high speed.

A tutorial on visual servo control

This article provides a tutorial introduction to visual servo control of robotic manipulators by reviewing the prerequisite topics from robotics and computer vision, including a brief review of coordinate transformations, velocity representation, and a description of the geometric aspects of the image formation process.