• Corpus ID: 18404693

Detection and Tracking of Liquids with Fully Convolutional Networks

  title={Detection and Tracking of Liquids with Fully Convolutional Networks},
  author={Connor Schenck and Dieter Fox},
Recent advances in AI and robotics have claimed many incredible results with deep learning, yet no work to date has applied deep learning to the problem of liquid perception and reasoning. In this paper, we apply fully-convolutional deep neural networks to the tasks of detecting and tracking liquids. We evaluate three models: a single-frame network, multi-frame network, and a LSTM recurrent network. Our results show that the best liquid detection results are achieved when aggregating data over… 

Figures from this paper

Incorporating side-channel information into convolutional neural networks for robotic tasks

Empirical tests on robot collision prediction and control problems compare the proposed architectures in terms of learning speed, memory usage, learning capacity, and susceptibility to overfitting.

Efficient and robust deep networks for semantic segmentation

This paper explores and investigates deep convolutional neural network architectures to increase the efficiency and robustness of semantic segmentation tasks and introduces a new part segmentation dataset, the Freiburg City dataset, which is designed to bring semantic segmentsation to highly realistic robotics scenarios.

Physics perception in sloshing scenes with guaranteed thermodynamic consistency

This work proposes a strategy to learn the full state of sloshing liquids from measurements of the free surface based on recurrent neural networks that project the limited information available to a reduced-order manifold so as to not only reconstruct the unknown information, but also to be capable of performing fluid reasoning about future scenarios in real time.

Liquid Pouring Monitoring via Rich Sensory Inputs

This work trains a hierarchical LSTM with late fusion for monitoring and proposes two auxiliary tasks during training: inferring the initial state of containers and forecasting the one-step future 3D trajectory of the hand with an adversarial training procedure to improve the robustness of the system.

Physically sound, self-learning digital twins for sloshing fluids

A novel self-learning digital twin strategy is developed for fluid sloshing phenomena and real-time prediction of the fluid response is obtained from a reduced order model (ROM) constructed by means of thermodynamics-informed data-driven learning.

Accurate Pouring with an Autonomous Robot Using an RGB-D Camera

A novel approach to autonomous pouring that tracks the liquidlevel using an RGB-D camera and adapts the rate of pouring based on the liquid level feedback and is able to pour liquids to a target height with an accuracy of a few millimeters.

Automated Liquid-Level Monitoring and Control using Computer Vision

A generalizable computer-vision based system capable of monitoring and controlling liquid-level across a variety of chemistry applications and successful deployment in three experimental use cases which require continous stirring.

Computer Vision for Recognition of Materials and Vessels in Chemistry Lab Settings and the Vector-LabPics Data Set

This work presents the Vector-LabPics data set, which consists of 2187 images of materials within mostly transparent vessels in a chemistry lab and other general settings, and trained neural networks achieved good accuracy in detecting and segmenting vessels and material phases, and in classifying liquids and solids.

Image processing for hydraulic jump free-surface detection: coupled gradient/machine learning model

High-frequency oscillations and high surface aeration, induced by strong turbulence, make water depth measurement for hydraulic jumps a persistently challenging task. The investigation of hydraulic

Robotic Comprehension of Viscosity

The program created performs a series of behaviors at different speeds and is successful in collecting large amounts of effort, force, and position data from a Kinova robotic arm, with four different stirring implements used to create motions in four distinct substances.



End-to-End Training of Deep Visuomotor Policies

This paper develops a method that can be used to learn policies that map raw image observations directly to torques at the robot's motors, trained using a partially observed guided policy search method, with supervision provided by a simple trajectory-centric reinforcement learning method.

Action-Conditional Video Prediction using Deep Networks in Atari Games

This paper is the first to make and evaluate long-term predictions on high-dimensional video conditioned by control inputs and proposes and evaluates two deep neural network architectures that consist of encoding, action-conditional transformation, and decoding layers based on convolutional neural networks and recurrent neural networks.

Fully convolutional networks for semantic segmentation

The key insight is to build “fully convolutional” networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning.

Recurrent Instance Segmentation

This work proposes a new instance segmentation paradigm consisting in an end-to-end method that learns how to segment instances sequentially, based on a recurrent neural network that sequentially finds objects and their segmentations one at a time.

Brain tumor segmentation with Deep Neural Networks

LSTM: A Search Space Odyssey

This paper presents the first large-scale analysis of eight LSTM variants on three representative tasks: speech recognition, handwriting recognition, and polyphonic music modeling, and observes that the studied hyperparameters are virtually independent and derive guidelines for their efficient adjustment.

Incorporating Failure-to-Success Transitions in Imitation Learning for a Dynamic Pouring Task

An imitation learning approach for a dynamic fluid pouring task that allows learning from errors made by humans and how they recovered from these errors subsequently is presented.

Caffe: Convolutional Architecture for Fast Feature Embedding

Caffe provides multimedia scientists and practitioners with a clean and modifiable framework for state-of-the-art deep learning algorithms and a collection of reference models for training and deploying general-purpose convolutional neural networks and other deep models efficiently on commodity architectures.

Long Short-Term Memory

A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.