Towards Learning to Perceive and Reason About Liquids

@article{Schenck2016TowardsLT,
  title={Towards Learning to Perceive and Reason About Liquids},
  author={Connor Schenck and Dieter Fox},
  journal={ArXiv},
  year={2016},
  volume={abs/1608.00887}
}
Recent advances in AI and robotics have claimed many incredible results with deep learning, yet no work to date has applied deep learning to the problem of liquid perception and reasoning. In this paper, we apply fully-convolutional deep neural networks to the tasks of detecting and tracking liquids. We evaluate three models: a single-frame network, multi-frame network, and a LSTM recurrent network. Our results show that the best liquid detection results are achieved when aggregating data over… Expand
Perceiving and reasoning about liquids using fully convolutional networks
TLDR
This paper used fully convolutional neural networks to learn to detect and track liquids across pouring sequences, and shows that these networks are able to perceive and reason about liquids, and that integrating temporal information is important to performing such tasks well. Expand
Visual closed-loop control for pouring liquids
  • C. Schenck, D. Fox
  • Computer Science, Engineering
  • 2017 IEEE International Conference on Robotics and Automation (ICRA)
  • 2017
TLDR
This paper develops methods for robots to use visual feedback to perform closed-loop control for pouring liquids using both a model-based and model-free method utilizing deep learning for estimating the volume of liquid in a container. Expand
Learning Robotic Manipulation of Granular Media
TLDR
This paper empirically demonstrate that explicitly predicting physical mechanics results in a policy that out-performs both a hand-crafted dynamics baseline, and a "value-network", which must otherwise implicitly predict the same mechanics in order to produce accurate value estimates. Expand
Reasoning About Liquids via Closed-Loop Simulation
  • C. Schenck, D. Fox
  • Computer Science, Engineering
  • Robotics: Science and Systems
  • 2017
TLDR
The results show that closed-loop simulation is an effective way to prevent large divergence between the simulated and real liquid states, and can enable reasoning about liquids that would otherwise be infeasible due to large divergences, such as reasoning about occluded liquid. Expand
Transparent Liquid Segmentation Without Annotations
  • 2021
Liquid state estimation is important for robotics tasks such as pour1 ing; however, estimating the state of transparent liquids is a challenging problem. 2 We propose a novel segmentation pipelineExpand
Modeling and Control for Robotic Assistants: Single and Multi-robot Manipulation
TLDR
Considerations and implementations for making robots capable of performing complex tasks with an emphasis on combining model-based and data-driven approaches for best performance are considered. Expand
Water Streak Detection with Convolutional Neural Networks for Scrubber Dryers
TLDR
A compact,Low-cost, low-energy water streak detection system for the use with existing and new cleaning machines, which comprises a Raspberry Pi with an Intel Movidius Neural Compute Stick, an illumination source, and a camera to observe the floor after cleaning. Expand
Computer Vision Systems: 12th International Conference, ICVS 2019, Thessaloniki, Greece, September 23–25, 2019, Proceedings
In many industrial sectors such as aeronautics, power generation, oil & gas, complex metal parts especially the critical ones are constructed and manufactured for a very long lifespan (more than 10Expand
Unsupervised Selection of Negative Examples for Grounded Language Learning
TLDR
An unsupervised system that learns visual classifiers associated with words, using semantic similarity to automatically choose negative examples from a corpus of perceptual and linguistic data is described. Expand
Precision Pouring into Unknown Containers by Service Robots
TLDR
Two approaches for controlling the motion of a service robot as it pours liquid precisely from an unknown container into another unknown container without the need of any external tools are proposed. Expand
...
1
2
...

References

SHOWING 1-10 OF 27 REFERENCES
End-to-End Training of Deep Visuomotor Policies
TLDR
This paper develops a method that can be used to learn policies that map raw image observations directly to torques at the robot's motors, trained using a partially observed guided policy search method, with supervision provided by a simple trajectory-centric reinforcement learning method. Expand
Action-Conditional Video Prediction using Deep Networks in Atari Games
TLDR
This paper is the first to make and evaluate long-term predictions on high-dimensional video conditioned by control inputs and proposes and evaluates two deep neural network architectures that consist of encoding, action-conditional transformation, and decoding layers based on convolutional neural networks and recurrent neural networks. Expand
Incorporating Failure-to-Success Transitions in Imitation Learning for a Dynamic Pouring Task
We present an imitation learning approach for a dynamic fluid pouring task. Our approach allows learning from errors made by humans and how they recovered from these errors subsequently. We collectExpand
Learning to pour with a robot arm combining goal and shape learning for dynamic movement primitives
TLDR
The combination of goal and shape learning is stable and robust within large parameter regimes, which makes this combined method suitable for robotic applications. Expand
Force-based robot learning of pouring skills using parametric hidden Markov models
TLDR
Computational and experimental results show that the robot is able to learn to pour drinks using the proposed framework, outperforming other approaches such as the classical hidden Markov models in that it requires less training, yields more compact encodings and shows better generalization capabilities. Expand
LSTM: A Search Space Odyssey
TLDR
This paper presents the first large-scale analysis of eight LSTM variants on three representative tasks: speech recognition, handwriting recognition, and polyphonic music modeling, and observes that the studied hyperparameters are virtually independent and derive guidelines for their efficient adjustment. Expand
Long Short-Term Memory
TLDR
A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units. Expand
Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning
TLDR
The central idea is to use the slow planning-based agents to provide training data for a deep-learning architecture capable of real-time play, and proposed new agents based on this idea are proposed and shown to outperform DQN. Expand
Caffe: Convolutional Architecture for Fast Feature Embedding
TLDR
Caffe provides multimedia scientists and practitioners with a clean and modifiable framework for state-of-the-art deep learning algorithms and a collection of reference models for training and deploying general-purpose convolutional neural networks and other deep models efficiently on commodity architectures. Expand
Vision based behavior verification system of humanoid robot for daily environment tasks
TLDR
This paper presents a design and implementation of sensor based behavior verification system using an environment and manipulation knowledge, which is also used in manipulation motion planner, and presents software architecture that allows for a single stream code to perform complex concurrent humanoid motions. Expand
...
1
2
3
...