Skip to search formSkip to main contentSkip to account menu

Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

This paper proposes soft actor-critic, an off-policy actor-Critic deep RL algorithm based on the maximum entropy reinforcement learning framework, and achieves state-of-the-art performance on a range of continuous control benchmark tasks, outperforming prior on-policy and off- policy methods.
View PDF on arXiv (opens in a new tab)

Soft Actor-Critic Algorithms and Applications

Soft Actor-Critic (SAC), the recently introduced off-policy actor-critic algorithm based on the maximum entropy RL framework, achieves state-of-the-art performance, outperforming prior on-policy and off- policy methods in sample-efficiency and asymptotic performance.
View PDF on arXiv (opens in a new tab)

Reinforcement Learning with Deep Energy-Based Policies

A method for learning expressive energy-based policies for continuous states and actions, which has been feasible only in tabular domains before, is proposed and a new algorithm, called soft Q-learning, that expresses the optimal policy via a Boltzmann distribution is applied.
View PDF on arXiv (opens in a new tab)

Latent Space Policies for Hierarchical Reinforcement Learning

This work addresses the problem of learning hierarchical deep neural network policies for reinforcement learning by constraining the mapping from latent variables to actions to be invertible, and shows that this method can solve more complex sparse-reward tasks by learning higher-level policies on top of high-entropy skills optimized for simple low-level objectives.
View PDF on arXiv (opens in a new tab)

Composable Deep Reinforcement Learning for Robotic Manipulation

This paper shows that policies learned with soft Q-learning can be composed to create new policies, and that the optimality of the resulting policy can be bounded in terms of the divergence between the composed policies.
View on IEEE (opens in a new tab)

Backprop KF: Learning Discriminative Deterministic State Estimators

This work presents an alternative approach where the parameters of the latent state distribution are directly optimized as a deterministic computation graph, resulting in a simple and effective gradient descent algorithm for training discriminative state estimators.
View PDF on arXiv (opens in a new tab)

Learning to Walk via Deep Reinforcement Learning

A sample-efficient deep RL algorithm based on maximum entropy RL that requires minimal per-task tuning and only a modest number of trials to learn neural network policies is proposed and achieves state-of-the-art performance on simulated benchmarks with a single set of hyperparameters.
View PDF on arXiv (opens in a new tab)

Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discovery

This paper studies how to automatically learn dynamical distances: a measure of the expected number of time steps to reach a given goal state from any other state, which can be used to provide well-shaped reward functions for reaching new goals, making it possible to learn complex tasks efficiently.
View PDF on arXiv (opens in a new tab)

From Motor Control to Team Play in Simulated Humanoid Football

This work optimized teams of agents to play simulated football via reinforcement learning, constraining the solution space to that of plausible movements learned using human motion capture data, resulting in a team of coordinated humanoid football players that exhibit complex behavior at different scales, quantified by a range of analysis and statistics.
View on PubMed (opens in a new tab)

Acquiring Diverse Robot Skills via Maximum Entropy Deep Reinforcement Learning

This thesis studies how maximum entropy framework can provide efficient deep reinforcement learning algorithms that solve tasks consistently and sample efficiently, and devise new algorithms based on this framework, starting from soft Q-learning that learns expressive energy-based policies, to soft actor-critic that provides simplicity and convenience of actor-Critic methods.
View Paper (opens in a new tab)
By clicking accept or continuing to use the site, you agree to the terms outlined in our Privacy Policy (opens in a new tab), Terms of Service (opens in a new tab), and Dataset License (opens in a new tab)