Learning Representations and Generative Models for 3D Point Clouds
- Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, L. Guibas
- Computer ScienceInternational Conference on Machine Learning
- 8 July 2017
A deep AutoEncoder network with state-of-the-art reconstruction quality and generalization ability is introduced with results that outperform existing methods on 3D recognition tasks and enable shape editing via simple algebraic manipulations.
Manifold Mixup: Better Representations by Interpolating Hidden States
- Vikas Verma, Alex Lamb, Yoshua Bengio
- Computer ScienceInternational Conference on Machine Learning
- 13 June 2018
Manifold Mixup, a simple regularizer that encourages neural networks to predict less confidently on interpolations of hidden representations, improves strong baselines in supervised learning, robustness to single-step adversarial attacks, and test log-likelihood.
Negative Momentum for Improved Game Dynamics
- Gauthier Gidel, Reyhane Askari Hemmat, Ioannis Mitliagkas
- Computer ScienceInternational Conference on Artificial…
- 12 July 2018
It is proved that alternating updates are more stable than simultaneous updates and both theoretically and empirically that alternating gradient updates with a negative momentum term achieves convergence in a difficult toy adversarial problem, but also on the notoriously difficult to train saturating GANs.
Representation Learning and Adversarial Generation of 3D Point Clouds
- Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, L. Guibas
- Computer ScienceArXiv
- 8 July 2017
This paper introduces a deep autoencoder network for point clouds, which outperforms the state of the art in 3D recognition tasks, and designs GAN architectures to generate novel point-clouds.
Asynchrony begets momentum, with an application to deep learning
- Ioannis Mitliagkas, Ce Zhang, Stefan Hadjis, C. Ré
- Computer ScienceAllerton Conference on Communication, Control…
- 31 May 2016
It is shown that running stochastic gradient descent in an asynchronous manner can be viewed as adding a momentum-like term to the SGD iteration, and an important implication is that tuning the momentum parameter is important when considering different levels of asynchrony.
Memory Limited, Streaming PCA
- Ioannis Mitliagkas, C. Caramanis, Prateek Jain
- Computer ScienceNIPS
- 28 June 2013
An algorithm is presented that uses O(kp) memory and is able to compute the k-dimensional spike with O(p log p) sample-complexity - the first algorithm of its kind.
Accelerated Stochastic Power Iteration
- Peng Xu, Bryan D. He, Christopher De Sa, Ioannis Mitliagkas, Christopher Ré
- Computer ScienceInternational Conference on Artificial…
- 10 July 2017
This work proposes a simple variant of the power iteration with an added momentum term, that achieves both the optimal sample and iteration complexity, and constructs stochastic PCA algorithms, for the online and offline setting, that achieve an accelerated iteration complexity O ( 1 / Δ ) .
YellowFin and the Art of Momentum Tuning
- Jian Zhang, Ioannis Mitliagkas, C. Ré
- Computer ScienceConference on Machine Learning and Systems
- 12 June 2017
This work revisits the momentum SGD algorithm and shows that hand-tuning a single learning rate and momentum makes it competitive with Adam, and designs YellowFin, an automatic tuner for momentum and learning rate in SGD.
Joint Power and Admission Control for Ad-Hoc and Cognitive Underlay Networks: Convex Approximation and Distributed Implementation
- Ioannis Mitliagkas, N. Sidiropoulos, A. Swami
- Computer ScienceIEEE Transactions on Wireless Communications
- 24 October 2011
A centralized approximate solution to power control in interference-limited cellular, ad-hoc, and cognitive underlay networks is developed, which alternates between distributed approximation and distributed deflation - reaching consensus on a user to drop, when needed.
Omnivore: An Optimizer for Multi-device Deep Learning on CPUs and GPUs
- Stefan Hadjis, Ce Zhang, Ioannis Mitliagkas, C. Ré
- Computer ScienceArXiv
- 14 June 2016
The novel understanding of the interaction between system and optimization dynamics to provide an efficient hyperparameter optimizer is used, demonstrating that the most popular distributed deep learning systems fall within the tradeoff space, but do not optimize within the space.
...
...