• Corpus ID: 46959657

Localized Structured Prediction

  title={Localized Structured Prediction},
  author={Carlo Ciliberto and Francis R. Bach and Alessandro Rudi},
Key to structured prediction is exploiting the problem structure to simplify the learning process. A major challenge arises when data exhibit a local structure (e.g., are made by "parts") that can be leveraged to better approximate the relation between (parts of) the input and (parts of) the output. Recent literature on signal processing, and in particular computer vision, has shown that capturing these aspects is indeed essential to achieve state-of-the-art performance. While such algorithms… 

Figures from this paper

A Structured Prediction Approach for Conditional Meta-Learning

This work proposes task-adaptive structured meta- learning (TASML), a principled estimator that weighs meta-training data conditioned on the target task to design tailored meta-learning objectives and introduces algorithmic improvements to tackle key computational limitations of existing methods.

Manifold Structured Prediction

A structured prediction approach to manifold-valued regression is studied and a class of problems for which the considered approach is statistically consistent is characterized and how geometric optimization can be used to compute the corresponding estimator is studied.

Fine-grained Generalization Analysis of Structured Output Prediction

This paper significantly improves the state of the art by developing novel high-probability bounds with a logarithmic dependency on d and leverages the lens of algorithmic stability to develop generalization bounds in expectation without any dependency ond.

A General Framework for Consistent Structured Prediction with Implicit Loss Embeddings

A large class of loss functions is identified and study that implicitly defines a suitable geometry on the problem that is the key to develop an algorithmic framework amenable to a sharp statistical analysis and yielding efficient computations.

Structured Prediction for Conditional Meta-Learning

TASML is derived, a principled framework that yields task-specific objective functions by weighing meta-training data on target tasks and improves the performance of existing meta-learning models, and outperforms the state-of-the-art on benchmark datasets.

Deep Structured Prediction with Nonlinear Output Transformations

A novel model is developed which generalizes existing approaches, such as structured prediction energy networks, and a formulation is discussed which maintains applicability of existing inference techniques.

Towards Sharper Generalization Bounds for Structured Prediction

This paper investigates the generalization performance of structured prediction learning and obtains state-of-the-art generalization bounds from three different perspectives: Lipschitz continuity, smoothness, and space capacity condition.


  • Computer Science
  • 2020
PABI is proposed, a unified informativeness measure motivated by PAC-Bayesian theory, characterizing the reduction in uncertainty that indirect, weak signals provide, and its use in quantifying various types of incidental signals including partial labels, noisy labels, constraints, cross-domain signals, and combinations of these.

Foreshadowing the Benefits of Incidental Supervision

A unified PAC-Bayesian Informativeness measure (PABI) is proposed, characterizing the reduction in uncertainty that incidental supervision signals provide and demonstrating PABI's use in quantifying various types of incidental signals.

Foreseeing the Benefits of Incidental Supervision

A unified PAC-Bayesian motivated informativeness measure, PABI, is proposed that characterizes the uncertainty reduction provided by incidental supervision signals, and its effectiveness is demonstrated by quantifying the value added by various types of incidental signals to sequence tagging tasks.



Manifold Structured Prediction

A structured prediction approach to manifold-valued regression is studied and a class of problems for which the considered approach is statistically consistent is characterized and how geometric optimization can be used to compute the corresponding estimator is studied.

A Generalized Kernel Approach to Structured Output Learning

This work proposes a covariance-based operator-valued kernel that allows for the decoupling between outputs in the image space and the inability to use a joint feature space, and introduces a variant of the KDE method based on the conditional covariance operator that in addition to the correlation between the outputs takes into account the effects of the input variables.

Consistent Multitask Learning with Nonlinear Output Relations

This paper proposes a novel algorithm for learning multiple tasks which are related by a system of nonlinear equations that their joint outputs need to satisfy and shows that the algorithm is consistent and can be efficiently implemented.

Output Fisher embedding regression

A novel, general and versatile method called output Fisher embedding regression is introduced, based on a probabilistic modeling of training output data and the minimization of a Fisher loss, which shows that the pre-image problem enjoys a closed-form solution with an appropriate choice of the embedding.

Structured output regression for detection with partial truncation

We develop a structured output model for object category detection that explicitly accounts for alignment, multiple aspects and partial truncation in both training and inference. The model is

Learning to Localize Objects with Structured Output Regression

This work proposes to treat object localization in a principled way by posing it as a problem of predicting structured data: it model the problem not as binary classification, but as the prediction of the bounding box of objects located in images.

Large Margin Methods for Structured and Interdependent Output Variables

This paper proposes to appropriately generalize the well-known notion of a separation margin and derive a corresponding maximum-margin formulation and presents a cutting plane algorithm that solves the optimization problem in polynomial time for a large class of problems.

Leveraging Low-Rank Relations Between Surrogate Tasks in Structured Prediction

An efficient algorithm based on trace norm regularization which does not require explicit knowledge of the coding/decoding functions of the surrogate framework is proposed, which can be applied to the broad class of problems in which the surrogate space is large or even infinite dimensional.

Sharp Analysis of Learning with Discrete Losses

A least-squares framework is studied to systematically design learning algorithms for discrete losses, with quantitative characterizations in terms of statistical and computational complexity, to improve existing results by providing explicit dependence on the number of labels.

A Structured Prediction Approach for Label Ranking

We propose to solve a label ranking problem as a structured output regression task. In this view, we adopt a least square surrogate loss approach that solves a supervised learning problem in two