Corpus ID: 102352072

The Information Complexity of Learning Tasks, their Structure and their Distance

@article{Achille2019TheIC,
  title={The Information Complexity of Learning Tasks, their Structure and their Distance},
  author={Alessandro Achille and Giovanni Paolini and Glen Mbeng and Stefano Soatto},
  journal={ArXiv},
  year={2019},
  volume={abs/1904.03292}
}
  • Alessandro Achille, Giovanni Paolini, +1 author Stefano Soatto
  • Published in ArXiv 2019
  • Mathematics, Computer Science
  • We introduce an asymmetric distance in the space of learning tasks, and a framework to compute their complexity. These concepts are foundational to the practice of transfer learning, ubiquitous in Deep Learning, whereby a parametric model is pre-trained for a task, and then used for another after fine-tuning. The framework we develop is intrinsically non-asymptotic, capturing the finite nature of the training dataset, yet it allows distinguishing learning from memorization. It encompasses, as… CONTINUE READING

    Create an AI-powered research feed to stay up to date with new papers like this posted to ArXiv

    Citations

    Publications citing this paper.
    SHOWING 1-6 OF 6 CITATIONS

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 14 REFERENCES

    Kolmogorov's structure functions and model selection

    VIEW 15 EXCERPTS
    HIGHLY INFLUENTIAL

    Task2Vec: Task Embedding for Meta-Learning

    VIEW 4 EXCERPTS

    Critical Learning Periods in Deep Neural Networks

    VIEW 4 EXCERPTS

    Taskonomy: Disentangling Task Transfer Learning

    VIEW 1 EXCERPT

    Emergence of Invariance and Disentanglement in Deep Representations

    VIEW 2 EXCERPTS