Author pages are created from data sourced from our academic publisher partnerships and public sources.

- Publications
- Influence

The information bottleneck method

- Naftali Tishby, Fernando C Pereira, W. Bialek
- Physics, Computer Science
- ArXiv
- 24 April 2000

We define the relevant information in a signal $x\in X$ as being the information that this signal provides about another signal $y\in \Y$. Examples include the information that face images provide… Expand

The Hierarchical Hidden Markov Model: Analysis and Applications

- S. Fine, Y. Singer, Naftali Tishby
- Computer Science
- Machine Learning
- 1 July 1998

We introduce, analyze and demonstrate a recursive hierarchical generalization of the widely used hidden Markov models, which we name Hierarchical Hidden Markov Models (HHMM). Our model is motivated… Expand

The power of amnesia: Learning probabilistic automata with variable memory length

- D. Ron, Y. Singer, Naftali Tishby
- Computer Science
- Machine Learning
- 1 December 1996

We propose and analyze a distribution learning algorithm for variable memory length Markov processes. These processes can be described by a subclass of probabilistic finite automata which we name… Expand

Opening the Black Box of Deep Neural Networks via Information

- Ravid Shwartz-Ziv, Naftali Tishby
- Computer Science
- ArXiv
- 2 March 2017

Despite their great success, there is still no comprehensive theoretical understanding of learning with Deep Neural Networks (DNNs) or their inner organization. Previous work proposed to analyze DNNs… Expand

Selective Sampling Using the Query by Committee Algorithm

- Y. Freund, H. S. Seung, E. Shamir, Naftali Tishby
- Computer Science
- Machine Learning
- 1 September 1997

We analyze the “query by committee” algorithm, a method for filtering informative queries from a random stream of inputs. We show that if the two-member committee algorithm achieves information gain… Expand

Agglomerative Information Bottleneck

- N. Slonim, Naftali Tishby
- Computer Science
- NIPS
- 29 November 1999

We introduce a novel distributional clustering algorithm that maximizes the mutual information per cluster between data and given categories. This algorithm can be considered as a bottom up hard… Expand

Margin based feature selection - theory and algorithms

- Ran Gilad-Bachrach, A. Navot, Naftali Tishby
- Computer Science
- ICML '04
- 4 July 2004

Feature selection is the task of choosing a small set out of a given set of features that capture the relevant properties of the data. In the context of supervised classification problems the… Expand

Information Bottleneck for Gaussian Variables

- Gal Chechik, A. Globerson, Naftali Tishby, Yair Weiss
- Mathematics, Computer Science
- J. Mach. Learn. Res.
- 9 December 2003

The problem of extracting the relevant aspects of data was addressed through the information bottleneck (IB) method, by (soft) clustering one variable while preserving information about another -… Expand

Distributional Clustering of English Words

- Fernando C Pereira, Naftali Tishby, Lillian Lee
- Computer Science
- ACL
- 22 June 1993

We describe and evaluate experimentally a method for clustering words according to their distribution in particular syntactic contexts. Words are represented by the relative frequency distributions… Expand

Deep learning and the information bottleneck principle

- Naftali Tishby, Noga Zaslavsky
- Computer Science, Mathematics
- IEEE Information Theory Workshop (ITW)
- 9 March 2015

Deep Neural Networks (DNNs) are analyzed via the theoretical framework of the information bottleneck (IB) principle. We first show that any DNN can be quantified by the mutual information between the… Expand