#### Filter Results:

- Full text PDF available (8)

#### Publication Year

1973

2007

- This year (0)
- Last 5 years (0)
- Last 10 years (0)

#### Publication Type

#### Co-author

#### Journals and Conferences

Learn More

- Imre Csiszár, Paul C. Shields
- Foundations and Trends in Communications and…
- 2004

This tutorial is concerned with applications of information theory concepts in statistics, in the finite alphabet setting. The information measure known as information divergence or Kullback-Leibler distance or relative entropy plays a key role, often with a geometric flavor as an analogue of squared Euclidean distance, as in the concepts of I-projection,… (More)

We announce two results on the problem of estimating the order of a Markov chain from observation of a sample path. First is that the Bayesian Information Criterion (BIC) leads to an almost surely consistent estimator. Second is that the Bayesian minimum description length estimator, of which the BIC estimator is an approximation, fails to be consistent for… (More)

- David L. Donoho, Robert M. Gray, +5 authors David G. Stork
- 2002

Given noisy observations Xi = θi + Zi, i = 1, . . . , n, with noise Zi iid ∼ N(0, σ), we wish to recover the signal θ with small mean-squared error. We consider the Minimum Kolmogorov Complexity Estimator (MKCE), defined roughly as the n-vector θ̂(X) solving the problem min Y K(Y ) subject to ‖X − Y ‖2l2n ≤ σ 2 · n, where K(Y ) denotes the length of the… (More)

- Paul C. Shields
- IEEE Trans. Information Theory
- 1993

- Paul C. Shields
- IEEE Trans. Information Theory
- 1998

Information theorists frequently use the ergodic theorem ; likewise entropy concepts are often used in information theory. Recently the two subjects have become partially intertwined as deeper results from each discipline nd use in the other. A brief history of this interaction is presented in this paper, together with a more detailed look at three areas of… (More)

- Imre Csiszár, Paul C. Shields
- IEEE Trans. Information Theory
- 1996

Upper and lower bounds, both of order 6, are obtained on minimax redundancy of universal lossless codes for the class of renewal processes. This is the first example of an interesting model class with strong redundancy rate o(n) but not 0 (logn). For the same class, the nonexistence of weak-rate bounds of smaller order than v'% is also shown. The methods… (More)

- Paul C. Shields
- IEEE Trans. Information Theory
- 1999

- David L. Neuhoff, Paul C. Shields
- IEEE Trans. Information Theory
- 1979

- David L. Neuhoff, Paul C. Shields
- IEEE Trans. Information Theory
- 1998

A conceptually simple coding method may be described as follows. The source sequence is parsed into xed length blocks and a list of these blocks is placed in a dictionary. In the lossless case, the dictionary is transmitted and each successive block is encoded by giving its dictionary location. In the lossy case, the smallest collection of blocks such that… (More)

- Paul C. Shields
- IEEE Trans. Information Theory
- 1991