Paul C. Shields

Learn More
This tutorial is concerned with applications of information theory concepts in statistics, in the finite alphabet setting. The information measure known as information divergence or Kullback-Leibler distance or relative entropy plays a key role, often with a geometric flavor as an analogue of squared Euclidean distance, as in the concepts of I-projection,(More)
We announce two results on the problem of estimating the order of a Markov chain from observation of a sample path. First is that the Bayesian Information Criterion (BIC) leads to an almost surely consistent estimator. Second is that the Bayesian minimum description length estimator, of which the BIC estimator is an approximation, fails to be consistent for(More)
Given noisy observations Xi = θi + Zi, i = 1, . . . , n, with noise Zi iid ∼ N(0, σ), we wish to recover the signal θ with small mean-squared error. We consider the Minimum Kolmogorov Complexity Estimator (MKCE), defined roughly as the n-vector θ̂(X) solving the problem min Y K(Y ) subject to ‖X − Y ‖2l2n ≤ σ 2 · n, where K(Y ) denotes the length of the(More)
Information theorists frequently use the ergodic theorem ; likewise entropy concepts are often used in information theory. Recently the two subjects have become partially intertwined as deeper results from each discipline nd use in the other. A brief history of this interaction is presented in this paper, together with a more detailed look at three areas of(More)
Upper and lower bounds, both of order 6, are obtained on minimax redundancy of universal lossless codes for the class of renewal processes. This is the first example of an interesting model class with strong redundancy rate o(n) but not 0 (logn). For the same class, the nonexistence of weak-rate bounds of smaller order than v'% is also shown. The methods(More)
A conceptually simple coding method may be described as follows. The source sequence is parsed into xed length blocks and a list of these blocks is placed in a dictionary. In the lossless case, the dictionary is transmitted and each successive block is encoded by giving its dictionary location. In the lossy case, the smallest collection of blocks such that(More)