Probability Seminar

Donald J. Jacobs,  Department of Physics and Optical Science, UNC Charlotte

Abstract:  Given a sample of independent and identically distributed random variables {V_k}, a novel maximum entropy method is employed to estimate the underlying probability density function (pdf). A key concern of the method is not to over fit the data. In the nonparametric approach taken here, the logarithm of the pdf is expressed as a series expansion over a complete set of orthogonal functions. The expansion coefficients are iteratively optimized to minimize specially constructed error metrics based on binning and order statistics of transformed random variables {U_k}. The set {U_k} is obtained by the isomorphic transformation of V_k –> U_k using the cumulative distribution function of the trial pdf. The specially designed error metrics are functions of sample size, which have been empirically determined for binning and order statistics describing random numbers drawn from a uniform probability density on [0,1]. These metrics serve as a universal reference to quantify when the transformed random sample, {U_k}, exhibits typical fluctuations. Distinguishing between expected and atypical levels of fluctuations prevents over and under fitting the data. Multiple solutions for the pdf are generated to reflect intrinsic uncertainties due to incompleteness in finite sampling, and taken together, are used to boost confidence levels. Benchmark tests show that solutions converge toward the true pdf as sample size increases in accordance with the law of large numbers. Robust results are demonstrated on a variety of notoriously problematic test probability densities that include multi-modal and multi-resolution data, discontinuities, divergences, heavy tail extreme statistics and extreme degeneracies.