What is the Uniform Distribution?
Posted by Tom Leinster
Today I gave the Statistics and Data Science seminar at Queen Mary University of London, at the kind invitation of Nina Otter. There I explained an idea that arose in work with Emily Roff. It’s an answer to this question:
What is the “canonical” or “uniform” probability distribution on a metric space?
You can see my slides here, and I’ll give a lightning summary of the ideas now.
Let be a compact metric space.
Step 1 The uniform probability distribution (or more formally, probability measure) on should be one that’s highly spread out. So, we need to be able to quantify the “spread” of a probability distribution on a metric space.
There are many such measures of spread — a whole one-parameter family of them, in fact. They’re the diversities . Or if you prefer, you can work with the entropies ; it makes little difference.
Step 2 We now appear to have a problem. Different values of give different diversity measures , so it seems to be hoping for way too much for there to be a probability measure on that maximizes for all uncountably many s at once.
But miraculously, there is! Call it the maximizing measure on .
Step 3 Statisticians are very familiar with the idea of a maximum entropy distribution as being somehow canonical or preferable. But it’s not what we should call the uniform measure, as it’s not scale-invariant. For example, converting our metric from centimetres to inches would change the maximizing measure, and that’s not good.
The idea now is to take the large-scale limit. In other words, for each scale factor , write for the maximizing measure on the scaled space , and define the uniform measure on to be . This is scale-invariant.
Step 4 Let’s check this gives sensible results. We already know what “uniform distribution” should mean when is finite, or homogeneous (it should mean Haar measure), or a subset of Euclidean space (it should mean normalized Lebesgue measure). Does our general definition of uniform measure give the right thing in these cases? Yes, it does!
There’s also a connection between uniform measures and the Jeffreys prior, an “objective” or “noninformative” prior derived from Fisher information.
You can find all this and more in the slides.
Re: What is the Uniform Distribution?
Very nice! I’d be very interested to hear if you get any good suggested answers to your questions on the last slide, especially