Rényi Entropy and Free Energy
Posted by John Baez
David Corfield used to study statistical inference and machine learning. This led him to become interested in some variants of the ordinary notion of entropy — for example, Tsallis entropy and Rényi entropy. They’re very odd-looking concepts when you first see them, especially if you’ve spent a lot of time studying the ordinary Boltzmann–Gibbs–Shannon–von Neumann entropy. So, I’ve never been happy with them. Whenever I saw them, I thought: what do they mean?
I just got a lot happier with Renyi entropy. Thanks to conversations at the Entropy Club here at the Centre for Quantum Technologies, especially with Oscar Dahlsten, I noticed that Renyi entropy is just a slightly disguised version of a more familiar concept in thermodynamics: free energy.
Unsurprisingly, somebody already knew this.
However, further discussions revealed that Rényi entropy is just the natural q-deformation of the ordinary notion of entropy!
For more, check out my blog post and brief paper. I’m only reblogging here to get David’s attention — it makes more sense to put all the comments in one place.
And for even more, see David’s blog posts on Tsallis entropy and Rényi entropy. Amusingly, I didn’t remember these blog posts, or the comments I made on them, until just now!