Skip to the Main Content

Note:These pages make extensive use of the latest XHTML and CSS Standards. They ought to look great in any standards-compliant modern browser. Unfortunately, they will probably look horrible in older browsers, like Netscape 4.x and IE 4.x. Moreover, many posts use MathML, which is, currently only supported in Mozilla. My best suggestion (and you will thank me when surfing an ever-increasing number of sites on the web which have been crafted to use the new standards) is to upgrade to the latest version of your browser. If that's not possible, consider moving to the Standards-compliant and open-source Mozilla browser.

February 11, 2011

Rényi Entropy and Free Energy

Posted by John Baez

David Corfield used to study statistical inference and machine learning. This led him to become interested in some variants of the ordinary notion of entropy — for example, Tsallis entropy and Rényi entropy. They’re very odd-looking concepts when you first see them, especially if you’ve spent a lot of time studying the ordinary Boltzmann–Gibbs–Shannon–von Neumann entropy. So, I’ve never been happy with them. Whenever I saw them, I thought: what do they mean?

I just got a lot happier with Renyi entropy. Thanks to conversations at the Entropy Club here at the Centre for Quantum Technologies, especially with Oscar Dahlsten, I noticed that Renyi entropy is just a slightly disguised version of a more familiar concept in thermodynamics: free energy.

Unsurprisingly, somebody already knew this.

However, further discussions revealed that Rényi entropy is just the natural q-deformation of the ordinary notion of entropy!

For more, check out my blog post and brief paper. I’m only reblogging here to get David’s attention — it makes more sense to put all the comments in one place.

And for even more, see David’s blog posts on Tsallis entropy and Rényi entropy. Amusingly, I didn’t remember these blog posts, or the comments I made on them, until just now!

Posted at February 11, 2011 2:42 AM UTC

TrackBack URL for this Entry:

0 Comments & 0 Trackbacks