Back in April 2022, the CDF experiment, which operated at the long-ago-closed Tevatron particle collider. presented the world’s most precise measurement of the mass of the particle known as the “W boson“. Their result generated some excited commentary, because it disagreed by 0.1% with the prediction of the Standard Model of particle physics. Even though the mismatch was tiny, it was significant, because the CDF measurement was so exceptionally precise. Any disagreement of such high significance would imply that something has to give: either the Standard Model is missing something, or the CDF measurement is incorrect.

Like most of my colleagues, I was more than a little skeptical about CDF’s measurement. This was partly because it disagreed with the average of earlier, less precise measurements, but mainly because of the measurement’s extreme challenges. To quote a commentary that I wrote at the time,

• “A natural and persistent question has been: “How likely do you think it is that this W boson mass result is wrong?” Obviously I can’t put a number on it, but I’d say the chance that it’s wrong is substantial. Why? This measurement, which took several many years of work, is probably among the most difficult ever performed in particle physics. Only first-rate physicists with complete dedication to the task could attempt it, carry it out, convince their many colleagues on the CDF experiment that they’d done it right, and get it through external peer review into Science magazine. But even first-rate physicists can get a measurement like this one wrong. The tiniest of subtle mistakes will undo it.”

In the weeks following CDF’s announcement, I attended a detailed presentation about the measurement. The physicist who gave it tried to convince us that everything in the measurement had been checked, cross-checked, and understood. However, I did not find the presentation exceptionally persuasive, so my confidence in it did not increase.

But so what? It doesn’t matter what I think. All a theorist like me can do, seeing a measurement like this, is check to see if it is logically possible and conceptually reasonable for the W boson mass to shift slightly without messing up other existing measurements. And it is.

(In showing this is true, I took the opportunity to explain more about how the Standard Model works, and specifically how the W boson’s mass arises from simple math, before showing how the mass could be shifted upwards. Some of you may still find these technical details interesting, even though the original motivation for this series of articles is no longer what it was.)

Instead, what really matters is for other experimental physicists to make the same measurement, to see if they get the same answer as CDF or not. Because of the intricacy of the measurement, this was far easier said than done. But it has now happened.

In the past year, the ATLAS collaboration at the Large Hadron Collider [LHC] presented a new W boson mass measurement consistent with the Standard Model. But because their uncertainties were 60% larger than CDF’s result, it didn’t entirely settle the issue.

Now the CMS collaboration, ATLAS’s competitor at the LHC, has presented their measurement. They have managed to be almost as precise at that of CDF — a truly impressive achievement. And what do they find? Their result, in red below, is fully consistent with the Standard Model, shown as the vertical grey band, and with ATLAS, the bar line just above the red one. The CDF measurement is the bar outlying to the right; it is the only one in disagreement with the Standard Model.

Since the ATLAS and CMS results are both consistent with all other previous measurements as well as with the Standard Model, and since CMS has even reached the same level of uncertainty obtained by CDF, this makes CDF by far the outlier, as you can see above. The tentative but reasonable conclusion is that the CDF measurement is not correct.

Of course, the CDF experimentalists may argue that it is ATLAS and CMS that have made an error, not CDF. One shouldn’t instantly dismiss that out of hand. It’s worth remembering that ATLAS and CMS use the same accelerator to gather their data, and might have used similar logic in the design of their analysis, so it’s not completely impossible for them to have made correlated mistakes. Still, this is far from plausible, so the onus will be on CDF to directly pinpoint an error in their competitors’ work.

Even if the mistake is CDF’s, it’s worth noting that we still have no idea what exactly it might have been. A long chain of measurements and calibrations are required to determine the W boson mass at this level of precision (about one part in ten thousand). It would be great if the error within this chain could be tracked down, but no one may have the stamina to do that, and it is possible that we will never know what went wrong.

But the bottom line is that the discrepancy suggested by the CDF measurement was always a long shot. I don’t think many particle physicists are surprised to see its plausibility fading away.

Guest post by Matt Kukla and Tanjona Ralaivaosaona

Double limits capture the notion of limits in double categories. In ordinary category theory, a limit is the best way to construct new objects from a given collection of objects related in a certain way. Double limits, extend this idea to the richer structure of double categories. For each of the limits we can think of in an ordinary category, we can ask ourselves: how do these limits look in double categories?

In ordinary category theory, many results can be extended to double categories. For instance, in an ordinary category, we can determine if it has all limits (resp. finite limits) by checking if it has all products and equalizers (resp. binary products, a terminal object, and equalizers) (see Thm 5.1.26 in [3]). In a double category, we need to introduce a new notion of limit, known as a tabulator. One of the main theorems by Grandis and Paré states that a double category has all small double limits if and only if it has small double products, double equalizers, and tabulators. Therefore, these components are sufficient to construct small double limits. To explain this concept thoroughly, we will introduce their definitions in this post. There are various definitions depending on your focus, but for the sake of simplicity, this guide aims to be accessible to anyone with a background in category theory. For an introduction to double categories, see here.

We give an overview of how limits behave in this two-dimensional setting, following Grandis and Paré’s “Limits in double categories.” In particular, we make several definitions more explicit for use in further computations.

Introduction

Recall that double categories consist of two types of morphisms, horizontal and vertical, which interact in a compatible way. Often, composition of one arrow type is weaker than the other. Therefore, we may also think of limits in two different directions. However, limits with respect to the weaker class of morphisms tend to be badly behaved. Hence, in this post, we will only focus on horizontal double limits.

Throughout this article, we will refer to the class of morphisms with strong composition as “arrows,” written horizontally, with composition denoted by $\circ$. The weaker arrows will be called “proarrows,” written as vertical dashed arrows, and with composition denoted by $\bullet$. Identity arrows/proarrows for an object $X$ will be written $\mathbf{1}_X$ and $\mathbf{1}_X^\bullet$ respectively. Sometimes, we will also refer to the identity cell associated to an arrow $f:X \to Y$. This is obtained by taking both proarrow edges to be the respective vertical identities on objects:

There’s an analogous construction for proarrows, but we won’t need it in this article.

Double limits are defined for double diagrams and a double diagram is a double functor from an indexing double category $\mathbb{I}$ to an arbitrary double category $\mathbb{A}$. A limit for a given double diagram $D$ is a universal double cone over $D$. This is a very high-level definition, but we will try to explain each unfamiliar term and illustrate it with examples.

The first thing we need to understand is a double diagram for which we take the limits.

Diagrams

A double diagram in $\mathbb{A}$ of shape $\mathbb{I}$ is a double functor $F: \mathbb{I}\to \mathbb{A}$ between double categories $\mathbb{I}$ and $\mathbb{A}$. In strict double categories, a double functor is simultaneously a functor on the horizontal and vertical structures, preserving cells as well as their vertical compositions, horizontal compositions, identities. That is, for every cell $\alpha: u\to v$,

and for every composable pair of cells $\alpha: u\to v$ and $\beta: v\to w$

1. preserve horizontal compositions of cells: $F (\beta \circ \alpha) = F(\beta) \circ F(\alpha)$,

2. preserve vertical compositions of cells: $F (\gamma \bullet \alpha) = F(\gamma) \bullet F(\alpha)$,

3. preserve cell-wise horizontal identity: for each proarrow $u: A\nrightarrow B$, $F(1_u) = 1_{F u}$,

4. preserve cell-wise vertical identity: for each arrow $f: A\to A'$, $F(1^{\bullet}_f) = 1^{\bullet}_{F f}$,

We will also need the notion of a double natural transformation. These are defined componentwise, much in the same way as ordinary natural transformations. For double functors $F, G: \mathbb{I} \to \mathbb{A}$, a horizontal transformation $H: F \Rightarrow G$ is given by the following data:

• horizontal $\mathbb{A}$-arrows $H i: F i \to G i$ for every object $i \in \mathbb{I}$

• an $\mathbb{A}$-cell $H u$ for every proarrow $u:i \nrightarrow j$ in $\mathbb{I}$ of the shape

• Identities and composition are preserved.

• For every cell $\alpha \in \mathbb{I}$ with proarrow edges $u, v$ and arrow edges $f, g$, the component cells of $u$ and $v$ satisfy $(F\alpha | H v) = (H u | G \alpha)$

Vertical transformations satisfy analogous requirements with respect to vertical morphisms, given Section 1.4 of [1].

We will also use the notion of a modification to define double limits. Suppose we have double functors $F, F', G, G': \mathbb{I} \to \mathbb{A}$, horizontal transformations $H:F \Rightarrow G, K: F' \Rightarrow G'$ and vertical transformations $U:F \Rightarrow F', V: G \Rightarrow G'$. A modification is an assignment of an $\mathbb{A}$-cell $\mu i$ to each object $i \in \mathbb{I}$:

such that, for every horizontal $f:i \to j$, $(\mu i| V f) = (U f|\mu j)$:

Double limits will be defined as a universal double cone. But what are cones or double cones in double categories? You may ask.

Like ordinary categories, cones for a functor $F$ in double categories also consist of an object $X$ and morphisms from $X$ to the objects $F i$, for each object $i$ of $\mathbb{I}$. Note that there two types of morphisms, those of horizontal direction or arrows and those of vertical direction or proarrows. The morphisms involved in cones are the horizontal ones but must be compatible with vertical ones. Let’s dive into the definition to see how that works.

A double cone for a double functor $F: \mathbb{I}\to \mathbb{A}$ consists of an $X$ with arrows $p_i: X\to F i$ for each object $i$ of $\mathbb{I}$, and cells $p u: \mathbf{1}^{\bullet}_X \to F u$ for each every proarrow $u:i\nrightarrow j$, satisfying the following axioms:

1. for each object $i$ in $\mathbb{I}$, $$p(\mathbf{1}^{\bullet}_i)= \mathbf{1}^{\bullet}_{p_i}$$

2. for each composable pair of proarrows $u$ and $v$ in $\mathbb{I}$, $$p(v\bullet u)=p v\bullet p u$$

3. for every cell $\alpha: u\to v$ in $\mathbb{I}$, $$(p u | F\alpha) = p v$$

Note that this implies that $F f\circ p_i = p_j$ and $Fg\circ p_k = p_l$. We can observe that the cells $pu$ for every $u$ are made of two green arrows and $F u$, which is indeed a cell such that the horizontal source of $p u$ is the identity proarrow $\mathbf{1}^{\bullet}_{X}$.

For example, let’s take cones for the functor $F$ from an indexing double category which is the discrete double category (made of only two objects $i$ and $j$), to an arbitrary double category, defined such that $F i= A$ and $F j= B$. Then, a double cone $X$ for $F$ is a candidate product for $A$ and $B$.

Notice that the above description of a double cone satisfies the requirements of a horizontal transformation. We can consider a constant functor $D A: \mathbb{I} \to \mathbb{A}$ at an object $A$ of $\mathbb{A}$, then the data of a double cone with vertex $A$ is determined by a horizontal transformation $x:D A \Rightarrow F$. The componentwise definition of $x$ unrolls to precisely the conditions specified above.

We have now all the setup needed for defining double limits, since as we mentioned above, double limits are universal double cones. That is, a double cone for an underlying functor $F$ through which any other double cones factor.

Double Limits

Limits

Let $F: \mathbb{I} \to \mathbb{A}$ be a double functor. The (horizontal) double limit of $F$ is a universal cone $(A,x)$ for $F$.

Explicitly, this requires several things:

• For any other double cone $(A', x')$, there exists a unique arrow $c:A' \to A$ in $\mathbb{A}$ with $x \circ D c = x'$ (where $D$ is the constant functor at the vertex of $A$)

• Let $(A', x'), (A'', x'')$ be double cones with a proarrow $u: A' \nrightarrow A''$. For every collection of cell $\eta_i$ where $i$ is an object of $\mathbb{I}$, associated to components of each cone, which organize into a modification, there exists a unique $\mathbb{A}$-cell $\tau$ such that $(\tau | x i) = \eta_i$:

In other words, a cell built from a proarrow and the components of two cones (viewed as natural transformations) can be factored uniquely via $\tau$ and $1^\bullet$.

To get a better feel for double limits in practice, let’s examine (binary) products in a double category. Just as in 1-category theory, products are constructed as the double limit of the diagram $\bullet \ \bullet$ (two discrete objects). Spelling out the universal properties of a double limit, the (double) product of objects $A, B \in \mathbb{A}$ consists of an object $A \times B$ which satisfies the usual requirements for a product with respect to horizontal morphisms (with projection maps $\pi_A, \pi_B$. Additionally, given cells $\alpha, \beta$ as below:

there exists a unique cell $\alpha \times \beta$ such that

An identical condition must also hold for $B$ and $\pi_B$.

Equalizers can be extended to the double setting in a similar manner. Taking the double limit of the diagram $\bullet \rightrightarrows \bullet$ yields double equalizers. For horizontal $f,g: A \rightrightarrows B$ in $\mathbb{A}$, the double equalizer of $f$ and $g$ consists of an object $Eq(f,g)$ equipped with a horizontal arrow $e:Eq(f,g) \to A$, which is the equalizer of $f,g$ in the ordinary sense with respect to horizontal arrows. Additionally, for every cell $\eta$ with $(\eta | \mathbf{1}^\bullet_f) = (\eta | \mathbf{1}^\bullet_g)$, there exists a unique $\tau$ such that $(\tau | \mathbf{1}^\bullet) = \eta$:

Tabulators

Until now, we have considered examples of double limits of diagrams built from horizontal morphisms. Tabulators bring proarrows into the mix. They are an interesting case obtained as the limit over the diagram consisting of a single proarrow: $\bullet \nrightarrow \bullet$.

Suppose that $u:A \nrightarrow B$ is a proarrow. The tabulator of $u$ is the double limit of the diagram consisting of just $u$. Unrolling the limit, this amounts to an object $T u$ along with a cell $\tau$:

such that, for any cell $\eta$ of the following shape,

there exists a unique horizontal morphism $f: C \to T$ such that $(1^\bullet_f | \tau) = \eta$:

Additionally, any proarrow $v: C \nrightarrow D$ with horizontal morphisms to $A$ and $B$ forming a tetrahedron can be uniquely factored through $T u$:

In an ordinary category, the existence of all finite products and equalizers is enough to guarantee the existence of all limits. However, in the double setting, we need something extra: tabulators. The following result gives us a similar condition for limits in double categories.

Theorem (5.5 in [1]): A double category $\mathbb{A}$ has all small double limits if and only if it has small double products, equalizers, and tabulators.

Examples in $\mathbb{R}\text{elset}$

In this section, we consider the double category $\mathbb{R}\text{elset}$ of sets with functions as horizontal morphisms and relations as vertical morphisms, for more information see [1].

Tabulators

A tabulator for a proarrow or relation $R\subseteq A\times B$ is $R$ itself with the projection maps $p_1: R\to A$ and $p_2: R\to B$. For every other double cone $(C, q)= (C,q_1,q_2)$ of $R$, there exists a unique function or arrow $h= \langle q_1, q_2\rangle : C\to T R$ ($T R= R$), such that $q_i = p_i\circ h$; and for every relation $S\subseteq C\times D$ and such that $(D, t)= (D,t_1,t_2)$ is also a double cone for $R$, there exists a unique cell $\eta = (S R): S\to \mathbf{1}^{\bullet}_{R}$, such that $(\eta | p R) = q_1 \nrightarrow t_2$.

Product

The double product of two sets $A$ and $B$ is the cartesian product with the usual projection maps and we also have the following:

References

[1] Grandis, Marco, and Robert Paré. "Limits in double categories." Cahiers de topologie et géométrie différentielle catégoriques 40.3 (1999): 162-220.

[2] Patterson, Evan. “Products in double categories, revisited.” arXiv preprint arXiv:2401.08990 (2024).

[3] Leinster, Tom. “Basic category theory.” arXiv preprint arXiv:1612.09375 (2016).|

Why not run a quantum-steampunk creative-writing course?

Quantum steampunk, as Quantum Frontiers regulars know, is the aesthetic and spirit of a growing scientific field. Steampunk is a subgenre of science fiction. In it, futuristic technologies invade Victorian-era settings: submarines, time machines, and clockwork octopodes populate La Belle Èpoque, a recently liberated Haiti, and Sherlock Holmes’s London. A similar invasion characterizes my research field, quantum thermodynamics: thermodynamics is the study of heat, work, temperature, and efficiency. The Industrial Revolution spurred the theory’s development during the 1800s. The theory’s original subject—nineteenth-century engines—were large, were massive, and contained enormous numbers of particles. Such engines obey the classical mechanics developed during the 1600s. Hence thermodynamics needs re-envisioning for quantum systems. To extend the theory’s laws and applications, quantum thermodynamicists use mathematical and experimental tools from quantum information science. Quantum information science is, in part, the understanding of quantum systems through how they store and process information. The toolkit is partially cutting-edge and partially futuristic, as full-scale quantum computers remain under construction. So applying quantum information to thermodynamics—quantum thermodynamics—strikes me as the real-world incarnation of steampunk.

But the thought of a quantum-steampunk creative-writing course had never occurred to me, and I hesitated over it. Quantum-steampunk blog posts, I could handle. A book, I could handle. Even a short-story contest, I’d handled. But a course? The idea yawned like the pitch-dark mouth of an unknown cavern in my imagination.

But the more I mulled over Edward Daschle’s suggestion, the more I warmed to it. Edward was completing a master’s degree in creative writing at the University of Maryland (UMD), specializing in science fiction. His mentor Emily Brandchaft Mitchell had sung his praises via email. In 2023, Emily had served as a judge for the Quantum-Steampunk Short-Story Contest. She works as a professor of English at UMD, writes fiction, and specializes in the study of genre. I reached out to her last spring about collaborating on a grant for quantum-inspired art, and she pointed to her protégé.

Who won me over. Edward and I are co-teaching “Writing Quantum Steampunk: Science-Fiction Workshop” during spring 2025.

The course will alternate between science and science fiction. Under Edward’s direction, we’ll read and discuss published fiction. We’ll also learn about what genres are and how they come to be. Students will try out writing styles by composing short stories themselves. Everyone will provide feedback about each other’s writing: what works, what’s confusing, and opportunities for improvement. 

The published fiction chosen will mirror the scientific subjects we’ll cover: quantum physics; quantum technologies; and thermodynamics, including quantum thermodynamics. I’ll lead this part of the course. The scientific studies will interleave with the story reading, writing, and workshopping. Students will learn about the science behind the science fiction while contributing to the growing subgenre of quantum steampunk.

We aim to attract students from across campus: physics, English, the Jiménez-Porter Writers’ House, computer science, mathematics, and engineering—plus any other departments whose students have curiosity and creativity to spare. The course already has three cross-listings: Arts and Humanities 270, Physics 299Q, Computer Science 298Q, and Chemistry 299Q. More may join the list, and we’re petitioning to satisfy General Education requirements.¹ Undergraduate and graduate students are welcome. QuICS—the Joint Center for Quantum Information and Computer Science, my home base—is paying Edward’s salary through a seed grant. Ross Angelella, the director of the Writers’ House, arranged logistics and doused us with enthusiasm. I’m proud of how organizations across the university are uniting to support the course.

The diversity we seek, though, poses a challenge. The course lacks prerequisites, so I’ll need to teach at a level comprehensible to the non-science students. I’d enjoy doing so, but I’m concerned about boring the science students. Ideally, the science students will help me teach, while the non-science students will challenge us with foundational questions that force us to rethink basic concepts. Also, I hope that non-science students will galvanize discussions about ethical and sociological implications of quantum technologies. But how can one ensure that conversation will flow?

This summer, Edward and I traded candidate stories for the syllabus. Based on his suggestions, I recommend touring science fiction under an expert’s guidance. I enjoyed, for a few hours each weekend, sinking into the worlds of Ted Chiang, Ursula K. LeGuinn, N. K. Jemison, Ken Liu, and others. My scientific background informed my reading more than I’d expected. Some authors, I could tell, had researched their subjects thoroughly. When they transitioned from science into fiction, I trusted and followed them. Other authors tossed jargon into their writing but evidenced a lack of deep understanding. One author nailed technical details about quantum computation, initially impressing me, but missed the big picture: his conflict hinged on a misunderstanding about entanglement. I see all these stories as affording opportunities for learning and teaching, in different ways.

Students can begin registering for “Writing Quantum Steampunk: Science-Fiction Workshop” on October 24. We can offer only 15 seats, due to Writers’ House standards, so secure yours as soon as you can. Part of me still wonders how the Hilbert space I came to be co-teaching a quantum-steampunk creative-writing course.² But I look forward to reading with you next spring!

¹We expect the course to satisfy the requirement Distributive Studies: Scholarship in Practice (DSSP), but check back for the verdict.

²A Hilbert space is a mathematical object that represents a quantum system. But you needn’t know that to succeed in the course.

Given a smooth compact Riemannian manifold , the incompressible Euler equations can be written in abstract index notation as

Conjecture 1 (Finite time blowup) There exists a manifold and a smooth solution to the Euler equations that blows up at some finite time .

This remains open, however there has been progress on rougher versions of this problem. For instance, there is the well-known result of Elgindi (discussed in this previous post) that when and is sufficiently small, there exists a solution to the Euler equations on that blows up in finite time. There has also been progress in establishing various “universality” properties of the Euler flow on manifolds (which informally state that “fluid computers” are possible); see for instance this recent survey of Cardona, Miranda, and Peralta-Salas. Unfortunately, these “fluid computers” do not combine well with scaling symmetries, and so thus far have not been able to produce (finite energy) blowups.

I have been playing with one approach to this conjecture, which reduces to solving a certain underdetermined system of partial differential equations, and then establishing some stability result for the resulting solution. However, I have not been able to make headway on solving this latter system despite its underdetermined nature; so I thought I would record my partial attempt here in case anyone is interested in pursuing it further (and also to contribute to the practice of sharing unsuccessful attempts to solve a problem, which is still quite infrequently done in our community).

To avoid technicalities let us simplify the problem by adding a forcing term :

Theorem 2 (Finite time blowup for the forced equation) There exists a smooth solution to the forced Euler equations on that exhibits finite time blowup, in which the forcing term stays uniformly bounded in for any .

Roughly speaking, their argument proceeds by a multiscale construction, in which the solution is set up to eventually have some presence at a spatial scale , which is conducive to generating an exponential “stretching” of a small forcing term at a much higher spatial scale , which one then introduces to then set up the solution for the next scale.

As a model problem, I tried to reproduce this type of result from a more geometric perspective, trying to aim for a more “self-similar” blowup than a “multi-scale” one, in the hope that this latter type of blowup might be more tractable to analyze and eventually resolve Conjecture 1. I didn’t fully succeed; but I think the approach I outline below is in principle feasible.

The manifold I will work on is a cylinder , where is a smooth compact manifold, and the metric on is just the sum of the standard metric on the first coordinate and :

If we now use Greek indices to only denote coordinates in the “vertical” coordinate , the velocity field now becomes , and the Euler equations now split as

It is now tempting to try to set up an approximately scale-invariant blowup solution. It seems that the first step in this is to construct a “soliton” type localized steady state solution, that is a solution , to the equation

Analytically, this is not a particularly pleasant equation to try to solve; one can substitute the second equation into the first to obtain a single equation

Nevertheless, one can still attempt to solve this equation by separation of variables. If one makes the ansatz

The metric is hidden in this system through the covariant derivative . To eliminate the metric, we can lower indices to write

Remark 3 One can also try to directy create a self-similar blowup to (1), (2), for instance by making the ansatz

for and some fields and . This particular ansatz seems consistent with all known conservation laws; however it works out to basically be ten vector equations (plus some additional scalar constraints) on ten vector field unknowns, so is just barely overdetermined. I have not been able to locate a self-similar blowup ansatz that is underdetermined.

Ben Krause, Hamed Mousavi, Joni Teräväinen, and I have just uploaded to the arXiv the paper “Pointwise convergence of bilinear polynomial averages over the primes“. This paper builds upon a previous result of Krause, Mirek, and myself, in which we demonstrated the pointwise almost everywhere convergence of the ergodic averages

The basic strategy is to try to insert the weight everywhere in the proof of the convergence of (1) and adapt as needed. The weighted averages are bilinear averages associated to the bilinear symbol

2ND UPDATE: Auroras are indeed being observed. (I myself am a bit too far south and skies are hazy, making the moonlight blinding; but I am reading reports from northern Europe.)

UPDATE: Something has happened at the ACE satellite around 2300 UTC (0100 Europe time, 7 pm New York time. See the plot added to the bottom of this post.

———-

Still waiting for a possible outbreak of auroras (northern/southern lights) tonight; a tremendous blast from the Sun, launched from a sunspot two days ago, is believed likely to make a glancing impact on the Earth, and to do so within the next 12 hours or so. That means a possibility of bright northern lights tonight if you’re north of, say, New York City’s latitude.

BUT always keep in mind that forecasting auroras is part science, part art, part luck. Our chances are decent, but the forecast can always be wrong.

As far as timing, the best way to monitor what’s going on, I’ve found, is to use https://www.swpc.noaa.gov/products/ace-real-time-solar-wind and look for sudden activity in multiple data channels. If that happens, then the ACE satellite (about a million miles away) has detected a sudden change in the solar wind, and a geomagnetic storm is likely to start at Earth within an hour or so.

Whether you will see auroras or not during the storm depends on how powerful it is, which determines how far from the poles the auroras will reach and how bright they will be. While the forecast is for a strong storm, we’ll just have to see…

At 2300 UTC (about one hour before this posting) you can see jumps occurred in many channels below. That means that the solar storm may begin right around now (0000 UTC, 8 pm New York Time)

I’m trying to work out how classical statistical mechanics can reduce to thermodynamics in a certain limit. I sketched out the game plan in Part 1 but there are a lot of details to hammer out. While I’m doing this, let me stall for time by explaining more precisely what I mean by ‘thermodynamics’. Thermodynamics is a big subject, but I mean something more precise and limited in scope.

Thermostatic systems

A lot of what we call ‘thermodynamics’, or more precisely ‘classical thermodynamics’, has nothing to do with dynamics. It’s really about systems in equilibrium, not changing, so it actually deserves to be called ‘thermostatics’. Here’s one attempt to formalize a core idea:

Definition. A thermostatic system is a convex space together with a concave function We call the space of states, and call the entropy of the state

There’s a lot packed into this definition:

1. The general concept of convex space: it’s roughly a set where you can take convex combinations of points like where
2. How we make into a convex space: it’s pretty obvious, except that beats in convex combinations, like
3. What is a ‘concave’ function : it’s a function with

          for all and

To see all the details spelled out with lots of examples, try this:

• John Baez, Owen Lynch and Joe Moeller, Compositional thermostatics, J. Math. Phys. 64 (2023) 023304. (Blog articles here.)

We actually defined a category of thermostatic systems and maps between them.

What you can do with a thermostatic system

For now I will only consider thermostatic systems where made into a convex set in the usual way. The idea here is that a state is solely determined by its energy I’m trying to keep things as simple as possible, and generalize later only if my overall plan actually works.

Here’s what people do in this very simple setting. Our thermostatic system is a concave function

describing the entropy of our system when it has energy But often entropy is also a strictly increasing function of energy, with as In this case, it’s impossible for a system to literally maximize entropy. What it does instead is maximize ‘entropy minus how much it spends on energy’ — just as you might try to maximize the pleasure you get from eating doughnuts minus your displeasure at spending money. Thus, if is the ‘cost’ of energy, our system tries to maximize

The cost is the reciprocal of a quantity called temperature:

So, should be called inverse temperature, and the rough intuition you should have is this. When it’s hot, energy is cheap and our system’s energy can afford to be high. When it’s cold, energy costs a lot and our system will not let its energy get too high.

If as a function of is differentiable and has a maximum, the maximum must occur at a point where

or

This gives the fundamental relation between energy, entropy and temperature:

However, the math will work better for us if we use the inverse temperature

Suppose we have a system maximizing for some value of The maximum value of is called free entropy and denoted In short:

or if you prefer

This way of defining in terms of is called a Legendre–Fenchel transform, though conventions vary about the precise definition of this transform, and also its name. Since I’m lazy, I’ll just call it the Legendre transform. For more, read this:

• Simon Willerton, Enrichment and the Legendre–Fenchel transform: Part I, Part II.

The great thing about Legendre transforms is that if a function is convex and lower semicontinuous, when you take its Legendre transform twice you get that function back! This is part of the Fenchel–Moreau theorem. So under these conditions we automatically get another formula that looks very much like the one we’ve just seen:

When has a minimum as a function of and it’s differentiable there, this minimum must occur at a point where

or

Summary

I’m plotting a difficult course between sticking with historical conventions in thermodynamics and trying to make everything mathematically elegant. Everything above looks more elegant if we work with minus the free entropy, Starting from a thermostatic system we then get a beautifully symmetrical pair of relations:

If the infimum at left is achieved at some energy and is differentiable there, then

at this value of and this formula lets us compute the inverse temperature as a function of Similarly, if the infimum at right is achieved at some and is differentiable there, then

at this value of and this formula lets us compute as a function of

When we describe a thermostatic system as a limit of classical statistical mechanical systems, these are the formulas we’d like to see emerge in the limit!

Appendix: the traditional formalism

If you’ve never heard of ‘free entropy’, you may be relieved to hear it’s a repackaging of the more familiar concept of ‘free energy’. The free energy or more specifically the Helmholtz free energy, is related to the free entropy by

Unless you’re a real die-hard fan of thermodynamics, don’t read the following stuff, since it will only further complicate the picture I’ve tried to paint above, which is already blemished by the fact that physicists prefer to I will not provide any profound new insights: I will merely relate what I’ve already explained to an equivalent but more traditional formalism.

I’ve been treating entropy as a function of energy: this is the so-called entropy scheme. But it’s traditional to treat energy as a function of entropy: this is called the energy scheme.

The entropy scheme generalizes better. In thermodynamics we often want to think about situations where entropy is a function of several variables: energy, volume, the amounts of various chemicals, and so on. Then we should work with a thermostatic system where is a convex subset of Everything I did generalizes nicely to that situation, and now will be one of quantities that arise by taking a Legendre transform of

But when entropy is a function of just one variable, energy, people often turn the tables and try to treat energy as a function of entropy, say They then define the free energy as a function of temperature by

This is essentially a Legendre transform — but notice that inside the parentheses we have instead of We can fix this by using a sup instead of an inf, and writing

It’s actually very common to define the Legendre transform using a sup instead of an inf, so that’s fine. The only wrinkle is that this Legendre transform gives us instead of

When the supremum is achieved at a point where is differentiable we have

at that point. When is concave and lower semicontinuous, taking its Legendre transform twice gets us back where we started:

And when this supremum is achieved at a point where is differentiable, we have

To top it off, physicists tend to assume and take values where the suprema above are achieved, and not explicitly write what is a function of what. So they summarize everything I just said with these equations:

 

If instead we take the approach I’ve described, where entropy is treated as a function of energy, it’s natural to focus on the negative free entropy and inverse temperature If we write the equations governing these in the same slapdash style as those above, they look like this:

 

Less familiar, but more symmetrical! The two approaches are related by

I’m trying to work out how classical statistical mechanics can reduce to thermodynamics in a certain limit. I sketched out the game plan in Part 1 but there are a lot of details to hammer out. While I’m doing this, let me stall for time by explaining more precisely what I mean by ‘thermodynamics’. Thermodynamics is a big subject, but I mean something more precise and limited in scope.

Thermostatic systems

A lot of what we call ‘thermodynamics’, or more precisely ‘classical thermodynamics’, has nothing to do with dynamics. It’s really about systems in equilibrium, not changing, so it actually deserves to be called ‘thermostatics’. Here’s one attempt to formalize a core idea:

Definition. A thermostatic system is a convex space $X$ together with a concave function $S \colon X \to [-\infty,\infty]$. We call $X$ the space of states, and call $S(x)$ the entropy of the state $x \in X$.

There’s a lot packed into this definition:

1. The general concept of convex space: it’s roughly a set where you can take convex combinations of points $x,y$, like $a x + (1-a) y$ where $0 \le a \le 1$.
2. How we make $[-\infty,\infty]$ into a convex space: it’s pretty obvious, except that $-\infty$ beats $\infty$ in convex combinations, like $\frac{1}{3} (-\infty) + \frac{2}{3} \infty = -\infty$.
3. What is a ‘concave’ function $S \colon X \to [-\infty,\infty]$: it’s a function with

$$S(a x + (1-a) y) \ge a S(x) + (1-a) S(y) \qquad \text{for} \; 0 \le a \le 1$$

To see all the details spelled out with lots of examples, try this:

• John Baez, Owen Lynch and Joe Moeller, Compositional thermostatics, J. Math. Phys. 64 (2023) 023304. (Blog articles here.)

We actually defined a category of thermostatic systems and maps between them.

What you can do with a thermostatic system

For now I will only consider thermostatic systems where $X = \mathbb{R}$, made into a convex set in the usual way. In these examples a state is solely determined by its energy $E \in \mathbb{R}$. I’m trying to keep things as simple as possible, and generalize later only if my overall plan actually works.

Here’s what people do in this very simple setting. Our thermostatic system is a concave function

$$S \colon \mathbb{R} \to [-\infty, \infty]$$

describing the entropy $S(E)$ of our system when it has energy $E$. But often entropy is also a strictly increasing function of energy, with $S(E) \to \infty$ as $E \to \infty$. In this case, it’s impossible for a system to literally maximize entropy. What it does instead is maximize ‘entropy minus how much it spends on energy’ — just as you might try to maximize the pleasure you get from eating doughnuts minus your displeasure at spending money. Thus, if $C$ is the ‘cost’ of energy, our system tries to maximize

$$S(E) - C E$$

The cost $C$ is the reciprocal of a quantity called temperature:

$$C = \frac{1}{T}$$

So, $C$ should be called inverse temperature, and the rough intuition you should have is this. When it’s hot, energy is cheap and our system’s energy can afford to be high. When it’s cold, energy costs a lot and our system will not let its energy get too high.

If $S(E) - C E$ as a function of $E$ is differentiable and has a maximum, the maximum must occur at a point where

$$\frac{d}{d E} \left(S(E) - C E \right) = 0$$

or

$$\frac{d}{d E} S(E) = C$$

This gives the fundamental relation between energy, entropy and temperature:

$$\frac{d}{d E} S(E) = \frac{1}{T}$$

However, the math will work better for us if we use the inverse temperature.

Suppose we have a system maximizing $S(E) - C E$ for some value of $C$. The maximum value of $S(E) - C E$ is called free entropy and denoted $\Phi$. In short:

$$\Phi(C) = \sup_E \left(S(E) - C E \right)$$

or if you prefer

$$-\Phi(C) = \inf_E \left(C E - S(E) \right)$$

This way of defining $-\Phi$ in terms of $S$ is called a Legendre–Fenchel transform, though conventions vary about the precise definition of this transform, and also its name. Since I’m lazy, I’ll just call it the Legendre transform. For more, read this:

• Simon Willerton, Enrichment and the Legendre–Fenchel transform: Part I, Part II.

The great thing about Legendre transforms is that if a function is convex and lower semicontinuous, when you take its Legendre transform twice you get that function back! This is part of the Fenchel–Moreau theorem. So under these conditions we automatically get another formula that looks very much like the one we’ve just seen:

$$S(E) = \inf_C \left(C E + \Phi(C) \right)$$

When $C E + \Phi(C)$ has a minimum as a function of $C$ and it’s differentiable there, this minimum must occur at a point where

$$\frac{d}{d C} \left(C E + \Phi(C) \right) = 0$$

or

$$\frac{d}{d C} \Phi(C) = -E$$

Summary

I’m plotting a difficult course between sticking with historical conventions in thermodynamics and trying to make everything mathematically elegant. Everything above looks more elegant if we work with minus the free entropy, $\Psi = -\Phi$. Starting from a thermostatic system $S \colon \mathbb{R} \to [-\infty,\infty]$ we then get a beautifully symmetrical pair of relations:

$$\Psi(C) = \inf_E \left(C E - S(E) \right)$$ $$S(E) = \inf_C \left(C E - \Psi(C) \right)$$

If the first infimum is achieved at some energy $E$ and $S$ is differentiable there, then

$$S'(E) = C$$

at this value of $E$, and this formula lets us compute the inverse temperature $C$ as a function of $E$. Similarly, if the second infimum is achieved at some $C$ and $\Psi$ is differentiable there, then

$$\Psi'(C) = E$$

at this value of $C$, and this formula lets us compute $E$ as a function of $C$.

When we describe a thermostatic system as a limit of classical statistical mechanical systems, these are the formulas we’d like to see emerge in the limit!

Appendix: the traditional formalism

If you’ve never heard of ‘free entropy’, you may be relieved to hear it’s a repackaging of the more familiar concept of ‘free energy’. The free energy $F$, or more specifically the Helmholtz free energy, is related to the free entropy by

$$F = - T \Phi$$

Unless you’re a real die-hard fan of thermodynamics, don’t read the following stuff, since it will only further complicate the picture I’ve tried to paint above, which is already blemished by the fact that physicists prefer $\Phi$ to $-\Phi = \Psi$. I will not provide any profound new insights: I will merely relate what I’ve already explained to an equivalent but more traditional formalism.

I’ve been treating entropy as a function of energy: this is the so-called entropy scheme. But it’s traditional to treat energy as a function of entropy: this is called the energy scheme.

The entropy scheme generalizes better. In thermodynamics we often want to think about situations where entropy is a function of several variables: energy, volume, the amounts of various chemicals, and so on. Then we should work with a thermostatic system $S \colon X \to [-\infty,\infty]$ where $X$ is a convex subset of $\mathbb{R}^n$. Everything I did generalizes nicely to that situation, and now $\Psi$ will be one of $n$ quantities that arise by taking a Legendre transform of $S$.

But when entropy is a function of just one variable, energy, people often turn the tables and try to treat energy as a function of entropy, say $E(S)$. They then define the free energy as a function of temperature by

$$F(T) = \inf_S (E(S) - T S)$$

This is essentially a Legendre transform — but notice that inside the parentheses we have $E(S) - T S$ instead of $T S - E(S)$. We can fix this by using a sup instead of an inf, and writing

$$-F(T) = \sup_S (T S - E(S))$$

It’s actually very common to define the Legendre transform using a sup instead of an inf, so that’s fine. The only wrinkle is that this Legendre transform gives us $-F$ instead of $F$.

When the supremum is achieved at a point where $E$ is differentiable we have

$$\displaystyle{ \frac{d}{d S} E(S) = T }$$

at that point. When $E$ is concave and lower semicontinuous, taking its Legendre transform twice gets us back where we started:

$$E(S) = \sup_T (T S + F(T))$$

And when this supremum is achieved at a point where $F$ is differentiable, we have

$$\displaystyle{ \frac{d}{d T} F(T) = - S }$$

To top it off, physicists tend to assume $S$ and $T$ take values where the suprema above are achieved, and not explicitly write what is a function of what. So they would summarize everything I just said with these equations:

$$F = E - T S , \qquad E = T S + F$$

$$\displaystyle{ \frac{d F}{d T} = - S , \qquad \frac{d S}{d E} = T }$$

If instead we take the approach I’ve described, where entropy is treated as a function of energy, it’s natural to focus on the negative free entropy $\Psi$ and inverse temperature $C$. If we write the equations governing these in the same slapdash style as those above, they look like this:

$$\Psi = C E - S, \qquad S = C E - \Psi$$

$$\displaystyle{ \frac{d \Psi}{d C} = E, \qquad \frac{d S}{d E} = C }$$

Less familiar, but more symmetrical! The two approaches are related by

$$\displaystyle{ C = \frac{1}{T}, \qquad \Psi = \frac{F}{T} }$$

Thermodynamics is a funny subject. The first time you go through it, you don’t understand it at all. The second time you go through it, you think you understand it, except for one or two points.The third time you go through it, you know you don’t understand it, but by that time you are so used to the subject, it doesn’t bother you anymore. — Arnold Sommerfeld

 Two very brief items of interest:

• This article is a nice popular discussion of the history of fiber optics and the remarkable progress it's made for telecommunications.  If you're interested in a more expansive but very accessible take on this, I highly recommend City of Light by Jeff Hecht (not to be confused with Eugene Hecht, author of the famous optics textbook).
• I stumbled upon an interesting effort by Yokogawa, the Japanese electronics manufacturer, to provide an alternative path for semiconductor device prototyping that they call minimal fab.  The idea is, instead of prototyping circuits on 200 mm wafers or larger (the industry standard for large scale production is 200 mm or 300 mm.  Efforts to go up to 450 mm wafers have been shelved for now.), there are times when it makes sense to work on 12.5 mm substrates.  Their setup uses maskless photolithography and is intended to be used without needing a cleanroom.  Admittedly, this limits it strongly in terms of device size to 1970s-era micron scales (presumably this could be pushed to 1-2 micron with a fancier litho tool), and it's designed for single-layer processing (not many-layer alignments with vias).  Still, this could be very useful for startup efforts, and apparently it's so simple that a child could use it.

In Part 1, I explained my hopes that classical statistical mechanics reduces to thermodynamics in the limit where Boltzmann’s constant approaches zero. In Part 2, I explained exactly what I mean by ‘thermodynamics’. I also showed how, in this framework, a quantity called ‘negative free entropy’ arises as the Legendre transform of entropy.

In Part 3, I showed how a Legendre transform can arise as a limit of something like a Laplace transform.

Today I’ll put all the puzzle pieces together. I’ll explain exactly what I mean by ‘classical statistical mechanics’, and how negative free entropy is defined in this framework. Its definition involves a Laplace transform. Finally, using the result from Part 3, I’ll show that as negative free entropy in classical statistical mechanics approaches the negative free entropy we’ve already seen in thermodynamics!

Thermodynamics versus statistical mechanics

In a certain important approach to thermodynamics, called classical thermodynamics, we only study relations between the ‘macroscopic observables’ of a system. These are the things you can measure at human-sized distance scales, like the energy, temperature, volume and pressure of a canister of gas. We don’t think about individual atoms and molecules! We say the values of all the macroscopic observables specify the system’s macrostate. So when I formalized thermodynamics using ‘thermostatic systems’ in Part 2, the ‘space of states’ was really a space of macrostates. Real-valued functions on were macroscopic observables.

I focused on the simple case where the macrostate is completely characterized by a single macroscopic observable called its energy In this case the space of macrostates is If we can understand this case, we can generalize later.

In classical statistical mechanics we go further and consider the set of microstates of a system. The microstate specifies all the microscopic details of a system! For example, if our system is a canister of helium, a microstate specifies the position and momentum of each atom. Thus, the space of microstates is typically a high-dimensional manifold — where by ‘high’ I mean something like On the other hand, the space of macrostates is often low-dimensional — where by ‘low’ I mean something between 1 and 10.

To connect thermodynamics to classical statistical mechanics, we need to connect macrostates to microstates. The relation is that each macrostate is a probability distribution of microstates: a probability distribution that maximizes entropy subject to constraints on the expected values of macroscopic observables.

To see in detail how this works, let’s focus on the simple case where our only macroscopic observable is energy.

Classical statistical mechanical systems

Definition. A classical statistical mechanical system is a measure space equipped with a measurable function

We call the set of microstates, call the Hamiltonian, and call the energy of the microstate

It gets tiring to say ‘classical statistical mechanical system’, so I’ll abbreviate this as classical stat mech system.

When we macroscopically measure the energy of a classical stat mech system to be what’s really going on is that the system is in a probability distribution of microstates for which the expected value of energy is A probability distribution is defined to be a measurable function

with

The expected energy in this probability distribution is defined to be

So what I’m saying is that must have

But lots of probability distributions have Which one is the physically correct one? It’s the one that maximizes the Gibbs entropy:

Here is a unit of entropy called Boltzmann’s constant. Its value doesn’t affect which probability distribution maximizes the entropy! But it will affect other things to come.

Now, there may not exist a probability distribution that maximizes subject to the constraint but there often is — and when there is, we can compute what it is. If you haven’t seen this computation, you can find it in my book What is Entropy? starting on page 24. The answer is the Boltzmann distribution:

Here is a number called the inverse temperature. We have to cleverly choose its value to ensure That might not even be possible. But if we get that to happen, will be the probability distribution we seek.

The normalizing factor in the formula above is called the partition function

and it turns out to be important in its own right. The integral may not always converge, but when it does not we’ll just say it equals so we get

One reason the partition function is important is that

where and are computed using the Boltzmann distribution for the given value of For a proof see pages 67–71 of my book, though beware that I use different notation. The quantity above is called the negative free entropy of our classical stat mech system. In my book I focus on a closely related quantity called the ‘free energy’, which is the negative free entropy divided by Also, I talk about instead of the inverse temperature

Let’s call the negative free entropy so

I’ve already discussed negative free entropy in Part 2, but that was for thermostatic systems, and it was defined using a Legendre transform. This new version of negative free entropy applies to classical stat mech systems, and we’ll see it’s defined using a Laplace transform. But they’re related: we’ll see the limit of the new one as is the old one!

The limit as

To compute the limit of the negative free entropy as it will help to introduce some additional concepts.

First, given a classical stat mech system with measure space and Hamiltonian let

be the measure of the set of microstates with energy This is an increasing function of which is right-continuous, so it defines a Lebesgue–Stieltjes measure on the real line. Yes, I taught real analysis for decades and always wondered when I’d actually use this concept in my own work: today is the day!

The reason I care about this measure is that it lets us rewrite the partition function as an integral over the nonnegative real numbers:

Very often the measure is absolutely continuous, which means that

for some locally integrable function I will assume this from now on. We thus have

Physicists call the density of states because if we integrate it over some interval we get ‘the number of states’ in that energy range. At least that’s what physicists say. What we actually get is the measure of the set

Before moving on, a word about dimensional analysis. I’m doing physics, so my quantities have dimensions. In particular, and have units of energy, while the measure is dimensionless, so the density of states has units of energy^-1.

This matters because right now I want to take the logarithm of yet the rules of dimensional analysis include a stern finger-wagging prohibition against taking the logarithm of a quantity unless it’s dimensionless. There are legitimate ways to bend these rules, but I won’t. Instead I’ll follow most physicists and introduce a constant with dimensions of energy, called the energy width. It’s wise to think of this as an arbitrary small unit of energy. Using this we can make all the calculations to come obey the rules of dimensional analysis. If you find that ridiculous, you can mentally set equal to 1.

With that said, now let’s introduce the so-called microcanonical entropy, often called the Boltzmann entropy:

Here we are taking Boltzmann’s old idea of entropy as times the logarithm of the number of states and applying it to the density of states. This allows us to define an entropy of our system at a specific fixed energy Physicists call the set of microstates with energy exactly equal to some number the microcanonical ensemble, and they say the microcanonical entropy is the entropy of the microcanonical ensemble. This is a bit odd, because the set of microstates with energy exactly typically has measure zero. But it’s a useful way of thinking.

In terms of the microcanonical entropy, we have

Combining this with our earlier formula

we get this formula for the partition function:

Now things are getting interesting!

First, the quantity should remind you of the formula we saw in Part 2 for the negative free entropy of a thermostatic system. Remember, that formula was

Second, we instantly get a beautiful formula for the negative free entropy of a classical stat mech system:

Using this we can show the following cool fact:

Main Result. Suppose is a concave function with continuous second derivative. Suppose that for some the quantity has a unique minimum as a function of and at that minimum. Then

The quantity at right deserves to be called the microcanonical negative free entropy. So, when the hypotheses hold,

As the free entropy of a classical statistical mechanical system approaches its microcanonical free entropy!

Here I’ve left off the word ‘negative’ twice, which is fine. But this sentence still sounds like a mouthful. Don’t feel bad if you find it confusing. But it could be the result we need to see how classical statistical mechanics approaches classical thermodynamics as So I plan to study this result further, and hope to explain it much better!

But today I’ll just prove the main result and quit. I figure it’s good to get the math done before talking more about what it means.

Proof of the main result

Suppose all the hypotheses of the main result hold. Spelling out the definition of the negative free entropy what we need to show is

 

You’ll notice that the left hand side involves the energy width In fact it involves the energy width twice: once in a visible way, and once in a concealed way, since These two occurrences of cancel out, so that the left hand side is independent of You can either check this directly, or note that the negative free entropy is and the partition function was originally defined in a way that didn’t involve

So, we are allowed to let be any positive number we want, and from now on I’ll take

Next, we need a theorem from Part 3. My argument for that theorem was not a full mathematical proof — I explained the hole I still need to fill — so I cautiously called it an ‘almost proved theorem’. Here it is:

Almost Proved Theorem. Suppose that is a concave function with continuous second derivative. Suppose that for some the function has a unique minimum at and Then as we have

Now let’s use this to prove our main result! To do this, take

Then we get

 

and this is exactly what we want… in the case where which is sufficiently general.       ∎

In Part 1, I explained my hopes that classical statistical mechanics reduces to thermodynamics in the limit where Boltzmann’s constant $k$ approaches zero. In Part 2, I explained exactly what I mean by ‘thermodynamics’. I also showed how, in this framework, a quantity called ‘negative free entropy’ arises as the Legendre transform of entropy.

In Part 3, I showed how a Legendre transform can arise as a limit of something like a Laplace transform.

Today I’ll put all the puzzle pieces together. I’ll explain exactly what I mean by ‘classical statistical mechanics’, and how negative free entropy is defined in this framework. Its definition involves a Laplace transform. Finally, using the result from Part 3, I’ll show that as $k \to 0$, negative free entropy in classical statistical mechanics approaches the negative free entropy we’ve already seen in thermodynamics!

Thermodynamics versus statistical mechanics

In a certain important approach to thermodynamics, called classical thermodynamics, we only study relations between the ‘macroscopic observables’ of a system. These are the things you can measure at human-sized distance scales, like the energy, temperature, volume and pressure of a canister of gas. We don’t think about individual atoms and molecules! We say the values of all the macroscopic observables specify the system’s macrostate. So when I formalized thermodynamics using ‘thermostatic systems’ in Part 2, the ‘space of states’ $X$ was really a space of macrostates. Real-valued functions on $X$ were macroscopic observables.

I focused on the simple case where the macrostate is completely characterized by a single macroscopic observable called its energy $E \in [0,\infty)$. In this case the space of macrostates is $X = [0,\infty)$. If we can understand this case, we can generalize later.

In classical statistical mechanics we go further and consider the set $\Omega$ of microstates of a system. The microstate specifies all the microscopic details of a system! For example, if our system is a canister of helium, a microstate specifies the position and momentum of each atom. Thus, the space of microstates is typically a high-dimensional manifold — where by ‘high’ I mean something like $10^{23}$. On the other hand, the space of macrostates is often low-dimensional — where by ‘low’ I mean something between 1 and 10.

To connect thermodynamics to classical statistical mechanics, we need to connect macrostates to microstates. The relation is that each macrostate is a probability distribution of microstates: a probability distribution that maximizes entropy subject to constraints on the expected values of macroscopic observables.

To see in detail how this works, let’s focus on the simple case where our only macroscopic observable is energy.

Classical statistical mechanical systems

Definition. A classical statistical mechanical system is a measure space $(\Omega,\mu)$ equipped with a measurable function

$$H \colon \Omega \to [0,\infty)$$

We call $\Omega$ the set of microstates, call $H$ the Hamiltonian, and call $H(x)$ the energy of the microstate $x \in \Omega$.

It gets tiring to say ‘classical statistical mechanical system’, so I’ll abbreviate this as classical stat mech system.

When we macroscopically measure the energy of a classical stat mech system to be $E$, what’s really going on is that the system is in a probability distribution of microstates for which the expected value of energy is $E$. A probability distribution is defined to be a measurable function

$$p \colon \Omega \to [0,\infty)$$

with

$$\displaystyle{ \int_\Omega p(x) \, d\mu(x) = 1 }$$

The expected energy in this probability distribution is defined to be

$$\displaystyle{ \langle H \rangle = \int_\Omega H(x) \, p(x) \, d \mu(x) }$$

So what I’m saying is that $p$ must have

$$\langle H \rangle = E$$

But lots of probability distributions have $\langle H \rangle = E$. Which one is the physically correct one? It’s the one that maximizes the Gibbs entropy:

$$\displaystyle{ S = - k \int_\Omega p(x) \, \ln p(x) \, d\mu(x) }$$

Here $k$ is a unit of entropy called Boltzmann’s constant. Its value doesn’t affect which probability distribution maximizes the entropy! But it will affect other things to come.

Now, there may not exist a probability distribution $p$ that maximizes $S$ subject to the constraint $\langle H \rangle = E$, but there often is — and when there is, we can compute what it is. If you haven’t seen this computation, you can find it in my book What is Entropy? starting on page 24. The answer is the Boltzmann distribution:

$$\displaystyle{ p(x) = \frac{e^{-C H(x)/k}}{\int_\Omega e^{-C H(x)/k} \, d \mu(x)} }$$

Here $C$ is a number called the inverse temperature. We have to cleverly choose its value to ensure $\langle H \rangle = E$. That might not even be possible. But if we get that to happen, $p$ will be the probability distribution we seek.

The normalizing factor in the formula above is called the partition function

$$Z_k(C) = \int_\Omega e^{-C H(x)/k} \, d\mu(x)$$

and it turns out to be important in its own right. The integral may not always converge, but when it does not we’ll just say it equals $+\infty$, so we get

$$Z_k \colon [0,\infty) \to [0,\infty]$$

One reason the partition function is important is that

$$- k \ln Z_k(C) = C \langle H \rangle - S$$

where $\langle H \rangle$ and $S$ are computed using the Boltzmann distribution for the given value of $C$ For a proof see pages 67–71 of my book, though beware that I use different notation. The quantity above is called the negative free entropy of our classical stat mech system. In my book I focus on a closely related quantity called the ‘free energy’, which is the negative free entropy divided by $C$. Also, I talk about $\beta = 1/k T$ instead of the inverse temperature $C = 1/T$.

Let’s call the negative free entropy $\Psi_k(C)$, so

$$\displaystyle{ \Psi_k(C) = -k \ln Z_k(C) = - k \ln \int_\Omega e^{-C H(x)/k} \, d\mu(x) }$$

I’ve already discussed negative free entropy in Part 2, but that was for thermostatic systems, and it was defined using a Legendre transform. This new version of negative free entropy applies to classical stat mech systems, and we’ll see it’s defined using a Laplace transform. But they’re related: we’ll see the limit of the new one as $k \to 0$ is the old one!

The limit as $k \to 0$

To compute the limit of the negative free entropy $\Psi_k(C)$ as $k \to 0$ it will help to introduce some additional concepts.

First, given a classical stat mech system with measure space $(\Omega, \mu)$ and Hamiltonian $H \colon \Omega \to \mathbb{R}$, let

$$\nu(E) = \mu(\{x \in \Omega \vert \; H(x) \le E \}$$

be the measure of the set of microstates with energy $\le E$. This is an increasing function of $E \in \mathbb{R}$ which is right-continuous, so it defines a Lebesgue–Stieltjes measure $\nu$ on the real line. Yes, I taught real analysis for decades and always wondered when I’d actually use this concept in my own work: today is the day!

The reason I care about this measure $\nu$ is that it lets us rewrite the partition function as an integral over the nonnegative real numbers:

$$\displaystyle{ Z_k(C) = \int_0^\infty e^{-C E/k} \, d\nu(E) }$$

Very often the measure $\nu$ is absolutely continuous, which means that

$$d\nu(E) = g(E) \, d E$$

for some locally integrable function $g \colon \mathbb{R} \to \mathbb{R}$. I will assume this from now on. We thus have

$$\displaystyle{ Z_k(C) = \int_0^\infty e^{-C E/k} \, g(E) \, d E }$$

Physicists call $g$ the density of states because if we integrate it over some interval $[E, E + \Delta E]$ we get ‘the number of states’ in that energy range. At least that’s what physicists say. What we actually get is the measure of the set

$$\{x \in X: \; E \le H(x) \le E + \Delta E \}$$

Before moving on, a word about dimensional analysis. I’m doing physics, so my quantities have dimensions. In particular, $E$ and $d E$ have units of energy, while the measure $d\nu(E)$ is dimensionless, so the density of states $g(E)$ has units of energy^-1.

This matters because right now I want to take the logarithm of $g(E)$, yet the rules of dimensional analysis include a stern finger-wagging prohibition against taking the logarithm of a quantity unless it’s dimensionless. There are legitimate ways to bend these rules, but I won’t. Instead I’ll follow most physicists and introduce a constant with dimensions of energy, $w$, called the energy width. It’s wise to think of this as an arbitrary small unit of energy. Using this we can make all the calculations to come obey the rules of dimensional analysis. If you find that ridiculous, you can mentally set $w$ equal to 1.

With that said, now let’s introduce the so-called microcanonical entropy, often called the Boltzmann entropy:

$$S_{\mathrm{micro}}(E) = k \ln (w g(E))$$

Here we are taking Boltzmann’s old idea of entropy as $k$ times the logarithm of the number of states and applying it to the density of states. This allows us to define an entropy of our system at a specific fixed energy $E$. Physicists call the set of microstates with energy exactly equal to some number $E$ the microcanonical ensemble, and they say the microcanonical entropy is the entropy of the microcanonical ensemble. This is a bit odd, because the set of microstates with energy exactly $E$ typically has measure zero. But it’s a useful way of thinking.

In terms of the microcanonical entropy, we have

$$\displaystyle{ g(E) = \frac{1}{w} e^{S_{\mathrm{micro}}(E)/k} }$$

Combining this with our earlier formula

$$\displaystyle{ Z_k(C) = \int_0^\infty e^{-C E/k} g(E) \, d E }$$

we get this formula for the partition function:

$$\displaystyle{ Z_k(C) = \int_0^\infty e^{-(C E - S_{\mathrm{micro}}(E))/k} \, \frac{d E}{w} }$$

Now things are getting interesting!

First, the quantity $C E - S_{\mathrm{micro}}(E)$ should remind you of the formula we saw in Part 2 for the negative free entropy of a thermostatic system. Remember, that formula was

$$\Psi(C) = \inf_E (C E - S(E))$$

Second, we instantly get a beautiful formula for the negative free entropy of a classical stat mech system:

$$\displaystyle{ \Psi_k(C) = - k \ln Z_k(C) = - k \ln \int_0^\infty e^{-(C E - S_{\mathrm{micro}}(E))/k} \, \frac{d E}{w} }$$

Using this we can show the following cool fact:

Main Result. Suppose $S_{\mathrm{micro}} \colon [0,\infty) \to \mathbb{R}$ is a concave function with continuous second derivative. Suppose that for some $C \gt 0$ the quantity $C E - S_{\mathrm{micro}}(E)$ has a unique minimum as a function of $E$, and $S''_{\mathrm{micro}} \lt 0$ at that minimum. Then

$$\displaystyle{ \lim_{k \to 0} \Psi_k(C) \quad = \quad \inf_E \left(C E - S_{\mathrm{micro}}(E)\right) }$$

The quantity at right deserves to be called the microcanonical negative free entropy. So, when the hypotheses hold,

As $k \to 0$, the free entropy of a classical statistical mechanical system approaches its microcanonical free entropy!

Here I’ve left off the word ‘negative’ twice, which is fine. But this sentence still sounds like a mouthful. Don’t feel bad if you find it confusing. But it could be the result we need to see how classical statistical mechanics approaches classical thermodynamics as $k \to 0$. So I plan to study this result further, and hope to explain it much better!

But today I’ll just prove the main result and quit. I figure it’s good to get the math done before talking more about what it means.

Proof of the main result

Suppose all the hypotheses of the main result hold. Spelling out the definition of the negative free entropy $\Psi_k(C)$, what we need to show is

$$\displaystyle{ \lim_{k \to 0} - k \ln \int_0^\infty e^{-(C E - S_{\mathrm{micro}}(E))/k} \, \frac{d E}{w} \quad = \quad \inf_E \left(C E - S_{\mathrm{micro}}(E)\right) }$$

You’ll notice that the left hand side involves the energy width $w$. In fact it involves the energy width twice: once in a visible way, and once in a concealed way, since $S_{\mathrm{micro}}(E) = k \ln (w g(E))$. These two occurrences of $w$ cancel out, so that the left hand side is independent of $w$. You can either check this directly, or note that the negative free entropy is $-k \ln Z_k(C)$ and the partition function $Z_k$ was originally defined in a way that didn’t involve $w$.

So, we are allowed to let $w$ be any positive number we want, and from now on I’ll take $w = 1$.

Next, we need a theorem from Part 3. My argument for that theorem was not a full mathematical proof — I explained the hole I still need to fill — so I cautiously called it an ‘almost proved theorem’. Here it is:

Almost Proved Theorem. Suppose that $f \colon [0,\infty) \to \mathbb{R}$ is a concave function with continuous second derivative. Suppose that for some $s \gt 0$ the function $s x - f(x)$ has a unique minimum at $x_0$, and $f''(x_0) \lt 0$. Then as $\beta \to +\infty$ we have

$$\displaystyle{ \lim_{\beta \to +\infty} -\frac{1}{\beta} \ln \int_0^\infty e^{-\beta (s x - f(x))} \, d x \; = \; \inf_x \left( s x - f(x)\right) }$$

Now let’s use this to prove our main result! To do this, take

$$s = C, \quad x = E, \quad f(x) = S_{\mathrm{micro}}(E), \quad \beta = 1/k$$

Then we get

$$\displaystyle{\lim_{k \to 0} - k \ln \int_0^\infty e^{(C E - S_{\mathrm{micro}}(E))/k} \, d E \quad = \quad \inf_E \left(C E - S_{\mathrm{micro}}(E) \right) }$$

and this is exactly what we want… in the case where $w = 1$, which is sufficiently general.       ∎

Dan Faggella recorded an unusual podcast with me that’s now online. He introduces me as a “quantum physicist,” which is something that I never call myself (I’m a theoretical computer scientist) but have sort of given up on not being called by others. But the ensuing 85-minute conversation has virtually nothing to do with physics, or anything technical at all.

Instead, Dan pretty much exclusively wants to talk about moral philosophy: my views about what kind of AI, if any, would be a “worthy successor to humanity,” and how AIs should treat humans and vice versa, and whether there’s any objective morality at all, and (at the very end) what principles ought to guide government regulation of AI.

So, I inveigh against “meat chauvinism,” and expand on the view that locates human specialness (such as it is) in what might be the unclonability, unpredictability, and unrewindability of our minds, and plead for comity among the warring camps of AI safetyists.

The central point of disagreement between me and Dan ended up centering around moral realism: Dan kept wanting to say that a future AGI’s moral values would probably be as incomprehensible to us as are ours to a sea snail, and that we need to make peace with that. I replied that, firstly, things like the Golden Rule strike me as plausible candidates for moral universals, which all thriving civilizations (however primitive or advanced) will agree about in the same way they agree about 5 being a prime number. And secondly, that if that isn’t true—if the morality of our AI or cyborg descendants really will be utterly alien to us—then I find it hard to have any preferences at all about the future they’ll inhabit, and just want to enjoy life while I can! That which (by assumption) I can’t understand, I’m not going to issue moral judgments about either.

Anyway, rewatching the episode, I was unpleasantly surprised by my many verbal infelicities, my constant rocking side-to-side in my chair, my sometimes talking over Dan in my enthusiasm, etc. etc., but also pleasantly surprised by the content of what I said, all of which I still stand by despite the terrifying moral minefields into which Dan invited me. I strongly recommend watching at 2x speed, which will minimize the infelicities and make me sound smarter. Thanks so much to Dan for making this happen, and let me know what you think!

Added: See here for other podcasts in the same series and on the same set of questions, including with Nick Bostrom, Ben Goertzel, Dan Hendrycks, Anders Sandberg, and Richard Sutton.

read more

An older professor in my field has a quirk: every time he organizes a conference, he publishes all the talks in a conference proceeding.

In some fields, this would be quite normal. In computer science, where progress flows like a torrent, new developments are announced at conferences long before they have the time to be written up carefully as a published paper. Conference proceedings are summaries of what was presented at the conference, published so that anyone can catch up on the new developments.

In my field, this is rarer. A few results at each conference will be genuinely new, never-before-published discoveries. Most, though, are talks on older results, results already available online. Writing them up again in summarized form as a conference proceeding seems like a massive waste of time.

The cynical explanation is that this professor is doing this for the citations. Each conference proceeding one of his students publishes is another publication on their CV, another work that they can demand people cite whenever someone uses their ideas or software, something that puts them above others’ students without actually doing any extra scientific work.

I don’t think that’s how this professor thinks about it, though. He certainly cares about his students’ careers, and will fight for them to get cited as much as possible. But he asks everyone at the conference to publish a proceeding, not just his students. I think he’d argue that proceedings are helpful, that they can summarize papers in new ways and make them more accessible. And if they give everyone involved a bit more glory, if they let them add new entries to their CV and get fancy books on their shelves, so much the better for everyone.

My guess is, he really believes something like that. And I’m fairly sure he’s wrong.

The occasional conference proceeding helps, but only because it makes us more flexible. Sometimes, it’s important to let others know about a new result that hasn’t been published yet, and we let conference proceedings go into less detail than a full published paper, so this can speed things up. Sometimes, an old result can benefit from a new, clearer explanation, which normally couldn’t be published without it being a new result (or lecture notes). It’s good to have the option of a conference proceeding.

But there is absolutely no reason to have one for every single talk at a conference.

Between the cynical reason and the explicit reason, there’s the banal one. This guy insists on conference proceedings because they were more useful in the past, because they’re useful in other fields, and because he’s been doing them himself for years. He insists on them because to him, they’re a part of what it means to be a responsible scientist.

And people go along with it. Because they don’t want to get into a fight with this guy, certainly. But also because it’s a bit of extra work that could give a bit of a career boost, so what’s the harm?

I think something similar to this is why academic journals still work the way they do.

In the past, journals were the way physicists heard about new discoveries. They would get each edition in the mail, and read up on new developments. The journal needed to pay professional copyeditors and printers, so they needed money, and they got that money from investors by being part of for-profit companies that sold shares.

Now, though, physicists in my field don’t read journals. We publish our new discoveries online on a non-profit website, formatting them ourselves with software that uses the same programming skills we use in the rest of our professional lives. We then discuss the papers in email threads and journal club meetings. When a paper is wrong, or missing something important, we tell the author, and they fix it.

Oh, and then after that we submit the papers to the same for-profit journals and the same review process that we used to use before we did all this, listing the journals that finally accept the papers on our CVs.

Why do we still do that?

Again, you can be cynical. You can accuse the journals of mafia-ish behavior, you can tie things back to the desperate need to publish in high-ranked journals to get hired. But I think the real answer is a bit more innocent, and human, than that.

Imagine that you’re a senior person in the field. You may remember the time before we had all of these nice web-based publishing options, when journals were the best way to hear about new developments. More importantly than that, though, you’ve worked with these journals. You’ve certainly reviewed papers for them, everyone in the field does that, but you may have also served as an editor, tracking down reviewers and handling communication between the authors and the journal. You’ve seen plenty of cases where the journal mattered, where tracking down the right reviewers caught a mistake or shot down a crackpot’s ambitions, where the editing cleaned something up or made a work more appear more professional. You think of the journals as having high standards, standards you have helped to uphold: when choosing between candidates for a job, you notice that one has several papers in Physical Review Letters, and remember papers you’ve rejected for not meeting what you intuited were that journal’s standards. To you, journals are a key part of being a responsible scientist.

Does any of that make journals worth it, though?

Well, that depends on costs. It depends on alternatives. It depends not merely on what the journals catch, but on how often they do it, and how much would have been caught on its own. It depends on whether the high standards you want to apply to job applicants are already being applied by the people who write their recommendation letters and establish their reputations.

And you’re not in a position to evaluate any of that, of course. Few people are, who don’t spend a ton of time thinking about scientific publishing.

And thus, for the non-senior people, there’s not much reason to push back. One hears a few lofty speeches about Elsevier’s profits, and dreams about the end of the big for-profit journals. But most people aren’t cut out to be crusaders or reformers, especially when they signed up to be scientists. Most people are content not to annoy the most respected people in their field by telling them that something they’ve spent an enormous amount of time on is now pointless. Most people want to be seen as helpful by these people, to not slack off on work like reviewing that they argue needs doing.

And most of us have no reason to think we know that much better, anyway. Again, we’re scientists, not scientific publishing experts.

I don’t think it’s good practice to accuse people of cognitive biases. Everyone thinks they have good reasons to believe what they believe, and the only way to convince them is to address those reasons.

But the way we use journals in physics these days is genuinely baffling. It’s hard to explain, it’s the kind of thing people have been looking quizzically at for years. And this kind of explanation is the only one I’ve found that matches what I’ve seen. Between the cynical explanation and the literal arguments, there’s the basic human desire to do what seems like the responsible thing. That tends to explain a lot.

Q1: Who will you be voting for in November?

A: Kamala Harris (and mainstream Democrats all down the ballot), of course.

Q2: Of course?

A: If the alternative is Trump, I would’ve voted for Biden’s rotting corpse. Or for Hunter Biden. Or for…

Q3: Why can’t you see this is just your Trump Derangement Syndrome talking?

A: Look, my basic moral commitments remain pretty much as they’ve been since childhood. Namely, that I’m on the side of reason, Enlightenment, scientific and technological progress, secular government, pragmatism, democracy, individual liberty, justice, intellectual honesty, an American-led peaceful world order, preservation of the natural world, mitigation of existential risks, and human flourishing. (Crazy and radical, I know.)

Only when choosing between candidates who all espouse such values, do I even get the luxury of judging them on any lower-order bits. Sadly, I don’t have that luxury today. Trump’s values, such as they are, would seem to be “America First,” protectionism, vengeance, humiliation of enemies, winning at all costs, authoritarianism, the veneration of foreign autocrats, and the veneration of himself. No amount of squinting can ever reconcile those with the values I listed before.

Q4: Is that all that’s wrong with him?

A: No, there are also the lies, and worst of all the “Big Lie.” Trump is the first president in US history to incite a mob to try to overturn the results of an election. He was serious! He very nearly succeeded, and probably would have, had Mike Pence been someone else. It’s now inarguable that Trump rejects the basic rules of our system, or “accepts” them only when he wins. We’re numb from having heard it so many times, but it’s a big deal, as big a deal as the Civil War was.

Q5: Oh, so this is about your precious “democracy.” Why do you care? Haven’t you of all people learned that the masses are mostly idiots and bullies, who don’t deserve power? As Curtis Yarvin keeps trying to explain to you, instead of “democracy,” you should want a benevolent king or dictator-CEO, who could offer a privileged position to the competent scientists like yourself.

A: Yeah, so how many examples does history furnish where that worked out well? I suppose you might make a partial case for Napoleon, or Ataturk? More to the point: even if benevolent, science-and-reason-loving authoritarian strongmen are possible in theory, do you really expect me to believe that Trump could be one of them? I still love how Scott Alexander put it in 2016:

Can anyone honestly say that Trump or his movement promote epistemic virtue? That in the long-term, we’ll be glad that we encouraged this sort of thing, that we gave it power and attention and all the nutrients it needed to grow? That the road to whatever vision of a just and rational society we imagine, something quiet and austere with a lot of old-growth trees and Greek-looking columns, runs through LOCK HER UP?

I don’t like having to vote for the lesser of two evils. But at least I feel like I know who it is.

Q6: But what about J. D. Vance? He got his start in Silicon Valley, was championed by Peter Thiel, and is obviously highly intelligent. Doesn’t he seem like someone who might listen to and empower tech nerds like yourself?

A: Who can say what J. D. Vance believes? Here are a few choice quotes of his from eight years ago:

I’m obviously outraged at Trump’s rhetoric, and I worry most of all about how welcome Muslim citizens feel in their own country. But I also think that people have always believed crazy shit (I remember a poll from a few years back suggesting that a near majority of democratic voters blame ‘the Jews’ for the financial crisis). And there have always been demagogues willing to exploit the people who believe crazy shit.

The more white people feel like voting for trump, the more black people will suffer. I really believe that.

[Trump is] just a bad man. A morally reprehensible human being.

To get from that to being Trump’s running mate is a Simone-Biles-like feat of moral acrobatics. Vance reminds me of the famous saying by L. Ron Hubbard from his pre-Dianetics days: “If a man really wants to make a million dollars, the best way would be to start his own religion.” (And I feel like Harris’s whole campaign strategy should just be to replay Vance’s earlier musings in wall-to-wall ads while emphasizing her agreement with them.) No, Vance is not someone I trust to share my values, if he has values at all.

Q7: What about the other side’s values, or lack thereof? I mean, don’t you care that the whole Democratic establishment—including Harris—colluded to cover up that Biden was senile and cognitively unfit to be president now, let alone for another term?

A: Look, we’ve all seen what happens as a relative gets old. It’s gradual. It’s hard for anyone to say at which specific moment they can no longer drive a car, or be President of the United States, or whatever. This means that I don’t necessarily read evil intent into the attempts to cover up Biden’s decline—merely an epic, catastrophic failure of foresight. That failure of foresight itself would’ve been a huge deal in normal circumstances, but these are not normal circumstances—not if you believe, as I do, that the alternative is the beginning of the end of a 250-year-old democratic experiment.

Q8: Oh stop being so melodramatic. What terrible thing happened to you because of Trump’s first term? Did you lose your job? Did fascist goons rough you up in the street?

A: Well, my Iranian PhD student came close to having his visa revoked, and it became all but impossible to recruit PhD students from China. That sucked, since I care about my students’ welfare like I care about my own. Also, the downfall of Roe v. Wade, which enabled Texas’ draconian new abortion laws, made it much harder for us to recruit faculty at UT Austin. But I doubt any of that will impress you. “Go recruit American students,” you’ll say. “Go recruit conservative faculty who are fine with abortion being banned.”

The real issue is that Trump was severely restrained in his first term, by being surrounded by people who (even if, in many cases, they started out loyal to him) were also somewhat sane and valued the survival of the Republic. Alas, he learned from that, and he won’t repeat that mistake the next time.

Q9: Why do you care so much about Trump’s lies? Don’t you realize that all politicians lie?

A: Yes, but there are importantly different kinds of lies. There are white lies. There are scheming, 20-dimensional Machiavellian lies, like a secret agent’s cover story (or is that only in fiction?). There are the farcical, desperate, ever-shifting lies of the murderer to the police detective or the cheating undergrad to the professor. And then there are the lies of bullies and mob bosses and populist autocrats, which are special and worse.

These last, call them power-lies, are distinguished by the fact that they aren’t even helped by plausibility. Often, as with conspiracy theories (which strongly overlap with power-lies), the more absurd the better. Obama was born in Kenya. Trump’s crowd was the biggest in history. The 2020 election was stolen by a shadowy conspiracy involving George Soros and Dominion and Venezuela.

The central goal of a power-lie is just to demonstrate your power to coerce others into repeating it, much like with the Party making Winston Smith affirm 2+2=5, or Petruchio making Katharina call the sun the moon in The Taming of the Shrew. A closely-related goal is as a loyalty test for your own retinue.

It’s Trump’s embrace of the power-lie that puts him beyond the pale for me.

Q10: But Scott, we haven’t even played our “Trump” card yet. Starting on October 7, 2023, did you not witness thousands of your supposed allies, the educated secular progressives on “the right side of history,” cheer the sadistic mass-murder of Jews—or at least, make endless excuses for those who did? Did this not destabilize your entire worldview? Will you actually vote for a party half of which seems at peace with the prospect of your family members’ physical annihilation? Or will you finally see who your real friends now are: Arkansas MAGA hillbillies who pray for your people’s survival?

A: Ah, this is your first slash that’s actually drawn blood. I won’t pretend that the takeover of part of the US progressive coalition by literal Hamasniks hasn’t been one of the most terrifying experiences of my life. Yes, if I had to be ruled by either (a) a corrupt authoritarian demagogue or (b) an idiot college student chanting for “Intifada Revolution,” I’d be paralyzed. So it’s lucky that I don’t face that choice! I get to vote, once more, for a rather boring mainstream Democrat—alongside at least 70% of American Jews. The idea of Harris as an antisemite would be ludicrous even if she didn’t have a Jewish husband or wasn’t strongly considering a pro-Israel Jew as her running mate.

Q11: Sure, Kamala Harris might mouth all the right platitudes about Israel having a right to defend itself, but she’ll constantly pressure Israel to make concessions to Hamas and Hezbollah. She’ll turn a blind eye to Iran’s imminent nuclearization. Why don’t you stay up at night worrying that, if you vote for a useful idiot like her, you’ll have Israel’s annihilation and a second Holocaust on your conscience forever?

A: Look, oftentimes—whenever, for example, I’m spending hours reading anti-Zionists on Twitter—I feel like there’s no limit to how intensely Zionist I am. On reflection, though, there is a limit. Namely, I’m not going to be more Zionist than the vast majority of my Israeli friends and colleagues—the ones who served in the IDF, who in some cases did reserve duty in Gaza, who prop up the Israeli economy with their taxes, and who will face the consequences of whatever happens more directly than I will. With few exceptions, these friends despise the Trump/Bibi alliance with white-hot rage, and they desperately want more moderate leadership in both countries.

Q12: Suppose I concede that Kamala is OK on Israel. We both know that she’s not the future of the Democratic Party, any more than Biden is. The future is what we all saw on campuses this spring. “Houthis Houthis make us proud, turn another ship around.” How can you vote for a party whose rising generation seems to want you and your family dead?

A: Let me ask you something. When Trump won in 2016, did that check the power of the campus radicals? Or as Scott Alexander prophesied at the time, did it energize and embolden them like nothing else, by dramatically confirming their theology of a planet held hostage by the bullying, misogynistic rich white males? I fundamentally reject your premise that, if I’m terrified of crazy left-wing extremists, then a good response is to vote for the craziest right-wing extremists I can find, in hopes that the two will somehow cancel each other out. Instead I should support a coherent Enlightenment alternative to radicalism, or the closest thing to that available.

Q13: Even leaving aside Israel, how can you not be terrified by what the Left has become? Which side denounced you on social media a decade ago, as a misogynist monster who wanted all women to be his sex slaves? Which side tried to ruin your life and career? Did we, the online rightists, do that? No. We did not. We did nothing worse to you than bemusedly tell you to man up, grow a pair, and stop pleading for sympathy from feminists who will hate you no matter what.

A: I’ll answer with a little digression. Back in 2017, when Kamala Harris was in the Senate, her office invited me to DC to meet with them to provide advice about the National Quantum Initiative Act, which Kamala was then spearheading. Kamala herself sent regrets that she couldn’t meet me, because she had to be at the Kavanaugh hearings. I have (nerdy, male) friends who did meet her about tech policy and came away with positive impressions.

And, I dunno, does that sound like someone who wants me dead for the crime of having been born a nerdy heterosexual male? Or having awkwardly and ineptly asked women on dates, including the one who became my wife? OK, maybe Amanda Marcotte wants me dead for those crimes. Maybe Arthur Chu does (is he still around?). Good that they’re not running for president then.

Q14: Let me try one more time to show you how much your own party hates you. Which side has been at constant war against the SAT and other standardized tests, and merit-based college admissions, and gifted programs, and academic tracking and acceleration, and STEM magnet schools, and every single other measure by which future young Scott Aaronsons (and Saket Agrawals) might achieve their dreams in life? Has that been our side, or theirs?

A: To be honest, I haven’t seen the Trump or Harris campaigns take any position on any of these issues. Even if they did, there’s very little that the federal government can do: these battles happen in individual states and cities and counties and universities. So I’ll vote for Harris while continuing to advocate for what I think is right in education policy.

Q15: Can you not see that Kamala Harris is a vapid, power-seeking bureaucratic machine—that she has no fixed principles at all? For godsakes, she all but condemned Biden as a racist in the 2020 primary, then agreed to serve as his running mate!

A: I mean, she surely has more principles than Vance does. As far as I can tell, for example, she’s genuinely for abortion rights (as I am). Even if she believed in nothing, though, better a cardboard cutout on which values I recognize are written, than a flesh-and-blood person shouting values that horrify me.

Q16: What, if anything, could Republicans do to get you to vote for them?

A: Reject all nutty conspiracy theories. Fully, 100% commit to the peaceful transfer of power. Acknowledge the empirical reality of human-caused climate change, and the need for both technological and legislative measures to slow it and mitigate its impacts. Support abortion rights, or at least a European-style compromise on abortion. Republicans can keep the anti-wokeness stuff, which actually seems to have become their defining issue. If they do all that, and also the Democrats are taken over by frothing radicals who want to annihilate the state of Israel and abolish the police … that’s, uh, probably the point when I start voting Republican.

Q17: Aha, so you now admit that there exist conceivable circumstances that would cause you to vote Republican! In that case, why did you style yourself “Never-Trump From Here to Eternity”?

A: Tell you what, the day the Republicans (and Trump himself?) repudiate authoritarianism and start respecting election outcomes, is the day I’ll admit my title was hyperbolic.

Q18: In the meantime, will you at least treat us Trump supporters with civility and respect?

A: Not only does civil disagreement not compromise any of my values, it is a value to which I think we should all aspire. And to whatever extent I’ve fallen short of that ideal—even when baited into it—I’m sorry and I’ll try to do better. Certainly, age and experience have taught me that there’s hardly anyone so far gone that I can’t find something on which I agree with them, while disagreeing with most of the rest of the world.

Update: I’d been wavering—should I vote for the terrifying lunatic, ranting about trans criminal illegal aliens cooking cat meat, or for the nice woman constantly making faces as though the lunatic was completely cracking her up? But when the woman explicitly came out in favor of AI and quantum computing research … that really sealed the deal for me.

Between roughly 2001 and 2018, I’ve happy to have done some nice things in quantum computing theory, from the quantum lower bound for the collision problem to the invention of shadow tomography.  I hope that’s not the end of it.  QC research brought me about as much pleasure as anything in life did.  So I hope my tired brain can be revved up a few more times, between now and whenever advances in AI or my failing health or the collapse of civilization makes the issue moot. If not, though, there are still many other quantum activities to fill my days: teaching (to which I’ve returned after two years), advising my students and postdocs, popular writing and podcasts and consulting, and of course, learning about the latest advances in quantum computing so I can share them with you, my loyal readers.

On that note, what a time it is in QC!  Basically, one experimental milestone after another that people talked about since the 90s is finally being achieved, to the point where it’s become hard to keep up with it all. Briefly though:

A couple weeks ago, the Google group announced an experiment that achieved net gain from the use of Kitaev’s surface code, using 101 physical qubits to encode 1 logical qubit. The headline result here is that, in line with theory, they see the performance improve as they pass to larger codes with more physical qubits and higher distance. Their best demonstrated code has a distance of 7, which is enough to get “beyond break-even” (their logical qubit lasts more than twice as long as the underlying physical qubits), and is also enough that any future improvements to the hardware will get amplified a lot. With superconducting qubits, one is (alas) still limited by how many one can cram onto a single chip. On paper, though, they say that scaling the same setup to a distance-27 code with ~1500 physical qubits would get them down to an error rate of 10^-6, good enough to be a building block in a future fault-tolerant QC. They also report correlated bursts of errors that come about once per hour, from a still-unknown source that appears not to be cosmic rays. I hope it’s not Gil Kalai in the next room.

Separately, just this morning, Microsoft and Quantinuum announced that they entangled 12 logical qubits on a 56-physical-qubit trapped-ion processor, building on earlier work that I blogged about in April. They did this by applying a depth-3 logical circuit with 12 logical CNOT gates, to prepare a cat state. They report an 0.2% error rate when they do this, which is 11x better than they would’ve gotten without using error-correction. (Craig Gidney, in the comments, says that these results still involve postselection.)

The Microsoft/Quantinuum group also did what they called a “chemistry simulation” involving 13 physical qubits. The latter involved “only” 2 logical qubits and 4 logical gates, but 3 of those gates were non-Clifford, which are the hard kind when one is doing error-correction using a transversal code. (CNOT, by contrast, is a Clifford gate.)

Apart from the fact that Google is using superconducting qubits while Microsoft/Quantinuum are using trapped ions, the two results are incomparable in terms of what they demonstrate. Google is just scaling up a single logical qubit, but showing (crucially) that their error rate decreases with increasing size and distance. Microsoft and Quantinuum are sticking with “small” logical qubits with insufficient distance, but they’re showing that they can apply logical circuits that entangle up to 12 of these qubits.

Microsoft also announced today a new collaboration with the startup company Atom Computing, headquartered near Quantinuum in Colorado, which is trying to build neutral-atom QCs (like QuEra in Boston). Over the past few years, Microsoft’s quantum group has decisively switched from a strategy of “topological qubits or bust” to a strategy of “anything that works,” although they assure me that they also remain committed to the topological approach.

Anyway, happy to hear in the comments from anyone who knows more details, or wants to correct me on any particular, or has questions which I or others can try our best to answer.

Let me end by sticking my neck out. If hardware progress continues at the rate we’ve seen for the past year or two, then I find it hard to understand why we won’t have useful fault-tolerant QCs within the next decade. (And now to retreat my neck a bit: the “if” clause in that sentence is important and non-removable!)

In Part 1, I explained how statistical mechanics is connected to a rig whose operations depend on a real parameter and approach the ‘tropical rig’, with operations and as I explained my hope that if we take equations from classical statistical mechanics, expressed in terms of this -dependent rig, and let we get equations in thermodynamics. That’s what I’m slowly trying to show.

As a warmup, last time I explained a bit of thermodynamics. We saw that some crucial formulas involve Legendre transforms, where you take a function and define a new one by

I’d like the Legendre transform to be something like a limit of the Laplace transform, where you take a function and define a new one by

Why do I care? As we’ll see later, classical statistical mechanics features a crucial formula that involves a Laplace transform. So it would be great if we could find some parameter in that formula, take the limit and get a corresponding equation in thermodynamics that involves a Legendre transform!

As a warmup, let’s look at the purely mathematical question of how to get the Legendre transform as a limit of the Laplace transform — or more precisely, something like the Laplace transform. Once we understand that, we can tackle the physics in a later post.

From Legendre to Laplace: the rough idea

In Part 1 we saw that for each we can make into a rig with addition

As this operation approaches

and we get a version of the tropical rig. (There’s a more popular version using instead of but I decided to use )

Suppose we want to define a notion of integration with replacing ordinary addition. We could call it ‘β-integration’ and denote it by Then it’s natural to try this:

If the function is nice enough, we could hope that

Then, we could hope to express the Legendre transform

as the limit of some transform involving β-integration.

Indeed, in Section 6 here, Litvinov claims that the Legendre transform is the analogue of the Laplace transform:

• Grigory L. Litvinov, Tropical mathematics, idempotent analysis, classical mechanics and geometry.

But he doesn’t state any result saying that it’s a limit of the Laplace transform, or something like that.. Touchette states a result along these lines here:

• Hugo Touchette, Legendre–Fenchel transforms in a nutshell.

He even applies it to classical statistical mechanics! But he’s operating as a physicist, not a mathematician, so he doesn’t state a precise theorem. I’d like to take a crack at that, just to be sure I’m not fooling myself.

The Legendre transform as a limit

Touchette’s formula gives an Legendre transform involving a sup rather than an inf. I slightly prefer a version with an inf. We’ll get the Legendre transform as a limit of something that is not exactly a Laplace transform, but close enough for our physics application:

Almost Proved Theorem. Suppose that is a concave function with continuous second derivative. Suppose that for some the function has a unique minimum at and Then as we have

Almost Proof. Laplace’s method should give the asymptotic formula

Taking the logarithm of both sides and dividing by we get

since

Since has a minimum at we get the desired result:

The only tricky part is that Laplace’s method as proved here requires finite limits of integration, while we are integrating from all the way up to However, the function is concave, with a minimum at and it has positive second derivative there since Thus, it grows at least linearly for large so as the integral

can be arbitrary well approximated by an integral over some finite range       ∎

Someone must have studied the hell out of this issue somewhere — do you know where?

Now, let’s look at the key quantity in the above result:

We’ve got a and then the logarithm of an integral that is not exactly a Laplace transform… but it’s close! In fact it’s almost the Laplace transform of the function

since

The right hand side here would be the Laplace transform of if it weren’t for that in the exponential.

So, it seems to be an exaggeration to say the Legendre transform is a limit of the Laplace transform. It seems to be the limit of something that’s related to a Laplace transform, but more complicated in a number of curious ways.

This has made my life difficult (and exciting) for the last few weeks. Right now I believe that all the curious complications do exactly what we want them to in our physics applications. But I should hold off on declaring this until I write up all the details: I keep making computational mistakes and fixing them, with a roller-coaster of emotions as things switch between working and not working.

In Part 1, I explained how statistical mechanics is connected to a rig whose operations depend on a real parameter $\beta$ and approach the ‘tropical rig’, with operations $\min$ and $+$, as $\beta \to +\infty$. I explained my hope that if we take equations from classical statistical mechanics, expressed in terms of this $\beta$-dependent rig, and let $\beta \to +\infty$, we get equations in thermodynamics. That’s what I’m slowly trying to show.

As a warmup, last time I explained a bit of thermodynamics. We saw that some crucial formulas involve Legendre transforms, where you take a function $f \colon \mathbb{R} \to [-\infty,\infty]$ and define a new one $\tilde{f} \colon \mathbb{R} \to [-\infty,\infty]$ by

$$\tilde{f}(s) = \inf_{x \in \mathbb{R}} (s x - f(x))$$

I’d like the Legendre transform to be something like a limit of the Laplace transform, where you take a function $f$ and define a new one $\hat{f}$ by

$$\hat{f}(s) = \int_{-\infty}^\infty e^{-s x} f(x) \, d x$$

Why do I care? As we’ll see later, classical statistical mechanics features a crucial formula that involves a Laplace transform. So it would be great if we could find some parameter $\beta$ in that formula, take the limit $\beta \to +\infty$, and get a corresponding equation in thermodynamics that involves a Legendre transform!

As a warmup, let’s look at the purely mathematical question of how to get the Legendre transform as a limit of the Laplace transform — or more precisely, something like the Laplace transform. Once we understand that, we can tackle the physics in a later post.

From Legendre to Laplace: the rough idea

In Part 1 we saw that for each $\beta \gt 0$ we can make $(0,\infty]$ into a rig with addition

$$x \oplus_\beta y = -\frac{1}{\beta} \ln(e^{-\beta x} + e^{-\beta y})$$

As $\beta \to +\infty$, this operation approaches

$$x \oplus_\infty y = x \min y$$

and we get a version of the tropical rig. (There’s a more popular version using $\max$ instead of $\min$, but I decided to use $\min$.)

Suppose we want to define a notion of integration with $\oplus_\beta$ replacing ordinary addition. We could call it ‘$\beta$-integration’ and denote it by $\int_\beta$. Then it’s natural to try this:

$$\int_\beta f(x) \, d x = - \frac{1}{\beta} \ln \int e^{-\beta f(x)} d x$$

If the function $f$ is nice enough, we could hope that

$$\lim_{\beta \to +\infty} \int_\beta f(x) \, d x \; = \; \inf_x f(x)$$

Then, we could hope to express the Legendre transform

$$\tilde{f}(s) = \inf_{x \in \mathbb{R}} (s x - f(x))$$

as the $\beta \to + \infty$ limit of some transform involving $\beta$-integration.

Indeed, in Section 6 here, Litvinov claims that the Legendre transform is the $\beta = \infty$ analogue of the Laplace transform:

• Grigory L. Litvinov, Tropical mathematics, idempotent analysis, classical mechanics and geometry.

But he doesn’t state any result saying that it’s a limit of the Laplace transform, or something like that.. Touchette states a result along these lines here:

• Hugo Touchette, Legendre–Fenchel transforms in a nutshell.

He even applies it to classical statistical mechanics! But he’s operating as a physicist, not a mathematician, so he doesn’t state a precise theorem. I’d like to take a crack at that, just to be sure I’m not fooling myself.

The Legendre transform as a limit

Touchette’s formula gives an Legendre transform involving a sup rather than an inf. I slightly prefer a version with an inf. We’ll get the Legendre transform as a limit of something that is not exactly a Laplace transform, but close enough for our physics application:

Almost Proved Theorem. Suppose that $f \colon [0,\infty) \to \mathbb{R}$ is a concave function with continuous second derivative. Suppose that for some $s \gt 0$ the function $s x - f(x)$ has a unique minimum at $x_0$, and $f''(x_0) \lt 0$. Then as $\beta \to +\infty$ we have

$$\displaystyle{ -\frac{1}{\beta} \ln \int_0^\infty e^{-\beta (s x - f(x))} \, d x \quad \longrightarrow \quad \inf_x \left( s x - f(x)\right) }$$

Almost Proof. Laplace’s method should give the asymptotic formula

$$\displaystyle{ \int_0^\infty e^{-\beta (s x - f(x))} \, d x \quad \sim \quad \sqrt{\frac{2\pi}{-\beta f''(x_0)}} \; e^{-\beta(s x_0 - f(x_0)} }$$

Taking the logarithm of both sides and dividing by $-\beta$ we get

$$\displaystyle{\lim_{\beta \to +\infty} -\frac{1}{\beta} \ln \int_0^\infty e^{-\beta(s x - f(x))} \, d x \; = \; s x_0 - f(x_0) }$$

since

$$\displaystyle{ \lim_{\beta \to +\infty} \frac{1}{\beta} \; \ln \sqrt{\frac{2\pi}{-\beta f''(x_0)}} = 0 }$$

Since $s x - f(x)$ has a minimum at $x_0$ we get the desired result:

$$\displaystyle{ -\frac{1}{\beta} \ln \int_0^\infty e^{-\beta (s x - f(x))} \, d x \quad \longrightarrow \quad \inf_x \left( s x - f(x)\right) }$$

The only tricky part is that Laplace’s method as proved here requires finite limits of integration, while we are integrating from $0$ all the way up to $\infty$. However, the function $s x - f(x)$ is concave, with a minimum at $x_0$, and it has positive second derivative there since $f''(x_0) \lt 0$. Thus, it grows at least linearly for large $x$, so as $\beta \to +\infty$ the integral

$$\displaystyle \int_0^\infty e^{-\beta(s x - f(x))} \, d x$$

can be arbitrarily well approximated by an integral over some finite range $[0,a]$.       ∎

Someone must have studied the hell out of this issue somewhere — do you know where?

Now, let’s look at the key quantity in the above result:

$$\displaystyle{ - \frac{1}{\beta} \ln \int_0^\infty e^{-\beta (s x - f(x))} \, d x }$$

We’ve got a $-1/\beta$ and then the logarithm of an integral that is not exactly a Laplace transform… but it’s close! In fact it’s almost the Laplace transform of the function

$$g_\beta(x) = e^{\beta f(x)}$$

since

$$\displaystyle{ \int_0^\infty e^{-\beta (s x - f(x))} \, d x = \int_0^\infty e^{-\beta s x} \, g_\beta(x) \, d x }$$

The right hand side here would be the Laplace transform of $g_\beta$ if it weren’t for that $\beta$ in the exponential.

So, it seems to be an exaggeration to say the Legendre transform is a limit of the Laplace transform. It seems to be the limit of something that’s related to a Laplace transform, but more complicated in a number of curious ways.

This has made my life difficult (and exciting) for the last few weeks. Right now I believe that all the curious complications do exactly what we want them to in our physics applications. But I should hold off on declaring this until I write up all the details: I keep making computational mistakes and fixing them, with a roller-coaster of emotions as things switch between working and not working.

Besides learning about individual physical theories, students learn different frameworks in which physical theories are formulated. I’m talking about things like this:

• classical statics
• classical mechanics
• quantum mechanics
• thermodynamics
• classical statistical mechanics
• quantum statistical mechanics

A physical framework often depends on some physical constants that we can imagine varying, and in some limit one framework may reduce to another. This suggests that we should study a ‘moduli space’ or ‘moduli stack’ of physical frameworks. To do this formally, in full generality, we’d need to define what counts as a ‘framework’, and what means for two frameworks to be equivalent. I’m not ready to try that yet. So instead, I want to study an example: a 1-parameter family of physical frameworks that includes classical statistical mechanics — and, I hope, also thermodynamics!

Physicists often say things like this:

“Special relativity reduces to Newtonian mechanics as the speed of light, approaches ”

“Quantum mechanics reduces to classical mechanics as Planck’s constant approaches ”

“General relativity reduces to special relativity as Newton’s constant approaches ”

Sometimes they try to elaborate this further with a picture called Bronstein’s cube or the CGh cube:

This is presumably hinting at some 3-dimensional space where and can take arbitrary nonnegative values. This would be an example of what I mean by a ‘moduli space of physical frameworks’.

But right now I want to talk about talk about a fourth dimension that’s not in this cube. I want to talk about whether classical statistical mechanics reduces to thermodynamics as where is Boltzmann’s constant.

Since thermodynamics and statistical mechanics are often taught in the same course, you may be wondering how I distinguish them. Here are my two key principles: anything that involves probability theory or Boltzmann’s constant I will not call thermodynamics: I will call it statistical mechanics. For example, in thermodynamics we have quantities like energy entropy temperature obeying rules like

But in classical statistical mechanics becomes a random variable and we instead have

In classical statistical mechanics we can also compute the variance of and this is typically proportional to Boltzmann’s constant. As this variance goes to zero and we’re back to thermodynamics! Also, in classical statistical mechanics entropy turns out to be given by

where is some probability distribution on some measure space of states

I want to flesh out how classical statistical mechanics reduces to thermodynamics as and my hope is that this is quite analogous to how quantum mechanics reduces to classical mechanics as :

taking the limit of	taking the limit of
Quantum Mechanics	Classical Statistical Mechanics
gives	gives
Classical Mechanics	Thermodynamics

Here’s the idea. Quantum fluctuations are a form of randomness inherent to quantum mechanics, described by complex amplitudes. Thermal fluctuations are a form of randomness inherent to classical statistical mechanics, described by real probabilities. Planck’s constant sets the scale of quantum fluctuations, and as these go away and quantum mechanics reduces to classical mechanics. Boltzmann’s constant sets the scale of thermal fluctuations, and as these go away and classical statistical mechanics reduces to thermodynamics.

If this idea works, the whole story for how quantum mechanics reduces to classical mechanics as may be a Wick rotated version of how classical statistical mechanics reduces to thermodynamics as In other words, the two stories may be formally the same if we replace everywhere with

However, there are many obstacles to getting the idea to to work — or at least apparent obstacles, much as walls can feel like ‘obstacles’ when you’re trying to walk through a wide open door at night. Even before we meet the technical problems with Wick rotation, there’s the preliminary problem of getting thermodynamics to actually arise as the limit of classical statistical mechanics!

So despite the grand words above, it’s that preliminary problem that I’m focused on now. It’s actually really interesting.

Today I’ll just give a bit of background.

The math: deformation of rigs

Deformation quantization, is a detailed analysis of how quantum mechanics reduces to classical mechanics as and how you can try to reverse this process. If you’ve thought about this a lot, may have bumped into ‘idempotent analysis’: a version of analysis where you use minimization as a replacement for addition of real numbers, and addition as a replacement for multiplication. This works because addition distributes over minimization:

and it’s called ‘idempotent’ because minimization obeys

When we use minimization as addition we don’t get additive inverses, so numbers form a rig, meaning a ‘ring without negatives’. You also need to include to serve as an additive identity for minimization.

Idempotent analysis overlaps with ‘tropical mathematics’, where people use this number system, called the ‘tropical rig’, to simplify problems in algebraic geometry. People who do idempotent analysis are motivated more by applications to physics:

• Grigory L. Litvinov, Tropical mathematics, idempotent analysis, classical mechanics and geometry.

The basic idea is to study a 1-parameter family of rigs which for finite are all isomorphic to with its usual addition and multiplication, but in the limit approach a rig isomorphic to with ‘min’ as addition and the usual as multiplication.

Let me describe this in more detail, so you can see exactly how it works. In classical statistical mechanics, the probability of a system being in a state of energy decreases exponentially with energy, so it’s proportional to

where is some constant we’ll discuss later. Let’s write

but let’s extend to a bijection

sending to This says that states of infinite energy have probability zero.

Now let’s conjugate ordinary addition and multiplication, which make into a rig, by this bijection :

These conjugated operations make into a rig. Explicitly, we have

So the multiplication is always the usual on — yes, I know this is confusing — while the addition is some more complicated operation that depends on We get different rig structures on for different values of but these rigs are all isomorphic because we got them all from the same rig structure on

However, now we can take the limit as and get operations we call and If we work these out we get

These give a rig structure on that’s not isomorphic to any of those for finite This is the tropical rig.

(Other people define the tropical rig differently, using different conventions, but usually theirs are isomorphic to this one.)

The physics: classical statistical mechanics

What does all of this mean for classical statistical mechanics? The idea is that with its usual and is the rig of unnormalized probabilities. I’ll assume you know why we add probabilities for mutually exclusive events and multiply probabilities for independent events. But probabilities lie in which is not closed under addition. To get a rig, we work instead with ‘unnormalized’ probabilities, which lie in We add and multiply these just like probabilities. When we have a list of unnormalized probabilities we can convert them to probabilities by dividing each one by their sum. We do this normalization only after all the addition and multiplication is done and we want to make predictions.

In classical statistical mechanics, a physical system has many states, each with its own energy The unnormalized probability that the system is in a state of energy is

The case of infinite energy is not ordinarily considered, but it’s allowed by the the math here, and this gives an unnormalized probability of zero.

We can reason with these unnormalized probabilities using addition and multiplication — or, equivalently, we can work directly with the energies using the operations and on

In short, we’ve enhanced the usual machinery of probability theory by working with unnormalized probabilities, and then transferred it over to the world of energies.

The physical meaning of

All very nice. But what’s the physical meaning of and the limit? This is where things get tricky.

First, what’s ? In physics we usually take

where is temperature and is Boltzmann’s constant. Boltzmann’s constant has units of energy/temperature — it’s about joules per kelvin — so in physics we use it to convert between energy and temperature. has units of energy, so has units of 1/energy, and is dimensionless. That’s important: we’re only allowed to exponentiate dimensionless quantities, and we want to make sense.

One can imagine doing physics using some interpretation of the deformation parameter other than But let’s take to be Then we can still understand the limit in more than one way! We can

1. hold constant and let
2. hold constant and let

We could also try other things, like simply letting and do whatever they want as long as their product approaches zero. But let’s just consider these two options.

Hold constant and let

This option seems to make plenty of sense. It’s called Boltzmann’s ‘constant’, after all. So maybe we should hold it constant and let approach zero. In this case we’re taking the low temperature limit of classical statistical mechanics.

It’s sad that we get the tropical rig as a low-temperature limit: it should have been called the arctic rig! But the physics works out well. At temperature systems in classical statistical mechanics minimize their free energy where is energy and is entropy. As free energy reduces to simply the energy, Thus, in the low temperature limit, such systems always try to minimize their energy! In this limit we’re doing classical statics: the classical mechanics of systems at rest.

These ideas let us develop this analogy:

taking the limit of	taking the limit of
Quantum Mechanics	Classical Statistical Mechanics
gives	gives
Classical Mechanics	Classical Statics

Blake Pollard and I explored this analogy extensively, and discovered some exciting things:

• Quantropy:

  o Part 1: the analogy between quantum mechanics and statistical mechanics, and the quantum analogue of entropy: quantropy.

  o Part 2: computing the quantropy of a quantum system starting from its partition function.

  o Part 3: the quantropy of a free particle.

  o Part 4: a paper on quantropy, written with Blake Pollard.

But while this analogy is mathematically rigorous and leads to new insights, it’s not completely satisfying. First, and just feel different. Planck’s constant takes the same value everywhere while temperature is something we can control. It would be great if next to the thermostat on your wall there was a little box where you could adjust Planck’s constant, but it just doesn’t work that way!

Second, there’s a detailed analogy between classical mechanics and thermodynamics, which I explored here:

• Classical mechanics versus thermodynamics:

  o Part 1: Hamilton’s equations versus the Maxwell relations.

  o Part 2: the role of symplectic geometry.

  o Part 3: a detailed analogy between classical mechanics and thermodynamics.

  o Part 4: what is the analogue of quantization for thermodynamics?

Most of this was about how classical mechanics and thermodynamics share common mathematical structures, like symplectic and contact geometry. These structures arise naturally from variational principles: principle of least action in classical mechanics, and the principle of maximum entropy in thermodynamics. By the end of this series I had convinced myself that thermodynamics should appear as the limit of some physical framework, just as classical mechanics appears as the limit of quantum mechanics. So, let’s look at option 2.

Hold constant and let

Of course, my remark about holding constant because it’s called Boltzmann’s constant was just a joke. Planck’s constant is a constant too, yet it’s very fruitful to imagine treating it as a variable and letting it approach zero.

But what would it really mean to let ? Well, first let’s remember what it would mean to let

I’ll pick a specific real-world example: a laser beam. Suppose you’re a physicist from 1900 who doesn’t know quantum mechanics, who is allowed to do experiments on a beam of laser light. If you do crude measurements, this beam will look like a simple solution to the classical Maxwell equations: an electromagnetic field oscillating sinusoidally. So you will think that classical physics is correct. Only when you do more careful measurements will you notice that this wave’s amplitude and phase are a bit ‘fuzzy’: you get different answers when you repeatedly measure these, no matter how good the beam is and how good your experimental apparatus is. These are ‘quantum fluctuations’, related to the fact that light is made of photons.

The product of the standard deviations of the amplitude and phase is bounded below by something proportional to So, we say that sets the scale of the quantum fluctuations. If we could let these fluctuations would become ever smaller, and in the limit the purely classical description of the situation would be exact.

Something similar holds with Boltzmann’s constant. Suppose you’re a physicist from 1900 who doesn’t know about atoms, who has somehow been given the ability to measure the pressure of a gas in a sealed container with arbitrary accuracy. If you do crude measurements, the pressure will seem to be a function of the temperature, the volume of the container, and the amount of gas in the container. This will obey the laws of thermodynamics. Only when you do extremely precise experiments will you notice that the pressure is fluctuating. These are ‘thermal fluctuations’, caused by the gas being made of molecules.

The variance of the pressure is proportional to So, we say that sets the scale of the thermal fluctuations. If we could let these fluctuations would become ever smaller, and in the limit the thermodynamics description of the situation would be exact.

The analogy here is a bit rough at points, but I think there’s something to it. And if you examine the history, you’ll see some striking parallels. Einstein discovered that light is made of photons: he won the Nobel prize for his 1905 paper on this, and it led to a lot of work on quantum mechanics. But Einstein also wrote a paper in 1905 showing how to prove that liquid water is made of atoms! The idea was to measure the random Brownian motion of a grain of pollen in water — that is, thermal fluctuations. In 1908, Jean Perrin carried out this experiment, and he later won the Nobel “for his work on the discontinuous structure of matter”. So the photon theory of light and the atomic theory of matter both owe a lot to Einstein’s work.

Planck had earlier introduced what we now call Planck’s constant in his famous 1900 paper on the quantum statistical mechanics of light, without really grasping the idea of photons. Remarkably, this is also the paper that first introduced Boltzmann’s constant Boltzmann had the idea that entropy was proportional to the logarithm of the number of occupied states, but he never estimated the constant of proportionality or gave it a name: Planck did both! So Boltzmann’s constant and Planck’s constant were born hand in hand.

There’s more to say about how to correctly take the limit of a laser beam: we have to simultaneously increase the expected number of photons, so that the rough macroscopic appearance of the laser beam remains the same, rather than becoming dimmer. Similarly, we have to be careful when taking the limit of a container of gas: we need to also increase the number of molecules, so that the average pressure remains the same instead of decreasing.

Getting these details right has exercised me quite a bit lately. This is what I want to talk about.

Besides learning about individual physical theories, students learn different frameworks in which physical theories are formulated. I’m talking about things like this:

• classical statics
• classical mechanics
• quantum mechanics
• thermodynamics
• classical statistical mechanics
• quantum statistical mechanics

A physical framework often depends on some physical constants that we can imagine varying, and in some limit one framework may reduce to another. This suggests that we should study a ‘moduli space’ or ‘moduli stack’ of physical frameworks. To do this formally, in full generality, we’d need to define what counts as a ‘framework’, and what means for two frameworks to be equivalent. I’m not ready to try that yet. So instead, I want to study an example: a 1-parameter family of physical frameworks that includes classical statistical mechanics — and, I hope, also thermodynamics!

Physicists often say things like this:

“Special relativity reduces to Newtonian mechanics as the speed of light, $c$, approaches $\infty$.”

“Quantum mechanics reduces to classical mechanics as Planck’s constant $\hbar$ approaches $0$.”

“General relativity reduces to special relativity as Newton’s constant $G$ approaches $0$.”

Sometimes they try to elaborate this further with a picture called Bronstein’s cube or the CGh cube:

This is presumably hinting at some 3-dimensional space where $1/c, \hbar$ and $G$ can take arbitrary nonnegative values. This would be an example of what I mean by a ‘moduli space of physical frameworks’.

But right now I want to talk about talk about a fourth dimension that’s not in this cube. I want to talk about whether classical statistical mechanics reduces to thermodynamics as $k \to 0$, where $k$ is Boltzmann’s constant.

Since thermodynamics and statistical mechanics are often taught in the same course, you may be wondering how I distinguish them. Here are my two key principles: anything that involves probability theory or Boltzmann’s constant I will not call thermodynamics: I will call it statistical mechanics. For example, in thermodynamics we have quantities like energy $E$, entropy $S$, temperature $T$, obeying rules like

$$d E = T d S$$

But in classical statistical mechanics $E$ becomes a random variable and we instead have

$$d \langle E \rangle = T d S$$

In classical statistical mechanics we can also compute the variance of $E$, and this is typically proportional to Boltzmann’s constant. As $k \to 0$, this variance goes to zero and we’re back to thermodynamics! Also, in classical statistical mechanics entropy turns out to be given by

$$S = - k \int_X p(x) \ln(p(x)) \, d\mu(x)$$

where $p$ is some probability distribution on some measure space of states $(X,\mu)$.

I want to flesh out how classical statistical mechanics reduces to thermodynamics as $k \to 0$, and my hope is that this is quite analogous to how quantum mechanics reduces to classical mechanics as $\hbar \to 0$:

Here’s the idea. Quantum fluctuations are a form of randomness inherent to quantum mechanics, described by complex amplitudes. Thermal fluctuations are a form of randomness inherent to classical statistical mechanics, described by real probabilities. Planck’s constant $\hbar$ sets the scale of quantum fluctuations, and as $\hbar \to 0$ these go away and quantum mechanics reduces to classical mechanics. Boltzmann’s constant $k$ sets the scale of thermal fluctuations, and as $k \to 0$ these go away and classical statistical mechanics reduces to thermodynamics.

If this idea works, the whole story for how quantum mechanics reduces to classical mechanics as $\hbar \to 0$ may be a Wick rotated version of how classical statistical mechanics reduces to thermodynamics as $k \to 0$. In other words, the two stories may be formally the same if we replace $k$ everywhere with $i \hbar$.

However, there are many obstacles to getting the idea to to work — or at least apparent obstacles, much as walls can feel like ‘obstacles’ when you’re trying to walk through a wide open door at night. Even before we meet the technical problems with Wick rotation, there’s the preliminary problem of getting thermodynamics to actually arise as the $k \to 0$ limit of classical statistical mechanics!

So despite the grand words above, it’s that preliminary problem that I’m focused on now. It’s actually really interesting.

Today I’ll just give a bit of background.

The math: deformation of rigs

Deformation quantization, is a detailed analysis of how quantum mechanics reduces to classical mechanics as $\hbar \to 0$, and how you can try to reverse this process. If you’ve thought about this a lot, may have bumped into ‘idempotent analysis’: a version of analysis where you use minimization as a replacement for addition of real numbers, and addition as a replacement for multiplication. This works because addition distributes over minimization:

$$x + (y \, \min \, z) = (x + y) \, \min \, (x + z)$$

and it’s called ‘idempotent’ because minimization obeys

$$x \, \min \, x = x$$

When we use minimization as addition we don’t get additive inverses, so numbers form a rig, meaning a ‘ring without negatives’. You also need to include $+\infty$ to serve as an additive identity for minimization.

Idempotent analysis overlaps with ‘tropical mathematics’, where people use this number system, called the ‘tropical rig’, to simplify problems in algebraic geometry. People who do idempotent analysis are motivated more by applications to physics:

• Grigory L. Litvinov, Tropical mathematics, idempotent analysis, classical mechanics and geometry.

The basic idea is to study a 1-parameter family of rigs $R_\beta$ which for finite $\beta \gt 0$ are all isomorphic to $(-\infty,\infty]$ with its usual addition and multiplication, but in the limit $\beta \to +\infty$ approach a rig isomorphic to $[0,\infty)$ with ‘min’ as addition and the usual $+$ as multiplication.

Let me describe this in more detail, so you can see exactly how it works. In classical statistical mechanics, the probability of a system being in a state of energy $E$ decreases exponentially with energy, so it’s proportional to

$$e^{-\beta E}$$

where $\beta \gt 0$ is some constant we’ll discuss later. Let’s write

$$f_\beta(E) = e^{-\beta E}$$

but let’s extend $f_\beta$ to a bijection

$$f_\beta: (-\infty, \infty] \to [0,\infty)$$

sending $\infty$ to $0$. This says that states of infinite energy have probability zero.

Now let’s conjugate ordinary addition and multiplication, which make $[0,\infty)$ into a rig, by this bijection $f_\beta$:

$$x \oplus_\beta y = f_\beta^{-1} (f_\beta(x) + f_\beta(y))$$   $$x \odot_\beta y = f_\beta^{-1} (f_\beta(x) \cdot f_\beta(y))$$

These conjugated operations make $(-\infty,\infty]$ into a rig. Explicitly, we have

$$x \oplus_\beta y = -\frac{1}{\beta} \ln(e^{-\beta x} + e^{-\beta y})$$   $$x \odot_\beta y = x + y$$

So the multiplication is always the usual $+$ on $(-\infty,\infty]$ — yes, I know this is confusing — while the addition is some more complicated operation that depends on $\beta$. We get different rig structures on $(-\infty,\infty]$ for different values of $\beta \gt 0$, but these rigs are all isomorphic because we got them all from the same rig structure on $[0,\infty)$.

However, now we can take the limit as $\beta \to +\infty$ and get operations we call $\oplus_\infty$ and $\odot_\infty$. If we work these out we get

$$x \oplus_\infty y = x \min y$$   $$x \odot_\infty y = x + y$$

These give a rig structure on $(-\infty,\infty]$ that’s not isomorphic to any of those for finite $\beta$. This is the tropical rig.

(Other people define the tropical rig differently, using different conventions, but usually theirs are isomorphic to this one.)

The physics: classical statistical mechanics

What does all of this mean for classical statistical mechanics? The idea is that $[0,\infty)$ with its usual $+$ and $\times$ is the rig of unnormalized probabilities. I’ll assume you know why we add probabilities for mutually exclusive events and multiply probabilities for independent events. But probabilities lie in $[0,1]$, which is not closed under addition. To get a rig, we work instead with ‘unnormalized’ probabilities, which lie in $[0,\infty)$. We add and multiply these just like probabilities. When we have a list of unnormalized probabilities $p_1, \dots, p_n$, we can convert them to probabilities by dividing each one by their sum. We do this normalization only after all the addition and multiplication is done and we want to make predictions.

In classical statistical mechanics, a physical system has many states, each with its own energy $E \in (-\infty,\infty]$. The unnormalized probability that the system is in a state of energy $E$ is

$$f_\beta(E) = e^{-\beta E}$$

The case of infinite energy is not ordinarily considered, but it’s allowed by the the math here, and this gives an unnormalized probability of zero.

We can reason with these unnormalized probabilities using addition and multiplication — or, equivalently, we can work directly with the energies $E$ using the operations $\oplus_\beta$ and $\odot_\beta$ on $(-\infty,\infty]$.

In short, we’ve enhanced the usual machinery of probability theory by working with unnormalized probabilities, and then transferred it over to the world of energies.

The physical meaning of $\beta \to +\infty$

All very nice. But what’s the physical meaning of $\beta$ and the $\beta \to +\infty$ limit? This is where things get tricky.

First, what’s $\beta$? In physics we usually take

$$\beta = \frac{1}{k T}$$

where $T$ is temperature and $k$ is Boltzmann’s constant. Boltzmann’s constant has units of energy/temperature — it’s about $1.38 \cdot 10^{-23}$ joules per kelvin — so in physics we use it to convert between energy and temperature. $k T$ has units of energy, so $\beta$ has units of 1/energy, and $\beta E$ is dimensionless. That’s important: we’re only allowed to exponentiate dimensionless quantities, and we want $e^{-\beta E}$ to make sense.

One can imagine doing physics using some interpretation of the deformation parameter $\beta$ other than $1/k T$. But let’s take $\beta$ to be $1/k T$. Then we can still understand the $\beta \to +\infty$ limit in more than one way! We can