Skip to the Main Content

Note:These pages make extensive use of the latest XHTML and CSS Standards. They ought to look great in any standards-compliant modern browser. Unfortunately, they will probably look horrible in older browsers, like Netscape 4.x and IE 4.x. Moreover, many posts use MathML, which is, currently only supported in Mozilla. My best suggestion (and you will thank me when surfing an ever-increasing number of sites on the web which have been crafted to use the new standards) is to upgrade to the latest version of your browser. If that's not possible, consider moving to the Standards-compliant and open-source Mozilla browser.

December 6, 2020

Mathematical Phantoms

Posted by John Baez

A ‘mathematical phantom’ is a mathematical object that doesn’t exist in a literal sense, but nonetheless acts as if it did, casting a spell on surrounding areas of mathematics. The most famous example is the field with one element. Another is Deligne’s St, the symmetric group on tt elements, where tt is not a natural number. Yet another is G3, a phantom Lie group related to G2, the automorphism group of the octonions.

What’s your favorite mathematical phantom? My examples are all algebraic. Does only algebra have enough rigidity to create the patterns that summon up phantom objects? What about topology or combinatorics or analysis? Okay, G3 is really a creature from homotopy theory, but of a very algebraic sort.

Last night I met another phantom.

A while back David Corfield interrogated me about the mathematical phantom called SL(2,𝕆)SL(2,\mathbb{O}), which — if this made sense! — would consist of 2×22 \times 2 matrices with determinant 1 having octonion entries. Since the octonions are nonassociative, any direct attempt to describe such a group runs into a brick wall, and yet there are many reasons to want such a group, to complete various patterns in mathematics. A lot of people say SL(2,𝕆)SL(2,\mathbb{O}) should be the group Spin(9,1)Spin(9,1). See for example this, and the many references therein:

This makes a lot of things work. But is it the final answer?

Going slightly crazy from the boredom of coronavirus-induced lockdown, I’ve been browsing the arXiv while watching TV with my wife at night. I’m not sure that’s a good idea, but it’s more fun than social media. Last night I ran into this:

He’s unsatisfied with usual idea that SL(2,𝕆)SL(2,\mathbb{O}) is Spin(9,1)Spin(9,1), so he’s proposing a new candidate. Hitchin doesn’t like the usual idea because Spin(9,1)Spin(9,1) is 45-dimensional and the space of 2×22 \times 2 matrices of octonions is only 32-dimensional. His own preferred candidate has what might seem like an even worse problem: it’s not even a group! But it has some interesting properties and he makes a good argument for it.

I was planning to explain this and say more about how Hitchin’s work connects to another mathematical phantom: ‘octonionic twistors’. But I realize now that I’m not ready. So, I’ll think about this more, and turn the present blog post into a request: please tell me about your favorite mathematical phantoms!

Posted at December 6, 2020 7:01 PM UTC

TrackBack URL for this Entry:   https://golem.ph.utexas.edu/cgi-bin/MT-3.0/dxy-tb.fcgi/3276

67 Comments & 0 Trackbacks

Phantoms

Does the zeroth power of a real number count as an example? How about the factorial of 0?

In other words, do mathematical phantoms usually turn out to have at least one valid “solidification” —a consistent definition when seen from a more general point of view, such as “in the limit” or “in agreement with a set of axioms”? That would also allow things like those infinite series that “converge” to values of the Riemann zeta function; we really mean that the definition of that series is taken to be the output of the function.

Posted by: Stefan Forcey on December 6, 2020 9:26 PM | Permalink | Reply to this

Re: Phantoms

Indeed, many mathematical creature which originally manifested as phantoms are now respectable citizens of the mathematical community. Examples include irrational numbers, negative numbers and imaginary numbers.

This is why mathematical phantoms are important: they call for us to expand our way of thinking!

Posted by: John Baez on December 6, 2020 10:03 PM | Permalink | Reply to this

Re: Phantoms

What about dark natural numbers, the dark matter of arithmetic? It has left various traces in set theory, see https://www.hs-augsburg.de/~mueckenh/Transfinity/Transfinity/pdf, pp. 212ff.

Consider for example the endsegments

E(n) = {n, n+1, n+2, …}

of |N. The sequence of endsegments is strictly monotonically decreasing by one natural number per term

E(n+1) = E(n) \ {n} (*).

If the intersection of all terms is empty, as is easily proved by the fact that every n has a last appearance, then this can only be accomplished, when obeying (*), by the existence of endsegments having finite intersections. These however cannot be observed. They must be dark. This is also proved by the fact that for all observable natural numbers k, the intersection of endsegments

Int{E(1), E(2), …, E(k)}

is not empty but infinite. Note that in order to use an infinite set, it is never possible to address all elements individually but only as a collection. That is not a matter of lacking time or ressources.

Posted by: Wolfgang Mückenheim on February 17, 2021 4:39 PM | Permalink | Reply to this

Re: Mathematical Phantoms

Hmmm.

Maybe “the sporadic SIC in dimension 24” would fit the bill. In the argot of quantum information theory, a SIC is a set of d 2d^2 equiangular lines in d\mathbb{C}^d. The known examples fall into a mainline family (the “Weyl–Heisenberg SICs”) and a set of sporadic solutions, the SICs in dimensions 2 and 3 as well as one set of those in dimension 8. Among the weird things that happen with the sporadic SICs are unexpected relations to real equiangular lines in nearby dimensions, particularly a link between SICs in 8\mathbb{C}^8 and the maximal set of equiangular lines in 7\mathbb{R}^7. That construction in 7\mathbb{R}^7 is also extractable from the E 8\mathrm{E}_8 lattice, just as the maximal set in 23\mathbb{R}^{23} can be obtained from the Leech lattice. (See arXiv:2008.13288 for details and literature pointers.) So, it would be nice if there were a sporadic SIC up in 24\mathbb{C}^{24} which, when poked, yielded the maximal set of equiangular lines in 23\mathbb{R}^{23}.

Zhu (2015) proved that there is no doubly transitive SIC in 24\mathbb{C}^{24}. This rules out an exact analogy between the 8 7\mathbb{C}^8 \to \mathbb{R}^7 case and 24 23\mathbb{C}^{24} \to \mathbb{R}^{23}, since the sporadic SICs in 8\mathbb{C}^8, the SICs of Hoggar type, are doubly transitive. So, the sporadic SIC in 24\mathbb{C}^{24} feels more like a “phantom” than merely an unknown. Chatting informally with people who know a lot about this, the sense is that if something unusual happens, it would most naturally happen in dimension 24, but it can’t be the same kind of unusual thing already observed in dimensions 2, 3 and 8.

The symmetry groups of the doubly-transitive SICs relate in a nice way to lattices of integers in \mathbb{C}, \mathbb{H} and 𝕆\mathbb{O} (Eisenstein, Hurwitz and Cayley respectively). So, asking for this pattern to continue one step further is a bit like what happens with G 3\mathrm{G}_3: There isn’t another real normed division algebra for it to be the automorphism group of.

Posted by: Blake Stacey on December 6, 2020 9:52 PM | Permalink | Reply to this

Re: Mathematical Phantoms

We have a page for SL(2,O) where Hitchin’s paper is mentioned. Always plenty to add of course.

Posted by: David Corfield on December 6, 2020 10:11 PM | Permalink | Reply to this

Re: Mathematical Phantoms

Wow, I didn’t know about that page. I know boatloads of references to SL(2,𝕆)SL(2,\mathbb{O}), I work with this group daily now. I’ll add some.

Posted by: John Baez on December 6, 2020 10:46 PM | Permalink | Reply to this

Phantom Lie groups

It’s easy to say which homotopy types contain groups – they are exactly the loop spaces – and harder to say which ones contain manifolds. There’s been a bunch of work on figuring out which contain both; e.g. “A finite loop space not rationally equivalent to a compact Lie group”. Obviously the homotopy type of a Lie group contains both, but it’s interesting to find manifolds that are homotopic to groups but not to Lie groups. IIRC they automatically have trivial tangent bundle, and one can develop a notion of maximal torus for them.

Dyson’s “Missed opportunities” suggests a phantom 26-dim Lie group. Has it been “found” since then?

Posted by: Allen Knutson on December 6, 2020 10:41 PM | Permalink | Reply to this

Re: Phantom Lie groups

In Topology 43 (2004) Tilman Bauer constructs a (pp-local) framed manifold associated to a pp-compact group; hence G 3G_3 has an associated framed cobordism class in the 2-component (Z 16×Z 2 3\Z_{16} \times \Z_2^3) of the stable homotopy group of spheres in dimension 45. As far as I know, though, this manifestation could be zero.

Posted by: jackjohnson on December 7, 2020 9:29 PM | Permalink | Reply to this

Re: Mathematical Phantoms

I did wording that something that exists acts as if it did problematic. Something that doesn’t exist cannot act because it doesn’t exist. So what does the acting? Meaning, of we peel away the “figurative” wording, what could we say that would make better sense?

Posted by: Fryc de Lanckoronski on December 7, 2020 3:01 AM | Permalink | Reply to this

Re: Mathematical Phantoms

I was trying to recall a conference I co-organised – Two streams in the philosophy of mathematics. I see I was comparing Alexandre Borovik’s phantoms and Michael Harris’s avatars:

For me two of the most interesting issues to emerge during the conference was Borovik’s ‘phantoms’ and Harris’s ‘avatars’. The first of these may occur when there is a question as to whether a certain entity exists. Even if it does not, it may transpire that some counterpart of this nonexistent entity exists elsewhere. The setting of finite simple groups is a rich environment for this phenomenon.

In the case of avatars, on the other hand, they all exist, but they indicate the existence of a not yet expressible universal object. Grothendieck’s theory of motives is the classic example, and indeed it was here that he coined the term ‘avatar’ to describe an instantiation of a motive in a particular cohomological setting.

Michael’s talk is still available, here.

Another instance, in the penultimate paragraph of this section of his page on a proposal to characterise M-theory, Urs compares construction of the later to the construction of absolute geometry over 𝔽 1\mathbb{F}_1.

Posted by: David Corfield on December 7, 2020 7:40 AM | Permalink | Reply to this

Re: Mathematical Phantoms

I thought you’d mentioned mathematical phantoms somewhere but I couldn’t find where! So they go back to Alexandre Borovik.

I ran into a paper titled Mathematical phantoms but it’s about something else.

Posted by: John Baez on December 7, 2020 7:26 PM | Permalink | Reply to this

Re: Mathematical Phantoms

The only trace I have of what he might have meant in the context of finite simple groups is what I wrote back here, including his remarks on Israel Gelfand’s prophecy that

Sporadic simple groups are not groups, they are objects from a still unknown infinite family, some number of which happened to be groups, just by chance.

Posted by: David Corfield on December 8, 2020 7:58 AM | Permalink | Reply to this

Re: Mathematical Phantoms

Has there been any progress in the past decade and a half in that direction regarding the classification of the sporadic simple groups?

Posted by: Madeleine Birchfield on December 9, 2020 12:10 AM | Permalink | Reply to this

Re: Mathematical Phantoms

I haven’t heard of any progress in that direction regarding the classification of finite simple groups. The most promising avenue toward understanding them in new ways — as far as I know, and I’m certainly no expert — is the work on generalized Moonshine, Mathieu moonshine and related phenomena.

Posted by: John Baez on December 9, 2020 12:25 AM | Permalink | Reply to this

Re: Mathematical Phantoms

Further down that thread, there’s a talk abstract by Andrew Chermak which bring us back to G 3G_3:

The Dwyer-Wilkerson 2-adic loop space is a “2-local group”, by work of Levi and Oliver, and of Aschbacher and Chermak. Moreover, this object contains, as subgroups “at the prime 2”, the sporadic groups Co 3Co_3, J 2J_2, and ONO'N. Since O’Nan’s is a “pariah” (i.e. not involved in the Monster) it may be of interest to have a context in which it lives in harmony with Co 3Co_3 and J 2J_2, which are not pariahs.

Posted by: David Corfield on December 8, 2020 8:32 AM | Permalink | Reply to this

Re: Mathematical Phantoms

I’ve always liked that quote of Gelfand, ever since I heard it from David:

Sporadic simple groups are not groups, they are objects from a still unknown infinite family, some number of which happened to be groups, just by chance.

I have always vaguely hoped that the “still unknown infinite family” is connected to modular tensor categories. Modular tensor categories can be seen as a generalization of finite groups. Every finite group gives a modular tensor category called the the representation category of its quantum double, but there are also lots of other modular tensor categories.

However, I forget if finite simple groups give so-called ‘prime’ modular tensor categories. That would be the first thing to check; someone must have thought about this.

What makes it all even more fun is that the classification of modular tensor categories is closely tied to the classification of topological phases of matter! As Rowell puts it:

Topological phases of matter are like artificial elements. The only known topological phases of matter are fractional quantum Hall liquids: electron systems confined on a disk immersed in a strong perpendicular magnetic field at extremely low temperatures. Electrons in the disk, pictured classically as orbiting inside concentric annuli around the origin, organize themselves into some topological order. Therefore, the classification of topological phases of matter resembles the periodic table of elements. The periodic table does not go on forever, and simpler elements are easier to find. The topological quantum computing project is to find modular tensor categories in Nature, in particular those with non-abelian anyons. Therefore, it is important that we know the simplest modular tensor categories in a certain sense because the chance for their existence is better.

Posted by: John Baez on December 8, 2020 9:43 PM | Permalink | Reply to this

Re: Mathematical Phantoms

Following on the idea of pariah groups and “Happy Family” groups living together in harmony, the octonions show up again, or more specifically, the Cayley integers (which Conway and Smith call the “octavians”). The automorphism group of these integers, variously denoted G 2(2)G_2(2)' or U 3(3)U_3(3) or PSU(3,3)PSU(3,3), appears in the construction of the Hall–Janko group (a subgroup of the Monster) and also of the Rudvalis group (a pariah). Another point of analogy is that the Rudvalis group can be constructed as a rank-3 permutation group acting on 4060 points, where the stabilizer of a point is the Ree group 2F 4(2)^2 F_4(2). Per Wilson (2010), this group is in turn given by the symmetries of a “generalized octagon”. (Generalized nn-gons abstract the properties of the more familiar nn-gons that, as incidence graphs, they have diameter nn and their shortest cycles have length 2n2n. For example, the Fano plane is a generalized triangle.) Similarly, the Hall–Janko group has a rank-3 permutation representation on 100 points where the point stabilizer is U 3(3)U_3(3), and U 3(3)U_3(3) is furnished by the symmetries of a generalized hexagon.

Posted by: Blake Stacey on December 9, 2020 7:09 PM | Permalink | Reply to this

Re: Mathematical Phantoms

Similarly, the Hall–Janko group has a rank-3 permutation representation on 100 points…

I should learn this stuff when I grow an extra brain. This is the only mathematically significant appearance of the number 100 that I can think of offhand! It’s funny, given how much humans like it.

Posted by: John Baez on December 9, 2020 7:41 PM | Permalink | Reply to this

Re: Mathematical Phantoms

According to legend, the appearance of the number 100 struck group theorists as sufficiently interesting that they went looking for any other rank-3 permutation representations on 100 points, which led to the discovery of the Higman–Sims group.

Posted by: Blake Stacey on December 9, 2020 8:30 PM | Permalink | Reply to this

Re: Mathematical Phantoms

Griess tells the story of finding 100 an interesting number on page 24 here.

Posted by: Blake Stacey on March 29, 2021 8:14 PM | Permalink | Reply to this

Re: Mathematical Phantoms

I believe infinitesimals qualify as phantoms, at least in the mainstream approach to calculus. They ‘ought to’ exist, and are even manipulated to informally justify many arguments, but they don’t actually exist since in classical mathematics R is a field implies all infinitesimals are 0. If you work within constructive mathematics though, you can axiomatize fields in such a way that nonzero nilpotent elements cannot be chased away by the axiom of invertibility. Hence there are models of the real numbers with infinitesimals! And in that case, phantoms become very much… real (numbers).

Writing this makes me think: are there any other phantoms which are phantoms of constructive theories (killed by LEM)?

For example, the non-existence of the field with one element also leans on the axiom of invertibility of a field which forces a field to have two elements. Maybe one can define F1 to be the Heyting field where 0 is not apart from 1?

Posted by: Matteo Capucci on December 7, 2020 8:05 AM | Permalink | Reply to this

Re: Mathematical Phantoms

FWIW, I don’t think it is ever the case that the real numbers, as usually defined (e.g. via Cauchy sequences or Dedekind cuts), can contain infinitesimals. The “line objects” in synthetic differential geometry that contain infinitesimals do not coincide with the internally defined “real number object”. But that doesn’t detract much from the point. (Although I don’t expect one can say anything useful about 𝔽 1\mathbb{F}_1 this way.)

Posted by: Mike Shulman on December 7, 2020 4:20 PM | Permalink | Reply to this

Re: Mathematical Phantoms

I’ve wondered for a while if we could define a “smooth real line” which classically would be equivalent to the real line but which would live somewhere between the Cauchy and Dedekind reals, and which would in a suitable smooth topos be representable.

Starting with sheaves on the smooth Euclidean spaces n\mathbb{R}^n, it seems at least plausible that we could cut down the Dedekind real number object (which is “represented” by continuous maps into \mathbb{R}) into a smooth real number object represented by the \mathbb{R} in the site.

Posted by: David Jaz Myers on December 8, 2020 2:40 AM | Permalink | Reply to this

Re: Mathematical Phantoms

Thanks for the remark, Mike, I was actually quite convinced that some models of the KL axioms used internal real numbers are the line object.

Maybe one can concoct some models as topoi over some smooth cohesive topos? The idea being that real numbers from Grothendieck topoi inherit too much discreteness from sets, and you really need a smooth basis to make your reals smooth on the nose.

(There’s a high chance the above doesn’t make any sense, I apologize if this is the case).

Posted by: Matteo Capucci on December 10, 2020 11:01 AM | Permalink | Reply to this

Re: Mathematical Phantoms

Actually you can prove in constructive mathematics that the real numbers contain no infinitesimals: if xx and yy are real numbers such that |xy|<1n|x-y|\lt\frac{1}{n} for all positive integers nn, then x=yx=y. For Cauchy reals this is roughly the definition of when two Cauchy sequences are equivalent; for Dedekind reals it’s basically because they’re cuts of rationals and this property holds for rationals. Since an infinitesimal (and in particular a nilpotent) must by definition satisfy |x|<1n|x|\lt\frac{1}{n} for all nn, the only infinitesimal real number is zero.

I find this very unsatisfying too. It would be really nice if, as David suggested, we could define some kind of “real number object” internally in constructive mathematics that would give us smooth lines in smooth toposes. The closest thing I know of is Fourman’s definition of a smooth structure on a topos. Unfortunately even this approach can’t give us infinitesimals, since any subring of the Dedekind reals will inherit its property of containing no infinitesimals.

Posted by: Mike Shulman on December 11, 2020 2:10 AM | Permalink | Reply to this

Re: Mathematical Phantoms

I have a certain fondness for theory of real closed fields. A real closed field is a field such that:

  • 1-1 is not the square of anything,
  • for all xx either xx or x-x is the square of something,
  • every nonconstant polynomial of odd degree has a root.

Every such field has a total order such that xyx \ge y iff xyx - y has a square root.

Tarski showed the theory of real closed fields is complete and decidable. It’s elementarily equivalent to the first-order theory of the reals: i.e., a sentence in the first-order language of fields is valid in all real closed fields iff it’s valid in the reals.

Of course this theory is too weak to do a lot of analysis. But there are lots of real closed fields with infinitesimals, like the surreal numbers and the hyperreal numbers.

Posted by: John Baez on December 11, 2020 7:03 PM | Permalink | Reply to this

Re: Mathematical Phantoms

Berkeley certainly thought infinitesimals were phantoms. In The Analyst he wrote:

And what are these Fluxions? The Velocities of evanescent Increments? And what are these same evanescent Increments? They are neither finite Quantities nor Quantities infinitely small, nor yet nothing. May we not call them the ghosts of departed quantities?

By the way, the subtitle of this book was “A Discourse Addressed to an Infidel Mathematician. Wherein It is examined whether the Object, Principles, and Inferences of the modern Analysis are more distinctly conceived, or more evidently deduced, than Religious Mysteries and Points of Faith”.

Posted by: John Baez on December 7, 2020 7:30 PM | Permalink | Reply to this

Re: Mathematical Phantoms

The problems with defining the field of one element through field theory go back to commutative ring theory, in that any commutative ring that has 1=01 = 0 has a characteristic of 1, and every commutative ring with characteristic of 1 is isomorphic to the terminal ring. The terminal ring isn’t even a cancellative ring, let alone a field. This doesn’t change regardless if the external logic used is Heyting or Boolean.

Posted by: Madeleine Birchfield on December 8, 2020 10:25 PM | Permalink | Reply to this

Re: Mathematical Phantoms

And of course the terminal ring in the category of commutative rings is the trivial ring.

Posted by: Madeleine Birchfield on December 8, 2020 10:42 PM | Permalink | Reply to this

Re: Mathematical Phantoms

I think you’re onto something with the suggestion that the rigidity of algebra has to do with the plethora of phantoms in that subject. I’ve been trying to coming up with phantoms in analysis and probability, and mostly I’ve come up with things that probably seemed like phantoms at some point in time, but have become perfectly respectable mathematical objects via one kind of completion process or another. Examples would include:

  • Irrational numbers (as you mentioned above)
  • Distributions, like the Dirac delta function and its derivatives
  • Infinitesimals (as Matteo Capucci mentioned above)
  • Graph limits

Some other things I’ve thought of are examples of things we would like to exist, but somehow don’t come close enough even to have a phantom existence, like Lebesgue measure on an infinite dimensional Banach space.

One example that might count is infinite dimensional limits of random matrices, which often “exist” as noncommutative random variables in free probability theory. Or maybe those should actually be thought of as avatars rather than phantoms.

Posted by: Mark Meckes on December 7, 2020 8:40 AM | Permalink | Reply to this

Re: Mathematical Phantoms

Thinking about examples of this sort, I’m tempted to suggest that the “rigidity” of algebra is not what creates the patterns that summon phantoms, but rather what prevents things that “should” exist from actually existing in any real sense, making them into phantoms instead.

Posted by: Mike Shulman on December 7, 2020 4:07 PM | Permalink | Reply to this

Re: Mathematical Phantoms

I think this suggestion meshes well with my feeling that one of the main differences between algebra(-like subfields) and analysis(-like subfields) is that in the latter, if you know a theorem of the form “PP implies QQ”, it’s likely that various theorems of the form “(approximately PP) implies (approximately QQ)” are also true. Whereas in the former, it’s harder even to make sensible statements of that form.

Posted by: Mark Meckes on December 8, 2020 3:49 PM | Permalink | Reply to this

Re: Mathematical Phantoms

Sounds related to comments of Terry Tao I gathered here on open and closed conditions.

Posted by: David Corfield on December 8, 2020 4:50 PM | Permalink | Reply to this

Re: Mathematical Phantoms

Yes, definitely related. I don’t think it’s the whole story, but part of what’s behind the distinction I described is that (as Terry writes) algebra is heavy on closed conditions, whereas analysis is heavier on open conditions.

Of course the boundary between algebra and analysis is arbitrary and fuzzy. I think that this closed/open condition business is actually part of how we decide, as a community, which parts of mathematics we choose to describe with which label.

Posted by: Mark Meckes on December 8, 2020 5:44 PM | Permalink | Reply to this

Re: Mathematical Phantoms

Perhaps an example of this sort from topology is a p-compact group.

Posted by: Mike Shulman on December 7, 2020 4:22 PM | Permalink | Reply to this

Re: Mathematical Phantoms

I think the uniform probability distribution on the real line is a continuous phantom. It is somewhat resistant to understanding via completion processes. Or maybe someone here knows how to make sense of it.

Posted by: Jonathan Kirby on December 7, 2020 10:02 AM | Permalink | Reply to this

Re: Mathematical Phantoms

I think that’s a really good example! Slightly better, maybe, is its cousin the uniform probability distribution on the natural numbers, which plays a major role in analytic number theory. The notion of density serves as a good substitute for it in some contexts, but among other problems fails to actually be a measure.

Posted by: Mark Meckes on December 7, 2020 10:13 AM | Permalink | Reply to this

Re: Mathematical Phantoms

Can you say more, Jonathan? I’d be interested to know what you have in mind.

Part of the reason why I’m interested is that I’ve come to think of the uniform distribution on the real line as being “the” normal distribution. (I put “the” in quotes because, of course, you have to choose a mean and a variance, but that feels like a relatively minor consideration to do with the symmetry group of \mathbb{R}.)

All I ever seem to write about on this blog these days is entropy, so I feel like a stuck record, but the reason why I think of the normal distribution on \mathbb{R} as the morally uniform distribution is as follows.

On a finite set {1,,n}\{1, \ldots, n\}, the probability distribution (p 1,,p n)(p_1, \ldots, p_n) that maximizes the entropy p ilogp i- \sum p_i \log p_i is the uniform distribution. For a probability density function ff on \mathbb{R}, you can try to maximize the differential entropy flogf-\int f \log f. It’s clear that this is impossible, because e.g. scaling \mathbb{R} by a factor of 22 doubles the entropy of any distribution. And similarly, translating doesn’t change entropy. But if you fix the mean and variance, there is a distribution that maximizes entropy, and it’s the normal distribution. This is the justification for thinking of it as the analogue of the uniform distribution.

But you may be coming at this from a completely different angle, which I’m curious to hear.

Posted by: Tom Leinster on December 7, 2020 11:22 AM | Permalink | Reply to this

Re: Mathematical Phantoms

I think it’s fair to call the mean a minor consideration in this context, but not so much the variance. I’d rather say that for many purposes, “the uniform distribution on \mathbb{R}“is the same as “the normal distribution with infinite variance”.

As for the maximum entropy argument, instead of fixing the variance you could assume that ff is supported on a compact interval II; differential entropy is then maximized by the uniform distribution on II. Or you could fix the first absolute moment, and get an exponential distribution, etc., etc.

This is not to say there aren’t good reasons for focusing on the variance as the best normalizing condition for a probability distribution a lot of the time. But it’s certainly not the only one.

Posted by: Mark Meckes on December 8, 2020 3:28 PM | Permalink | Reply to this

Re: Mathematical Phantoms

Thanks, Mark. That’s clarifying.

I’m writing this off the top of my head so it may be screwy, but let me try out the following thought.

Let’s say that a “good” map \mathbb{R} \to \mathbb{R} is a diffeomorphism ϕ\phi such that pushing a probability measure forward along ϕ\phi has a predictable effect on entropy. I say “diffeomorphism” rather than “measure-isomorphism” as we’re talking about differential entropy. Formally, I mean that ϕ\phi is good if for every probability measure ν\nu on \mathbb{R} with a differentiable probability density function, the entropy of ν\nu determines the entropy of the pushforward ϕ *ν\phi_\ast \nu.

An equivalent definition: a diffeomorphism ϕ:\phi: \mathbb{R} \to \mathbb{R} is good if whenever ff is a differentiable probability density function on \mathbb{R}, the entropy of ff determines the entropy of the PDF ϕ *f:xϕ(x)f(ϕ(x))\phi^\ast f : x \mapsto \phi'(x)f(\phi(x)).

The good maps \mathbb{R} \to \mathbb{R} form a group under composition.

For example, any translation ϕ:xx+a\phi: x \mapsto x + a is good, because H(ϕ *f)=H(f)H(\phi^\ast f) = H(f) for all PDFs ff. Any dilation (dilatation?) ϕ:xax\phi: x \mapsto ax is good, because H(ϕ *f)=|a|H(f)H(\phi^\ast f) = |a| H(f) for all PDFs ff. Since the good maps form a group, any affine map ϕ:\phi : \mathbb{R} \to \mathbb{R} is good.

Are the affine maps the only good maps? I can’t think of any others.

Now when we’re looking to find “the” maximum entropy distribution on \mathbb{R}, or indeed any space, it’s natural to seek a way of ignoring the good maps — of saying “yes yes, we all know what happens to the entropy when you compose with a good map, but let’s put that aside”.

The group of good maps acts on the set of differentiable PDFs, and if it’s true that good == affine then one of the orbits is the set of normal distributions of arbitrary mean and variance, another is the set of two-sided exponential distributions of arbitrary mean and variance, and so on.

We can’t ask whether one orbit has greater entropy than another, because within a single orbit the different PDFs have different entropies. Instead, we can choose a representative from each orbit (like each country choosing a single Olympic shot-putter) and let those compete against each other.

Now if good == affine then I think each orbit contains exactly one member of mean 00 and variance 11, say (or any other constants). So we could use those as the representatives. Then the normal orbit would win.

But I guess each orbit also contains exactly one member of mean 00 and first absolute moment 11. If we use those as the representatives, a different orbit would win.

It seems to me that focusing on distributions supported on a compact interval is different in flavour: not every orbit contains such a distribution.

Not sure where I’m going with this, but does it make sense?

Posted by: Tom Leinster on December 8, 2020 7:58 PM | Permalink | Reply to this

Re: Mathematical Phantoms

It’s not actually that different: not every distribution has finite variance, either. So (assuming good = affine; I haven’t thought carefully about that but I suspect it’s true):

  • Each orbit which contains a member with finite variance (and therefore consists only of PDFs with finite variance) contains exactly one member of mean 0 and variance 1.
  • Each orbit which contains a member with finite first absolute moment (and therefore consists only of PDFs with finite first absolute moment) contains exactly one member of median 0 and first absolute moment 1. (Median is more natural than mean here, but this is a minor point.)
  • Each orbit which contains a member with compact support (and therefore consists only of PDFs with compact support) contains exactly one member, with, say, [0,1][0,1] as its minimal supporting closed interval.

And we can say which orbit’s special member maximizes entropy in each case.

If you like, in the third case the quantity “diameter of the support” plays an analogous role to “standard deviation” in the first case and “mean absolute deviation” in the second. It’s easy to be misled by the most common examples, but all three of those may be infinite for an arbitrary PDF.

Posted by: Mark Meckes on December 9, 2020 9:10 AM | Permalink | Reply to this

Re: Mathematical Phantoms

I concede! Thanks; I’ve really learned from this.

Posted by: Tom Leinster on December 9, 2020 11:25 AM | Permalink | Reply to this

Re: Mathematical Phantoms

I really like this idea, and I think it relates to what Stefan Forcey says above: Some phantoms have a canonical solidification. Which in this case would mean that one could consider a normal distribution as the canonical solidification of the uniform distribution on \mathbb{R}.

Posted by: Abel Jansma on December 14, 2020 1:39 PM | Permalink | Reply to this

Re: Mathematical Phantoms

My favourite phantoms (maybe ex-phantoms now) are non-wellfounded sets. Azcel’s 1988 book brought together the theory of how to get this working, by identifying sets with decorated graphs; Barwise & Moss 1996 give some applications (see here). Conway’s hyperreals are kind of similar.

Recently, some philosophers got interested in impossible worlds: these are metaphysical phantoms. Though the ultimate metaphysical phantom, imo, is the global truth predicate (only “local approximations” exist).

Posted by: Jeffrey Ketland on December 7, 2020 10:32 AM | Permalink | Reply to this

Re: Mathematical Phantoms

The hyperreals are Robinson’s; it’s the surreals that are Conway’s.

Posted by: Mike Shulman on December 7, 2020 4:09 PM | Permalink | Reply to this

Re: Mathematical Phantoms

Oops - yes, you’re right!

Posted by: Jeffrey Ketland on December 7, 2020 7:36 PM | Permalink | Reply to this

Re: Mathematical Phantoms

Jeffrey:

Though the ultimate metaphysical phantom, imo, is the global truth predicate (only “local approximations” exist).

I like the idea that truth is the ultimate mathematical phantom. (I know that’s not what you said.) Rota said “Mathematics is not about truth, it’s about the game of truth.”

Posted by: John Baez on December 14, 2020 9:03 PM | Permalink | Reply to this

Re: Mathematical Phantoms

I’ve thought of modern formal mathematics before as somewhat of a logical sandbox game, in that one selects the building tools in the game in the form of foundational axioms for mathematics (whether set theory, first order logic, type theory, etc), and from there, construct mathematical objects and proofs from the foundational axioms, with no in-game end goals, only user created goals.

Posted by: Madeleine Birchfield on December 16, 2020 7:44 AM | Permalink | Reply to this

Re: Mathematical Phantoms

Probably my favorite mathematical phantom is the set of all sets. (Or the (2-)category of all categories, or the type of all types, etc. — I’m talking about size issues, not categorical dimension.)

Posted by: Mike Shulman on December 7, 2020 4:09 PM | Permalink | Reply to this

Re: Mathematical Phantoms

I think we’ve all walked across this phantom bridge at some time. Just like the wiley coyote, if you don’t look down you never seem to fall.

Posted by: David Jaz Myers on December 8, 2020 2:44 AM | Permalink | Reply to this

Re: Mathematical Phantoms

Here are some more phantoms from the Category Theory Community Server, where my link to this post triggered a lively discussion. Life is getting complicated now, with two active category-oriented forums to attend to.

Nathanael Arkor wrote:

“Small (nonposetal) cocomplete category” has a similar flavour to some of the examples mentioned here (but may actually be reified in a constructive setting). The study of locally presentable categories is in part inspired by this search. Are there any other categorical examples?

Matteo Capucci wrote:

In fact, I’d say pure phantoms do not last long since once you realize you may want them you’re halfway through formalizing them. A different kettle of fish is to formalize them right. E.g., 𝔽 1\mathbb{F}_1​ has been formalized for at least 25 years but we are still in the pre-paradigmatic phase of the associated Kunnian revolution. Infinitesimals are probably stuck there too.

Amar Hadzhisanovic wrote:

So mathematical phantoms go the other way compared to regular phantoms: they are spirits of the not-born-yet rather than spirits of the dead…

I suppose pure motives are another quintessential mathematical phantom.

Or maybe that should be mixed motives. I don’t really know much about it.

Jules Hedges wrote:

A general class of examples is “things that exist in a conservative extension of your theory”.

For example if you’re working in ZFC then you can pretty much get away with pretending that proper classes exist most of the time, even though plain ZFC doesn’t think they exist.

Morgan Rogers wrote:

Or classifying objects which aren’t representable (things which exist in e.g. a topos associated to your category, but don’t pull back to concrete things in the category).

Peter Arndt wrote:

The machinery of topos theoretic spectra is a midwife helping unborn spirits into existence: E.g. the initial local ring under a given ring in general doesn’t exist … in the category of sets — but it does exist in another topos!

Posted by: John Baez on December 7, 2020 6:43 PM | Permalink | Reply to this

Re: Mathematical Phantoms

Also on the Category Theory Community Server, Reid Barton adds:

In homotopy theory there is a phenomenon that if you look at (say) the homotopy groups of spheres at a fixed prime pp, there are certain patterns where nothing appears until degree roughly pp, then the next term is in degree roughly 2p2p, and in general there is a simple description up to degree around p 2p^2, and then a more complicated description until around p 3p^3, and so on. But for any fixed prime pp, this description breaks down at some point because the powers of pp don’t grow quickly enough.

So there’s this funny idea of a hypothetical “infinite prime” (due I think to Haynes Miller?) at which all of these components would be spread out infinitely far from each other, and could never interfere.

I’m not familiar enough with this area to know whether the “infinite prime” is really a “phantom” in the sense discussed here, though, or some other kind of hypothetical object.

Some discussion followed in which this “infinite prime” was distinguished from the “infinite prime” in number theory. The latter is another interesting phantom: see e.g.

Posted by: John Baez on December 9, 2020 7:57 PM | Permalink | Reply to this

Re: Mathematical Phantoms

Haynes and I, and others I’m sure, learned of the infinite prime in stable homotopy theory from Michael Barratt and Mark Mahowald back in the day. There has [IMO] been some serious progress on this question by Barthel, Schlank, and Stapleton, cf

and

Posted by: jackjohnson on December 9, 2020 9:24 PM | Permalink | Reply to this

Re: Mathematical Phantoms

Cool! This very much fits into the theme we’ve been seeing, of “monsters as limiting objects”.

Inspired by the Ax–Kochen isomorphism theorem, we develop a notion of categorical ultraproducts to capture the generic behavior of an infinite collection of mathematical objects. We employ this theory to give an asymptotic solution to the approximation problem in chromatic homotopy theory. More precisely, we show that the ultraproduct of the E(n,p)E(n,p)-local categories over any non-prinicipal ultrafilter on the set of prime numbers is equivalent to the ultraproduct of certain algebraic categories introduced by Franke. This shows that chromatic homotopy theory at a fixed height is asymptotically algebraic.

Posted by: John Baez on December 9, 2020 9:51 PM | Permalink | Reply to this

Re: Mathematical Phantoms

Quantum Field Theory!

Posted by: AB on December 7, 2020 8:57 PM | Permalink | Reply to this

Re: Mathematical Phantoms

That is only the case in 4 (3+1) dimensions and higher. Quantum field theories have already been axiomised by the Wightman axioms and models of QFT have already been constructed in 2 (1+1) and 3 (2+1) dimensions.

Posted by: Madeleine Birchfield on December 8, 2020 6:28 AM | Permalink | Reply to this

Re: Mathematical Phantoms

I think “space isomorphic with its own function space” provides another nice example of a mathematical phantom, whose non-existence haunted at Dana Scott before he was finally able to prove its existence. He tells this story in a 1976 lecture:

Returning to 1969, what I started to do was to show Strachey that he was all wrong and that he ought to do things in quite another way. He had originally had his attention drawn to the λ-calculus by Roger Penrose and had developed a handy style of using this notation for functional abstraction in explaining programming concepts. It was a formal device, however, and I tried to argue that it had no mathematical basis. I have told this story before, so to make it short, let me only say that in the first place I had actually convinced him by “superior logic” to give up the type-free λ-calculus. But then, as one consequence of my suggestions followed the other, I began to see that computable functions could be defined on a great variety of spaces. The real step was to see that function-spaces were good spaces, and I remember quite clearly that the logician Andrzej Mostowski, who was also visiting Oxford at the time, simply did not believe that the kind of function spaces I defined had a constructive description. But when I saw they actually did, I began to suspect that the possibilities of using function spaces might just be more surprising than we had supposed. Once the doubt about the enforced rigidity of logical types that I had tried to push onto Strachey was there, it was not long before I had found one of the spaces isomorphic with its own function space, which provides a model of the “type-free” λ-calculus. The rest of the story is in the literature.

(Incidentally, the reference to Penrose might be surprising, but I was just listening to a panel discussion about Christopher Strachey where Penrose tells this story of how he managed (after several attempts!) to get Strachey interested in λ-calculus.)

Posted by: Noam Zeilberger on December 7, 2020 9:16 PM | Permalink | Reply to this

Re: Mathematical Phantoms

I first learned about the λ\lambda-calculus from Penrose’s 1989 book The Emperor’s New Mind, in a log cabin on the banks of Loch Awe.

Posted by: Tom Leinster on December 7, 2020 10:11 PM | Permalink | Reply to this

A topological phantom

Closely related to infinitesimals suggested by Capucci, my favorite non-algebraic example of a phantom is a nearness relation in topology.

One likes to say that continuous map take nearby values at nearby points. Naively, one could imagine a formalization where a topological space consists of a set S, along with a distinguished binary relation ≈ (nearness), and the continuous maps between topological spaces are just the functions that preserve this relation.

To the regret of Real Analysis 101 students everywhere, this naive formalization does not work: there is no binary relation ≈ on the real numbers such that f: ℝ→ ℝ is continuous precisely if it preserves ≈.

But (as Borovik pointed out) even if a desired entity does not exist, it may transpire that some counterpart of the nonexistent entity exists elsewhere. Classical set theory provides nearness relations for a well-known class, the so-called Alexandroff spaces. For functions between Alexandroff spaces, the specialization preorder ≤ acts as a nearness relation.

Unfortunately, one cannot go further: Top is not equivalent to any full subcategory of the category of (reflexive) graphs, and in fact the category of Alexandroff spaces is maximal with this propery.

However, a bit of non-standard analysis takes us really close: if we replace the notion of binary relation definable in ZFC with binary relation definable in Nelson’s Internal Set Theory, we get nearness relations that work for standard continuous functions between standard topological spaces. I.e. in IST we can uniformly equip each standard topological space S with a relation ≈S such that a standard function f: A→B between two spaces is continuous precisely if x ≈A y implies f(x) ≈B f(y) for all x,y in A.

These new relations are very nice to have: using these, one can write the topological semantics for intuitionistic logic in Kripke style, and present sheaves as pointwise assignments of stalks to points and transition functions to pairs of nearby points (the way Michael Robinson prefers to present sheaves on Alexandroff spaces).

Posted by: Zoltan A. Kocsis on December 8, 2020 5:05 PM | Permalink | Reply to this

Re: A topological phantom

I like the idea of ‘nearness relations’ as being a phantom that the usual approach to topology sidesteps. I often try to motivate the concept of ‘neighborhood’ by saying a neighborhood of xx is a way of saying when a point yy is near xx. So in this approach we give up trying to have a single concept ‘yy is near xx’, instead having many such concepts, one for each neighborhood of xx.

Posted by: John Baez on December 8, 2020 9:48 PM | Permalink | Reply to this

Re: A topological phantom

That many-instead-of-single brings to mind the concept of “approximate identity” (under convolution), defined as a sequence of functions converging appropriately to the delta “function” (a phantom if ever I saw one).

Posted by: Allen Knutson on December 9, 2020 2:21 AM | Permalink | Reply to this

Re: A topological phantom

Yes! I guess one of the most fundamental ‘many-instead-of-single’ tricks is to use any ϵ>0\epsilon \gt 0 as a stand-in for an infinitesimal.

There’s a cool paper called something like “Ghosts of vanished monsters” that talks about the opposite of the Dirac delta: the limit of a sequence of nonnegative functions that get more and more spread-out and flat while keeping their integral equal to 1. I can’t find it anymore.

Posted by: John Baez on December 9, 2020 2:37 AM | Permalink | Reply to this

Re: A topological phantom

I’d like to see that paper! That’s almost exactly the same as the phantom Jonathan Kirby suggested up above.

Posted by: Mark Meckes on December 9, 2020 9:23 AM | Permalink | Reply to this

Re: A topological phantom

I used Google Scholar to search for articles containing the phrase “vanished monsters”, and after sifting through a bunch of irrelevant junk I finally found what seems like the right one:

  • Bogdan Mielnik, Space echo, Letters in Mathematical Physics 12 (1986), 49–56.

Abstract. Residual effects of vanishing fields are shown to provide a formal analogue of the spin echo effect for the nonrelativistic spinless particle. The possible significance to the problem of nonlinear distributions is discussed.

It has a promising reference to this:

  • A. Alonso, Shrinking Potentials in the Schrödinger’s Equation, Ph.D. thesis, Princeton University, 1980.
Posted by: John Baez on December 9, 2020 4:05 PM | Permalink | Reply to this

Re: A topological phantom

Thanks! That paper says that the phrase “vanished monsters” comes from the title of a 1972 talk by John Klauder in Warsaw.

A little more googling also shows that a more accurate version of that promising reference would be: A. Alonso y Coria, …, 1978. I haven’t found my way to the actual thesis yet, though.

Posted by: Mark Meckes on December 9, 2020 5:06 PM | Permalink | Reply to this

Re: Mathematical Phantoms

A bit late to the party, but my favorite might be “the space of random sequences” - with, of course, its non-phantasmal counterpart, the locale of random sequences. Alex Simpson’s paper on the topic was a significant influence in getting me interested in locale theory.

Posted by: Alex Elsayed on February 10, 2021 6:23 AM | Permalink | Reply to this

Re: Mathematical Phantoms

The way I have understood ‘the field of order 1’ (to the extent I have understood it, which isn’t great) is in terms of counting formulas.

There isn’t a literal field of order 1, so, in the metric defined on the real numbers, the order qq of a finite field has |q1|1|q-1| \geq 1. But that’s not the end of the story. There are other metric completions of \mathbb{Q}, one for each prime pp, which are the pp-adic completions of \mathbb{Q}. The pp-adic distance || p||_{p} has a ‘blind spot’, which is that any (positive integral) power of pp has distance 1 from 1. But it is possible for powers of other primes to converge to 1 pp-adically; this can happen in sequences whether or not more than one prime appears this way.

If you fix a single prime rr distinct from pp when doing that, something nice happens:

Let o p(r)o_{p}(r) be the order of rr, considered as an element of (/(p)) ×(\mathbb{Z}/(p))^{\times}. Let {a n}\{ a_n \} be a sequence of positive integers. Then the sequence r a nr^{a_n} converges pp-adically to 1 if and only if o p(r)a no_{p}(r) \mid a_n for all sufficiently large nn, and the sequence a na_n itself converges to 0 pp-adically. Each r a nr^{a_n} is the order of a characteristic rr finite field, and considering such a sequence of characteristic rr fields aligns nicely with the description of their algebraic closure as being the direct limit of all finite fields of characteristic rr.

Posted by: David Harden on February 7, 2023 9:47 AM | Permalink | Reply to this

Post a New Comment