A mean field cannot get out of the way

Correlation energy is defined as a subtraction: the exact energy less what a single determinant can manage. Every essay on correlation so far has treated it as a quantity to be measured — how it scales, how it splits into kinds, what an occupation number says about it. None has asked the flat-footed question of how big it gets, and the answer is that there is no limit at all.

At a repulsion of thirty-two times the hopping the mean field misses 27.82 and the exact energy is −0.2946. The error is ninety-four times the answer, and doubling the repulsion again would roughly double it.

That is not a statement about this model being unrealistic. It is a statement about what a mean field is, and the reason is simple enough to write down before anything is computed.

The count that is the slope #

A mean field spreads each electron over the whole system and lets it move in the average of the others. So the probability of finding an up electron and a down electron on the same site is the product of two independent densities, summed over the sites,

$$D_{\text{mf}} = \sum_i n_{i\uparrow} n_{i\downarrow}$$

and for a uniform density — $n/2N$ of each spin on each of $N$ sites — that is $n^2/4N$.

Nothing in that expression is an energy. It is a count of coincidences: how often two electrons that are not paying any attention to each other happen to be in the same place. And it does not fall when the repulsion rises, because a symmetric system’s mean field has nowhere to move its density to. Every site is like every other; spreading out evenly is what it already does.

So the mean field pays $U$ times a fixed number, and its error grows linearly with a slope that can be written down from the electron count and the site count alone.

Two electrons on four sites gives 0.25 and measures 0.2477. Four on four gives 1.00 and measures 0.9820. Six on six gives 1.50 and measures 1.4715. Doubling the electron count quadruples the slope, which is the $n^2$ in the formula showing itself: 0.9820 against 0.2477 is a ratio of 3.96.

The exact state does the thing the mean field cannot #

The other half of the divergence is what the exact state does with its freedom, and it is worth drawing because it is the whole of the physics.

The exact double occupancy starts at 1.0000, which is $n^2/4N$ — the two agree at $U = 0$ because they must — and reaches 0.0091 at $U = 32t$, a fall of a hundred and ten times. Between $U = 4t$ and $U = 32t$ it falls by 2.7, then 3.5, then 3.9 for each doubling of the repulsion, on its way to the factor of four that says it goes as $1/U^2$.

That is the electrons learning to avoid each other. It costs them kinetic energy — an electron that keeps out of another’s way is an electron that is not hopping freely — and the exact state pays exactly as much of that as the saving is worth. What comes out is an energy that saturates: at $U = 32t$ the exact energy is −0.2946, and at infinite repulsion it would be zero, because a state in which no two electrons ever meet pays no repulsion at all.

A mean field cannot do any of this. Avoiding a particular electron is a correlation between two coordinates, and a product of one-electron densities has no such correlation in it by construction. That is not a defect of the approximation, it is the approximation.

Where the density can move, the slope changes #

There is one way a restricted mean field can reduce its coincidence count, and it is worth separating out because it is the exception that fixes the rule: it can make the density non-uniform.

A four-site chain with two electrons has more density on its middle sites than on its ends, so its coincidence count starts above the uniform value at 0.3000. Raising the repulsion pushes the density flat, the count falls to the uniform 0.25, and there it stops — because a uniform density is the least coincidental arrangement available to a product of densities, and past that point there is nothing left to do.

At half filling even that escape is closed. The density is already uniform at every repulsion, so the slope is $N/4$ from the first step and the error is a straight line from the origin.

The error, split into the two things it is made of #

The account above has two halves — the mean field pays repulsion it need not, and the exact state gives up kinetic energy to avoid paying it — and both are separately computable, because the energy of this model is a kinetic term plus $U$ times the double occupancy and nothing else.

At $U = 32t$ the mean field overpays by 31.71 in repulsion and the exact state has given up 3.89 in kinetic energy to arrange it, leaving 27.82. The first number grows linearly for ever and the second saturates — the exact state cannot give up more kinetic energy than it has — so the slope of the total is very slightly below $n^2/4N$, which is exactly the discrepancy the measurement shows.

The rightmost columns carry a fact that is easy to miss and is the whole argument in one line. The mean field’s kinetic energy is −4.4721 at every repulsion in the table, which is its value with no repulsion at all. It never moves, because the density never moves; there is nothing for the repulsion to push. The exact state’s kinetic energy rises from −4.4721 to −0.5870 over the same range, which is what avoidance costs.

The escape that is not available, and the one that is #

There is a second escape, and it has already appeared. Release the constraint that the two spins share their spatial orbitals and the mean field can put up electrons on alternate sites and down electrons on the others, which drives $\sum_i n_{i\uparrow} n_{i\downarrow}$ down without correlating anything.

The released field’s error saturates. That is the same mechanism as the exact state’s — a smaller coincidence count — arrived at by a different route, and the route matters: the exact state reduces the count by correlating two coordinates, while the released mean field reduces it by giving the molecule a spin density it does not have.

So the growth described in this essay is a property of the restricted mean field, and the escape from it costs a description that is wrong in a way the energy cannot report. That trade is worked out in detail. The claim here is narrower and firmer: as long as the mean field is required to describe the state’s symmetry correctly, its error is $U$ times a count and there is no bound on it.

Why this is a fact about the model and not about a molecule #

The temptation at this point is to conclude that mean-field theory is useless for a strongly repelling system, and it is worth being careful about what has and has not been shown.

The repulsion here is on-site only. In a real molecule the dominant part of the repulsion is between electrons on different centres, and that part is described well by a mean field — it is a classical electrostatic energy between two densities, which is exactly what a product of densities computes correctly. What is missing is the correlation hole: the reduction in the chance of finding a second electron near the first. This model is built so that the hole is the only thing there is.

The absolute size is in units of the hopping. Saying the error is 27.82 at $U = 32t$ is saying it is 27.82 hoppings, and a real system with $U/t = 32$ is a system where the bonding has almost stopped. For most molecules $U/t$ is nearer one, where the error is a tenth of a hopping and the mean field is very good indeed.

What survives is the shape. The error is linear in the repulsion with a computable slope, so it does not converge, does not average out, and does not become negligible for a large system: doubling the sites at fixed filling doubles $N/4$. That is the property that matters for reading a calculation, and it is why energies from a mean field are compared with each other rather than used absolutely.

The counterweight to all of the above is worth stating plainly, because it is what keeps mean-field methods in use. The energy is second order in the error in the wavefunction and everything else is first order, so a mean field whose energy is badly wrong may still have most other things approximately right — and, less comfortably, one whose energy is nearly right can be wrong about everything else.

The connection to a gap that has no business existing #

The same count explains a result three fields away. A half-filled band in a one-electron model is a metal, because the highest occupied level and the lowest empty one are the same level. Put a repulsion in and the exact answer develops a gap.

The sharpest single case is the charge gap. A half-filled four-site ring has an exact gap that grows without limit with the repulsion, with a slope of one at large U — moving an electron onto an already-occupied site costs U — and the one-electron answer is exactly zero at every repulsion whatever. That is not a quantitative failure of a mean field; it is a quantity the mean field does not have.

The gap grows as $U$ for the same reason the error does: at large repulsion the cost of putting two electrons on one site is $U$, and every quantity that requires doing so inherits it. A mean field that cannot stop doing it pays that cost in its ground-state energy; the exact state pays it only when it is made to, which is what a charge gap measures. The insulator band theory cannot see is the same arithmetic told as a fact about conduction.

Where all of it starts is the smallest system that has the effect at all: two sites and two electrons, whose four exact states can be written down. The state with the spins parallel does not move as the repulsion is raised, because two parallel spins cannot occupy one site and so pay nothing — a coincidence count of exactly zero, arrived at without any calculation.

The same defect where there is nothing to correlate with #

The slope measured here counts coincidences that a spread-out density cannot avoid, and the count is a property of the density rather than of the interaction. That framing has a consequence which is worth following, because it produces the sharpest possible test of a mean field: apply it to one electron.

One electron cannot coincide with anything. Its exact correlation energy is zero — not small, zero, by the definition of the quantity, since the exact wavefunction of a one-electron system is a single determinant and the subtraction has nothing to subtract. Any method that reports a non-zero correlation energy for a hydrogen atom has reported the energy of an electron interacting with itself.

Hartree–Fock passes. Its exchange term is constructed so that the part of the Coulomb energy in which an electron repels its own density is cancelled exactly, term by term, and the cancellation is algebraic rather than numerical. That is why a Hartree–Fock calculation on hydrogen returns $-0.5$ hartree exactly and a correlation energy of nothing.

The approximate density functionals in everyday use do not pass. They replace the exchange term by a functional of the density, and a functional of the density has no way of knowing which part of the Coulomb repulsion was an electron meeting itself. So the cancellation is approximate, and what is left over shows up as a correlation energy for a system that has none: a few thousandths of a hartree for the gradient-corrected functionals, several times that for the simplest local one. On a hydrogen atom, where the right answer is zero.

That residue has a name — self-interaction error — and its consequences are not confined to one-electron systems, which is what makes the test worth running.

An electron repelling itself is pushed outward, so the density spreads further than it should. Every symptom of an over-delocalised density follows: barriers to reaction that come out too low, because a transition state has its charge spread over more centres than the reactants do; band gaps that come out too small; charge transferred too readily between fragments.

And the failure is worst exactly where this essay’s mean field is best. A spread-out density is what a mean field prefers, and a spurious self-repulsion rewards spreading out further. So the two errors do not cancel: the coincidence count measured above says a uniform density pays too much repulsion, and the self-interaction says the density is driven to be more uniform than it should be.

The stretched one-electron bond is the standing example. A hydrogen molecule ion pulled apart must end as a hydrogen atom and a bare proton, with the electron on one side or the other. An approximate functional puts it too low in energy with half an electron on each nucleus, because half an electron repelling half an electron is a smaller self-interaction than a whole one repelling itself. That is one electron, no correlation, and a qualitatively wrong dissociation — the same shape of failure as the mean field’s here, with the sign of the error reversed.

Which gives correlation a symmetry worth stating plainly. A spin-paired mean field cannot let two electrons avoid each other, and an approximate functional cannot stop one electron from avoiding itself. The first misses an energy that grows without limit as the repulsion rises; the second manufactures one where no repulsion should exist at all. Both are failures of the same kind — a description in terms of a density alone, being asked a question about which electron is where — and both are invisible to the variational principle, since neither method’s energy is a bound on anything once the exchange has been approximated.

The one-electron test is the cheapest diagnostic in the subject and it is nearly free: run the method on a hydrogen atom and see whether it reports a correlation energy. The exact answer is known to infinite precision, it is zero, and a method that fails there fails everywhere for the same reason.

What was computed, and how #

Every energy above is an eigenvalue. The exact ones come from diagonalising the full matrix over configurations — 36 of them for four electrons on four sites, 400 for six on six — and the mean-field ones from iterating a one-electron problem until the density it is built from agrees with the density it produces.

Four things are checked, and the fourth is the one that could have failed.

At zero repulsion the two agree exactly. They must: the exact ground state of a system with no interaction is a single determinant, and the mean field is searching the space of single determinants. Agreement to 10⁻⁹ there is the check that the two calculations are being done on the same system.

The error is never negative. A mean field is variational, so it cannot beat the exact answer, and an implementation that did would be reporting an arithmetic fault rather than a discovery.

The exact double occupancy falls by more than twenty times over the range while the mean field’s settles within 0.02 of the uniform count. Both halves are required, because a comparison in which neither side moves proves nothing.

And the measured slope agrees with $n^2/4N$ to within 0.03 for three systems, including one with a different number of sites. That is the check with the most ways to fail: the closed form is derived from the density and contains no eigenvalue, so if the account above were wrong the number would simply not match.

Still open: the count with long-range repulsion #

The natural extension is the one this model cannot reach. Real repulsion is long-ranged, so the coincidence count is not a sum over sites but an integral over pairs of positions, and turning the closed form above into an integral over that interaction is the calculation that would say whether the slope stays a count. The self-interaction discussed above is the other half of the same arithmetic, and neither half is computed here.

The nearer question is what the count does at fillings between the two computed here. Two electrons on four sites and four on four differ by a factor of four in slope and by a qualitative feature — whether the density can flatten at all — and the boundary between them is where a mean field stops having anywhere to move. Finding it would need odd electron counts, which a spin-paired field cannot describe without breaking the symmetry the whole comparison depends on.