Where the electrons are, without subtracting anything

The correlation energy can be taken apart by its energy, its wavefunction, its scaling and its reference. Each shows the same shape of trouble, and the reference shows it worst: a correlation energy is a subtraction, exact minus mean-field, and two defensible choices of mean field disagree about the answer by a factor of 259.

There is an object that has none of that trouble. Correlation is a statement about where the electrons are relative to one another, and there is a function that says so directly. The pair distribution needs no reference, no determinant and no subtraction: it is a property of the exact state alone, and it is computable exactly for every system here.

Computed, it does two things. It measures the correlation as a probability rather than as an energy — and it separates two systems the energy cannot tell apart.

The object #

Given the exact ground state, the pair distribution is the probability of finding an electron on site $j$ given one on site $i$, divided by what that probability would be if the two were independent. A value of one means no correlation at that separation, below one means avoidance, above one means the opposite.

Three of them are worth having, and they say different things.

Opposite spins is the one a repulsion acts on. On a Hubbard model the repulsion is paid only when an up and a down electron are on the same site, so the on-site value of this function is the quantity the energy is a sum over.

Same spins is the control. Two electrons of one spin can never be on one site whatever the repulsion is, so its on-site value is exactly zero — and it is checked to be zero at every repulsion tested, to twelve decimal places. A calculation that produced anything else there would be wrong in a way no comparison with another calculation would reveal.

Charge–charge is the sum, and is what a density–density measurement sees.

All three are computed by summing over the configurations of the exact ground state with the weights the state gives them, which is one loop and no subtraction.

One piece of bookkeeping in that loop is worth naming, because getting it wrong produces a plausible answer. A pair distribution counts pairs of different electrons, so the on-site entries have to use $n(n-1)$ rather than $n^2$ — otherwise a site holding one up electron reports a same-spin pair with itself, the control comes out at two rather than zero, and the exclusion principle appears to have been violated by the arithmetic. The control exists to catch exactly that, and it caught it.

What it says #

At no repulsion it is exactly one at every separation. That is the tripwire and it is a strong one: an exact ground state of non-interacting electrons is a single determinant, so the pair distribution has to be flat, and any error in the bookkeeping shows up as a deviation from one rather than as a plausible-looking number.

The hole at contact deepens monotonically. At a repulsion of sixteen the chance of finding two opposite spins on one site is four per cent of what independence would give. It does not reach zero and it is not expected to: an electron that never shared a site with another would not be able to move at all, so the residual double occupancy is the price the system pays for a kinetic energy, and it is the same trade the double occupancy curve prices as an energy.

And what leaves the site turns up next door. The value at separation one rises from 1.0000 to 1.5947 over the same range. The electrons are not pushed apart; they are moved along by one, and the alternation that produces — enhanced at odd separations and depleted at even ones — is the antiferromagnetic arrangement the spin correlations measure separately, seen in the charge rather than in the spin.

The two systems the energy cannot tell apart #

The finding about references is that a correlation energy is a report on a choice as much as on a system. This is the sharper version: two systems that produce the same correlation energy have their electrons in different places.

Take a plain ring at a repulsion of 4. Then take the same ring with its sites pulled apart in energy — every other site raised, every other lowered — and search for the repulsion at which its correlation energy per site is the same. It is 9.55.

Both give −0.084863 per site, to a part in a million. Their on-site pair distributions are 0.2873 and 0.1183 — a factor of 2.4, which is not a subtlety.

The second system’s electrons avoid each other much more thoroughly than the first’s, and its correlation energy says they do not. That is not a defect of either calculation; it is the difference between a number and a function. The energy is one integral of the distribution against one weight, and many distributions integrate to the same thing.

What the energy was actually measuring #

Putting the two findings together gives a description of the correlation energy that is worth stating plainly.

It is not a measure of correlation. It is the energetic consequence of the correlation, measured against a reference that has to be chosen, on a system whose repulsion sets the weight.

And the two systems above show why that matters. The modulated ring has a larger repulsion and less double occupancy, and the two effects cancel in the energy: a smaller number of expensive pairs costs the same as a larger number of cheap ones. Nothing about the energy announces which of the two situations it is in.

This is the same failure met before in a different place. A weight that depends on how it is weighed is about a decomposition; the reference decides the correlation is about a subtraction; and this is about a projection — a function reduced to a scalar by integrating it against something.

What a practitioner should take from it #

A correlation energy is comparable between systems only when the reference and the interaction are the same. Two molecules described at the same level with the same basis are fair game; two methods, two references or two different couplings are not.

The distribution is more expensive and says more. It needs the exact state rather than an energy, which is why nobody computes it for a large molecule — but where it can be computed it settles questions the energy leaves open, and it does so without any convention in it at all.

And the on-site value is a measurement in its own right. Double occupancy is accessible experimentally in cold-atom realisations of exactly this model, so the one point of the function that carries the whole energy is also the one point that can be checked.

There is a fourth thing, which is about how results of this kind should be reported rather than about what they mean. A correlation energy quoted without its reference is not a number anybody else can use; a double occupancy quoted without anything is. The rule generalises: prefer the quantity that has no convention in it, even when it is harder to compute, because it is the one that survives being read by somebody who made different choices.

A third kind of quantity #

It is worth placing the pair distribution against two kinds of quantity worth keeping apart.

Some numbers are conventions. A partial charge, an oxidation state, an ionic weight, a correlation energy: each is the output of a rule somebody chose, and two people who chose differently get different answers about the same molecule. A third kind of correlation is a decomposition of exactly that sort.

Some numbers are measurements. A spectrum, a bond length, a magnetic moment: what the object does, whatever anybody thinks.

And some are computed properties of an exact state that nobody has to choose anything to define. A density is one; a pair distribution is another. They are not measurements — no instrument returns them directly — and they are not conventions either, and that middle category is where the most useful theoretical quantities sit.

The whole of this essay is the observation that the usual account of correlation sits in the first category and does not have to.

What is quoted, and what is computed #

Nothing is quoted. There is no molecule and no measurement in this essay. The ring, its filling, the repulsions and the modulation are the model’s parameters; every ground state, every weight, every pair distribution and the matched repulsion are computed.

The two systems in the matching are compared through a quantity — the correlation energy — that does involve a mean field, because that is the quantity being criticised. The distributions it is criticised with involve none.

One of these two quantities is an observable and the other is not #

The case for looking at the pair distribution has been made on grounds of convenience — no reference, no subtraction, no dependence on somebody’s choice of determinant. There is a stronger version of it, and it is worth stating because it settles the matter rather than arguing it.

A correlation energy is not an observable. Its first term is: the exact ground-state energy of a system is an eigenvalue of its Hamiltonian and a spectroscopic measurement can reach it. Its second term is not. The Hartree–Fock energy is the output of a procedure applied to a model space — it depends on the basis, on whether spin symmetry was imposed, on whether the procedure found the global minimum — and no experiment measures it, because there is no operator whose expectation value it is. A difference between an observable and a non-observable is a non-observable.

That is a stronger complaint than the factor of 259. The factor of 259 says two conventions disagree; this says there is no measurement that could adjudicate between them, because neither is a prediction about anything.

A pair distribution is an observable. It is the expectation value of an operator — the density of pairs at a stated separation — evaluated on the exact state, and expectation values of operators on exact states are exactly what measurements return. Nothing in it is a procedure.

And it is measurable in practice as well as in principle, which is not always true of quantities in that class. The static structure factor of a system’s electrons is the Fourier transform of its density–density correlation, and the structure factor is what an inelastic X-ray scattering experiment returns once its energy loss spectrum is integrated. Compton scattering reaches a related object in momentum space. These are difficult measurements and they are made, on solids and on liquids, and what they return is the correlation between the positions of two electrons — the thing this essay has computed and the thing a correlation energy is a proxy for.

So the choice between the two quantities is not a matter of which is more convenient to compute. One of them is a property of nature and the other is a property of a calculation, and the collection’s standing habit — prefer the number that could be wrong against a measurement — picks the first without needing an argument.

None of which retires the correlation energy. It is the quantity every method is built to recover, it is what a benchmark table reports, and its usefulness does not depend on being observable — a number can organise a subject without being measurable, and this one has organised quantum chemistry for eighty years. What it means is that when the two disagree, or when a correlation energy behaves strangely, the pair distribution is the place to look for what is actually happening, and not the other way round.

The two systems above that agree about the energy to a part in a million and disagree about the electrons by more than half are the case in miniature: an unmeasurable quantity matched exactly, and a measurable one not matched at all.

What this cannot say #

A lattice model has no distance in it. “Separation one” is a bond, not an ångström, so the hole here has no width — which is exactly what a real Coulomb hole has and is the most conspicuous absence in the model. The hole that is not repulsion is where this collection separates the exchange hole from the correlation one, and both are spatial objects there.

One filling and small rings. Six sites at half filling for the survey and four for the matching, which is what an exact diagonalisation reaches. A different filling changes the alternation, and a large system would change how far the hole’s effects reach. The smallest many-electron calculation is where this limit was set and why.

The matching uses a broken-symmetry reference. Which reference is used changes the correlation energy — that is the whole point about references — so the matched repulsion of 9.55 would be a different number with a different reference. The disagreement between the two distributions would not, since neither of them knows a reference exists.

And a pair distribution is not an observable either, quite. It is an expectation value of a well-defined operator, which is much better than a partial charge, and measuring it needs a two-particle probe.

Nor does it settle what a mean field is for. A mean field is cheap and an exact state is not, so a quantity that needs the exact state cannot replace the correlation energy in any calculation large enough to matter. What it can do is calibrate: on systems where both are available, it says what the energy was and was not reporting — which is what a mean field cannot get out of the way is about from the other side.

What the distribution requires #

Two electrons of one spin are never on one site, at every repulsion, to twelve decimal places. That is the exclusion principle and it is the control: it must hold whatever the repulsion does.

With no repulsion the opposite spins are independent at every separation, to nine decimal places — the tripwire, because a flat distribution is what a single determinant gives and anything else would be an error in the bookkeeping.

They avoid each other more at every larger repulsion, at every step of the scan.

At a large repulsion they are rarely on one site at all, below 0.35 of the uncorrelated value.

And the pair missing from contact turns up further out, at every repulsion — checked separately, because a distribution that had simply lost weight would satisfy the previous three.

Two systems matched to the same correlation energy per site agree to a part in a million, which is the check on the search.

And they still disagree about where the electrons are, by more than 0.05 at contact. Measured: 0.1690, which is more than half of the first one’s value.

Still open: the width of a Coulomb hole #

The obvious open question is the width the model does not have. A real Coulomb hole is a region of space with a size, and its size is what distinguishes the short-range correlation that transfers between molecules from the near-degenerate kind that does not. A lattice hole is one bond wide by construction, so the distinction drawn energetically — two kinds of correlation, and only one is small — cannot be drawn spatially here. Doing it would need a continuum model, and the smallest honest one is two electrons in a box.

The nearer question is about the alternation. The distribution is enhanced at odd separations and depleted at even ones, which is an antiferromagnetic arrangement in the charge, and its amplitude at separation one saturates by a repulsion of eight while the on-site hole is still deepening at sixteen. Two features of one function moving at different rates is a decomposition the energy cannot see, and asking which of the two the correlation energy is mostly made of — by integrating the distribution against the repulsion piece by piece — would say what a correlation energy is a measurement of, in the only terms that do not need a reference.