The correction that was computed somewhere else

Two wrong numbers and a right difference measured error cancellation between two calculations by the same method: two wrong numbers whose difference is right, and the conditions under which the wrongness is the same wrongness. It ended by naming the other half of the practice.

The other half is cancellation between two methods, and it is what every composite scheme in computational chemistry is built on: compute a small correction at a high level and a large one at a low level, and assume the errors of the two are separable.

That assumption has a shape:

$$E_{\text{composite}}(\text{target}) = E_{\text{low}}(\text{target}) + \big[\,E_{\text{high}}(\text{reference}) - E_{\text{low}}(\text{reference})\,\big]$$

and its whole content is that the bracket belongs to the method pair rather than to the system it was computed on. If it does, the recipe delivers high-level accuracy at low-level cost, which is why every practical calculation of a thermochemical quantity is built this way. A better energy is not a better answer is the other side of the same coin: what a calculation is being asked for decides which of its errors matter. If it does not, the recipe delivers something with no error bound at all.

The smallest system that can carry one #

The test needs the exact answer for both the reference and the target, which is why it is done on four sites.

A four-site Hubbard ring at half filling has a four-hundred-dimensional configuration space, and diagonalising it exactly is a few seconds. The low level is the unrestricted mean field — one determinant, self-consistent, allowed to break spin symmetry — and the high level is that diagonalisation. The difference between them is the correlation energy, exactly and by definition.

The reference is the symmetric ring: four identical sites. The target is the same ring with its two sublattices pulled apart in energy by ε, which is what an electronegativity difference is in this model and is the parameter a charge transfer is built on. So the correction is computed on the homonuclear molecule and used on the heteronuclear one, which is exactly the shape of the real practice — a correction from a small symmetric case applied to a larger unsymmetrical one.

The recipe is exact where it was built #

At ε = 0 the reference and the target are the same system, so the composite reproduces the exact energy to the last bit. That is the tripwire: it says the recipe has been assembled the right way round, and a version with a sign error or a mismatched reference would fail here rather than somewhere subtle.

A little way from the reference it does what it is famous for. At ε = 0.5 the mean field is wrong by 0.338566 hartree — a large error, a sixth of the total — and the composite is wrong by 0.000884. It has removed 99.74 per cent of it.

That is a real result and it is why the method is used. A correction worth a sixth of the total energy, computed once on a small system and reused, is a very large amount of accuracy for a very small amount of work.

And then it stops #

The rest of the table is the assumption failing.

The exact correlation energy of the target does not stay put. It falls from −0.339451 at ε = 0 to −0.323274 at ε = 2, to −0.209045 at ε = 4, to −0.020578 at ε = 8, to −0.000472 at ε = 16. The transferred number is −0.339451 throughout, because it was computed once.

The reason is not subtle and is worth stating in words. Correlation energy is what a single determinant cannot describe, and what it cannot describe is the choice between two configurations of comparable weight. In the symmetric ring the two sublattices are equivalent, so the configurations with the electrons on one sublattice and on the other are degenerate, and no single determinant can hold both. Pull the sublattices apart and one configuration becomes much lower than the other; the ground state becomes nearly a single configuration; and a single determinant describes it almost perfectly.

So the correlation energy is a measure of how nearly degenerate the reference is, and asymmetry destroys degeneracy. A correction computed on a symmetric system is a correction for a problem the asymmetric system does not have.

At ε = 8 the mean field is wrong by 0.020578 and the composite is wrong by 0.318873. The recipe has made the answer worse by a factor of 15.5, and it did so while every individual calculation in it was correct.

Why nothing warns #

The failure has no symptom inside the calculation, and that is the part worth carrying away.

The mean field converges. The exact diagonalisation converges. The bracket is a well-defined number. The composite energy is a smooth function of ε with no discontinuity, no oscillation and no obviously wrong magnitude. A practitioner with only the composite number in hand — which is the ordinary situation, since the exact answer is what the recipe exists to avoid computing — sees nothing at all.

The one internal signal available is the quality of the reference determinant, and it is available. The mean field on the symmetric ring has to break spin symmetry to find its minimum, and a broken-symmetry solution is the standard sign that a single determinant is struggling. On the asymmetric ring it does not. So the diagnostic exists — is the low-level description of the reference qualitatively the same as its description of the target? — and it is a comparison the recipe does not require anybody to make.

The same failure, read as a statement about references #

There is a way of seeing this that connects it to three other arguments, and it is the useful generalisation.

A composite scheme is a subtraction with two references: the high-level and low-level answers on the reference system. A resonance energy is a subtraction with one reference, and how much a resonance energy depends on which reference is chosen has already been measured — benzene comes out at 2.0000β or 1.0121β depending. The reference decides the correlation makes the same point about a correlation energy directly: the quantity is defined as what the reference misses, so it is a property of the pair rather than of the molecule.

Put those together and this result is not a surprise. It is the same statement one step further out: if a correlation energy is a property of a reference, then transferring one between systems is transferring a property of one system to another, and there is no reason it should survive the journey. What is new here is the size — a factor of fifteen the wrong way — and the fact that it can be measured rather than argued about.

What would make it work #

The failure is diagnostic rather than fatal, and it says what a well-chosen reference has to satisfy.

The bracket has to be a property both systems share, so the reference has to resemble the target in the respect the correction is about — not in size, not in chemical formula, but in how nearly degenerate its low-lying configurations are. Two systems with the same near-degeneracy structure will transfer a correction between them well however different they look; two with different near-degeneracy structures will not, however similar they look.

That is a testable criterion and it is not the criterion in use. In practice a reference is chosen for being small, and the assumption is that a correction from a small analogue transfers to a large molecule of the same kind. When the molecule has a near-degeneracy the analogue does not — a stretched bond, a transition metal, a diradical — the transfer fails in exactly the way measured here, and the literature’s name for those cases is that they are multireference. The smallest many-electron calculation is where this collection first met the distinction, on two sites rather than four.

How much charge a site-energy difference actually moves against how much repulsion opposes it is the same axis used here, and the reason the correlation energy dies along it is visible there: the electrons stop meeting, so there is less left to correlate.

The two ways of being wrong, side by side #

It is worth putting this result next to error cancellation within one method, because the two failures are not the same failure and are easily confused.

Cancellation within a method works when the same approximation is made twice and the two errors are similar. That is a statement about one method applied to two systems, and it works better the more alike the systems are — which is why an isodesmic reaction, where the same bonds appear on both sides, is the workhorse of practical thermochemistry.

Transfer between methods works when the difference between two approximations is similar for two systems. That is a statement about a difference of differences, and it is a much stronger requirement: it needs not only that the two systems be alike but that they be alike in the specific respect the two methods disagree about.

The table above is the second failing while the first would not. At ε = 8 the mean field’s own error is tiny — 0.020578, a fifth of a per cent of the total — so the low level is doing very well on the target. It is the reference that the low level does badly on, and the recipe is carrying the reference’s difficulty over to a system that does not have it.

Another quantity where a single-determinant answer and an exact one part company on the same ring: Koopmans’ theorem is exact for nothing, and how far off it is depends on the same variable. A correction transferred between systems inherits every one of those dependences.

How the correlation energy scales in the two regimes it has is the quantity being transferred: where the exponent changes, the character of the correlation changes with it, and a correction fitted on one side of that change is fitted to a different thing.

A composite scheme also assumes additivity — that the methods it stacks give the same answer for two separated systems as for the two separately — and that assumption is measurably false. Two independent failures, stacked.

What this cannot say #

Two kinds of correlation, and only one is here. Two kinds of correlation, and only one is small separated them; what is transferred here is entirely the near-degenerate kind.

Four sites is not a molecule. The Hubbard model has one orbital per site and one repulsion integral, and its correlation energy is entirely of the near-degeneracy kind. A real molecule’s correlation energy is mostly dynamic — the short-range keeping-apart of electrons — which is far more transferable than what is measured here, and is the reason composite methods work as well as they do.

One low level, one high level. The pair here is unrestricted mean field and exact. A real composite scheme stacks several pairs, and the errors of the stack are not the sum of the errors of the steps.

The asymmetry is the only thing varied. A real transfer changes basis set, molecule size and geometry at once, and separating those is what a proper study of this would do.

And nothing here is a criticism of any particular method. What is being tested is the assumption, in a system where it can be tested, and the finding is that the assumption is a property of the reference rather than a general fact.

Why the practice survives a result this bad #

A table showing a recipe fifteen times worse than the calculation it was improving invites an obvious objection, and the objection is a good one. Every serious thermochemical protocol in use is built on exactly this assumption, and the good ones reach a few kilojoules a mole on hundreds of molecules. Either the measurement above is unrepresentative, or a large and well-tested body of practice is wrong. It is the first, and saying precisely how it is unrepresentative is what turns the result into a rule rather than a scare.

Correlation energy is not one quantity. It splits into two with different physics, and the split decides everything about transferability.

Near-degeneracy correlation is a single determinant’s inability to hold two configurations of comparable weight. It is a global property of the state, it appears and disappears with a degeneracy, and it is what the table above measures. Pulling the sublattices apart removes the degeneracy, so it removes the quantity, which is why the transferred number describes a difficulty the target does not have.

Dynamic correlation is the short-range business of electrons keeping out of each other’s way — the Coulomb hole, the cusp, the part of the wavefunction that depends on the distance between two electrons. It is present in every system with more than one electron, it does not switch off, and it is nearly local: a pair of electrons in a given bonding environment contributes about the same amount to it wherever that environment occurs.

The model here has the first and none of the second, and it has none by construction rather than by accident. One orbital per site and a repulsion that acts only when two electrons are on the same site leaves electrons nowhere to avoid one another within a site, because a site has no interior. So the quantity being transferred in the table is one hundred per cent the part that does not transfer, and zero per cent of the part that does.

A closed-shell organic molecule near its equilibrium geometry is close to the opposite extreme. Its leading determinant carries most of the weight, its near-degeneracy correlation is small, and the bulk of its correlation energy is the short-range local kind — which is transferable for a reason that can be stated rather than hoped for. If each pair in a similar environment contributes a similar amount, the correction is roughly proportional to a count of such pairs, and a bracket formed on a small molecule carrying the same pairs carries the same count. That proportionality is the whole mechanism, and it is the same property as size consistency seen from the other side: an error that grows with the number of electrons in a regular way is an error that subtracts cleanly.

So the honest reading of the table is not that composite schemes fail. It is that they are protected by a quantity they never measure, and the protection is withdrawn in one specific circumstance: when the target has a near-degeneracy the reference does not, or the reference has one the target does not. That is the stretched bond, the diradical, the transition metal and the transition state — and those are exactly the cases the literature already lists as the ones composite methods are unreliable for. The value of four sites is that the failure can be watched happening against an exact answer instead of inferred from a pattern of disagreements.

Which makes the model’s poverty the point rather than a limitation. A test run on a system whose correlation was mostly the local kind would have removed ninety-nine per cent of the error at every asymmetry and demonstrated nothing at all. Choosing a model with no dynamic correlation whatever is the same choice as running the repulsion far past any real molecule’s: it is how a mechanism is made visible, and it is why the number that comes out is a diagnosis and not a measurement of any real method’s accuracy.

What was checked #

At the reference the composite is exact, to a nanohartree, which is the tripwire that the recipe is assembled correctly.

Near the reference it removes almost the whole of the low level’s error — above ninety per cent at ε = 0.5 — because a scheme that did not would not be worth examining.

Its error never shrinks as the asymmetry grows, at every step of the scan.

By the far end it removes less than half of the error, which is the statement that the transfer has failed rather than merely degraded.

And the refusal, which is the essay: the composite is fifteen times as wrong as the low level at ε = 8, both measured against the same exact answer — so a scheme that cannot be worse than its own starting point is refused by one table.

Still open: a diagnostic for transfer, and the other variable #

The obvious open question is the diagnostic. A broken-symmetry mean-field solution is a signal that a single determinant is struggling, and it is available before any high-level calculation is run. Whether the magnitude of the symmetry breaking predicts the transfer failure — whether a reference and a target with similar breaking transfer well and dissimilar ones do not — is a correlation that could be measured across a range of ε and U rather than assumed, and it would turn a piece of received practice into a rule with a number in it.

The nearer question is about additivity in the other variable. Everything here transfers a correction across a change of site energy at fixed repulsion. Transferring across a change of repulsion at fixed site energy is the other half of the same square, and whether the two failures compound or partly cancel is what a two-dimensional version of this table would say. A method that is not additive found that two corrections computed separately do not sum to the correction computed together; this essay finds that one correction does not move between systems. Whether those are two faces of one arithmetic is a question the same four sites could settle.