The worst case measured the reference

A composite method makes a cheap calculation expensive-looking by borrowing. It runs a mean field on the system it wants and adds a correlation correction computed somewhere else, where an exact answer could be afforded. The transferability square has asked, essay after essay, whether anything the mean field produces for nothing warns when the borrowed correction will be wrong. Its systems are rings of four at half filling with a repulsion and a staggered site energy; the correction has always been borrowed from one of them, at a repulsion of eight and no site energy.

The essay before this one found that the conventional mean-field start misses the lowest solution on a band of systems whose site energy sits one or two below the repulsion, and that the square’s largest numbers were those misses rather than failures of transfer. It closed by naming the question its own result made sharp. The reference sat well away from the band, where the two starts agree. A reference chosen on the band would carry a correction fitted to a different solution — and whether the square’s scores depend on that choice as strongly as they depended on the start is one run of the same calculation away.

That run is made here, on the same eighty-one systems, from eight references.

Eight places to borrow from #

On this grid — repulsions from one to twenty, site energies from zero to ten, nine of each — the conventional start lands above the lowest mean field at four systems: a repulsion of three with a site energy of two, four with three, six with four, and eight with six. The miss at six and four is 1.124, the largest; the others are 0.15, 0.14 and 0.50. Those four are the references on the band. Three more sit beside it, at the band’s repulsions of three, four and six with no site energy, where the two starts agree. The eighth is the published reference.

At every reference, every other system’s error is its mean-field energy plus the reference’s correction less its exact energy, and every cheap diagnostic is its mean-field quantity’s distance from the reference’s. Nothing else changes, and the published reference on the lowest field reproduces the published scores: a worst polarisation ratio of 14.7 and a largest error of 0.351.

The reference moves the worst case more than the start did #

The start was the headline of the essay before: fixing it moved the polarisation’s worst ratio on the grid from 48.0 to 14.7. Moving the reference, with the start fixed, moves it from 9.73 to 213, a factor of twenty-two against the start’s three and a quarter. The published reference sits near the bottom of that range, and the references off the band spread as widely as those on it: four with no site energy gives 28.4, six with no site energy 213.

The band references are not special on the lowest field. They give 19.7, 28.6, 124 and 9.73 — the best score of all eight belongs to a band reference — and the three beside them give 15.0, 28.4 and 213. What decides a reference’s score is not whether it sits on the band.

On the conventional start the band references are special, in the wrong direction, which comes later. Off the band the two starts carry the same correction, to the iteration’s tolerance, and differ only in the systems they are applied to.

The median does not care #

The median over comparable pairs — the score the essay before argued for beside the worst case — barely moves. The polarisation’s stays between 1.21 and 1.55 at all eight references; double occupancy’s between 1.21 and 1.61; their product’s between 1.12 and 1.55; the mean-field energy’s between 1.20 and 1.92. The highest occupied level’s sits between 2.26 and 3.20 at every reference, above all the others everywhere, as it sat when the medians were first taken.

So the typical comparison was robust to more than one bad system. It is robust to where the correction came from, which is a larger change: every error on the grid moves when the reference moves.

What a worst ratio divides by #

The worst ratio is the largest, over pairs of systems with nearly the same diagnostic value, of one error’s magnitude divided by the other’s. Its numerator is bounded: no error on the lowest-field grid exceeds 0.4 at any reference. Its denominator is not.

The forty scores fall on a fan below a line of 0.4 over the denominator, and the largest of them all divide by an error of a thousandth or less. At seven of the eight references, the polarisation’s worst pair has one member whose error is under 0.025, and at six with no site energy it is 0.0009. The one reference whose smallest error anywhere on the grid is as large as 0.012 — the band reference at eight and six — has the best worst ratio of all, 9.73. A worst ratio does measure a diagnostic’s failure to sort the errors, and it also measures how close the grid comes to an error of zero, which is a property of the grid and the reference rather than of the diagnostic.

Where the error passes through zero #

An error near zero is a system whose correlation energy happens to match the reference’s, and that can be read off the grid.

With either reference the errors take both signs, so somewhere they pass through zero. The light, positive errors form a column at and just beyond the reference’s own repulsion, and the smallest errors on the grid are the reference’s neighbours: from the published reference, eight with a site energy of a half at 0.00047 and eight with one at 0.0019; from six with no site energy, six with a half at 0.00027 and six with four at 0.00088. A reference with no site energy transfers almost perfectly to its own repulsion at a small site energy, because the correlation barely changes, and the system that receives the best transfer becomes the denominator of the worst ratio.

So a worst ratio over a grid that contains the reference’s neighbours is guaranteed a small denominator, and how small depends on how finely the grid samples near the reference. A composite method is usually judged by how well it transfers to systems like the reference; the worst-case score penalises it for transferring well.

A reference fitted to the wrong solution passes #

On the conventional start, the four band references carry a correction fitted to a mean field that is not the method’s own answer. The correction differs from the lowest field’s by exactly the miss at the reference — 1.124 at six and four.

Carried from six and four on the conventional start, the correction of −1.497 puts every system’s error between −1.50 and −0.60. The polarisation’s worst ratio falls to 2.05, under the census threshold of three, and the composite is wrong by more than half a unit everywhere. Carried from eight and six on the conventional start, the score is 1.95. A ratio is blind to a common offset: errors that are all large and alike have ratios near one, and a diagnostic scored by ratios passes on a square where the transfer has failed at every system. Two wrong numbers made a right difference only when both were wrong the same way; here the one wrong number is added to every system, and the score reads the uniformity of the failure as success.

The largest error is the correction #

On the lowest field the largest error on the grid equals the carried correction at every reference, to two thousandths. It sits at a system with almost no correlation energy of its own — for the published reference a repulsion of one and a site energy of ten — which receives the whole of the borrowed correction and needs none of it. So the square’s largest error is not a failure the diagnostics were asked to warn about; it is the size of the reference’s correlation energy, and it moves from 0.244 to 0.392 as the reference moves. On the conventional start the references off the band sit above the line, at the start’s own miss at six and four, which is the largest error the essay before took apart.

The eight references in one table #

What stands #

The verdicts stand and the numbers do not. At every reference on the lowest field, every cheap candidate’s worst ratio exceeds three, so the grid fails all of them wherever the correction is fitted; and at every reference the medians say the typical comparison is good for all but the highest occupied level. Those are the two statements the square has made, and neither depends on the reference. Every specific worst ratio quoted in the essays of this argument — 14.7, 48, 45.7, 2.61 — is a number about one reference, and the factor between two of them is as much a property of where the grid’s errors pass through zero as of the diagnostics.

The census’s rule — a candidate fails where its worst ratio exceeds three — is the one most exposed. It can be failed by transferring well to a neighbour and passed by transferring badly to everything.

What the earlier worst cases were reading #

Read with the denominator in view, several of the square’s earlier findings look different without becoming wrong.

The first cross found two systems at nearly the same polarisation difference whose errors differed by a factor of forty-six, and that pair became the polarisation’s one failure in the census of where the candidates fail. One of its two members was a missed mean field; the other had a small error. A worst ratio always has that shape — one member large and one small — and the essay before removed the large member’s inflation. The small member was never examined, and on this evidence it is the more movable of the two: a different reference would have put a different system near zero.

The grid’s finding that the interior was worse than the arms is, in part, the same thing. The arms are the systems at the reference’s repulsion and at no site energy, and they include the reference’s own neighbours, whose errors are the smallest on the grid. Comparisons confined to the arms pair small errors with small errors; comparisons into the interior pair them with large ones. Some of the interior’s excess is a statement about where the reference sits.

The warning that started the argument is untouched, because it was scored by rank agreement rather than by a worst ratio. Its claim — that the transfer fails where the broken-symmetry solution collapses — is about the size of the error, and the size of the error is the one quantity here that a reference cannot hide: the largest error on the grid is the carried correction at every reference, and it is largest at the systems with least correlation of their own.

None of this rescues a cheap diagnostic. Every one of them fails the grid at every reference. What changes is the reading of a single worst number: it is a property of a candidate, a grid and a reference together, and the last of those was never varied until now.

How it was computed #

The systems are rings of four sites at half filling with unit hopping, a Hubbard repulsion U and a staggered site energy ±ε/2, on the grid of nine repulsions by nine site energies the square’s grid essay used. The exact ground state is the full diagonalisation of the thirty-six-state problem; the unrestricted mean field is computed twice, from the conventional start — densities twenty per cent off uniform, alternating in spin — and as the lowest over forty-two starts. The composite energy at a system is its mean-field energy plus the reference’s exact-minus-mean-field correction; its error is that less the exact energy. Each diagnostic is the magnitude of a mean-field quantity’s difference from the reference’s. Comparable pairs are those whose diagnostic values agree within fifteen per cent, both above a hundredth of the largest; the worst ratio is the largest error ratio over them, and the median their median. The reference itself is excluded from its own grid, since its error is zero.

The checks, run wherever these figures are drawn: the published reference on the lowest field reproduces the published worst ratio and largest error to a per cent; off the band both starts carry the same correction; on the band they differ by exactly the miss; the reference moves the worst ratio by more than twice the start’s factor while the median stays between 1.15 and 1.6; every grid’s errors take both signs, every reference’s worst pair but one divides by an error under 0.025, and the exception has the largest smallest error and the best score; the largest error equals the carried correction on the lowest field; and the conventional start’s band reference at six and four passes the census with every error beyond a half. The refusal is the reference, which must be absent from its own grid.

Where the ring stops #

Four sites. Every number is a ring of four, where the exact answer is a small matrix. The structural points — that a ratio’s denominator can approach zero wherever errors change sign, and that a common offset hides in a ratio — are properties of the statistics, not the ring; the sizes are the ring’s.

Eight references, one grid. The references are the band’s four and their neighbours, not a survey of every system, and a finer grid near any reference would supply smaller denominators still. That is the point, and it means no worst ratio quoted here is a limit.

A composite method with one correction. Real composite schemes correct with several pieces fitted at several levels, and their errors need not change sign over a family the way these do.

Who found what #

Composite thermochemistry — mean-field or low-level energies corrected by differences computed at higher levels on smaller or simpler systems — is a long-standing practice, and the instability of unrestricted mean fields to their starting guess is long known. That ratios are blind to a common offset and explode near zero is elementary statistics. What is computed here is the square’s scores at eight references under both starts, the near-zero denominators behind its worst cases, and the passing score a reference on the wrong solution produces.

A score should not reward the reference #

The habit worth keeping is to ask of any score what it would do to a method that is perfect near the reference and bad everywhere else, and to one that is uniformly bad. A worst ratio fails the first and passes the second, which is backwards for a warning about transfer. A score that compares errors should be built so a uniform offset counts against the method rather than vanishing from it — an absolute error scale, or a comparison against the reference’s own correction — and so that a near-perfect transfer cannot become the denominator of a failure.

Still open: an absolute score, and two references #

The obvious open question is the score itself. The census threshold of three is a ratio, and both of the failures here are failures of ratios. Scoring each candidate by how well it predicts the error’s size in absolute terms — a rank correlation with the error’s magnitude is one such score already computed, and it ran from 0.47 to 0.94 for the polarisation across these references — would remove the near-zero denominators and the offset at once, and the question is whether the square’s verdicts survive a score that cannot be gamed either way.

The nearer question is a composite with two references. If the correction carried depends on the reference by as much as the largest error on the grid, a correction interpolated between two references — one with weak correlation and one with strong — is the natural next method, and the square’s systems are cheap enough to ask whether it shrinks the largest error from the size of one correction to the size of their difference.