Leverage catches the spikes and none of the rises
Worth reading first: The lookup was the problem, not the references · A reference at the worst miss made the worst miss worse.
A composite method borrows correlation energy from reference systems that a calculation can afford, and gives it to systems that it cannot — a correction computed somewhere else, carried to where it is needed. On the Hubbard transferability square — a ring of four sites at half filling, its repulsion U and staggered site energy ε each varied, every system’s exact correlation energy known — the plainest version gives each held-out system the correlation energy of whichever of the 81 coarse references is nearest to it in two cheap readings, its double occupancy and its highest occupied level, the pair that separates the square where it does not fold. The lookup was the problem, not the references found that a least-squares plane through the six nearest references does much better with the same 81: a worst miss over the 1,008 held-out systems of 0.088 and a median of 0.0050, against the nearest reference’s 0.166 and 0.013.
It also found the plane’s one failure. Placing references one at a time wherever the plane currently misses worst — the oracle placement, which knows every answer and so bounds what any rule could achieve — the worst miss at the thirty-ninth placement was 3.99 and at the forty-third 2.05, in units where every correlation energy on the square lies between −0.6 and zero. At those steps some system’s six nearest references lay almost on a line in the two readings, and a plane through points on a line is free to tilt about it. The essay ended by asking for a plane that sees that from its neighbours alone and declines to answer — and whether, with the spikes gone, adding a reference would stop making the worst miss worse.
The first half has a clean answer. The second half does not, and the reason it does not is the more useful finding.
Where the plane was guessing
The thirty-ninth placement’s spike is worth looking at directly, because its geometry says what any detector has to detect.
The system sits at U = 14.9, ε = 8.5, and its six nearest references are all at ε = 8, at repulsions from 10.8 to 20. In the two readings those six lie along a line: their highest-occupied-level coordinates agree to two parts in a thousand while their double occupancies spread across a tenth of the scale. Along that line their correlation energies change steadily, from −0.217 to −0.392, and the plane fits that change well. Across the line it has almost nothing to go on — six points whose variance across the line is a hundred-thousandth of their variance along it — and the system is not on the line. It lies 0.027 above it, which is as far away as its nearest reference.
The plane’s slope across the line is whatever the tiny scatter of the six about their line dictates, and evaluated 0.027 out it returns −4.30 for a system whose correlation energy is −0.310. The forty-third placement’s spike is the same picture one row down: six references at ε = 6, a system at ε = 6.5, a variance across the line seven hundred-thousandths of the variance along it, and an answer of −2.39 for −0.339.
That geometry does not need the answer to be detected. It needs only the six neighbours’ readings and the system’s own, which are what the plane is built from anyway.
The number the plane already computes
The question as it was posed suggested the most direct detector: the smaller eigenvalue of the neighbours’ spread in the two readings, relative to the larger. It is near zero exactly when they lie on a line. At the two spikes it is 1.5 × 10⁻⁵ and 7.4 × 10⁻⁵, and over every held-out system on the coarse references it never falls below 0.023.
There is a better number, and the plane computes it anyway. A least-squares plane centred on the system returns its intercept, and the intercept’s sensitivity to the data is a diagonal element of the inverse normal matrix, , for the design whose rows are — the neighbour’s offset from the system in the two readings. That element is the factor by which the value’s variance exceeds the variance of a single reference, and it is called the leverage of the evaluation point. For six neighbours spread symmetrically round the system it is one sixth, the variance factor of a plain average; for six neighbours on a line with the system beside them it is enormous. It also grows when the neighbours are well spread but the system lies off to one side of all of them — the other way a plane extrapolates — which the eigenvalue ratio cannot see.
On the coarse references the leverage of the six-neighbour plane at the 1,008 held-out systems has a median of 0.25 and a largest value of 1.88. At the two spikes it is 2,172 and 6,706. Three decades separate the spikes from every ordinary evaluation, with nothing in between.
The obvious alternative, and the one most often reached for, is distance: a lookup is surely guessing when the system is far from its references. It is not what went wrong here. Both spike systems are 0.028 from their nearest reference in the scaled readings, where the median held-out system is 0.023 away and the furthest 0.125. They sit in the crowd. The refusal makes that a test rather than a picture: a guard that falls back whenever the nearest reference is further than any coarse system’s nearest reference — the tightest distance threshold that leaves the coarse plane untouched — lets both spikes through, and its own placement reaches 3.99 just as the unguarded plane’s does. Distance measures how far the system is from its references; leverage measures how far it is from where its references can say anything.
Two detectors, one gap
The eigenvalue ratio and the leverage both put the spikes three decades beyond the coarse systems, and either would remove them. They part company on the systems in between, and the difference is the reason to prefer the second.
A neighbour set can lie nearly on a line with the system on that line. Then the plane is being asked to interpolate along the one direction it knows, its value is well determined, and its leverage is modest; the ratio of the two eigenvalues is just as small as at a spike. On the 17×17 grid used as references, the eigenvalue guard fires on ten held-out systems and leverage on none, and the ten it fires on were being answered well — falling back to the nearest reference there moves the median miss from 0.0015 to 0.0016. Under its own placement the eigenvalue guard fires on up to thirty-five held-out systems at once, the leverage guard on at most fifteen. The ratio measures the neighbours; the leverage measures the neighbours as seen from the system, which is the only thing the answer depends on.
The same distinction separates a warning that reads the cheap calculation from one that reads only where the references sit. A warning is useful in proportion to how few good answers it condemns, and on that count the leverage guard condemns none here.
A guard that changes only the spikes
A guarded plane computes the six-neighbour plane and its leverage, and if the leverage exceeds a threshold it does something else: either returns the nearest reference’s correlation energy, or widens the neighbourhood to nine and then twelve references and fits again. The eigenvalue guard is run the same way, with a threshold of 10⁻³, for comparison.
On the coarse references none of the guards fires on any held-out system, so all three return the unguarded plane’s 0.088 and 0.0050 exactly. The interesting question is the oracle placement.
All three remove both spikes. The guarded planes’ worst miss over 128 placements never exceeds 0.117, which is the plane’s first placement — the crest system at U = 8.85, ε = 10, whose reference tilts the plane for a neighbour — and the largest single rise falls from 3.97 to between 0.03 and 0.05. After 128 placements the plane with a wider fallback ends at 0.0101, the unguarded plane’s own value; the plane that falls back to the nearest reference ends at 0.0275, because by then five held-out systems trip the guard and the nearest reference is a worse answer for them than a wider plane.
The threshold barely matters. From 2 to 100 every number in the figure is the same, because there is nothing between 1.88 and 2,172 for a threshold to separate. Below 2 the guard begins to fire on coarse systems — six of them at 1.2, three at 1.5 — and since those include systems the plane was answering well, the coarse worst miss rises to 0.164. At 10⁴ the guard misses the larger spike and 3.99 returns. A guard on a fitted lookup is usually presented as a trade between accuracy and safety with a threshold to tune. Here, on this square, it is neither: the plane’s own leverage puts the failures three decades away from the successes.
That is a statement about this square and these 81 references, and it is a measured one rather than a general one. A set of references laid out on a coarser or a more irregular grid would put more systems at intermediate leverage, and the gap could close.
The rises were never the spikes
The second half of the question was whether, with the spikes gone, the plane’s own placement becomes monotone. A reference at the worst miss made the worst miss worse found the nearest-reference lookup’s worst miss rising twenty-two times in 128 placements, even along the line where the steep side of the square concentrates every large miss, and traced it to cells: a piecewise-constant lookup reassigns a whole region when a reference arrives. The plane was expected to be smoother. It was not — its worst miss rose thirty times — and the spikes made it easy to suppose that the plane’s rises were its extrapolations.
They were not. Two of the unguarded plane’s thirty rises are the spikes. The other twenty-eight have a median of 0.005, and the guarded plane has almost exactly the same rises — thirty-two of them, median 0.004 — at almost exactly the same placements. In every one of the sixty-two rises across the two, the reference just placed is one of the new worst system’s six nearest references. The rise happens where the neighbourhood changes, not somewhere else on the square.
The mechanism is the one the nearest reference had, one level up. A plane through the six nearest neighbours is a different plane for every set of six. As references are added, a held-out system’s six change, and when the new reference enters them the plane through them changes too — sometimes by tilting toward an answer that is right at the new reference and wrong a short distance away. The greedy placement puts its reference at the worst current miss, which is where the correlation energy is changing fastest, so the new reference is precisely the kind that tilts a neighbour’s plane. A lookup built from nearest neighbours of any kind is piecewise in the neighbour set, and cells of some kind are its price.
The leverage cannot see this. A neighbour set that has just changed is usually a perfectly good one, with modest leverage: the plane through it is well determined and simply different from the plane through the previous six. Leverage reports how uncertain the plane is about its own value, and these planes are certain and wrong in a new way.
How the guards were built and checked
The scaled readings, the references and the held-out pool are those of the essays on the square: the 81 coarse systems, the 1,008 systems of the finest grid that the coarse square does not contain, and the two readings each divided by its range over the coarse references. The plane is a least-squares fit of correlation energy to an intercept and two slopes, centred on the system, solved by elimination with partial pivoting; its value is the intercept, and its leverage the first diagonal element of the inverse of the 3 × 3 normal matrix, found by solving with the unit vector on the right.
The unguarded plane reproduces the published one to 10⁻⁹ at every system checked, and its own placement reaches 3.99 at the thirty-ninth step. The leverage of six points evenly spaced on a circle round the system was checked to be one sixth to 10⁻¹². Each guard leaves the coarse plane untouched, each guarded placement stays below the nearest reference’s coarse worst miss of 0.166, and the distance guard of the refusal — a threshold equal to the largest nearest-reference distance among coarse held-out systems, 0.125 in the scaled readings — fails to stop the spike. The oracle placements are greedy and deterministic, and the comparison between forms is between their own placements, each chasing its own worst miss.
What a guard on a ring of four cannot say
Every number here belongs to one model, a ring of four sites — the one system on the square with no threshold of its own to find — and to one reference layout, a nine-by-nine grid in U and ε. The gap of three decades between the spikes and the ordinary systems is a property of that layout: the spikes arise only when the greedy placement has put six references in one row of the grid near a system off the row, and a layout without rows — references scattered at random, or placed by a rule that avoids alignment — would put fewer systems at the extremes and more in between. Whether leverage would still separate failure from success on such a layout is not tested.
The readings are also exact here. A real composite reads its cheap quantities from an approximate calculation, with errors of its own, and leverage multiplies those errors as surely as it multiplies the scatter of the references. A system whose readings are uncertain sits at a range of leverages rather than at one.
Nor is the transferred quantity the only thing that can go wrong. Every score on the square carries one correlation correction fitted at a single reference system, and carried from elsewhere it changes the worst case by an order of magnitude; a guard on the lookup says nothing about whether the references themselves were the right ones to trust.
And the oracle placement is not a placement anyone can run. It knows every system’s correlation energy and spends that knowledge on choosing the next reference. What it shows is the worst a lookup’s own structure can do; a placement by physics or by a warning has different rises, and possibly fewer.
A lookup has two ways to be wrong
The guard separates them. One is the plane being asked a question its neighbours cannot answer, and that is announced in advance by a number the plane already computes: the guard costs nothing, needs no tuning on this square, and removes the only catastrophic misses the oracle ever found. The other is the neighbourhood itself changing under a system, and that is not announced at all, because a fresh neighbourhood looks exactly like a good one. The warning a cheap calculation gives and the spread among nearest references both read the answers of the neighbours, and neither was built to see that the neighbours have just changed.
The second kind is the smaller of the two by two orders of magnitude, and on this square no single rise of it exceeds about 0.05. It is also the only kind left.
Still open: a lookup that does not jump, and a guard that reads the readings’ errors
The obvious open question is whether any lookup built from references can be monotone under its own placement. A lookup whose answer changes continuously as a reference is added — weights that fall smoothly to zero at a radius, so that a new reference enters gradually rather than at once — would have no cells, and the question is whether its oracle then rises at all. If it does not, monotonicity is a matter of smoothness in the weights, and the plane’s rises are the price of its hard cutoff at six. If it still rises, the rises belong to greedy placement itself, which puts every reference where the surface is steepest, and no form of lookup removes them.
The nearer question is the readings’ own uncertainty. On this square the readings are exact. A composite that reads them from a mean-field calculation carries their error into the plane’s slopes, and leverage multiplies it. Adding the readings’ error to the guard — falling back when leverage times the readings’ variance exceeds the scatter of correlation energy among the neighbours — would turn the threshold from a number chosen above a gap into a number derived from the calculation, and say whether the gap survives once the readings are noisy.
Shares its objects with
Essays naming at least two of the same things, that neither author linked.
- The second number is the error, rearranged — both name composite method, correlation energy, hubbard model, mean-field approximation, transferability
- A blank is not a pass — both name composite method, correlation energy, mean-field approximation, transferability
- A constant ties the polarisation — both name composite method, correlation energy, mean-field approximation, transferability
- Five failures in five different places — both name composite method, correlation energy, mean-field approximation, transferability
- The arms were the kindest part of the square — both name composite method, correlation energy, mean-field approximation, transferability
- The worst system in the square was the solver's — both name composite method, correlation energy, mean-field approximation, transferability
Named objects
A dashed tag is an object no other essay names yet.
Composite methodCorrelation energyHubbard modelMean-field approximationTransferability