What is taught wrongly

A reference at the worst miss made the worst miss worse

A composite that borrows the correlation energy of the nearest reference system missed worst on a stretch of the transferability square where the correlation energy changes fast, and the hope was that a handful of references placed there would bring the worst miss down to the median's. The stretch is the line where the site energy equals the repulsion: all twenty-one misses above 0.08 lie near it. But no rule that can be run without knowing every answer gets the worst lower than 0.060, a rule that does know every answer needs 480 of the 1,008 systems to reach the median, and its very first placement raises the worst miss from 0.166 to 0.200.

Worth reading first: Two readings separate the square only where they do not fold · A constant ties the polarisation.

A composite method runs a mean field on the system it wants and adds a correlation correction computed somewhere else. On the transferability square — a ring of four at half filling, swept in its repulsion UU and a staggered site energy ε\varepsilon — the plainest such method is a lookup: compute two cheap readings of the mean field, the double occupancy and the highest occupied level, find the reference system whose readings are nearest, and borrow its correlation energy. Trained on the coarse square of eighty-one systems and asked about the other 1,008 of a grid refined twice, that lookup misses by a median of 0.013 in units of the hopping and at worst by 0.166.

The worst miss was not where the readings fail. The two readings separate the square — no two distant systems share them — and the worst miss was on a system inside the coarse square’s own range, where the correlation energy simply changes faster than eighty-one references can follow. That suggested the obvious repair: put a few more references there, rather than spreading them uniformly, and see whether the worst miss comes down to the median’s scale. It is the first time in this long argument about diagnostics a mean field gives for nothing that the answer might be a placement rather than a better diagnostic.

It comes down. It does not come down to the median, and the reason is not where the references go. A nearest-reference lookup has a worst case that more references can make worse, and its own best placement shows that most clearly.

The steep side is the line where the site energy meets the repulsion

The composite misses worst along the line where the site energy equals the repulsion. Each of the 1008 held-out systems of the square, placed by its repulsion and site energy and coloured by how far the nearest of the 81 coarse references misses its correlation energy: under 0.03, 0.03 to 0.08, over 0.08. The dashed line is ε = U. 32 of the 50 worst misses lie within a quarter of the repulsion of it, a band holding 120 of the 1008 systems. The worst, 0.166, is at U = 8.85, ε = 10.
Fig. 1 Every held-out system of the square at its repulsion and site energy, coloured by how far the nearest coarse reference misses its correlation energy, with the fifty worst ringed and the line ε = U dashed.

The misses are not scattered. Of the 1,008 held-out systems, 184 are missed by more than 0.03 and 21 by more than 0.08, and every one of those twenty-one lies within a quarter of the repulsion of the line ε=U\varepsilon = U. Of the fifty worst, thirty-two do, in a band that holds 120 of the 1,008 systems, an eighth of the square. Eight of the other eighteen are in one column at U=2.45U = 2.45 and small site energy, each missed by about 0.05 because the nearest reference is at U=2U = 2; there the highest occupied level passes through its maximum, so one reading stops moving while the correlation energy keeps changing by a tenth of the hopping per coarse step.

The line has a physical meaning. On a ring with a staggered site energy, a large repulsion makes a half-filled band an insulator by keeping one electron on every site, and a large site energy makes it an insulator a different way, by putting both electrons of each pair on the low sites. Where ε≈U\varepsilon \approx U the two costs balance, moving an electron from a high site to a low one costs nothing on balance, and the ground state is a mixture of both arrangements that changes character quickly as either parameter moves. The mean field describes each side tolerably and the crossover badly, so the correlation energy it misses is steepest exactly there. The coarse grid samples that line at only seven of its eighty-one points.

So the steep stretch is not a mystery, and a placement rule built on physics alone is available: put the new references nearest the line, without computing anything.

Six ways to choose where the next reference goes

Each rule adds kk systems from the 1,008 to the eighty-one coarse references, and the lookup is scored on whatever is left. The rules differ in what they are allowed to know.

At random, the blind baseline, as the median of nine seeded draws. On the 17×17 grid, the uniform refinement: 208 systems, a single budget. Far in the readings: each new reference is the system furthest, in the two scaled readings, from every reference so far, which fills the space the lookup searches and needs no correlation energy at all. Along ε = U: the systems nearest the crossover line, in order of ∣ε−U∣/U|\varepsilon - U|/U, which needs nothing but the two parameters. Where references disagree: the system whose three nearest references disagree most in correlation energy, which needs only the references’ own values and is the rule an active-learning scheme would use. At the worst miss: the system the lookup currently misses worst, which needs every system’s correlation energy and is therefore not a rule anyone could run. It is the bound on what any placement could achieve with this lookup.

Only a rule that knows every answer brings the worst miss down steadily, and even it stays above the median at 208. The composite's worst miss on the held-out systems against the number of references added to the coarse 81, for five placement rules, with the 17×17 grid's 208 added systems as a single point (0.130). at random: 4 0.166, 8 0.166, 16 0.166, 32 0.166, 64 0.166, 128 0.166, 208 0.164; far in the readings: 4 0.166, 8 0.166, 16 0.166, 32 0.130, 64 0.130, 128 0.130, 208 0.084; along ε = U: 4 0.166, 8 0.166, 16 0.173, 32 0.129, 64 0.089, 128 0.060, 208 0.109; where references disagree: 4 0.164, 8 0.164, 16 0.254, 32 0.254, 64 0.254, 128 0.129, 208 0.129; at the worst miss: 4 0.129, 8 0.125, 16 0.085, 32 0.054, 64 0.045, 128 0.031, 208 0.022. With none added, 0.1664.
Fig. 2 The worst held-out miss against the number of references added to the coarse eighty-one, for five rules, with the 17×17 grid’s 208 added systems as a single point and the coarse median dashed.

Random placement does nothing to the worst miss: 0.166 with none, 0.166 with 128 and 0.164 with 208. The uniform grid brings it to 0.130, the same figure the far-in-the-readings rule reaches with 32 and keeps until 128. The crossover rule is the best of the rules that can actually be run, as the map promised: 0.129 at 32, 0.089 at 64 and 0.060 at 128, less than half what the uniform grid achieves with 208. Then, past the 120 systems in its band, it starts adding systems far from the line and the worst miss rises again, to 0.109 at 208, at U=20U = 20 and ε=9\varepsilon = 9.

The turn is worth reading closely, because it is the first sign of what the rest of this essay is about. The 208th reference along the line sits at ∣ε−U∣=0.39 U|\varepsilon - U| = 0.39\,U, well off the crossover, at U=13.86U = 13.86 and ε=8.5\varepsilon = 8.5, with a correlation energy of −0.334. It is now the nearest reference in the readings to the system at U=20U = 20 and ε=9\varepsilon = 9, whose correlation energy is −0.226, and that system’s miss becomes the worst on the square. At 128 the worst had been a crest system at U=7.44U = 7.44 and ε=10\varepsilon = 10, borrowing from a reference in the band; at 208 it is a system the band never threatened, damaged by a reference that was added to help.

The bound is lower at every budget: 0.085 at 16, 0.054 at 32, 0.031 at 128 and 0.022 at 208. That is the answer to the question as asked. A handful of references does not bring the worst miss to the median’s scale under any rule, and the gap between the best runnable rule and the bound, at 128, is a factor of two.

The typical system was never the problem

Every rule moves the median miss by about the same amount; the rules differ at the worst. The composite's median miss on the held-out systems against the number of references added to the coarse 81, for five placement rules, with the 17×17 grid's 208 added systems as a single point (0.010). at random: 4 0.0131, 8 0.0128, 16 0.0126, 32 0.0120, 64 0.0117, 128 0.0102, 208 0.0098; far in the readings: 4 0.0133, 8 0.0131, 16 0.0131, 32 0.0124, 64 0.0120, 128 0.0122, 208 0.0103; along ε = U: 4 0.0131, 8 0.0133, 16 0.0133, 32 0.0126, 64 0.0122, 128 0.0109, 208 0.0103; where references disagree: 4 0.0128, 8 0.0126, 16 0.0126, 32 0.0127, 64 0.0120, 128 0.0106, 208 0.0088; at the worst miss: 4 0.0130, 8 0.0128, 16 0.0126, 32 0.0121, 64 0.0116, 128 0.0093, 208 0.0073. With none added, 0.0131.
Fig. 3 The median held-out miss against the number of references added, for the same five rules, with the 17×17 grid ringed.

The median tells the other half. Every rule moves it from 0.013 to between 0.010 and 0.012 at 128 references, and the bound reaches 0.0093 at 128 and 0.0073 at 208. Random placement does about as well for the median as any rule that can be run, and the uniform grid’s 0.0096 sits among them. The rules that concentrate references — along the crossover, far in the readings — are slightly worse for the median than random at most budgets, because the references they place are not where most systems are.

So the placement problem has two parts with opposite answers. The typical system is served by spreading references evenly; the worst system is served by concentrating them where the correlation energy is steep. A uniform grid does the first and not the second, and a rule built for the second gives up a little of the first. That trade is visible only because the median and the worst have been read separately, as the reference-free score learned to do when a single scalar was hiding a fold.

Even the bound needs half the square

Placing each reference at the worst miss, the worst reaches the median's scale only after 480 of the 1,008. The worst held-out miss as references are added one at a time at whichever system is currently missed worst, from none to 600: 0 0.166, 64 0.045, 128 0.031, 192 0.023, 256 0.027, 320 0.018, 384 0.015, 448 0.014, 512 0.011, 576 0.012. It first falls to the coarse composite's median miss, 0.0131, at 480 references.
Fig. 4 The worst held-out miss as references are added one at a time at whichever system is currently missed worst, from none to six hundred, with the coarse median dashed.

Carried further, the bound falls fast and then slowly: 0.045 at 64, 0.031 at 128, 0.023 at 192, and then a flat tail that crosses the coarse median, 0.0131, only at 480 references. By then the lookup holds 561 of the 1,089 systems in its table, and what is being measured is no longer a composite with a few well-placed references but a table that contains half the answers. Even the tail is not monotone: at 240 references the worst miss jumps from 0.021 to 0.045 before coming back down.

The lead’s hope was that the worst miss was a sampling problem — a stretch the coarse grid stepped over — and that a few references aimed there would close it. The steepness is real, but it is not localised enough for a few references to cover. The correlation energy changes quickly along the whole crossover line, from U=2U = 2 to U=10U = 10, and across it in a band whose width shrinks as the repulsion grows. A lookup needs a reference close to every system on that band, in the readings, before the worst miss can match the typical one.

Where the bound puts its references

Half of the first 64 references an oracle would add lie near ε = U, where an eighth of the systems are. The 81 coarse references (open circles) and the first 64 systems added by placing each new reference at the worst current miss (filled). 32 of the 64 lie within a quarter of the repulsion of the line ε = U, which holds 120 of the 1008 held-out systems.
Fig. 5 The eighty-one coarse references and the first sixty-four systems added by placing each reference at the worst current miss, with those near the line ε = U in the darker colour.

The first sixty-four placements of the bound confirm the map. Thirty-two of them lie within a quarter of the repulsion of the crossover line, four times the line’s share of the square, Most of the other thirty-two are at repulsions above ten, where the coarse grid has three columns, at 12, 16 and 20, to span a factor of two, and four are at small repulsion beside the column where the highest occupied level turns over. The bound and the physical rule agree about where references are needed. They disagree about how many and in what order, and the order matters more than it should.

A reference can make the worst miss worse

Adding a reference can make the worst miss worse, and the rule that knows most does it most often. The worst held-out miss after each of the first 128 placements, for three rules, drawn as steps. along ε = U: rises 3 times, the largest at placement 37; where references disagree: rises 2 times, the largest at placement 12; at the worst miss: rises 22 times, the largest at placement 28.
Fig. 6 The worst held-out miss after each of the first 128 placements for three rules, drawn as steps, with a dot at every placement after which the worst miss rose.

The bound’s very first placement is the published worst system, at U=8.85U = 8.85 and ε=10\varepsilon = 10, and adding it as a reference makes the worst miss larger: 0.200, on its neighbour at U=9.80U = 9.80 and ε=10\varepsilon = 10. That neighbour sits on the crest of the correlation energy, past anything the coarse square sampled, and the new reference is now the nearest one to it in the readings, with a correlation energy 0.20 away. Before the addition its nearest reference was further off in the readings and, by accident, closer in the answer. In the first 128 placements the bound’s worst miss rises twenty-two times.

The rule that uses the references’ disagreement does worse. At its twelfth placement it adds U=8.85U = 8.85 and ε=9.5\varepsilon = 9.5, and the worst miss goes to 0.254 on the system next to it on the crest; it stays at 0.254 for sixty-nine placements, until the eighty-first adds that system itself. The disagreement rule finds the right region, since three nearest references that disagree are a sign of a steep stretch. What it cannot know is that the system it picks will become the nearest reference for a system whose answer it does not resemble. The crossover rule rises three times, and the far-in-the-readings rule, which never places a reference close to an existing one, never rises at all.

This is a property of the lookup, not of the placement. A nearest-reference rule is piecewise constant: each reference owns a cell in the space of readings and lends its answer to everything in it. Adding a reference redraws the cells, and a system that was in a cell whose answer happened to suit it can find itself in a new cell whose answer does not. Nothing in a nearest-neighbour rule promises that more data lowers its worst error; it promises only that the error goes to zero when every system is its own reference. An interpolating composite — a weighted average of several neighbours, or a fit to the local slope — would not have this property in the same form, and would be the next thing to try.

How the placements were scored

Every rule at every budget: the worst miss, with the median beneath it. The worst and median held-out miss of the nearest-reference composite after adding 4, 8, 16, 32, 64, 128, 208 references by each rule; with none added, 0.166 and 0.0131. The 17×17 grid adds 208 and gives 0.130 and 0.0096.
Fig. 7 The worst and median held-out miss of the lookup after each rule adds 4 to 208 references, with the 17×17 grid’s single budget.

Every system’s exact ground-state energy comes from diagonalising the ring of four with two electrons of each spin, and its lowest unrestricted mean field from the search over starts that fixed the solver’s worst system; the correlation energy is the difference. The readings are the mean field’s double occupancy and the energy of its highest occupied level, each scaled by its range on the coarse square, and the distance between two systems is Euclidean in those two scaled numbers. The pool is the 1,008 systems of the 33×33 grid that the 9×9 coarse square does not contain, and each rule’s score is the lookup’s error on the pool less the systems it has added. The random rule’s figures are the median over nine seeded draws.

The checks run where the figures are drawn. With nothing added every rule reproduces the published lookup, worst 0.166 and median 0.013. The uniform refinement is scored on the 800 pool systems it leaves. The bound’s first pick is the published worst system. At every budget from sixteen to sixty-four no runnable rule beats the bound on the worst miss. And the refusal tests what makes the bound a bound: a rule that is just as greedy and knows just as much, but adds the pool system nearest in the readings to the current worst miss rather than the worst miss itself, must do worse than the bound at thirty-two references. It does, 0.130 against 0.054. If it had not, the bound’s advantage would have been its greed rather than its knowledge of the answers.

What a lookup on a ring of four cannot settle

The composite is the crudest one. A nearest-reference lookup is the fair test of what references can do with no assumptions about the shape of the correlation energy, and it is also the one composite whose worst case can grow with more data. A composite that interpolated between neighbours would not jump when a cell is redrawn, and its placement problem might have a different answer. The finding that a handful of references cannot close the worst miss belongs to the lookup.

The square has two parameters. On a real benchmark the systems vary in many more ways than a repulsion and a site energy, and a crossover line becomes a crossover surface of dimension one less than the space, which a fixed budget samples more thinly still. That makes the result here, if anything, the easy case.

The bound is a bound for this rule only. Placing at the worst miss is greedy, and a placement chosen jointly could do better at the same budget. It is the best ordering of single additions, not the best set.

The repair that moves the problem

There is a pattern in the long sequence of these diagnostics. Each candidate failed somewhere different, and each repair moved the failure rather than removing it: the arms of the square were kind to one reading and the interior was not; a worst case measured the reference rather than the diagnostic. Placement does the same thing in a new way. A reference at the worst miss repairs that system and hands the worst miss to its neighbour, and the neighbour is on the crest, beyond the range anything had sampled, where the lookup has no good answer to lend.

That is also the encouraging reading. The misses that remain after a good placement are on the crest of the correlation energy along the crossover line, a region that can be named in advance from the two parameters alone. A composite that knew where its references were thin — that reported, along with its answer, the distance to the nearest reference and the spread of the answers nearby — would flag those systems even when it could not fix them, and that is a different and more modest goal than bringing the worst miss to the median’s scale.

Still open: an interpolating lookup, and a warning instead of a fix

The obvious open question is the composite itself. Replacing the nearest reference with a weighted average of the three nearest, or a local linear fit in the two readings, removes the cell structure that makes a new reference able to raise the worst miss. Whether the crossover rule then becomes monotone, and whether the bound’s 480 falls to something a benchmark could afford, would say how much of the difficulty was the lookup’s and how much the correlation energy’s.

The nearer question is a warning. The spread of correlation energy among a system’s three nearest references was a poor placement rule, but it may be a good diagnostic: on the coarse square, before any placement, it can be computed for every held-out system from the references alone. If its largest values pick out the systems the lookup misses worst — the twenty-one above 0.08 along the crossover line — then a composite could say which of its answers not to trust, which is what this whole sequence of essays has been asking a cheap calculation to do.

What links here

Computed from the collection rather than written here: the essays that point at this one.

Reads more easily once this is understood

Essays that name this one as worth reading first.

Shares its objects with

Essays naming at least two of the same things, that neither author linked.

Named objects

A dashed tag is an object no other essay names yet.

Composite methodCorrelation energyHubbard modelMean-field approximationTransferability