Koopmans' theorem is exact for nothing

What a photoelectron spectrum measures opens the subject by insisting on the distinction its title makes: the bands of a photoelectron spectrum are ionisation energies, which are differences between the energies of two states of a molecule, and they are read off as orbital energies, which are eigenvalues of a one-electron operator belonging to one state.

The bridge between them is Koopmans’ theorem, and the usual account of why it works is a story about two errors cancelling. This essay computes both errors.

The three quantities #

For a system with $N$ electrons, three numbers are in play.

The exact ionisation energy is $E(N-1) - E(N)$, from two exact calculations.

The self-consistent answer is the same difference with both energies from a mean field — solve for the molecule, solve again for the ion, subtract. This is what is called a ΔSCF calculation, and it is a real calculation done twice.

The theorem’s answer is $-\varepsilon_{\text{HOMO}}$, the negative of the highest occupied orbital energy of the neutral molecule, and it costs nothing extra at all.

Koopmans’ theorem is the statement that the third equals the second when the ion’s orbitals are frozen at the molecule’s, and in the mean field used here that is exact rather than approximate: removing an electron from orbital $m$ changes the energy by exactly $-\varepsilon_m$, because the interaction term is bilinear in the up and down densities and only one of them changed. So the difference between the second and third numbers is entirely the ion’s relaxation, and the difference between the first and second is entirely correlation.

A matrix that depends on its own answer #

Most model energies come from diagonalising a matrix that was written down before the calculation started — a Hückel adjacency matrix, a ligand field, a force constant matrix. This one is different in the way that matters: the matrix depends on its own answer. Hückel theory and what it gets right is the extreme of the other kind — a matrix of ones and zeroes, written down before anything is computed. Each electron moves in a field made by the average density of the others, that density comes out of the eigenvectors, and the two have to be made to agree by iteration.

It is not the Gaussian-basis self-consistent field of production quantum chemistry, and the difference is why it can be trusted here. There are no two-electron integrals over basis functions to get wrong: the only interaction is a number attached to a site, so the Fock matrix is the hopping matrix with a diagonal added. What makes it worth doing at all is that the exact answer for the same model is available, so every quantity the mean field produces sits beside the thing it is approximating rather than beside another approximation.

The iteration needs one piece of care and it is worth recording. Linear mixing with a single fixed weight converges for a weak field and oscillates forever between two densities for a strong one; rather than pay for a small weight in every case, the mixing is halved and the whole iteration restarted until it settles, and which weight was needed is reported — because a case needing a very small one is a case sitting near a point where two solutions exchange stability, which is the subject of a later section.

What the three do #

At zero repulsion all three agree exactly, and they have to: with no interaction the mean field is exact and there is nothing to relax.

Relaxation is positive at every repulsion, meaning the theorem overestimates: $+0.010$ at $U = t$, $+0.337$ at $U = 4t$, $+1.764$ at $U = 8t$.

That sign is not an observation, it is a theorem, and checking it as one is the test that the calculation is consistent. The ion’s own field gives it a lower energy than the molecule’s field does, because the mean field is variational and doing the calculation twice can only improve the second answer. A scheme whose Fock operator was not the derivative of its own energy would break that, and a scheme like that produces a negative relaxation — which is how such an inconsistency is found.

Correlation is negative at every repulsion: $-0.057$, $-1.049$, $-4.055$ at the same three points. The molecule has more electrons than the ion and therefore more correlation energy, so the mean field is further above the truth for the molecule than for the ion, and the difference goes the other way.

And the residue grows. $-0.047$, $-0.712$, $-2.291$. The cancellation is real — the two errors are of opposite sign and the total is smaller than either — and it is not a cancellation anyone can rely on: at $U = 8t$ the two errors are $1.76$ and $4.06$ and what survives is $2.29$, which is more than half of the larger one.

Why the theorem is quoted anyway #

None of the above says Koopmans’ theorem is useless, and the practice it supports is defensible for a reason the model can also show.

It costs nothing. A ΔSCF calculation for every band of a spectrum is one calculation per band; the theorem gives the whole spectrum from one calculation of the neutral molecule.

It gets the order right far more often than it gets the values right. The relaxation of a tightly bound orbital and of a loosely bound one differ, but not usually enough to swap them, so the pattern of a spectrum survives. That is the property fewer bands than electrons actually uses when it counts bands and compares with a symmetry prediction: the count and the ordering, not the numbers.

And the cancellation is genuinely helpful in the range molecules are in. At $U \approx t$ — a reasonable ratio for a valence orbital — the residue is one part in a hundred of the ionisation energy, and that is a better error than either term alone would give.

What the model adds is the boundary. The cancellation is a numerical accident of a range, not a structural fact, and it fails first where the correlation is largest: a molecule with a small gap, a transition metal, an ion — which is where the insulator band theory cannot see finds a one-electron picture failing for the same underlying reason. Which is exactly where photoelectron spectra are hardest to assign, and where the assignments have historically been argued about.

The other thing a mean field does #

There is a second finding here and it is not about ionisation at all.

Every self-consistent field has to be told what symmetry to look for. The usual instruction — the two spins share their spatial orbitals — is a constraint rather than a property of the answer. Release it, and above a critical repulsion the same iteration on the same system finds a different solution: up electrons concentrated on alternate sites, down electrons on the others.

It has a lower energy. For the four-site chain, the polarised solution appears between $U = 1.5t$ and $U = 2t$ and is lower from there on. At $U = 16t$ the symmetric mean field gives $+11.53$ and the polarised one $-0.374$, against an exact $-0.583$ — so the released calculation is better by more than an order of magnitude in its error.

And it describes a molecule that is not there. The exact ground state of this system is a spin singlet with no spin density anywhere; the broken solution has spin density alternating from site to site. The energy it buys is real and the picture it buys is false, which is the precise sense in which a variational improvement can be a description getting worse.

That is the same shape as where molecular orbital theory dissociates’s problem seen from the other side. There a single determinant was wrong at dissociation and the repair was to add configurations; here the repair is to let the single determinant break a symmetry, which fixes the energy and not the state.

What this means for reading a spectrum #

Three practical statements follow, and they are the reason this belongs in the field of things taught confidently and wrongly.

A band is not an orbital energy. It is an ionisation energy, and the two differ by relaxation, which is a property of the ion and not of any orbital of the molecule.

An orbital energy is not observable at all — a point orbitals are not where the electron is makes about the orbital and this makes about its energy. It is an eigenvalue of an operator that depends on which mean field was chosen — and the previous section shows that even that is not unique, since a released field gives different orbitals with different energies for the same molecule.

Assignments that depend on small differences are the ones to doubt. Water’s lone pairs are not a pair turns on two bands being at different energies, and that conclusion is safe because the difference is large. A conclusion that turns on two bands being a few tenths of an electronvolt apart is a conclusion about relaxation and correlation differences, which is not what it is usually presented as.

What a self-consistent field is doing, in one paragraph #

The method deserves a plain statement, because everything above depends on what it is.

An electron in a many-electron system feels the other electrons. A mean field replaces that by having each electron feel the average of the others: the repulsion term becomes a number attached to each site, computed from the density, and the resulting one-electron problem is a matrix diagonalisation like any other. The catch is that the density comes out of the eigenvectors, so the matrix cannot be written down until the answer is known.

The resolution is to guess, solve, rebuild the matrix from the answer, and repeat until the density stops moving. That is the whole of it, and it is why the method is called self-consistent: what is achieved at the end is not correctness but agreement between the field and the density that produced it.

In practice the first guess is usually the density with the repulsion switched off, and at strong repulsion a plain repeat can swing between two densities without ever settling. Mixing part of the previous density into each new one damps the swing, and the converged density keeps no memory of how much was mixed — only of which self-consistent solution the iteration fell into, which is where the non-uniqueness below comes from.

Two things follow that the essay has already used. The scheme is variational — its energy is the expectation of the true Hamiltonian in a single determinant, so it cannot fall below the exact ground state, which is what makes the relaxation term positive. And it is not unique, because the iteration can converge to more than one self-consistent density, which is what the symmetry-breaking section is about.

What a mean field can never contain is two electrons dodging each other: each moves in an average, so their positions are independent given the average, and the energy that genuine avoidance saves is missing by construction. That missing energy is what “correlation” names, and its size is the gap between the two lower curves in this essay’s first figure.

Two errors with known signs give a bracket #

The cancellation is measured here and found to be partial, and there is a weaker statement available that survives everywhere the cancellation does not — because the signs of the two corrections are fixed even where their sizes are not.

Relaxation always lowers the ion’s energy. The remaining electrons rearrange into the hole, and rearranging can only help, because the unrelaxed arrangement is one of the options available to the relaxed calculation. So neglecting relaxation makes the ion look too high in energy and the ionisation energy too large.

Correlation is always larger in the neutral. The neutral molecule has one more electron pair to correlate, so its correlation energy is more negative than the ion’s, and neglecting correlation everywhere makes the neutral look too high — which makes the ionisation energy too small.

Fixing the two signs gives a bracket rather than a number. The true ionisation energy lies between the value with relaxation alone and the value with correlation alone, and the Koopmans value sits somewhere between them because it has neither.

That is worth having because it is unconditional. Whatever the repulsion, whatever the molecule, whichever correction dominates, the two bound the answer from opposite sides — so a calculation returning a value outside the bracket has an error in it rather than an approximation.

It also says which way each failure runs. Where relaxation dominates — a compact orbital, a core level, a metal centre — Koopmans overestimates. Where correlation dominates — a diffuse orbital, a weakly bound electron — it underestimates. Knowing which of the two a case belongs to gives the direction of the error before its size is known, which is more than the cancellation argument offers.

Where the model stops #

On-site repulsion only, one orbital per site. The usual caution for this model applies: no long-range repulsion, no exchange between different orbitals, and no geometry.

The site energy is a constant. A constant added to every site shifts every energy by one electron’s worth and cancels from every difference reported here; it is carried so that the numbers read as ionisation energies rather than as the negative ones a model with no nucleus produces.

Four sites. The exact solver is deliberately limited — the same six-site ceiling what couples two spins works under — and the trends here are established over one system size. Whether the residue grows or shrinks with system size is a real question this model cannot answer.

No orbital relaxation of the kind a real ion has. A real molecule’s ion contracts its remaining orbitals inwards towards the exposed nuclear charge, which is a change of shape in a basis. Here the orbitals can only redistribute among sites, so the relaxation computed is the smaller part of the real one — and the direction of the conclusion is therefore conservative.

One consequence for practice: the theorem’s error is not the same for every band of a spectrum, which is why it distorts a spectrum rather than merely shifting it. A tightly bound orbital leaves an ion whose remaining electrons relax more, because more of them are close to the hole; a loosely bound one leaves an ion that barely notices. So the relaxation is larger for the deeper bands, and the whole spectrum is stretched.

That is measurable in this model by removing an electron from the lower level rather than the higher one, and the direction comes out as expected. It is also why the theorem is at its most trustworthy where it is most used — on the outermost band, where the relaxation is smallest and the correlation difference is smallest too.

What computing both errors adds #

The simpler arguments establish that a photoelectron band is an ionisation energy rather than an orbital energy, that water’s two lone pairs are at different energies and are therefore not equivalent, and that methane has fewer bands than electrons for reasons of symmetry.

This one computes the size of the gap between the two quantities. The theorem’s error is the sum of a relaxation that always makes it too large and a correlation difference that always makes it too small; the two are of comparable size and cancel partially; and the residue at four times the hopping is already a fifth of the answer. Along the way comes a self-consistent field checked against an exact answer at every step, and the finding that a self-consistent field is not one thing, because releasing a symmetry constraint gives a better energy and a worse picture.

The open questions are the intensities of a photoelectron spectrum, which no argument here has touched, and to what a spectrum looks like when the ion is not in the state the theorem assumes it is.