Ten directions no frequency can see

More coordinates than motions found that methane has ten internal coordinates and nine ways to vibrate, and that the excess is a real combination of coordinates describing no displacement of any atom. It then showed what that does to a force field, which is where the difficulty the sharing-out of a mode among its coordinates has always had turns into an arithmetic one: adding $\lambda\, r r^{\mathsf T}$ to the force constant matrix, for the redundancy $r$, changes every frequency by five parts in a hundred million.

It called that the flat direction and left the obvious next question, which is what to do about it. The literature’s answer is to project the force constant matrix onto the space orthogonal to the redundancy, which picks one point out of the flat direction and reports it.

Two things turn out to be wrong with that sentence, and the second is worse than the first.

The flat direction is a ten-dimensional space #

The original argument was $B^{\mathsf T} r = 0$, so $B^{\mathsf T}(\lambda r r^{\mathsf T})B = \lambda (B^{\mathsf T}r)(B^{\mathsf T}r)^{\mathsf T} = 0$. That argument proves far more than it was used for. The Hessian is unchanged by adding

$$r s^{\mathsf T} + s r^{\mathsf T}$$

for any vector $s$ whatever, because both terms carry a factor of $B^{\mathsf T} r$. Taking $s = r$ recovers the original direction; taking $s$ anywhere else gives a different flat direction, and there are as many of them as there are coordinates.

Counted rather than argued: the map from symmetric force constant matrices to Cartesian Hessians is linear, so build it in the basis of the $n_c(n_c+1)/2$ symmetric matrices and count the zero singular values.

Both agree with the closed form $d\,n_c - d(d-1)/2$ for $d$ redundancies, which is carried alongside precisely because it is easy to derive and easy to get wrong. Two computations of one integer, by a rank and by algebra.

Eighteen per cent of methane’s force field is unmeasurable, and it is unmeasurable not because the experiment is hard but because those directions produce no motion of any atom. A normal mode is not a bond stretch and a force constant, it turns out, is not always a measurement.

A check along one direction would have passed on a calculation that handled only the $r r^{\mathsf T}$ direction, because that is the only direction such a check tries. Drawing $s$ from a generator instead moves every entry of the force field by an amount of order one — methane’s by up to 1.26 mdyn/Å, boron trifluoride’s by 1.79 — and the frequencies still do not move.

What a projection picks, and by which metric #

A projection is a choice of which member of the flat space to report, and a choice needs a rule. Three rules are defensible and a fourth is the one everybody would reach for.

Projected, Euclidean. $F \mapsto PFP$ with $P = I - r r^{\mathsf T}$. This is also the minimum-norm member of the flat space, because the projection is orthogonal in the ordinary inner product on matrices — so the smallest force field consistent with the spectrum and the projection orthogonal to the redundancy are the same convention under two descriptions, which is worth knowing since they sound like different principles.

The redundant constant zeroed. Set $r^{\mathsf T} F r = 0$ and change nothing else, which is what a fit with a single linear constraint does.

As fitted. Whatever the least-squares search returned, with no constraint at all — which is what a fit with more constants than frequencies produces when nothing stops it. The fitted methane field is here, and it is well behaved for a reason given below that is not to its credit.

Projected, G-weighted. The metric a vibrational analysis is actually written in — the one that turns internal coordinates into mass-weighted ones and makes the eigenvalue problem the one that gives frequencies. It is the natural choice, and it does not exist.

$$r^{\mathsf T} G r = 1.8 \times 10^{-16} \quad \text{(methane)}, \qquad -4.7 \times 10^{-27} \quad \text{(boron trifluoride)}.$$

Those are zeros, and they are zero by construction: a redundancy is defined as a null vector of $G$, since $G = BB^{\mathsf T}$ and $B^{\mathsf T}r = 0$. So the G-weighted projector divides by zero at every molecule that has a redundancy at all. Forcing the division through and letting the floating point produce whatever it produces gives a matrix that moves the frequencies by 506 per cent.

This is not a numerical difficulty to be worked around with a pseudo-inverse. It is the statement that the metric which makes the vibrational problem what it is has no opinion about the direction in question, which is exactly why the direction is flat.

What the surviving conventions quote #

Three conventions can be formed. All three reproduce every frequency of methane to nine decimal places. The constants they report:

The stretches are untouched, because the redundancy has no bond stretching in it at all — it says the six angles at a tetrahedral centre are not independent, and it lives entirely among them.

Everything that touches an angle moves. The bend constant differs by 16.7 per cent between two conventions. The stretch–bend constant differs by exactly a factor of two. And the bend–bend interaction goes from exactly zero to −0.0824, which is a change from this molecule has no bend–bend coupling to it has one worth a sixth of the bend constant, on identical data.

The factor of two is not a coincidence and it is worth deriving, because it says how the disagreement scales. A C–H stretch appears in three of the six angles at the carbon. The Euclidean projection subtracts the redundancy’s share, and the redundancy is an equal $1/\sqrt{6}$ part of each angle, so the stretch–bend constant is multiplied by $1 - 3/6$. For boron trifluoride each stretch is in two of three angles and the factor is $1 - 2/3$, which is a third:

The disagreement is larger for the smaller molecule, because the redundancy is spread over fewer coordinates and takes a bigger share of each. That is the opposite of the usual expectation about small molecules, which is that they are the determined ones — and it is the same reversal a pair of band measurements shows, where the cleaner regime is the worse-conditioned one.

The finding nobody fitting a force field wants #

Methane’s fitted field agrees with the projected one about the bend–bend constant, which looks like good news for it. It is not. Methane’s fit has no bend–bend constant to disagree about — its parameterisation carries four constants, and the flat direction moves the bend and bend–bend constants together, so a fit that cannot represent a bend–bend interaction cannot run along the flat direction.

A parameterisation too poor to contain the freedom is not a parameterisation that has removed it. It is one that has hidden it, and the hiding stops the moment somebody adds the constant that a slightly better fit would want.

The boron trifluoride field does carry a bend–bend constant. So it should be sitting somewhere on the flat line, at a place decided by the optimiser rather than by the spectrum — and it is. Refitting from four starting points that differ only in where the two bend constants began:

The bend constant ranges over 0.88 mdyn/Å, through a negative value that no referee would pass, and the fit is equally good at every point. Their difference is fixed at 0.511 and moves by five parts in a thousand across all four.

The spectrum fixes the difference. The two numbers a fit prints are the difference plus wherever the optimiser stopped. That is a defect in any such fit, found by this arithmetic and not by any test of agreement — every such test checks that the computed frequencies match the observed ones, which they do, beautifully, at every point along a line the data does not constrain.

What should be reported instead #

The honest object is the combination the data fixes, not the constants a convention produces from it.

For boron trifluoride that is $f_\alpha - f_{\alpha\alpha} = 0.511$, one number where two were printed. For methane the corresponding statements are that $f_\alpha - f_{\alpha\alpha} = 0.4941$ under every convention, and that the stretch–bend constant is determined only up to the same freedom.

This is the same recommendation an underdetermined structure arrives at from a different direction, and the same one a susceptibility curve forces when it is asked how many parameters it can carry: report what is measured, and say which combination it is. A table of individually meaningless numbers whose differences are meaningful — like a spectrum that counts environments rather than atoms, read as though it counted atoms — is a table that invites exactly the comparison it cannot support — two papers using two conventions, their bend constants differing by seventeen per cent, and a reader concluding that the molecules differ.

What is quoted, and what is computed #

Two things are quoted, and both are measurements: the observed frequencies of methane and of its deuterated form, and those of boron trifluoride. Everything else is computed — the coordinates from the structure, the B matrix from the coordinates, the redundancy as the null space of $BB^{\mathsf T}$, the flat space as the null space of the map from force constants to Hessians, and every projection.

The nullity is computed as a rank and checked against a closed form derived separately. Neither is the other read back to itself: one counts small eigenvalues of a 55 × 55 Gram matrix, the other is arithmetic on two integers.

The refits are genuine refits, not perturbations of a stored answer. Each starts the same least-squares search from a different point and runs it to convergence, which is why the residuals are four slightly different numbers rather than one.

What this cannot say #

The harmonic approximation is everywhere in this, and a real force field is fitted to frequencies that are not harmonic. That makes the reported constants wrong in a way this essay says nothing about, and it does not touch the argument: an anharmonic fit has the same $B$ matrix, the same redundancy and the same flat space.

The conventions here are three of many. A paper may constrain a constant to a value transferred from another molecule, or fit a symmetry-adapted set, or drop a coordinate. Each is a different rule for choosing a point, and the spread between three is a lower bound on the spread between all of them.

The G-weighted projection’s 506 per cent is not a physical number. It is what floating point does when a projector is formed by dividing by $10^{-16}$, and it is reported only to show that the failure is loud rather than subtle once the division is allowed.

And a molecule with no redundancy has none of this. Water and ammonia have independent coordinates and determined force fields, which is why the difficulty looks like an exotic case until the count is done — methane and boron trifluoride are not exotic molecules.

Where the number ten comes from #

Ten invisible directions in a fifty-five-dimensional space is a measured result, and it has an exact derivation that needs no metric at all — which matters, because the metric is precisely what the essay finds cannot define the projection.

Write $u$ for the redundant combination of coordinates: the vector that describes no displacement of any atom. Every set of coordinates has $u$ in its null space, so a force-constant matrix $F$ and a matrix $F'$ produce identical frequencies whenever they differ by something that only ever acts on $u$.

The symmetric matrices with that property have a form:

$$F' - F = u\,a^{\mathsf T} + a\,u^{\mathsf T},$$

for any vector $a$ whatever. Each such difference contributes nothing to the energy of any real displacement, because every real displacement is orthogonal to $u$ in the sense that matters — it has no component along the direction $u$ describes.

The dimension of that family is the number of free choices in $a$, which is the number of coordinates. Methane has ten internal coordinates, so the family is ten-dimensional, and ten is exactly the number measured here.

The arithmetic therefore has a shape: one redundancy times ten coordinates gives ten invisible directions, out of a total of $10 \times 11 / 2 = 55$. Not a coincidence of methane, and immediately generalisable — a molecule with $r$ redundancies among $n$ coordinates has an invisible subspace of dimension $rn - r(r-1)/2$, which for one redundancy is simply $n$.

That derivation also says why no metric is needed to count the directions, only to project onto their complement. The count is a statement about the null space of the coordinate transformation, which is fixed by the geometry alone. Choosing which of the many matrices differing by such a term to report is a different operation, it requires deciding what smallest means, and that is where a metric has to be supplied and where the natural one fails.

The form of the difference also says which constants are exposed and which are safe, before any projection is attempted. Every member of the family has $u$ as one of its two factors, so a constant is touched only if it connects a coordinate to something $u$ has a component on. A constant between two coordinates that both lie outside $u$’s support is untouchable by any member of the family, and is therefore determined by the frequencies alone.

For methane $u$ lives entirely among the six angles, so the four stretching coordinates are outside it, and every constant involving only stretches is safe. That is a prediction a table of convention-dependent constants can be checked against, and it is available from the shape of the redundancy rather than from a fit.

What was checked #

The flat space has the dimension the redundancy count predicts — ten of fifty-five and seven of twenty-eight — with the rank and the closed form computed by different routes.

A member of it drawn at random moves no frequency, while moving the force field by more than a whole unit. Both halves are checked: without the second, a bug that ignored the added term would pass.

Every formable convention reproduces every frequency to better than a part in a million, which is what makes the disagreement between them a disagreement about conventions rather than about arithmetic.

The bend constant differs between conventions by more than a tenth of itself and the stretch–bend constant by more than four tenths — stated as inequalities, so that a change to the fitted field cannot quietly make the finding go away.

The G-weighted projector cannot be formed, and the convention is refused rather than reported. A routine that returned its matrix would be returning the ratio of two roundings.

And the fitted boron trifluoride bend constant moves by more than half a unit with its starting point, its difference from the bend–bend constant does not move, and every fit reaches the same residual. All four are checked together, because any three of them without the fourth would read as a statement about optimiser noise.

Still open: transferability, and the determined combinations #

The obvious open question is transferability, which is what force constants are for. A constant transferred from one molecule to another is transferred under whatever convention its source used, and the tables above say that transferring a bend constant between two conventions costs seventeen per cent before any chemistry enters. Computing the same three conventions for a series — methane, silane, germane — and asking whether the convention spread is larger than the chemical spread down the group, in the way a molecule’s softest coordinate is compared against its stiffest, would say whether transferability is a real observation or an artefact of everybody using one program.

The nearer question is which combinations are determined, stated properly. The flat space is a subspace of the fifty-five-dimensional space of constants, so its orthogonal complement is a set of forty-five combinations that are fixed by the spectrum — and those combinations are computable, being the singular vectors belonging to the non-zero singular values. Writing out the handful of them that involve the bend constants, in a form a reader could compare between papers, would turn report what is measured from advice into a table.