Skip to main content
← All work

loci

Tap code treated as a channel with unequal symbol costs: how few taps a word can possibly cost, and a constructed code that gets within 0.03 of the bound.

Status

Finished as an investigation. Five interactive pages and three solvers, all reproducible from the repository. Not a product and not novel. The coding theory it rests on was settled between 1948 and 1998, and the write-up says so.

Channel capacity
1.0000 bits per tap
Floor
8.99 taps per word
Constructed code
9.02 taps per word
Standard grid
21.98 taps per word
Corpus
SUBTLEX-US, 37,058 words
Repository
16 commits, 8,598 lines
loading the lattice

The question

Tap code is the prisoner's alphabet: a five by five grid, tap the row, pause, tap the column. It has been in use since at least the Vietnam War and the layout has never changed: the alphabet goes in reading order, with C sharing K's cell so twenty-six letters fit twenty-five squares.

Read as a communication channel, that layout is strange. A cell costs its row number plus its column number, so the top left is worth two taps and the bottom right is worth ten. The standard grid spends the single cheapest cell in the entire system on the letter A.

The question this project answers is how much that costs, and then how few taps a word could possibly take if you stopped being sentimental about it.

Measuring the grid

Letter frequency was weighted by how often the containing word is actually spoken, using SUBTLEX-US subtitle frequencies rather than written-text counts, because tap code is speech.

The standard alphabetical grid costs 21.98 taps for the average spoken word. Reordering the same twenty-five cells so the cheapest hold the most-sent letters brings that to 16.65, a 24% cut for no structural change at all. Adding whole-word cells on a six by six grid reaches 15.94.

The word cells are the disappointing part, and the reason is worth stating: any single letter is sent far more often than any single word, so letters outbid words for the cheap cells, and common words are assembled from common letters and were therefore already cheap to spell. "The" is three of the top five letters. Reordering is worth 5.3 taps; the word shortcuts are worth 0.7.

The floor

A run of k taps carries one symbol and costs k, which makes this a channel whose symbols have unequal cost. Channels like that have a capacity: the C solving the sum over k of two to the minus C times k equals one. With unbounded runs that is exactly one bit per tap. Capped at five, about as far as a person counts without losing their place, it is 0.9752.

An English word drawn from spoken frequency carries 8.77 bits. Dividing gives 8.99 taps per word as a lower bound for any prefix code that treats words as independent events. The constructed code reaches 9.02.

The ladder between the grid and the bound is not closed by tapping more cleverly. Giving letters variable-length codewords instead of a grid actually scores slightly worse than the grid does, because twenty-five symbols of similar frequency are close to the best case for a fixed two-run code. Every real gain comes from not spelling: five hundred memorised word codes reach 12.44 taps, five thousand reach 10.02.

Two things the construction had to get right

Stopping at the Shannon codeword length leaves about a quarter of the Kraft budget unspent, which is a quarter of the available compression discarded. The leftover is spent greedily on whichever symbol buys the most taps per unit of budget.

More interesting, the Kraft-style inequality for unequal letter costs is necessary but not sufficient, and the construction has to know that. A set of codeword costs can satisfy the inequality and still have nowhere to sit in the code tree. The minimal counterexample is embarrassingly small: three codewords each of cost two. The inequality gives 0.776, comfortably under one, but there are exactly two strings of total cost two over runs of one to five, namely a run of two and two runs of one. Three codewords, two slots, no code. The assignment therefore walks the tree and pushes overflow symbols one tap further out rather than trusting the bound.

What the floor rests on

A bound is only as hard as its premises. Relaxing them one at a time: two distinguishable taps, knuckle against pad, doubles the channel capacity and halves the floor to 4.44. An agreed five hundred word vocabulary instead of the full corpus drops it to 7.53. Charging one tap-time for each inter-run gap, which is the honest thing to do if the metric is time rather than effort, pushes it up to 13.47.

And one premise does not bend, it breaks. Real tap code marks the end of a word with a longer pause, and that pause carries information nobody paid taps for. Allow it and the code no longer has to be prefix free, more codewords fit inside every budget, and the best code costs 6.24 taps per word, straight through the 8.99 floor. The bound was never violated; it simply never applied, because it was derived for a channel in which silence says nothing. Push the same trick and a message costs zero taps.

The page shows all four, because a bound presented without its premises is a decoration.

More than two dimensions

A grid is two runs because someone chose two. Three runs is a cube, four is a tesseract, and a cell still costs the sum of its coordinates.

Each dimension multiplies the reachable cells and charges a floor tax of one tap, because every dimension costs at least one. So a two by two by two by two by two hypercube holds thirty-two symbols and cannot send any of them for under five taps, which is worse than the alphabetical grid it was meant to improve on. Too few dimensions and the far corner runs away from you: a single row of twenty-five cells costs 31.59 taps per word.

The best dimensionality therefore rises with the size of the codebook. Two dimensions win at thirty-six cells, three from sixty-four to five hundred and twelve, four past a thousand. Shapes at each dimensionality were searched rather than guessed.

What the shape buys is not efficiency, it is structure. A cube cell is three small numbers that sit on a memory route and can be reconstructed when half forgotten; a variable-length codebook is a list of arbitrary strings with nothing to fall back on. For a system that has to live in someone's head that is usually the better trade, and it is not a trade the coding literature scores.

Prior art, checked

None of the coding theory here is new, and the citations were re-fetched rather than recalled. Shannon (1948) opens on this exact channel, symbols of unequal duration and capacity as the largest real root of a characteristic equation, in section 1 of the discrete noiseless channel, with the state-constrained form as his Theorem 1. Shannon (1951) is the source for English at roughly a bit per character; his measured figure is a bracket, 0.6 to 1.3 bits for a hundred characters of context, and "about one bit" is his own summary of it.

Karp (1961) formulated minimum-redundancy coding on a noiseless channel with unequal letter costs and, importantly for the construction here, gave conditions that are necessary and sufficient, a system of linear inequalities over the code tree's levels rather than one scalar test, then solved it as an integer program. Krause (1962) generalized McMillan's inequality to letters of unequal duration; his abstract is explicit that this is the converse direction, used to show no uniquely decipherable encoding raises the capacity. Varn (1971) gives the optimal code when codewords are equally probable, which is the opposite of the skew here. Golin and Rote (1998) construct optimal unequal-cost prefix codes by dynamic programming.

The trap in section four is also known, and is stated as an exercise rather than a theorem. Csiszar and Korner set the necessity direction as a corollary and then ask the reader to show the inequality is in general not sufficient for a prefix or separable code to exist. Beal, Berstel, Marcus, Perrin, Reutenauer and Siegel prove it by counterexample in their chapter on variable-length codes, and Marcus (1957) has the earliest version found, with a counterexample of the same shape as the one measured here. What no source states is the converse anyone reaching for the equal-cost result would assume, and that absence is the reason the construction walks the tree instead of trusting the bound.