System: one chip per layer

Each layer runs on its own chip; 18 chips hold DeltaNet layers, 6 hold attention layers. The embedding is a row lookup; the output layer is a capacitor grid read by an 8-bit ADC referenced to the peak logit (a winner-take-all circuit would suffice for greedy decoding).

Inside a layer chip

Weights never move: each matrix is a fixed grid. Signals stay analog inside the chip and become digital only at the memory blocks and the chip-to-chip link. DeltaNet memory is digital because an analog capacitor memory failed in simulation: every token writes every cell, so thermal noise piles up faster than any practical capacitor can hold it.

Every element and what it adds

element	what it does	built from	what it adds	errors / cost
Embedding table	Turns the token number into 1,024 numbers that describe the word	Memory, one row read per token	The starting meaning of the word	Memory size (254M values for Qwen3.5-0.8B)
Norm unit	Rescales the 1,024 signals to a fixed average size	RMS detector + divider + one gain resistor per line	Keeps signals inside the useful range: not clipped, not lost in noise	~0.5 % gain mismatch, 0.2 % RMS error
DAC / row driver	Turns a digital number into a voltage	Digital-to-analog converter per line (at the chip input)	The bridge from bits to the analog world	Rounding (10 bits), energy per conversion
Carrier + modulator	Makes a 100 MHz sine wave whose height is the signal (flipped wave = negative)	Oscillator shared by the chip, a mixer per row	Lets capacitors compute (they only pass changing signals) and lets the read-out reject offsets	~1 mW oscillator per chip, ~10 fJ per row per read
Capacitor grid	Multiplies every input by every weight and adds them up	A pair of capacitors per weight (C+ / C−), 0–255 unit capacitors each	The core: a whole matrix product in one step, weights never move	Mismatch ~0.13 % at 10 fF, stray capacitance, ~5 µm² per cell
Row and column wires	Carry the waves in and the summed currents out	Thin metal lines across the grid	The addition itself (Kirchhoff: currents into a wire add up)	Resistance delays the wave along the row; calibrated per column
Synchronous demodulator	Turns the summed wave current back into a steady value	Mixer driven by the same carrier + averaging	Keeps only the part in step with the carrier: offsets and slow noise are rejected	~10 fJ per output per read
Read-out amplifier + feedback resistor	Turns the small current into a voltage with the right scale	Transimpedance amplifier per column; the resistor value is part of the design	One scale per output column (the row's largest weight)	Noise (the main random error), 0.5 % gain mismatch, ~3.5 µW each
Rails	The highest voltage a circuit can produce	Supply limits of every amplifier	Nothing; it is a limit to design around	Signals beyond it are clipped; full scale is set to the largest signal seen
SiLU / sigmoid unit	The model's 'decision' curve: passes positive signals, mostly blocks negative ones	Transistors whose current curve has that shape	Non-linearity: without it the whole model would be one big matrix	~1 % gain mismatch, trimmed offsets
Gilbert multiplier	Multiplies two signals (one gates the other)	Classic analog multiplier cell	Gating: lets the model switch information on and off	~1 % gain mismatch, slight compression of large signals
Short conv (sample & hold)	Remembers the last 3 inputs for a tiny local mix	Capacitors that hold earlier values + resistor taps	Local word-order information	kT/C noise on the held values (very small)
ADC to memory / DAC back	Converts signals to bits for the memory and back	Converters at the memory boundary	Lets a precise digital memory work with the analog grids	Energy per conversion
DeltaNet memory (18 chips)	Remembers earlier words in a fixed-size table (16 × 128 × 128 numbers)	SRAM + digital multiply-add units	Memory of the whole text without growing	Most of the energy; analog versions failed (thermal noise piles up)
Attention unit (6 chips)	Looks back at every earlier word to find the exact ones that matter	KV cache in SRAM/DRAM + digital softmax	Exact recall (e.g. a number mentioned earlier)	Energy and memory grow with the text length
Gated norm	Rescales each memory head and gates it with a second signal	Norm unit + SiLU unit + Gilbert multiplier	Keeps the memory output well scaled	Same errors as its parts
Residual capacitors	Hold the main 1,024 signals and add each block's result	One capacitor node per line	Free addition: current in = charge added	Negligible
Chip-to-chip link	Moves the 1,024 signals to the next layer chip	10-bit ADC → serial link (64 lanes) → DAC	Separate chips per layer; bits clean away accumulated analog noise	~20 nJ per hop; 8-bit links cost +2.5 % quality
Output grid + ADC	Scores all 248,320 possible next tokens	Capacitor grid + 8-bit ADC referenced to the top score	The prediction	Largest single block (~2,700 mm²); a winner-take-all circuit could pick the winner directly
Calibration (after manufacturing)	Measures and removes predictable errors	Per-column phase reference, offset auto-zero, rail settings	Turns big systematic errors into small random ones	Test time per chip; must be repeated if conditions change

The AC capacitor grid

Capacitors instead of resistors: capacitors add no thermal (Johnson) noise, match better (~0.1 %), and their reactive current causes almost no wire voltage drop. Synchronous demodulation also rejects amplifier offsets and 1/f noise. The cost is a carrier generator, modulators and demodulators, and area (~5 µm² per 10 fF capacitor).

Component counts per chip

chip	weights	capacitors	amplifiers	area mm²
L00	21,528,576	21,373,857	17,440	229.1
L01	21,528,576	21,392,507	17,440	229.1
L02	21,528,576	21,392,099	17,440	229.1
L03	18,350,080	18,228,909	14,336	195.2
L04	21,528,576	21,385,721	17,440	229.1
L05	21,528,576	21,385,748	17,440	229.1
L06	21,528,576	21,382,071	17,440	229.1
L07	18,350,080	18,225,107	14,336	195.2
L08	21,528,576	21,388,216	17,440	229.1
L09	21,528,576	21,379,999	17,440	229.1
L10	21,528,576	21,378,534	17,440	229.1
L11	18,350,080	18,220,418	14,336	195.2
L12	21,528,576	21,382,810	17,440	229.1
L13	21,528,576	21,384,244	17,440	229.1
L14	21,528,576	21,368,961	17,440	229.1
L15	18,350,080	18,211,621	14,336	195.2
L16	21,528,576	21,381,768	17,440	229.1
L17	21,528,576	21,385,928	17,440	229.1
L18	21,528,576	21,385,014	17,440	229.1
L19	18,350,080	18,222,695	14,336	195.2
L20	21,528,576	21,387,290	17,440	229.1
L21	21,528,576	21,388,624	17,440	229.1
L22	21,528,576	21,381,133	17,440	229.1
L23	18,350,080	18,197,520	14,336	195.2
head	254,279,680	252,686,181	248,320	2,707.8

Area: two capacitor cells per weight (C/2 fF/µm² + 0.3 µm² overhead) plus ~50 µm² per amplifier. Converters, digital memory, control and I/O are not included.