Recorded from the emulator: the chip processed each text below, and the steps show the real numbers of that run for the last word.
Text: “The cat sat on the”
Follow the word ' the' through the chips
The model reads the whole text, but here we follow only the last piece, because that is where it guesses what comes next. All numbers below come from this run.
Step 1: Text becomes numbers (tokens)
In simple words: A computer cannot read letters. The text is cut into pieces called tokens, and each piece has a number, like a page number in a dictionary of 248,320 pieces.
In the chip: This is ordinary digital work, done before the chips.
The #760 cat #7993 sat #7338 on #383 the #279
Step 2: The token becomes a list of 1,024 numbers (embedding)
In simple words: Token #279 is looked up in a big table. Its row is a list of 1,024 numbers that describe the word, a bit like a fingerprint. Words with similar meaning get similar lists.
In the chip: The table is stored once; the row is read out and sent to the first layer chip.
first 8 of 1,024: +0.002 -0.045 +0.005 +0.015 +0.004 +0.014 +0.005 +0.021 …
Step 3: Volume control (norm unit)
In simple words: Some lists are loud, some are quiet. The norm unit turns the volume so the list always has the same average loudness, so the next parts are never overloaded or too quiet.
In the chip: An analog circuit measures the average size of all 1,024 signals (root-mean-square) and divides each signal by it; one small resistor per line sets the learned gain.
before: +0.002 -0.045 +0.005 +0.015 +0.004 +0.014 +0.005 +0.021 …
after: +0.218 -3.089 +0.433 +1.428 +0.281 +0.979 +0.330 +1.416 …
Step 4: Numbers become voltages, then waves (DAC + modulator)
In simple words: Electronics cannot use numbers directly, so each number becomes an electrical push: a voltage. Then the voltage is turned into a wave that wiggles 100 million times per second. A bigger number makes a taller wave; a negative number makes a wave that is flipped upside down.
In the chip: A digital-to-analog converter makes the voltage (full scale ±0.5 V for the largest signal 17.19); a modulator multiplies it by a 100 MHz carrier.
Step 5: The magic part: capacitors multiply, wires add (capacitor grid)
In simple words: Each input wave crosses many tiny capacitors. A capacitor lets a little wave-current through: more if the wave is taller (the input) and more if the capacitor is bigger (the learned weight). That is a multiplication, done by physics. All those little currents flow into one wire and simply add up, like streams joining a river. That is the sum. One wire = one answer, and thousands of wires work at the same time.
In the chip: Current through a capacitor: I = ω·C·V. Each weight is a pair of capacitors (C+ for positive, C− for negative) made of 0–255 unit capacitors of 39.2 aF. Kirchhoff's law adds all 1,024 currents on the column wire. Here: FFN gate grid of layer 0, output column 3, the 5 biggest of its 1,024 currents.
Step 6: Back from a wave to a number (synchronous demodulator + amplifier)
In simple words: The wire carries a wave, but the next part needs a plain number. The demodulator listens to the wave in perfect step with the original rhythm and measures how tall it is, ignoring noise that does not follow the rhythm. An amplifier then turns the small current into a usable voltage.
In the chip: The demodulator multiplies the column current by the carrier and averages it (only the in-phase part survives, so amplifier offsets and slow noise are rejected). The read-out amplifier's feedback resistor sets the scale. Real circuits are not perfect: capacitor mismatch, amplifier noise and wire delays shift the result a little.
exact math: +2.0788 | emulated chip: +2.0390 | difference -1.91 %
Step 7: A smart gate decides what passes (SiLU unit and multiplier)
In simple words: Not every signal should pass. A gate lets big positive signals through, almost blocks negative ones, and is gentle in between. Then a second signal (from the 'up' grid) is multiplied by the gate's answer: it is like a tap controlling how much water flows.
In the chip: SiLU is made from transistors whose current curve has the right shape; the multiplication is a Gilbert cell. Each unit has its own tiny manufacturing differences (gain about 1 %, trimmed offsets).
gate +2.079 → SiLU +1.848 (chip +1.775) → × up → +0.7182 (chip +0.6913)
Step 8: Adding the result to the main signal (residual)
In simple words: The layer does not replace the signal; it adds a correction to it, like adding a note to a page instead of rewriting the page. The 1,024 main signals carry the meaning from layer to layer.
In the chip: The down grid's output currents flow straight into the 1,024 residual capacitors: charge adds to charge, so the addition costs nothing.
main signal after layer 0, first 8: -0.561 +0.017 -0.100 -0.016 -0.090 +0.039 +0.006 +0.002 …
on the chip: -0.549 +0.015 -0.097 -0.014 -0.091 +0.033 +0.002 -0.002 …
Step 9: Remembering earlier words (digital memory)
In simple words: To understand a sentence the model must remember the words before. Each layer chip keeps a memory: 16 small tables of 128 × 128 numbers that are updated with every word and read to find what matters now.
In the chip: Analog memory turned out too noisy, so this memory is digital: signals go through an ADC into SRAM, the memory is updated with digital multiply-adds, and the answer goes back through a DAC to the analog part. The emulator rounds the signals at both converters; the digital arithmetic itself is exact.
memory read-out, first 8 numbers: -0.000 -0.000 -0.000 +0.000 -0.000 +0.000 +0.000 +0.000 …
on the chip: -0.000 -0.000 -0.000 +0.000 -0.000 +0.000 +0.000 +0.000 …
Step 10: Jumping to the next chip (ADC → bits → DAC)
In simple words: Each layer lives on its own chip. To travel between chips, every signal is turned into a short row of 0s and 1s (bits), sent along a fast cable, and turned back into a voltage. Bits are robust: noise on the cable cannot change a 0 into a 1 easily, so errors stop piling up.
In the chip: A 10-bit ADC per line, a serial link, and a DAC on the next chip. Here signal 0:
-0.5495 → code -243 →
→ back to -0.5496 on the next chip
Step 11: The same journey 24 times (all layer chips)
In simple words: Steps 3–10 happen on every chip, 24 times. Some chips mix information from earlier words (memory chips and attention chips), all of them refine the meaning. Small errors appear on each chip, but the norm units and the digital links keep them from growing out of control.
In the chip: 18 DeltaNet chips and 6 attention chips, each about 1.1 µs per token.
Step 12: Guessing the next word (output grid)
In simple words: The final signal is compared with all 248,320 known pieces at once. Each comparison gives a score; the highest scores are the best guesses for what comes next.
In the chip: The output grid has 248,320 columns (one per token). An 8-bit ADC measures the scores relative to the top one (a winner-take-all circuit could pick the winner directly).
| exact model | chance | emulated chip | chance |
|---|---|---|---|
| ' floor' | 15.3 % | ' floor' | 14.5 % |
| ' sofa' | 8.9 % | ' sofa' | 8.7 % |
| ' grass' | 5.7 % | ' chair' | 5.3 % |
| ' bed' | 4.6 % | ' bed' | 4.1 % |
| ' rug' | 3.5 % | ' bench' | 4.1 % |
Step 13: How long and how much energy
In simple words: The whole journey for this one word takes about 27 millionths of a second. In the time you blink (0.1 s) the chips could handle about 3,738 words. The energy is about 6.2 millionths of a joule — about what it takes to lift one grain of sugar by 106 cm.
In the chip: Emulated hardware: 26.7 µs and 6.24 µJ per token at context 5 (estimates from the circuit model of an unbuilt chip). The Mac needs far longer to simulate it.
Text: “The capital of France is”
Follow the word ' is' through the chips
The model reads the whole text, but here we follow only the last piece, because that is where it guesses what comes next. All numbers below come from this run.
Step 1: Text becomes numbers (tokens)
In simple words: A computer cannot read letters. The text is cut into pieces called tokens, and each piece has a number, like a page number in a dictionary of 248,320 pieces.
In the chip: This is ordinary digital work, done before the chips.
The #760 capital #6511 of #314 France #9338 is #369
Step 2: The token becomes a list of 1,024 numbers (embedding)
In simple words: Token #369 is looked up in a big table. Its row is a list of 1,024 numbers that describe the word, a bit like a fingerprint. Words with similar meaning get similar lists.
In the chip: The table is stored once; the row is read out and sent to the first layer chip.
first 8 of 1,024: -0.001 +0.019 -0.001 +0.021 +0.005 +0.027 -0.013 +0.001 …
Step 3: Volume control (norm unit)
In simple words: Some lists are loud, some are quiet. The norm unit turns the volume so the list always has the same average loudness, so the next parts are never overloaded or too quiet.
In the chip: An analog circuit measures the average size of all 1,024 signals (root-mean-square) and divides each signal by it; one small resistor per line sets the learned gain.
before: -0.001 +0.019 -0.001 +0.021 +0.005 +0.027 -0.013 +0.001 …
after: -0.096 +1.308 -0.110 +1.891 +0.375 +1.848 -0.866 +0.046 …
Step 4: Numbers become voltages, then waves (DAC + modulator)
In simple words: Electronics cannot use numbers directly, so each number becomes an electrical push: a voltage. Then the voltage is turned into a wave that wiggles 100 million times per second. A bigger number makes a taller wave; a negative number makes a wave that is flipped upside down.
In the chip: A digital-to-analog converter makes the voltage (full scale ±0.5 V for the largest signal 17.19); a modulator multiplies it by a 100 MHz carrier.
Step 5: The magic part: capacitors multiply, wires add (capacitor grid)
In simple words: Each input wave crosses many tiny capacitors. A capacitor lets a little wave-current through: more if the wave is taller (the input) and more if the capacitor is bigger (the learned weight). That is a multiplication, done by physics. All those little currents flow into one wire and simply add up, like streams joining a river. That is the sum. One wire = one answer, and thousands of wires work at the same time.
In the chip: Current through a capacitor: I = ω·C·V. Each weight is a pair of capacitors (C+ for positive, C− for negative) made of 0–255 unit capacitors of 39.2 aF. Kirchhoff's law adds all 1,024 currents on the column wire. Here: FFN gate grid of layer 0, output column 86, the 5 biggest of its 1,024 currents.
Step 6: Back from a wave to a number (synchronous demodulator + amplifier)
In simple words: The wire carries a wave, but the next part needs a plain number. The demodulator listens to the wave in perfect step with the original rhythm and measures how tall it is, ignoring noise that does not follow the rhythm. An amplifier then turns the small current into a usable voltage.
In the chip: The demodulator multiplies the column current by the carrier and averages it (only the in-phase part survives, so amplifier offsets and slow noise are rejected). The read-out amplifier's feedback resistor sets the scale. Real circuits are not perfect: capacitor mismatch, amplifier noise and wire delays shift the result a little.
exact math: -2.0119 | emulated chip: -2.0016 | difference +0.51 %
Step 7: A smart gate decides what passes (SiLU unit and multiplier)
In simple words: Not every signal should pass. A gate lets big positive signals through, almost blocks negative ones, and is gentle in between. Then a second signal (from the 'up' grid) is multiplied by the gate's answer: it is like a tap controlling how much water flows.
In the chip: SiLU is made from transistors whose current curve has the right shape; the multiplication is a Gilbert cell. Each unit has its own tiny manufacturing differences (gain about 1 %, trimmed offsets).
gate -2.012 → SiLU -0.237 (chip -0.239) → × up → -0.1076 (chip -0.1041)
Step 8: Adding the result to the main signal (residual)
In simple words: The layer does not replace the signal; it adds a correction to it, like adding a note to a page instead of rewriting the page. The 1,024 main signals carry the meaning from layer to layer.
In the chip: The down grid's output currents flow straight into the 1,024 residual capacitors: charge adds to charge, so the addition costs nothing.
main signal after layer 0, first 8: -0.425 -0.003 -0.050 +0.057 -0.056 -0.050 +0.061 -0.115 …
on the chip: -0.419 -0.007 -0.053 +0.052 -0.059 -0.052 +0.056 -0.117 …
Step 9: Remembering earlier words (digital memory)
In simple words: To understand a sentence the model must remember the words before. Each layer chip keeps a memory: 16 small tables of 128 × 128 numbers that are updated with every word and read to find what matters now.
In the chip: Analog memory turned out too noisy, so this memory is digital: signals go through an ADC into SRAM, the memory is updated with digital multiply-adds, and the answer goes back through a DAC to the analog part. The emulator rounds the signals at both converters; the digital arithmetic itself is exact.
memory read-out, first 8 numbers: -0.000 -0.000 -0.000 +0.000 -0.000 -0.000 +0.000 -0.000 …
on the chip: -0.000 -0.000 -0.000 +0.000 -0.000 -0.000 +0.000 -0.000 …
Step 10: Jumping to the next chip (ADC → bits → DAC)
In simple words: Each layer lives on its own chip. To travel between chips, every signal is turned into a short row of 0s and 1s (bits), sent along a fast cable, and turned back into a voltage. Bits are robust: noise on the cable cannot change a 0 into a 1 easily, so errors stop piling up.
In the chip: A 10-bit ADC per line, a serial link, and a DAC on the next chip. Here signal 0:
-0.4187 → code -185 →
→ back to -0.4184 on the next chip
Step 11: The same journey 24 times (all layer chips)
In simple words: Steps 3–10 happen on every chip, 24 times. Some chips mix information from earlier words (memory chips and attention chips), all of them refine the meaning. Small errors appear on each chip, but the norm units and the digital links keep them from growing out of control.
In the chip: 18 DeltaNet chips and 6 attention chips, each about 1.1 µs per token.
Step 12: Guessing the next word (output grid)
In simple words: The final signal is compared with all 248,320 known pieces at once. Each comparison gives a score; the highest scores are the best guesses for what comes next.
In the chip: The output grid has 248,320 columns (one per token). An 8-bit ADC measures the scores relative to the top one (a winner-take-all circuit could pick the winner directly).
| exact model | chance | emulated chip | chance |
|---|---|---|---|
| ' Paris' | 13.0 % | ' Paris' | 25.1 % |
| ' the' | 9.8 % | ' the' | 7.1 % |
| ' located' | 7.0 % | ':' | 7.1 % |
| ':' | 5.9 % | '\n' | 5.5 % |
| '\n' | 5.6 % | ' located' | 5.5 % |
Step 13: How long and how much energy
In simple words: The whole journey for this one word takes about 27 millionths of a second. In the time you blink (0.1 s) the chips could handle about 3,738 words. The energy is about 6.2 millionths of a joule — about what it takes to lift one grain of sugar by 106 cm.
In the chip: Emulated hardware: 26.7 µs and 6.24 µJ per token at context 5 (estimates from the circuit model of an unbuilt chip). The Mac needs far longer to simulate it.