Take two numbers: 0.6 and 0.6.
If I turn each into a Boolean first, using x > 0.5, they’re both true. Their AND is true too. But feed those same numbers straight into the Boolean-neuron formula a + b - 1.5 > 0, and the answer is false.
Nothing mysterious happened. I changed the input contract. 🔎
Ted Benson’s new post on embedding Boolean circuits in neural networks separates gate construction from connecting those gates to a trained model. That separation is the useful part. Here’s a small executable way to see why.
Original mathematical illustration of the stated formulas. The shaded regions are computed decision regions, not measured model behavior.
First, build the gates
Let step(z) return one when z > 0, otherwise zero. With inputs restricted to zero or one:
- AND uses
step(a + b - 1.5). - OR uses
step(a + b - 0.5). - NOT uses
step(0.5 - a).
For AND, only the input sum of two clears the threshold. For OR, either nonzero bit clears it. NOT reverses each allowed bit. These are fixed weights and biases; there’s no training loop. 💡
XOR can then combine an OR gate and an AND gate: turn on when “either” is one and “both” is zero. A hidden layer lets the network build that combination instead of asking a single linear boundary to separate opposite corners.
Run the whole truth table 🧪
This uses only Python’s standard library. Save it as boolean_check.py and run python3 boolean_check.py.
from itertools import product
def step(x):
return int(x > 0)
def gate_xor(a, b):
either = step(a + b - 0.5)
both = step(a + b - 1.5)
return step(either - both - 0.5)
print('a b AND OR XOR')
for a, b in product((0, 1), repeat=2):
both, either = step(a + b - 1.5), step(a + b - 0.5)
xor = gate_xor(a, b)
assert both == (a & b) and either == (a | b)
assert xor == (a ^ b)
assert step(0.5 - a) == (1 - a)
print(a, b, both, either, xor)
a = b = 0.6
raw_and = step(a + b - 1.5)
probed_and = step(step(a - 0.5) + step(b - 0.5) - 1.5)
assert (raw_and, probed_and) == (0, 1)
assert step(0.5 - 0.5) == 0
print('analog inputs:', raw_and, 'probe first:', probed_and)
print('NOT at 0.5:', step(0.5 - 0.5))
It prints:
a b AND OR XOR
0 0 0 0 0
0 1 0 1 1
1 0 0 1 1
1 1 1 1 0
analog inputs: 0 probe first: 1
NOT at 0.5: 0
✅ All Boolean cases match Python’s bitwise reference operations. The analog example deliberately disagrees.
The diagram explains the disagreement
The direct AND neuron asks whether a + b exceeds 1.5. Its positive region sits above a diagonal line.
Thresholding first asks whether each input exceeds 0.5. Its positive region is the upper-right rectangle. Both rules agree at the Boolean corners. Between the corners, they are different functions.
So a truth-table check proves the gate for its declared inputs. It doesn’t prove that arbitrary real activations behave like those inputs.
Even equality needs a choice: the strict rule sends 0.5 to zero in the NOT example. If a production probe is meant to include its threshold, its comparison must say so. ⚠️
Connecting this to a language model is another job
Benson proposes a probe that turns an activation into a bit, followed by a circuit and an additive intervention later in the network.1
My check covers the arithmetic circuit. It doesn’t establish that a chosen activation means a particular concept, that a threshold keeps that meaning across prompts, or that the intervention reliably changes an answer.
That’s where I’d want model evidence: labeled examples around the threshold, cases where the intended concept is absent, and checks of the downstream effect. A correct gate can faithfully act on a poorly chosen signal. The wiring deserves as much attention as the truth table.
References and asset credit
- Embedding boolean circuits in a neural network, Ted Benson. Source of the gate construction and proposed model connection.
The diagram, Python implementation, bitwise checks and analog counterexample are original explanatory material. No trained language model was evaluated.
AI disclosure
AI was used to draft the article.
Top comments (0)