In 1943 Warren McCulloch and Walter Pitts proposed a mathematical caricature of a brain cell: it adds up its weighted inputs and fires if the total crosses a threshold. In 1958 Frank Rosenblatt built the perceptron, a machine that learned those weights from examples, and the New York Times reported the Navy expected it to eventually walk, talk and be conscious of its existence. Expectations ran ahead of reality, as they would again.
You've met this already: with = sigmoid it is logistic regression, and with = identity it is linear regression. The real neuron analogy is loose. Biological neurons spike in time, have complex dendrites and learn by mechanisms we don't fully understand; "neural" networks borrowed the inspiration, not the mechanism.
The perceptron learning rule
Rosenblatt's perceptron uses a hard step: output 1 if , else 0. Its learning rule is simple: for each misclassified example, nudge the weights toward it if it should have fired, away from it if it shouldn't. If a straight line can separate the classes, this is guaranteed to find one in a finite number of updates (Novikoff's convergence theorem, 1962).
The XOR problem
That if is the catch. Exclusive-or outputs 1 when exactly one of two inputs is 1. Plot the four cases and no single line separates the ones from the zeros. In their 1969 book Perceptrons, Marvin Minsky and Seymour Papert proved this and other limits of single-layer perceptrons. Funding and interest in neural networks collapsed for over a decade, one of the "AI winters".
The fix was known even then: add a hidden layer. Two neurons can each draw a line, and a third combines them. What was missing was a way to train the hidden layer's weights, since the perceptron rule needs to know each neuron's correct output and hidden neurons don't have one. That method, backpropagation, became widely known in 1986. You'll derive it in the next unit.
A neuron is a weighted sum passed through an activation. One neuron draws one line, so it can't solve problems like XOR; layers of neurons can.