Perceptron

One neuron, one line

A perceptron takes some numbers (features), multiplies each by a weight, adds a bias and checks the sign:

ŷ = +1  if  w₁x₁ + w₂x₂ + b ≥ 0
    −1  otherwise

Geometrically, w·x + b = 0 is a straight decision line. Points on one side are class +1 and the rest class −1. The weight vector w is perpendicular to the line and points towards the +1 side.

Learning from mistakes

Start with arbitrary weights and sweep through the training points. Whenever a point is classified wrongly, nudge the line towards it:

for each point (x, y):
    if sign(w·x + b) ≠ y:
        w ← w + η · y · x
        b ← b + η · y

η is the learning rate. For a +1 point that was predicted −1 the update adds the point to w, which raises its score; a −1 point is subtracted. Repeat for several epochs until a full pass makes no mistakes.

Does it always work?

If the classes can be separated by a straight line (they are linearly separable), the perceptron convergence theorem says the algorithm stops after a finite number of updates. If they cannot be separated, it keeps wandering forever. The classic failure is XOR: no single line separates (0,0), (1,1) from (0,1), (1,0). That limitation motivated multi-layer networks (see neural networks).

Perceptron vs related models

Code

def train_perceptron(points, labels, lr=1.0, epochs=30):
    w, b = [0.0, 0.0], 0.0
    for _ in range(epochs):
        mistakes = 0
        for (x1, x2), y in zip(points, labels):
            pred = 1 if w[0] * x1 + w[1] * x2 + b >= 0 else -1
            if pred != y:
                w[0] += lr * y * x1
                w[1] += lr * y * x2
                b += lr * y
                mistakes += 1
        if mistakes == 0:
            break
    return w, b

Common mistakes

• Updating on every point instead of only on mistakes.
• Forgetting the bias b. Without it the line must pass through the origin.
• Expecting convergence on data that is not linearly separable.
• Using labels 0/1 with the formula w + η·y·x. This version needs labels +1/−1.