Multiclass Classification with Neural Networks

Extending Binary Classification

In binary classification, our hypothesis outputs a single value:

$$h_\Theta(x) \in [0,1]$$

For multiclass classification, instead of returning a single value,
our hypothesis returns a vector of probabilities.

---

Example: Four-Class Classification

Suppose we want to classify an image into one of four categories:

• Car
• Pedestrian
• Truck
• Motorcycle

Instead of one output unit, we use four output units.

---

Network Structure

$$\text{Input Image} \rightarrow \text{Hidden Layers} \rightarrow \text{4 Output Units}$$

Each output unit corresponds to one class.

---

Output Representation

Our hypothesis now returns:

$$h_\Theta(x) = \begin{bmatrix} h_\Theta(x)_1 \\ h_\Theta(x)_2 \\ h_\Theta(x)_3 \\ h_\Theta(x)_4 \end{bmatrix}$$

Where:

• $h_\Theta(x)_1$ → Probability of Car
• $h_\Theta(x)_2$ → Probability of Pedestrian
• $h_\Theta(x)_3$ → Probability of Truck
• $h_\Theta(x)_4$ → Probability of Motorcycle

---

Training Labels (One-Hot Encoding)

Each training example has a label vector:

$$y^{(i)} \in \mathbb{R}^4$$

Examples:

Car:

$$\begin{bmatrix} 1 \\ 0 \\ 0 \\ 0 \end{bmatrix}$$

Motorcycle:

$$\begin{bmatrix} 0 \\ 0 \\ 0 \\ 1 \end{bmatrix}$$

This is called one-hot encoding.

---

Example Output

Suppose the network outputs:

$$h_\Theta(x) = \begin{bmatrix} 0 \\ 0 \\ 1 \\ 0 \end{bmatrix}$$

This means:

$$h_\Theta(x)_3 = 1$$

So the predicted class is the third category.

If we defined:

1 → Car
2 → Pedestrian
3 → Truck
4 → Motorcycle

Then the model predicts:

$$\text{Truck}$$

---

Decision Rule

In practice, we select:

$$\text{Prediction} = \arg\max_k h_\Theta(x)_k$$

That is, we choose the class with the largest output value.

---

Key Idea

• Binary classification → 1 output unit
• Multiclass classification → K output units
• Output layer size = number of classes
• Final prediction = index of largest output

Neural networks naturally extend logistic regression to multiple classes by simply increasing the number of output neurons.