Neural Networks as Logical Gates

A single logistic neuron can simulate logical gates.

By adjusting:

• Bias (threshold)
• Weights (importance of inputs)

we can model:

• AND
• OR
• NOT

Neural networks are powerful because stacking these simple units allows us to represent much more complex functions.

Implementing the AND Operator $(x_1 \land x_2)$

The logical AND operator is true only when:

• $x_1 = 1$
• $x_2 = 1$

Otherwise, it is false.

$x_1$	$x_2$	Result
0	0	0
1	0	0
0	1	0
1	1	1

Network Structure

graph LR

%% Input Layer
    subgraph Input Layer
        x0(((x0)))
        x1(((x1)))
        x2(((x2)))
    end

%% Hidden Layer 1
    subgraph Hidden Layer 1
        a1{a1}
    end

%% Output Layer
    subgraph Output Layer
        y(((hθx)))
    end

%% Connections: Input → Hidden 1
    x0 --> a1
    x1 --> a1
    x2 --> a1
    
%% Connections: Hidden 2 → Output
    a1 --> y
 

Our small neural network looks like:

$$\begin{bmatrix} x_0 \\ x_1 \\ x_2 \end{bmatrix} \rightarrow g(z^{(2)}) \rightarrow h_\Theta(x)$$

Where: $x_0 = 1$ is the bias unit

Choosing the Weights

Consider weight matrix:

$$\Theta^{(1)} = \begin{bmatrix} -30 & 20 & 20 \end{bmatrix}$$

The hypothesis becomes:

$$h_\Theta(x) = g(-30 + 20x_1 + 20x_2)$$

Evaluating All Input Combinations

$x_1$	$x_2$	Expected	$h_\theta(x)$
0	0	0	$g(-30) \approx 0$
1	0	0	$g(-10) \approx 0$
0	1	0	$g(-10) \approx 0$
1	1	1	$g(10) \approx 1$

Conclusion

With this choice of weights:

$$\Theta^{(1)} = \begin{bmatrix} -30 & 20 & 20 \end{bmatrix}$$

the neural network behaves exactly like an AND gate.

---

Implementing the OR Operator $(x_1 \lor x_2)$

The logical OR operator is true when:

• $x_1 = 1$, or
• $x_2 = 1$, or both

$x_1$	$x_2$	Result
0	0	0
1	0	1
0	1	1
1	1	1

We can implement OR using a different set of weights:

$$\Theta^{(1)} = \begin{bmatrix} -10 & 20 & 20 \end{bmatrix}$$

The hypothesis becomes:

$$h_\Theta(x) = g(-10 + 20x_1 + 20x_2)$$

Evaluating All Input Combinations

$x_1$	$x_2$	Expected	$h_\theta(x)$
0	0	0	$g(-10) \approx 0$
1	0	0	$g(10) \approx 1$
0	1	0	$g(10) \approx 1$
1	1	1	$g(30) \approx 1$

Conclusion

With this choice of weights:

$$\Theta^{(1)} = \begin{bmatrix} -10 & 20 & 20 \end{bmatrix}$$

the same neural network behaves exactly like an OR gate.

---

Implementing Not Gate ($\neg x_1$ )

graph LR

%% Input Layer
    subgraph Input Layer
        x0(((x0)))
        x1(((x1)))
    end

%% Hidden Layer 1
    subgraph Hidden Layer 1
        a1{a1}
    end

%% Output Layer
    subgraph Output Layer
        y(((hθx)))
    end

%% Connections: Input → Hidden 1
    x0 --> a1
    x1 --> a1
    
%% Connections: Hidden 2 → Output
    a1 --> y
 

The logical NOT operator is true when:

• $x_1 = 0$

and vice versa

$x_1$	Result
0	1
1	0

We can implement NOT using weights:

$$\Theta^{(1)} = \begin{bmatrix} 10 & -20 \end{bmatrix}$$

The hypothesis becomes:

$$h_\Theta(x) = g(10 - 20x_1)$$

$x_1$	Expected	$h_\theta(x)$
0	1	$g(10) \approx 1$
1	0	$g(-10) \approx 0$

---

Summary

We can use weight to simulate Logic gates with Neural networks

AND

$$\Theta^{(1)} = \begin{bmatrix} -30 & 20 & 20 \end{bmatrix}$$

OR

$$\Theta^{(1)} = \begin{bmatrix} -10 & 20 & 20 \end{bmatrix}$$

NOT

$$\Theta^{(1)} = \begin{bmatrix} 10 & -20 \end{bmatrix}$$

NOR = NOT OR

$$\Theta^{(1)} = \begin{bmatrix} 10 & -20 & -20 \end{bmatrix}$$