Recommender Systems 🍿

A recommender system predicts how users would rate items they have not yet rated.

Example:

• $n_u$ = number of users 👤
• $n_m$ = number of movies 📺

Users rate some movies (1–5 stars), but many ratings are missing.
Goal: predict the missing ratings.

---

Content-Based Recommendation 🎬

The system recommends items whose content matches the user's preferences.

Predict ratings based on movie features ($x^{(i)}$) and user preferences ($\theta^{(j)}$).

1. Feature vector $(x^{(i)})$ 📺

Each movie has features describing its content.

Each movie is represented by a feature vector:

Example:

• $x_1$ = romance
• $x_2$ = action
• $x_3$ = genre
• $x_4$ = actors
• $x_5$ = etc.

$$x^{(i)} = \begin{bmatrix} 1 \\ x_1 \\ x_2 \end{bmatrix}$$

The first element is the bias feature:

$$x_0 = 1$$

If we have $n$ features, then:

$$x^{(i)} \in \mathbb{R}^{n+1}$$

Example movie features:

Movie	Romance ($x_1$)	Action ($x_2$)
Love at Last	0.9	0
Romance Forever	1.0	0.01
Swords vs Karate	0	0.9
Nonstop Car Chases	0.1	1.0
Cute Puppies of Love	0.99	0

---

2. User Preference Model $(\theta^{(j)})$ 👤

Each user $j$ has their own parameter vector:

It represents the user's preferences for features.

$$\theta^{(j)} \in \mathbb{R}^{n+1}$$

Example:

• Alice likes romance → high weight on $x_1$
• Bob likes action → high weight on $x_2$

---

Rating Prediction $(\hat{y}^{(i,j)})$

Predicted rating of movie $i$ for user $j$

$$\hat{y}^{(i,j)} = (\theta^{(j)})^T x^{(i)}$$

$y^{(i,j)}$ = actual rating by user $j$ on movie $i$ (if defined)

where:

• $\theta^{(j)}$ = user preference vector
• $x^{(i)}$ = movie feature vector

This is just linear regression.

Example

Movie: Cute Puppies of Love

Feature vector:

$$x = \begin{bmatrix} 1 \\ 0.99 \\ 0 \end{bmatrix}$$

Alice's preference vector:

$$\theta^{(1)} = \begin{bmatrix} 0 \\ 5 \\ 0 \end{bmatrix}$$

Prediction $\hat{y}^{(i,j)}$:

$$\hat{y} = (\theta^{(1)})^T x$$

Result:

$$\hat{y} = 5 \times 0.99 = 4.95$$

Predicted rating ≈ 5 stars.

---

Rating Prediction for single User

Let

• $i$ = movie/ item index
• $j$ = user index

Where:

• $m^{(j)}$ = number of movies rated by user $j$

Rating on movie $i$ by user $j$:

If user have rated movie $i$ $r(i,j)=1$

If user have rated movie $i$ $r(i,j)=0$

Actual Rating of movie $i$ given by user $j$ (if defined)

$y^{(i,j)} = 3$

Parameter vector for user $j$

kind of movies liked by user $j$

$\theta^{(j)}$

Predicted Rating

For use $j$, movie $i$ predicted rating

$$\hat{y}^{(i,j)} = (\theta^{(j)})^T x^{(i)}$$

💰 Cost Function

We want predictions close to actual ratings.

Predicted Rating

$$\hat{y}^{(i,j)} = (\theta^{(j)})^T x^{(i)}$$

Loss can be calculated as $Loss = Predicted \ Rating - Actual\ Rating$

Which is equals to

$$\hat{y}^{(i,j)} = (\theta^{(j)})^T x^{(i)} - y^{(i,j)}$$

For user $j$, minimize squared error:

$$\min_x \frac{1}{2} \sum_{(i,j):r(i,j)=1} \left( (\theta^{(j)})^T x^{(i)} - y^{(i,j)} \right)^2+ \frac{\lambda}{2} \sum_k (x_k^{(i)})^2$$

Goal:

• minimize prediction error
• regularize features

Cost function

$$J(\theta^{(j)}) = \frac{1}{2} \sum_{i:r(i,j)=1} \left((\theta^{(j)})^T x^{(i)} - y^{(i,j)}\right)^2+ \frac{\lambda}{2} \sum_{k=1}^{n}(\theta_k^{(j)})^2$$

First Term = Mean Square Error

Second term = regularization (prevents overfitting).

All Users

We learn parameters for all users:

$$J(\theta^{(1)},...,\theta^{(n_u)}) = \frac{1}{2} \sum_{j=1}^{n_u} \sum_{i:r(i,j)=1} \left((\theta^{(j)})^T x^{(i)} - y^{(i,j)}\right)^2+ \frac{\lambda}{2} \sum_{j=1}^{n_u}\sum_{k=1}^{n}(\theta_k^{(j)})^2$$

Minimize this to learn all user preferences.

---

Cost Optimization

Parameters are learned using:

• Gradient Descent
• or advanced optimizers (LBFGS, Conjugate Gradient)

Updates look similar to linear regression updates.

---

Limitation

Need to define features for items (movies).

Requires hand-crafted features for items.

In many real systems:

• features are missing
• hard to define

This leads to the next method: Collaborative Filtering described in the next post