FM model
"TLDR: This article introduces the FM model, which is a statistical model used to deal with high-dimensional sparse data and feature intersection, and can capture the relationship between features. The FM model reduces the number of computational parameters and increases the capture of feature interactions by introducing low-rank factor vectors, paying special attention to second-order interactions. The model uses gradient descent to optimize the loss function and learns the interaction between users and items through factor vectors. In recommendation systems, FM models can be used to predict users' ratings or click probabilities of items by analyzing the interaction of features between users and items."
#FMmodel
The FM model is a statistical model that is suitable for processing high-dimensional sparse data and feature intersections, and can effectively capture the relationship between features.
Principle
FM models can be viewed as an extension of linear models that capture second-order interactions between features.
The formula is as follows
Among them, w_0 is the bias term, w_i is the weight of feature i, v_i is the factor vector of feature i, and x_i is the value of feature i.
Factor vector v_i is used to capture the interaction between feature i and other features, and because the feature vector dimension is usually small, it helps reduce model complexity and the risk of overfitting.
The inner product of v_i, v_j reflects the strength of the interaction between these two features.
Overall, the feature interaction part is added to the linear model, and the feature interaction uses the inner product of the factor vector as the weight.
Processing high-dimensional sparse data
By introducing low-rank factor vectors to process sparse data, the number of parameters is reduced. Even with feature vectors of thousands of dimensions, the factor inner product used to calculate feature interactions is very small.
Compare linear models
The linear model only captures the first-order effects of different features, while FM has more second-order feature interactions.
Training
Like the linear model, the gradient descent method is used to optimize the loss function.
Factor vector
The factor vector is obtained by random initialization and optimized by gradient descent.
Application in recommendation system
The FM model can predict the user's rating or click probability of an item by learning the interaction between user features and item features.
Dimension of factor vector
Determine through cross-validation methods to balance model complexity and generalization ability
Cold start
Difference from DIN and DIEN
FM pays more attention to the interaction between features, especially the second-order interaction, while DIN and DIEN focus more on modeling user interests.
Example
Suppose we have a user behavior dataset where each user may have interacted with thousands of items, but each user has actually only interacted with a handful of items. This means that the vast majority of elements in the user-item matrix are 0, which is a typical sparse data set.
In the FM model, we can assign a factor vector to each user and each item. When predicting a user's interest in a product, we only need to consider the non-zero features corresponding to the user and the product (i.e., the products that have interacted), and estimate the interaction between them by calculating the inner product of these two factor vectors. In this way, even if the total number of users and items is very large, the FM model can still learn and predict efficiently.