Movie Rating Prediction on the MovieLens 32M Dataset Using Random Forests

Abstract

The challenges of user viewing decision-making in the global film industry and the insufficient accuracy of existing rating prediction models are addressed in this study. The investigation uses the MovieLens 32M dataset to explore movie rating prediction. Initially, stratified sampling was conducted on the dataset utilising Python scripts. Subsequent to the amalgamation of data, the implementation of a differentiated approach to the management of missing values, the purification of outliers, and the transformation of temporal features, the dataset was segmented into training and test sets at an 8:2 ratio, whilst ensuring the maintenance of consistent rating distributions. Consequently, a Random Forest regression model was constructed. GridSearchCV was employed to optimize hyperparameters such as the number of trees and maximum depth. The final model demonstrated excellent performance on the test set, with a coefficient of determination (R2) of 0.8776, a mean squared error (MSE) of 0.1399, and a mean absolute error (MAE) of 0.1697. This approach demonstrated a substantial improvement in performance when compared to established benchmark models such as linear regression and support vector machines. It effectively captured the nonlinear relationships present in the rating data, thus showcasing its ability to handle complex data structures. Feature importance analysis revealed that the user's average historical rating (importance score 0.7921) and the movie's average historical rating (0.0680) are the core factors influencing rating predictions, while the rating standard deviation and user ID have weaker impacts. The findings of this research provide quantitative evidence for the optimisation of scheduling strategies for film producers, the enhancement of personalised recommendation systems, and the evaluation of film value on content platforms.