This paper contributes improvements on both the defect of dot product and the imbalance of the datasets in matrix factorization. Above all, matrix factorization is still the most widely used technology in the recommendation system. However, its dot product does not satisfy the triangle inequality, which restricts the improvement of its recommendation effect. We take inspiration from the distance factor of metric learning, and convert the determinants of user-item relevance from the size of the dot product to the distance of the metric factorization. Furthermore, the number of positive examples is much smaller than the negatives in most datasets. Such an unbalanced scenario will affect the accuracy of recommendations. Inspired by the positive semidefinite matrix of the popular Mahalanobis distance in the field of metric learning, we have fully considered the interaction information between users and items and propose the concept of all-weighted matrix. Finally, the combination of the two improved techniques proposed the All-Weighted Metric Factorization (AWMF) method, which is applied to the personalized ranking task. Extensive experimental results on three real-world datasets demonstrate that our method outperforms the competitive baselines on several evaluation metrics.