Latent factor models have achieved great success in personalized recommendations, but they are also notoriously difficult to explain. In this work, we introduce a regression tree to guide the learning of latent factors and the procedure of tree construction could be utilized as explanations for the resulting latent profiles. Specifically, we build trees on users and items respectively with user-generated reviews and associate a latent profile to each node on the tree to represent users or items. With the growth of the regression tree, the latent profiles are gradually refined under the regularization of the tree. As a result, we are able to track the evolution of latent profiles by looking into the path of each profile on regression trees, which can serve as the explanations for the resulting recommendations. Extensive experiments on two large collections of Amazon and Yelp reviews demonstrate the advantage of our model over several competitive baseline algorithms. Besides, our extensive user study also confirms the practical value of explainable recommendations generated by our model.