The sparsity of user check-in trajectory data is a great challenge faced by point of interest(POI) recommendation. To alleviate the data sparsity, existing research often utilizes the geographic and time information in check-in trajectory data to discover the hidden spatio-temporal relations. However, existing models only consider the spatio-temporal relationship between locations, ignoring that between POI categories. To further reduce the negative impact of data sparsity, motivated by the method to integrate the spatio-temporal relationship by attention mechanism in LSTPM, this paper proposes a POI recommendation model based on double-level spatio-temporal relationship in locations and categories-POI2TS. POI2TS integrates the spatio-temporal relationship between locations and that between categories through attention mechanism to more accurately capture users' preferences. The test results on the NYC and TKY datasets show that POI2TS is more accurate compared with the state-of-the-art models, which verifies that integrating the spatio-temporal relationship between locations and that between categories can effectively improve POI recommendation models.