In the past decades, various approaches have been proposed to address the time series prediction problem, among which nonlinear autoregressive exogenous (NARX) models achieve great progresses in one-step time prediction. Although NARX models are capable of capturing long-term dependence of the time series data, the impact of associated attributes lacks enough attention. To cope with this issue, in this paper we propose a Multi-Attention algorithm based Recurrent Neural Network (RNN) to perform multivariate time series forecasting. In the first stage, given a raw multivariate time series segment, we obtain both relevant encoder hidden state and encoder hidden state of the associated attribute by employing input-attention and self-attention respectively. In the second stage, we use temporal-convolution-attention neural network to process the encoder hidden states and capture long-range temporal patterns. Finally, extensive empirical studies tested with four real world datasets (NASDAQ100, SML2010, Gas Sensor Array Temperature Modulation and Air Quality) demonstrate the effectiveness and robustness of our proposed approach.