An increasing amount of Internet traffic has its content encrypted. We address the question of whether it is possible to predict the activities taking place over an encrypted channel, in particular Microsoft's Remote Desktop Protocol. We show that the presence of five typical activities can be detected with precision greater than 97% and recall greater than 94% in 30-second traces. We also show that the design of the protocol exposes fine-grained actions such as keystrokes and mouse movements which may be leveraged to reveal properties such as lengths of passwords.