Recently, Lin and Pryadko presented the quantum two-block group algebra codes, a generalization of bicycle codes obtained from Cayley graphs of non-Abelian groups. We notice that their construction is naturally suitable to obtain a quantum equivalent of the well-known classical Margulis code. In this paper, we first present an alternative description of the two-block group algebra codes using the left-right Cayley complex; then, we show how to modify the construction of Margulis to get a two-block algebra code. Finally, we construct several quantum Margulis codes and evaluate their performance with numerical simulations.