Fusion of high-level symbolic reasoning with lower level signal-based reasoning has attracted significant attention. We propose an architecture that integrates the high-level symbolic domain knowledge using a hierarchical planner with a lower level reinforcement learner that uses hybrid data (structured and unstructured). We introduce a novel neuro-symbolic system, Hybrid Deep ReRPeL that achieves the best of both worlds---the generalization ability of the planner with the effective learning ability of deep RL. Our results in two domains demonstrate the superiority of our approach in terms of sample efficiency as well as generalization to increased set of objects.