Security code smells are receiving increasing attention in the domain of Android apps development. They serve as coding guidelines aimed at identifying vulnerabilities originating from the application’s source code. Numerous related tools have been proposed to align with DevSecOps guidelines for integration into the Android apps development process. However, there’s a lack of comparable effort in creating benchmarks for open-source apps, making the thorough evaluation of these tools challenging, often requiring manual effort and time-consuming processes. In this paper, we propose PrivBench, an evolving benchmark that captures vulnerabilities related to Android ecosystem. For instance, PrivBench focuses on Privilege Escalation (PE) vulnerabilities. It incorporates multiple code patterns for each vulnerability, demonstrating their potential existence within the app source code. To showcase the significance of PrivBench, we used it for evaluating two well-known tools used to identify Android security code smells. This evaluation allowed us to present their performance in detecting the vulnerabilities within their scopes. We believe that our benchmark can be useful for advancing the capabilities of state-of-the-art tools, enhancing their effectiveness in vulnerability detection, and increasing developers awareness to evade privilege escalation vulnerabilities.