The ability to monitor web and network performance becomes crucial to understand the reasons behind any service degradation. Such monitoring is also helpful to understand the relationship between the quality of experience of end users and the underlying network state. Many troubleshooting tools have been proposed recently. They mainly consist of conducting active network measurements from within the browser. However, most of these tools either lack accuracy, or perform measurements to a limited set of servers. They are also known to introduce non negligible overhead onto the network. The objective of this paper is to propose a new approach based on passive measurements freely available from within the web browser, and couple them to deep learning models to estimate the latency and bandwidth metrics of the underlying network without injecting any additional measurement traffic. We develop and implement our approach, and compare its estimation accuracy with the most known web-based network measurement techniques available nowadays. We follow a controlled experimental approach to derive our inference models. Results of our study show that our approach can give a very good accuracy compared to others, its accuracy is even higher than most standard techniques, and very close to the rest.