Smart contracts, which serve as the backbone of decentralized applications (dApps), self-execute key functions on blockchain platforms. They currently manage assets valued in the trillions of dollars in the cryptocurrency space. However, their immutability after deployment makes them particularly vulnerable to exploitation if any weaknesses are present. Identifying and addressing these vulnerabilities is critical to avoiding significant financial and reputational damage. One of the commonly used automated methods for fast and efficient vulnerability detection is Fuzzing. However, both traditional fuzzing techniques and also those based on machine learning encounter challenges, such as selecting ineffective transaction sequences, either by generating them randomly or pre-generating them before running the fuzzer. This leads to a failure in updating transaction sequences based on the dynamical smart contract states during fuzzing process. Additionally, some methods mutate test cases and store them in a pool, which becomes problematic when physical memory is no unlimited. In this paper, we present MADFuzz, a Multi-Agent Deep Reinforcement Learning (DRL)-based approach designed to address the challenges of smart contract fuzzing. To improve the selection of effective transaction sequences, we develop agents that dynamically generate optimal functions and arguments based on the current state of the smart contract. By utilizing DRL, our approach generates transaction sequences in real-time without the need for memory storage, efficiently overcoming the limitations of previous methods. Finally, we conduct experiments to compare MADFuzz with existing state-of-the-art techniques, and the results demonstrate that our approach significantly outperforms the competition.