With the rapid growth of multimedia data and the diversity of the available image contents, it becomes necessary to develop advanced machine learning algorithms for the purpose of categorizing and recognizing images. Hierarchical clustering methods have shown promising results in computer vision applications. In this paper, we present a new unsupervised image categorization technique in which we cluster images using an agglomerative hierarchical procedure and a dissimilarity metric is derived based on smoothed Dirichlet (SD) distribution. We propose a mixture of SD distributions and a maximum-likelihood learning framework, from which we derive a Kulback-Leibler divergence between two SD mixture models. Experiments on challenging images dataset that contains different indoor and outdoor places reveal the importance of the hierarchical clustering when categorizing images. The conducted tests prove the robustness of the proposed image categorization approach as compared to the other related-works.