Mirror detection is crucial for avoiding collisions and misrecognition of reflections in real-world scenes. Existing methods often rely on assumptions, such as semantic similarity between objects and their reflections or contextual correlations, which may not hold in cluttered environments, limiting generalization. Recent advances in foundation models like the Segment Anything Model (SAM) have boosted general-purpose segmentation, yet SAM struggles with mirrors due to prompt ambiguity and reflective complexity. Moreover, Prior methods neither exploit SAM nor generalize well to diverse scenes. To address this, we adapt SAM for mirror segmentation by introducing a Frequency-Chirality Adapter (FCA), which encodes frequency-domain textures and visual chirality to enhance mirror-specific perception. Additionally, we design a Prior-Aware Localization (PAL) module to provide automatic prompt guidance from effective semantic priors, eliminating the need for manual inputs. Experiments on PMD, MSD, and RGBD-Mirror datasets show that our method consistently outperforms previous state-of-the-art approaches, significantly enhancing SAM’s performance on mirror segmentation.