As-built deviation of BIM elements from the as-designed Building Information Model (BIM) remains a recurring concern in construction quality assurance. Classical Scan-vs-BIM methods rely on tripod-mounted Terrestrial Laser Scanning (TLS) paired with point-to-mesh deviation analysis, which is effective for aggregate dimensional compliance but does not yield discrete per-element verified-or-missing decisions under a unified geometric criterion. This study proposes a BIM-referenced verification framework that pairs lightweight LiDAR-Inertial-Visual Simultaneous Localization and Mapping (SLAM) capture with a 3D Intersection-over-Union (IoU) decision variable for BIM elements such as reinforced concrete columns. A handheld walkthrough yields a colorized SLAM point cloud, semantically enriched in parallel by lifting YOLOv12-prompted SAM-HQ instance masks from RGB to 3D, so that every captured point carries a BIM-element class label. The semantic point cloud is registered to the as-designed BIM through a CloudCompare coarse-to-fine workflow. The BIM is consumed in Industry Foundation Classes (IFC) form, from which each element is reduced to an oriented bounding box (OBB). The 3D IoU between the BIM OBB and the class-matched, BIM-OBB-bounded subset of semantic points, evaluated by uniform voxel occupancy, serves a dual role, as a spatial agreement metric and as a per-element decision variable: elements exceeding an initial threshold ( = 0.5, subject to future calibration) are confirmed verified, and per-category counts are reconciled against the BIM schedule, producing an auditable BIM-indexed verification report. The framework is demonstrated on a 4th-floor research laboratory of the Civil Engineering Research Building at National Taiwan University (NTU), where six in-situ reinforced concrete columns were verified end-to-end and yielded 3D IoU values from 0.7579 to 0.9840, all passing the τ=0.5 threshold.