NearSEE: Near-Storage Expert Execution for Edge Inference of Mixture-of-Experts LLMs

EasyChair Preprint 16034

Abstract

Mixture-of-Experts (MoE) language models reduce per-token computation by routing each token to a small subset of expert FFNs. However, sparse activation does not reduce the total expert-parameter footprint since the selected expert weights must still be accessed during inference. On edge-class systems with constrained host memory, autoregressive decoding repeatedly fetches these weights through SSD-backed offload, shifting the bottleneck from arithmetic throughput to data movement across the storage–memory hierarchy. This paper presents NearSEE, a computational-storage-based Near-Storage Expert Execution model for memory-constrained MoE inference. We implement NearSEE on Samsung SmartSSD devices for FP32 Mixtral-8x7B-Instruct and evaluate it under 32 GB and 64 GB host-memory budgets. Compared with the CPU/disk offload baseline, NearSEE-1CSD improves decode throughput by up to 2.14× under 32 GB and 1.84× under 64 GB, while NearSEE- 2CSD further improves decode throughput by up to 3.78× and 3.09×, respectively. Under the 64 GB configuration, NearSEE reduces measured host-and-CSD energy by up to 52% relative to the CPU/disk offload baseline. These results show that nearstorage expert execution directly targets the decode-stage offload bottleneck in memory-constrained MoE inference.

Keyphrases: Computational Storage, LLM Inference, Mixture-of-Experts (MoE)

@booklet{EasyChair:16034,
  author    = {Prapti Panigrahi and Wahid Uz Zaman and Mahmut Kandemir and Vijaykrishnan Narayanan},
  title     = {NearSEE: Near-Storage Expert Execution for Edge Inference of Mixture-of-Experts LLMs},
  howpublished = {EasyChair Preprint 16034},
  year      = {EasyChair, 2026}}