In-Situ Data Reduction via Incoherent Sensing

EasyChair Preprint no. 1314

Abstract

We present a framework for in-situ processing of large-scale simulation data that performs a universal data reduction. Instead of direct compression of the data, we propose a different approach that can be benefit from compressed sensing (CS) theory. Unlike the direct data compression techniques where the accuracy of recovery is fixed, the proposed framework enables more accurate recovery (after in situ data reduction), with using better sparse representations, that can be learned from and optimized for the simulation data. Moreover, we discuss the practical case when the assumption of sparsity doesn’t hold, the optimization-based recovery algorithm is able to recover the most important elements in the data (characterized by the best k term approximation), despite significant reduction in the data. We provide theoretical arguments from CS theory and demonstrate experimentally the error behavior exhibited by the proposed approach compared by the best k-term approximation. These arguments, together with our experiments, support the unique feature of the proposed in-situ data reduction: the accuracy of the recovery algorithm can be improved after data reduction by learning better representations for simulation data. The proposed approach provides opportunities for developing new data reduction mechanisms in high performance computing and simulation environments.

Keyphrases: compressed sensing, In-situ data reduction, volume rendering

@Booklet{EasyChair:1314,
  author = {Kai Zhang and Alireza Entezari},
  title = {In-Situ Data Reduction via Incoherent Sensing},
  howpublished = {EasyChair Preprint no. 1314},

  year = {EasyChair, 2019}}