| ||||
| ||||
![]() Title:On Writing SMT-LIB Scripts: Metrics and a New Dataset Conference:SMT 2025 Tags:dataset, evolution, novices and SMT-LIB Abstract: Satisfiability Modulo Theory (SMT) checking is concerned with checking the satisfiability of first-order formulas with respect to some background theories. The SMT-LIB format is a standardized language for scripts expressing SMT problems. Popular datasets of SMT-LIB scripts have been collected for benchmarking SMT solvers. Rather than focusing on evaluating SMT solvers, our work focuses on exploring how novice users write SMT-LIB scripts. We present a dataset of SMT-LIB scripts with fine-grained editing paths. The dataset consists of 2,415 editing paths with a total of 18,133 SMT-LIB scripts. All scripts were collected from a web-based interface for the Z3 SMT solver in educational settings. We analyze the dataset in terms of sizes of scripts, errors users make, similarities of consecutive scripts, editing distances, and edit steps required to fix errors. We make the dataset and the code for computing our metrics available for future research on language design, tool support, and teaching materials. On Writing SMT-LIB Scripts: Metrics and a New Dataset ![]() On Writing SMT-LIB Scripts: Metrics and a New Dataset | ||||
Copyright © 2002 – 2025 EasyChair |