Tags:Combinatorial Problems, Educational Timetabling Problems, empirical study, evaluation method, heuristic based evaluation, heuristic based evaluation method, heuristic based method, Heuristic Evaluation, heuristic method, information visualization, Information Visualization Evaluation, interactive visualization, NP-Hard, problem domain, rater confidence, timetabling problem, User Interface and visualization evaluation
Abstract:
The conception, and usage, of methods designed to evaluate information visualizations is a challenge that goes along with the development of these visualizations. In the scientific literature there is a myriad of proposals for such methods. However, none of them was able to pacify the field or establish itself as a "de facto" standard, due to difficulties like: (a) the complexity of its usage; (b) high financial and time costs; and (c) the need of a large number of raters to guarantee the reliability of the results. One way to circumvent such adversities is the usage of Heuristic-Based Evaluations thanks to its simplicity, low cost and speed of application and the quality of reached results. This article intends to conduct an empirical methodological study about the use of three heuristic-based methods (Zuk bet al., Forsell & Johansson and Wall et al.) for evaluation of visualizations in the context of Educational Timetabling Problems. Five different visualizations, extracted from literature, were evaluated using the original methods and versions modifiedby the authors (where an importance factor were assigned to each statement being evaluated, as well as the level of confidence of the rater in his/her appointment) in order to improve their efficiency when measuring the quality of visualizations. The experimental results demonstrated that for the two first heuristics, only the modification on the importance of the statements proved to be (statistically) relevant. For the third one, both factors did not induce different results.
An Empirical Methodological Study of Evaluation Methods Applied to Educational Timetabling Visualizations