Sliceplorer evaluation results
To perform a task-based evaluation, we investigated the different techniques in two different ways. First, we used a qualitative result inspection approach. We (the authors) iteratively analyzed the techniques with different datasets and summarized our discussion and analysis on a four point scale: “None” means that it is not possible to perform the task with the technique, “partly” means that it requires major interaction with the view to accomplish the task, “mostly” means that one can accomplish the task with little interaction, and “fully” means that this task is directly addressed by the technique.
Second, in order to get a more objective judgement we also asked four visualization experts familiar with examining multi-dimensional spaces like parameter space exploration to examine the eight datasets with different techniques and rate how well each task can be accomplished with each technique on the same scale. We averaged these results and show them along with the results of our qualitative result inspection in the table above.
The following table shows the results of evaluating how easy it was to perform each task with each technique on a 4-point scale where the colors represent:
- 0none
- 1partially
- 2mostly
- 3fully
| Task | Task description for discrete data items from Amar, Eagan, and Stasko | Our adaption to continuous scalar functions | Gerber et al. | Contour tree | Topological spines | HyperSlice | 1D slices |
|---|---|---|---|---|---|---|---|
| Retrieve value | "Given a set of specific cases, find attributes of those cases" | Given an x, what is the function value? | 1 | 1 | 1 | 3 | 3 |
| Filter | "Given some concrete conditions attribute values, find data cases satisfying those conditions." | For what parameter values is the function equal or over x? | 2 | 2 | 3 | 2 | 3 |
| Compute derived value | "Given a set of data cases, compute an aggregate numeric representation of those data cases" | Summary statistics: variance, mean, SA | 1 | 0 | 1 | 0 | 3 |
| Find extremum | "Find data cases possessing an extreme value of an attribute over its range within the data set" | Find local/global min/max | 3 | 3 | 3 | 0 | 3 |
| Determine range | "Given a set of data cases and an attribute of interest, find the span of values within the set" | What is the range of possible outputs? | 3 | 3 | 1 | 1 | 3 |
| Characterize Distribution | "Given a set of data cases and a quantitative attribute of interest, characterize the distribution of that attribute’s values over the set" | What types of shapes do the manifolds have | 0 | 0 | 0 | 1 | 3 |
| Find anomalies | "Identify any anomalies within a given set of data cases with respect to a given relationship or expectation, e.g. statistical outliers" | Do areas of the manifold have shapes unlike any others | 0 | 0 | 0 | 1 | 3 |
| Cluster | "Given a set of data cases, find clusters of similar attribute values" | Areas of the manifold have similar shapes | 1 | 0 | 0 | 0 | 3 |
| Correlate | "Given a set of data cases and two attributes, determine useful relationships between the values of those attributes" | 1D vs 2D relationships | 0 | 0 | 0 | 3 | 1 |
| Task | Task description for discrete data items from Amar, Eagan, and Stasko | Our adaption to continuous scalar functions | Gerber et al. | Contour tree | Topological spines | HyperSlice | 1D slices |
|---|---|---|---|---|---|---|---|
| Retrieve value | "Given a set of specific cases, find attributes of those cases" | Given an x, what is the function value? | 0 | 0 | 0 | 2 | 2 |
| Filter | "Given some concrete conditions attribute values, find data cases satisfying those conditions." | For what parameter values is the function equal or over x? | 0 | 1 | 0 | 1 | 2 |
| Compute derived value | "Given a set of data cases, compute an aggregate numeric representation of those data cases" | Summary statistics: variance, mean, SA | 1 | 0 | 0 | 2 | 2 |
| Find extremum | "Find data cases possessing an extreme value of an attribute over its range within the data set" | Find local/global min/max | 2 | 2 | 1 | 1 | 2 |
| Determine range | "Given a set of data cases and an attribute of interest, find the span of values within the set" | What is the range of possible outputs? | 1 | 1 | 0 | 2 | 2 |
| Characterize Distribution | "Given a set of data cases and a quantitative attribute of interest, characterize the distribution of that attribute’s values over the set" | What types of shapes do the manifolds have | 2 | 1 | 1 | 1 | 2 |
| Find anomalies | "Identify any anomalies within a given set of data cases with respect to a given relationship or expectation, e.g. statistical outliers" | Do areas of the manifold have shapes unlike any others | 1 | 0 | 1 | 1 | 2 |
| Cluster | "Given a set of data cases, find clusters of similar attribute values" | Areas of the manifold have similar shapes | 0 | 0 | 0 | 0 | 2 |
| Correlate | "Given a set of data cases and two attributes, determine useful relationships between the values of those attributes" | 1D vs 2D relationships | 0 | 0 | 0 | 2 | 1 |
Techniques
- Morse-Smale complex visualization by Gerber et al.
- Contour tree
- Topological spines
- HyperSlice
- Our 1D slice method
Datasets
Not all datasets could be rendered with all techniques due to software errors.