In trying to develop a visual statistical system, it is frustrating to deal with the many limitations of common statistical tests–normality, equal variance, or equal sample size conditions. These conditions make the tests brittle; that is, they only work on a subset of all interesting data sets, and which subset is difficult to determine. How should this brittleness be accounted for in a statistical system? One possibility is to run additional tests and to warn the user if some condition is not met. In practice warnings would occur frequently and it would be up to the user’s judgment to decide if they were meaningful. To overcome this issue, a statistical system could be imagined that would attempt to mimic the decisions of an expert statistician. It would look at a battery of test results and automatically determine what additional tests could be run depending on what conditions were met. How would one train such a system? And, I think, more importantly, how would one evaluate the uncertainty or error in the results of such a complicated system?

I recently came across the idea of robust statistics, which originated with Tukey and associated statisticians in the middle of the last century. This concept appears here in the introductory chapter to Mosteller and Tukey’s *Data Analysis and Regression*. The goal is to find statistical tests that work on a broad range of data distributions. From a system implementation standpoint, this approach is much preferable to user input or a complicated expert system. However, I have not seen these techniques used in any of my Stats classes. Have they been superseded by Bayesian or bootstrapping approaches?

The other principle topic in this first chapter is “vague concepts”. The authors give the example of standard deviation, which is a very specific method for measuring the spread of a distribution. However, to evaluate our use of the standard deviation we must be able to place it in context of all other possible measures of spread. This meta-level or “vague concept” is lost in many introductions to statistics.