Split-plot Design

I like this - a good robust defence!  Let the debate go on.

Designs that are in some sense balanced are, as far as I can judge,  
still the staple of agricultural trials.  The sorts of issues that  
arise in time series modeling must also be increasingly important.   
The thinking behind those designs is important, and ought to penetrate  
more widely.

That general style of design is not however limited to agricultural  
experimentation.  It is used widely, with different wrinkles, in  
psychology in industrial experimentation, in medicine, in laboratory  
experimentation, and with internet-based experimentation a new major  
area of application.  It ought to be used much more widely outside of  
agriculture.  Agriculture has been somewhat unique in having  
professional statisticians working alongside agricultural scientists  
for many decades now.

The wrinkles can be important.  Fisher famously commented on the  
frequent need to ask Nature more than one question at a time:
"No aphorism is more frequently repeated in connection with field  
trials, than that we must ask Nature few questions, or, ideally, one  
question, at a time. The writer is convinced that this view is wholly  
mistaken. Nature, he suggests, will best respond to a logical and  
carefully thought out questionnaire"

For each fixed effect and interaction, it is however necessary to be  
clear what is the relevant error term.  Some time ago, someone posted  
a question about the analysis of a psychology experiment where the  
number of possible error terms was very large, where some pooling was  
desirable, and where it was not at all clear what terms in the anova  
table to pool, and where d.f, were so small that the mean squares did  
not provide much clue.  (I looked for the email exchange, but could  
not immediately find it.) That is not a good design.  There may be  
much more potential for this sort of difficultly in psychology as  
compared to agriculture.  In agriculture the sites/blocks/plots/ 
subplots hierarchy is the order of the day, albeit often crossed with  
seasons to ensure that the analysis is not totally obvious.

The data sets are often larger than formerly. (But not always, again  
note some psychology experiments. Or they may not be large in the  
sense of allowing many degrees of freedom for error) Large datasets  
readily arise when, as for the internet-based experimentation,  
randomization can be done and data collected automatically.  (Some  
advertisers are randomizing their pop-up ads, to see which gives the  
best response.) With largish degrees of freedom for error, it is no  
longer necessary to worry about exact balance.  I consider Doug that  
you are too tough on the textbooks.  Maybe they ought to be branching  
out from agricultural experimentation more than they do; that is as  
much as I'd want to say.  There are any number of examples from  
psychology, some of them very interesting, in the psychology books on  
experimental design.  (Data from published papers ought nowadays as a  
matter of course go into web-based archives - aside from other  
considerations this would provide useful teaching resources. In some  
areas, this is already happening.)

Degrees of freedom make sense in crossed designs also; it is the F- 
statistics for SEs for fixed effects that can be problematic.  It may  
happen that one or two sources of error (maybe treatments by years)  
will dominate to such an extent that other sources can pretty much be  
ignored.  The conceptual simplification is worth having; its utility  
may not be evident from a general multi-level modeling perspective.

Maybe one does not want statisticians to be too dyed in the wool  
agricultural.  Still, a bit of that thinking goes a long way, not  
least among social scientists.  The arguments in Rosenbaum's  
"Observational Data" are much easier to follow if one comes to them  
with some knowledge and practical experience of old-fashioned  
experimental design.  That seems to me a good indication of the quite  
fundamental role of those ideas, even if one will never do a  
randomized experiment.

I'd have every novice statistician do apprenticeship's that include  
experience in horticultural science (they do not have a long tradition  
of working alongside professional statisticians), medicine and  
epidemiology, business statistics, and somewhere between health social  
science and psychology.  It is unfortunate that I did not myself have  
this broad training, which may go some way to explaining why I am not  
in complete agreement with Doug's sentiments!!

This is not to defend, in any large variety of places and  
circumstances, inference that relies on degrees of freedom.  Not that  
my point relate directly to degrees of freedom.  On degrees of  
freedom, I judge them to have a somewhat wider usefulness than Doug  
will allow.  It would be nice if lmer() were to provide Kenward &  
Rogers style degrees of freedom ( as the commercial ASReml-R software  
does), but Doug is right to press the dangers of giving information  
that can for very unbalanced designs be misleading.  Given a choice  
between mcmcsamp() and degrees of freedom, maybe I would choose  
mcmcsamp().

There's enlightening discussion of the opportunities that internet- 
based business offers for automated data collection and for  
experimentation on website visitors, in Ian Ayres "SuperCrunchers. Why  
thinking by numbers is the new way to be smart" (Bantam).   
Fortunately, the hype of the title pretty much goes once on gets into  
the text.

John Maindonald             email: john.maindonald at anu.edu.au
phone : +61 2 (6125)3473    fax  : +61 2(6125)5549
Centre for Mathematics & Its Applications, Room 1194,
John Dedman Mathematical Sciences Building (Building 27)
Australian National University, Canberra ACT 0200.

Split-plot Design

Thread (12 messages)