Meaning of /, :, and %in% in lmer

The short answer is that (1|A/B) is expanded to (1|A) + (1|A:B) so you
can choose whatever form makes sense to you.

Thanks, that was what I needed to hear.

There are different circumstances where a notation like (1|A/B) would
be used.  Some are reasonable choices and some are artifacts of
artificial ways of assigning labels to factor levels.  Rather than my
trying to guess what kind of application you have in mind, could you
describe a situation where you would want to fit an lmer model with
terms like that?

It's a virology experiment. We have two ancestral strains. From each of those 
we have derived several new strains, and then have made multiple fitness 
measurements on the new strains. We want to know whether the ancestral strain 
has an effect on the fitness of the derived strains. The model I'm using for 
that is
	fitness ~ ancestor + (1|ancestor:strain),
because strains are nested within ancestors. If I were using
	fitness ~ ancestor + (1|ancestor/strain),
then ancestor would get both a fixed and a random effect, which doesn't make 
sense.

I have a second question, related to the hypothesis testing of whether the 
fixed ancestor effect is significant. I've read all the threads about why it 
is problematic to do an F test to calculate a p value, and that it is better 
to do markov-chain monte carlo. My question is: Is there a proper reference I 
can cite to substantiate the claim that the standard (i.e., SAS) way of 
calculating significance in this case is problematic, or do I have to refer 
to the mailing list archive?

Thanks a lot,
  Claus

Claus Wilke
Section of Integrative Biology 
 and Center for Computational Biology and Bioinformatics 
University of Texas at Austin
1 University Station C0930
Austin, TX 78712
cwilke at mail.utexas.edu
512 471 6028