linear model and by()
On Nov 12, 2009, at 8:26 PM, Sam Albers wrote:
Hello R list, This is a question for anyone who has used the by() command. I would like to perform a regression on a data frame by several factors. Using by() I think that I have able to perform this using the following:
lm.r <- by(master, list(Sectionf=Sectionf, startd=startd), function(x) lm
(tot.c ~ starttime, data = x)) So that is, I would like to perform separate regressions for each level of Sectionf for each level of startd. Now I can get the coefficients and intercepts from all the fitted models. However, I am now unsure how to glean more information from the regressions. If I follow up with this command I get this:
summary(lm.r)
9-09-04.Length 9-09-04.Class 9-09-04.Mode 12 lm list 12 lm list 12 lm list 9-09-11.Length 9-09-11.Class 9-09-11.Mode 12 lm list 12 lm list 12 lm list Under normal circumstances, this usually ends up giving me more info about the fitted model, but some reason when using the by() command it doesn't.
Beacuase the results are a layer deeper than summary is describing; Try: lapply(lm.r, summary) (this theory tested with one of the standard datasets, Indometh: library(utils) data(Indometh) reglist <- by(Indometh, Indometh$Subject, function(x) lm(conc~time, data=x) ) > summary(reglist) Length Class Mode 1 12 lm list 4 12 lm list 2 12 lm list 5 12 lm list 6 12 lm list 3 12 lm list Whereas, without the assignment to reglist I got a series of expected lm output, so by "works".
Similarly, although I wasn't hopeful that this would work, when I tried to examine the residuals, I get these warning messages:
similarly: lapply(lm.r, plot)
plot(lm.r$fit,lm.r$res,xlab="Fitted",ylab="Residuals")
Error in plot.window(...) : need finite 'xlim' values In addition: Warning messages: 1: In min(x) : no non-missing arguments to min; returning Inf 2: In max(x) : no non-missing arguments to max; returning -Inf 3: In min(x) : no non-missing arguments to min; returning Inf 4: In max(x) : no non-missing arguments to max; returning -Inf So my question might be answered by someone saying that this isn't a good way to do this. Still, can anyone suggest some strategies that might help me out here? Specifically, how might I be able work with these regressions once I have fitted all the models. Obviously I need to examine the residuals, look at the model fit, etc. Any suggestions on what I might be doing wrong would be much appreciated. I am also including a subset of the data and the commands I used in case that is helpful. I am using Ubuntu 9.04 and R 2.8.1 Thanks in advance! Sam
Sorry to not use your data but it's not in a form that lends itself very well to quick testing. If you had included the input commands I might have tried it.
attach(master) Sectionf <- factor(Section) startd=as.Date(startdate, format="%d/%m/%Y")
lm.r <- by(master, list(Sectionf=Sectionf, startd=startd), function(x) lm
(tot.c ~ starttime, data = x)) Data: "Section","starttime","startdate","tot.c" "Upstream",0,04/09/09,0.17 "Upstream",0,04/09/09,0.19 "Upstream",0,04/09/09,0.14 "Middle",0,04/09/09,0.2 "Middle",0,04/09/09,0.13 "Middle",0,04/09/09,0.11 "Downstream",0,04/09/09,0.16 "Downstream",0,04/09/09,0.17 "Downstream",0,04/09/09,0.17 "Upstream",25,04/09/09,0.17 "Upstream",25,04/09/09,0.19 "Upstream",25,04/09/09,0.14 "Middle",25,04/09/09,0.2 "Middle",25,04/09/09,0.13 "Middle",25,04/09/09,0.11 "Downstream",25,04/09/09,0.16 "Downstream",25,04/09/09,0.17 "Downstream",25,04/09/09,0.17 "Upstream",50.06,04/09/09,0.17 "Upstream",50.06,04/09/09,0.19 "Upstream",50.06,04/09/09,0.14 "Middle",50.06,04/09/09,0.2 "Middle",50.06,04/09/09,0.13 "Middle",50.06,04/09/09,0.11 "Downstream",50.06,04/09/09,0.16 "Downstream",50.06,04/09/09,0.17 "Downstream",50.06,04/09/09,0.17 "Upstream",75.42,04/09/09,0.17 "Upstream",75.42,04/09/09,0.19 "Upstream",75.42,04/09/09,0.14 "Middle",75.42,04/09/09,0.2 "Middle",75.42,04/09/09,0.13 "Middle",75.42,04/09/09,0.11 "Downstream",75.42,04/09/09,0.16 "Downstream",75.42,04/09/09,0.17 "Downstream",75.42,04/09/09,0.17 "Upstream",100.14,04/09/09,0.17 "Upstream",100.14,04/09/09,0.19 "Upstream",100.14,04/09/09,0.14 "Middle",100.14,04/09/09,0.2 "Middle",100.14,04/09/09,0.13 "Middle",100.14,04/09/09,0.11 "Downstream",100.14,04/09/09,0.16 "Downstream",100.14,04/09/09,0.17 "Downstream",100.14,04/09/09,0.17 "Upstream",125.31,04/09/09,0.17 "Upstream",125.31,04/09/09,0.19 "Upstream",125.31,04/09/09,0.14 "Middle",125.31,04/09/09,0.2 "Middle",125.31,04/09/09,0.13 "Middle",125.31,04/09/09,0.11 "Downstream",125.31,04/09/09,0.16 "Downstream",125.31,04/09/09,0.17 "Downstream",125.31,04/09/09,0.17 "Upstream",150.29,04/09/09,0.17 "Upstream",150.29,04/09/09,0.19 "Upstream",150.29,04/09/09,0.14 "Middle",150.29,04/09/09,0.2 "Middle",150.29,04/09/09,0.13 "Middle",150.29,04/09/09,0.11 "Downstream",150.29,04/09/09,0.16 "Downstream",150.29,04/09/09,0.17 "Downstream",150.29,04/09/09,0.17 "Upstream",0,11/09/09,0.12 "Upstream",0,11/09/09,0.16 "Upstream",0,11/09/09,0.12 "Middle",0,11/09/09,0.08 "Middle",0,11/09/09,0.12 "Middle",0,11/09/09,0.1 "Downstream",0,11/09/09,0.11 "Downstream",0,11/09/09,0.13 "Downstream",0,11/09/09,0.13 "Upstream",25,11/09/09,0.12 "Upstream",25,11/09/09,0.16 "Upstream",25,11/09/09,0.12 "Middle",25,11/09/09,0.08 "Middle",25,11/09/09,0.12 "Middle",25,11/09/09,0.1 "Downstream",25,11/09/09,0.11 "Downstream",25,11/09/09,0.13 "Downstream",25,11/09/09,0.13 "Upstream",50,11/09/09,0.12 "Upstream",50,11/09/09,0.16 "Upstream",50,11/09/09,0.12 "Middle",50,11/09/09,0.08 "Middle",50,11/09/09,0.12 "Middle",50,11/09/09,0.1 "Downstream",50,11/09/09,0.11 "Downstream",50,11/09/09,0.13 "Downstream",50,11/09/09,0.13 "Upstream",75,11/09/09,0.12 "Upstream",75,11/09/09,0.16 "Upstream",75,11/09/09,0.12 "Middle",75,11/09/09,0.08 "Middle",75,11/09/09,0.12 "Middle",75,11/09/09,0.1 "Downstream",75,11/09/09,0.11 "Downstream",75,11/09/09,0.13 "Downstream",75,11/09/09,0.13 "Upstream",100,11/09/09,0.12 "Upstream",100,11/09/09,0.16 "Upstream",100,11/09/09,0.12 "Middle",100,11/09/09,0.08 "Middle",100,11/09/09,0.12 "Middle",100,11/09/09,0.1 "Downstream",100,11/09/09,0.11 "Downstream",100,11/09/09,0.13 "Downstream",100,11/09/09,0.13 "Upstream",125.04,11/09/09,0.12 "Upstream",125.04,11/09/09,0.16 "Upstream",125.04,11/09/09,0.12 "Middle",125.04,11/09/09,0.08 "Middle",125.04,11/09/09,0.12 "Middle",125.04,11/09/09,0.1 "Downstream",125.04,11/09/09,0.11 "Downstream",125.04,11/09/09,0.13 "Downstream",125.04,11/09/09,0.13 -- ***************************************************** Sam Albers Geography Program
David Winsemius, MD Heritage Laboratories West Hartford, CT