stats lm() function

Hi, 

Im using the lm() function where the formula is quite big (300 arguments) and the data is a frame of 3000 values. 

This is running in a loop where in each step the formula is reduced by one argument, and the lm command is called again (to check which arguments are useful) . 

This takes 1-2 minutes. 
Is there a way to speed this up? 
i checked the code of the lm function and its seems that its preparing the data and then calls lm.Fit(). i thought about just doing this praparing stuff first and only call lm.fit() 300 times. 
	[[alternative HTML version deleted]]

I think you will find that many readers of this list would rather try  to 
dissuade you from this misguided strategy. You are unlikely to get  to a 
sensible solution in using step-down procedures with this sort of 
situation (large number of predictors with modest size of data).

-- 
David Winsemius

On Mar 12, 2009, at 1:59 PM, Paul Hermes wrote:

Hi,

Im using the lm() function where the formula is quite big (300 
arguments) and the data is a frame of 3000 values.

This is running in a loop where in each step the formula is reduced  by 
one argument, and the lm command is called again (to check which 
arguments are useful) .

This takes 1-2 minutes.
Is there a way to speed this up?
i checked the code of the lm function and its seems that its  preparing 
the data and then calls lm.Fit(). i thought about just  doing this 
praparing stuff first and only call lm.fit() 300 times.
[[alternative HTML version deleted]]

David Winsemius, MD
Heritage Laboratories
West Hartford, CT

ok,
i think i have to be more precise of what we are doing.
first thing: this code is not from me, and Im new to R (and never touched 
anything like this)
Im just the lucky guy who has to maintain this crap :)
this call to the lm function is part of a code wich is used to predict the 
marketvalues from a bunch of our products.
as 'target' function it gets the past marketvalues we have in our 
database.(this is what goes into the 'data' parameter into the lm
function)

then we have allot other prices and enviromental data (like similar 
products, stock sizes, seasonal informations, .... )
with this, the big formula is created (y ~ x1 + x2 + x3 + x4 + x5 .......
+ 
x300)


all this goes into the lm call. then the result is somehow anaylsed to 
figure out wich input data-set had the least influence (or similaryti ) to 
the past marketvalues. this one gets eleminated and lm is called again 
wihout this data-set.
this is done until we just have a small number of datasets left.

could be that everything im writing here is totaly bullshit (cause im not 
shure if i got every thing right)
but this thing is working an creates very nice predictions ;)

i just fugured that the lm call's in this loop tooks the most time and i 
want to reduce this.
any ideas?

----- Original Message ----- 
From: "David Winsemius" <dwinsemius at comcast.net>
To: "Paul Hermes" <paul.hermes at analytic-company.com>
Cc: <r-help at r-project.org>
Sent: Thursday, March 12, 2009 3:42 PM
Subject: Re: [R] stats lm() function

I think you will find that many readers of this list would rather try  to 
dissuade you from this misguided strategy. You are unlikely to get  to a 
sensible solution in using step-down procedures with this sort of 
situation (large number of predictors with modest size of data).

-- 
David Winsemius

On Mar 12, 2009, at 1:59 PM, Paul Hermes wrote:

Hi,

Im using the lm() function where the formula is quite big (300 
arguments) and the data is a frame of 3000 values.

This is running in a loop where in each step the formula is reduced  by 
one argument, and the lm command is called again (to check which 
arguments are useful) .

This takes 1-2 minutes.
Is there a way to speed this up?
i checked the code of the lm function and its seems that its  preparing 
the data and then calls lm.Fit(). i thought about just  doing this 
praparing stuff first and only call lm.fit() 300 times.
[[alternative HTML version deleted]]

David Winsemius, MD
Heritage Laboratories
West Hartford, CT