Skip to content

Step Function Freezing R

7 messages · John, Jgabriel, Uwe Ligges

Jgabriel
# 
I have a question that might not be "kosher" here, but I'm running out
of options and need some help. Basically using the function "step" is
freezing R. I am running a model that includes a number of
interactions on a large data set with a number of dummy variables
representing whether an event occurred or not. Here's a simplified
example -

fit = lm(a~b*(c+d+e)*(f+g+h)*(i+j+k))
regs = step(fit)

at this point R freezes and alternates between "Not Responding" and
"Running" in Windows task manager. It wasn't always this way. I was
able to run the same model on the same dataset a couple weeks ago and
while it took the step function 20-30 minutes to compute the best AIC
and didn't exclude all the insignificant variables/interactions, it
served its purpose. Even after uninstalling and reinstalling R and
RExcel (which I used to load the data set), the problem persists. I
tried it on a smaller data set and still no luck. This is more of a
computing issue, so if anyone can point me to a forum better equipped
to help me out, I will owe you my eternal gratitude. I hate to fill up
space with such an uninspired question, but like I said I'm running
out of options.

Thanks,

JGS
John
# 
There are some points of information you should include.  One, have there been 
recent updates of Windows that may have altered system behaviour?  Two, have 
you updated R during the same period?  Three, have you installed software that 
uses system resources that were available to R before - e.g. has Norton or 
whatever you use to guard your system upgraded?  Four, what versions of 
Windows and Excel are you running?  Five, have you inspected your data for 
corruption?  Six, what changes (data entry, editing, recoding, etc.) have you 
done that might coincide with the changed behaviour?  Seven, have you allowed 
the process to simply run - say overnight?  Nine, how big is you database? 

In essence, your problem sounds like system problem.  The most likely cause is 
a change in software, but without some really clear information on changes on 
your system that could potentially have caused this, you are unlikely to find 
it easily.  One possibility is that a change in software has resulted in 
smaller available resource space.  

JWDougherty
On Thursday 12 November 2009 09:05:34 am Jgabriel wrote:
Jgabriel
# 
I can't fully answer all these questions, but I'll do my best - There
have not been any updates of Windows, and I did not update R during
the period, although I did reinstall it after the problem started.
There have been no changes to Norton or any other software that uses
system resources in that way. The one thing I can think of is that I
installed a program called Digitizer (creates data tables/ csds from
visually analyzing line charts) around the same time it started
freezing. I have completely uninstalled it and deleted all related
files. The problem should not be with the data, which is fine. I am
running Windows XP Professional and Excel 2007. I have allowed the
process to run overnight on two separate occasions. The first time it
was completely frozen and there was no progress. Last night it
actually made progress, but the fact remains that a process that used
to take an hour at the most still has not even halfway completed after
over 8 hours. I even installed extra RAM on the system so that there
is more than when the process used to work. I agree that it is
probably a change in software, but I can't figure out what has changed
or what I can do about it.

Thanks for your help,

JGS
On Nov 12, 7:05?pm, J Dougherty <j... at surewest.net> wrote:
Uwe Ligges
#
Jgabriel wrote:
Aha! Please ask your local IT experts to check the RAM and if that is 
fine ask them to proceed. I have already seen too many machines with RAM 
problems during the last years. It is hardly an R problem.

Uwe Ligges
Jgabriel
# 
Alright, here's the update: the IT guy insists that the RAM is not the
problem. RAM, according to him, is "pretty black and white". There
should be enough RAM on the system for the function to work, and he
was the one who installed the extra RAM in the first place. Could
there be another RAM issue besides the amount? Is there some special
way in which R uses RAM that could be causing the issue? He also wiped
the computer and rebuilt the whole system, to no avail. He's getting
pretty tired of my problems, to say the least. You say you've seen a
lot of RAM problems over the years, can you help me out?

On Nov 13, 1:15?pm, Uwe Ligges <lig... at statistik.tu-dortmund.de>
wrote:
John
#
On Friday 13 November 2009 07:17:28 am Jgabriel wrote:
OK, from what you are saying, it seems clear the problem is a system problem 
rather than an R issue.  MS issues patches every month, so there may not have 
been upgrades, but there could still be systemic changes.   As regards R, did 
you update R?  How big is the data table?  Has it grown?  Did you or someone 
else alter the available memory to R by an environment setting?  Have you 
searched "Memory" in R help and manuals?  Is this computer yours, or is it 
used by others who may have altered settings?  These are all questions that 
may be pertinent.  Good luck.


JWDougherty
2 days later
Jgabriel
# 
Can you think of any systemic changes that might interefere with R
besides Symantec EndPoint and LiveUpdate? I have removed those
programs and allocated more memory to R, but it is still way too
slow.
On Nov 13, 10:45?pm, J Dougherty <j... at surewest.net> wrote: