Memory(RAM) issues

On Jun 23, 2011, at 11:57 AM, Abhisek Saha wrote:

Hi Lui, Anupam and my other R-user friends,
Thanks ?for your previous suggestions. As for my issue, it is clearly
RAM problem as my code is running perfectly as long as my input data
size is small and code has been refined a number of times to increase
efficiency [ by using matrix more in the context of manipulations and
reducing order of time complexity wherever possible]. Yet ?it has
gotta run 2000 iterations and in each of the iterations it repeatedly
modifies ?few [0.5 Mn * 15] tables

After few basic investigation it seems that the problem MAY be
resolved if 64 bit version of R is installed in 64 bit OS as it claims
to be able use potentially a ?8TB RAM ( for my case I think 4 GB RAM
may serve the purpose but 2 GB is not enough) if available.I don't
have any domain knowledge in IT since I hail from a statistics
background. ?I work in a company as a statistical analyst. The IT
folks of my company are ready to deploy any free solution but I need
to tell them what software needs to be installed and what all OS
configurations would be required. They normally run all of the
applications on unix servors.

So I need to know if any free 64 bit of R version can be installed in
unix servor or if not unix, may be on other servor. So hereby I
request my R-user friends to please let me know if any free 64 bit
version of R is available

Yes.

that can be installed on any unix servor. If
that is not available is there any other FREE ?solution and if
available, how to get that and what all configuration is required.

There are a bunch of "UNIX-alike" platforms. Read the FAQ:

http://cran.r-project.org/doc/FAQ/R-FAQ.html#What-machines-does-R-run-on_003f

-- David.

Awaiting your replies,
Regards,
Abhisek

P.S. Please forward this mail to any other R-mailing list if you deem
it fit for any of them.

On Sat, Jun 11, 2011 at 4:38 PM, Lui ## <lui.r.project at googlemail.com>
wrote:

Hello Abhisek,

maybe you wanna try it on just a bigger machine (I guess you are
working at a university, so I hope you do have access to them). In
case getting computing time or the like is a major issue, you may
wanna try Amazon AWS: For a few rupees (about 50-100 per hour) you can
"rent" pretty fast computers (20 Ghz, 8BG of RAM). You may want to try
out the Windows version (little bit more expensive) which is easily
accessible via remote desktop. Installing Revolution (which is free
for academic purposes) (64 Bit Version) might give you a good start.
Maybe its not a viable option in a long term view (pricing), but it
might help to get a clue whether the problem can be solved on a bigger
machine and just trying it out without a bigger hastle...

Good luck!

Lui

On Sat, Jun 11, 2011 at 7:47 AM, Abhisek Saha <tading at gmail.com> wrote:

Thanks Anupam for your inputs. I believe there are two ways to
circumvent the issue...1> making the code more efficient 1> Increasing
the memory in some way.The reasons why I did not paste the code are 1>
It is long and using quite a number of functions that ?I created 2>
Secondly my intention is not to make the code more efficient if that's
possible. Here landing into a memory problem with 2 GB RAM is natural
as my analysis entails 1500 simulations from huge multivariate
distributions that change after every simulation and tomorrow I may
have to do similar analysis with 10 million observations * 20 columns.

In view of above I shall be needing more memory sometime later and my
IT friends are ready to support me for that(probably with a sandbox)
but I am not sure whether I can install probably a servor version of R
that can be capable of working with 8GB or so RAM. So it is more of
technical help I need and I have no clue regarding the plausibility of
the solution mentioned( i.e. a servor version of R that is capable of
more memory).

Regards,
Abhisek

On Sat, Jun 11, 2011 at 10:10 AM, Anupam <anupamtg at gmail.com> wrote:

It will be helpful on this forum to use metric measures: 12 Lakh is 1.2
million, thus your data is 1.2 million observations x 15 variables. I
do not
know the intricacies of R. You may have to wait for someone with that
knowledge to respond.

Including some relevant portions of error messages and code in your
query
can also help someone to respond to your message.

Anupam.
-----Original Message-----
From: r-help-bounces at r-project.org
[mailto:r-help-bounces at r-project.org] On
Behalf Of Abhisek Saha
Sent: Saturday, June 11, 2011 6:25 AM
To: r-help at r-project.org
Subject: [R] Memory(RAM) issues

Dear All,
I have been working with R(desktop version) in VISTA. I have the latest
version R 2.13.0. I have been working with few data-sets of size 12Lakh
* 15
and my code is quite computing intensive ( applying MCMC gibbs sampler
on a
posterior of 144 variables) that often runs into a memory issue such as
memory can not allocate the vector ,full size(shows to have reached
something like 1.5 GB) reached or something to this effect. I have a
RAM of
2GB. ?I checked with the option like memory.size and it says a 64 bit R
if
sat on 64 bit windows then max memory capability is 8TB.

Now I don't have ?background to understand the definitions and
differences
between 32 and 64 bit machines and other technical requirements like
servor
etc but it would be of great help if anyone can let me have a feel of
it.
Could any of you tell me whether some servor version of R would resolve
my
issue or not (I am not sure now what kind of servor my company would
allow R
to be installed at this point ,may be linux type) and if that's the
case
could any of you guide me about how to go about installing that onto a
sevor.

Thank you,
Abhisek

? ? ? [[alternative HTML version deleted]]

David Winsemius, MD
West Hartford, CT