for loop performance
On 04/13/2011 02:55 PM, Barth B. Riley wrote:
Dear list I am running some simulations in R involving reading in several hundred datasets, performing some statistics and outputting those statistics to file. I have noticed that it seems that the time it takes to process of a dataset (or, say, a set of 100 datasets) seems to take longer as the simulation progresses. Has anyone else noticed this? I am curious to know if this has to do with how R processes code in loops or if it might be due to memory usage issues (e.g., repeatedly reading data into the same matrix).
Hi Barth
The 'it gets slower' symptom is often due to repeatedly 'growing by 1' a
list or other data structure, e.g.,
m = matrix(100000, 100)
n = 20000
result = list()
system.time(for (i in seq_len(n)) result[[i]] = m)
versus 'pre-allocate and fill'
result = vector("list", n)
system.time(for (i in seq_len(n)) result[[i]] = m)
The former causes 'result' to be copied on each new assignment, and the
size of the copy gets larger each time.
Thanks in advance Barth PRIVILEGED AND CONFIDENTIAL INFORMATION This transmittal and any attachments may contain PRIVILEGED AND CONFIDENTIAL information and is intended only for the use of the addressee. If you are not the designated recipient, or an employee or agent authorized to deliver such transmittals to the designated recipient, you are hereby notified that any dissemination, copying or publication of this transmittal is strictly prohibited. If you have received this transmittal in error, please notify us immediately by replying to the sender and delete this copy from your system. You may also call us at (309) 827-6026 for assistance.
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Computational Biology Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N. PO Box 19024 Seattle, WA 98109 Location: M1-B861 Telephone: 206 667-2793