Query about memory used in list and dataframe

test.df <- data.frame(a1= 1:6, a2= 1:6, a3 = 1:6, a4 = 1:6, a5 = 1:6, a6 = 1:6

test.df

object.size(test.df)

x <- 1:6
test.df1 <- data.frame(x,x,x,x,x,x,x,x,x,x,x,x)
object.size(test.df1)

list.1 <- list(x,x,x,x,x,x,x,x,x,x,x,x)
object.size(list.1)

list.2 <- NULL
for (i in 1:12) list.2[[i]] <- x
object.size(list.2)

test.df3 <- as.data.frame(do.call(cbind, list.2))
object.size(test.df3)

Hi,

I had a query regarding which object, a list or a dataframe, consumes more
R memory. Let me clarify this:

For example, I have a df of 6 rows and 12 columns, say 'test'. I do
object.size() and find it uses 3.3 KB of memory.

I run a loop and make a list, say 'testlist', of 6 elements, each element
being the above mentioned df 'test'. The size of this list is 19.9 KB,
understandably.

Now I combine this list into a dataframe using rbind. The df formed has 12
cols and 36 rows. The size of this df is just 5.8 KB, almost a 75%
reduction in memory.

I had to work with a much larger list, and I thought of using the same
method to convert my bigger list (62 dataframes, each having 4 cols and
close to 200,000 rows) into a single dataframe. The big list, sat LIST A,
had a size of 571 MB. But when I convert it into a dataframe, say DF A,
using rbind, the object size increases to 1.35 GB. This was in
contradiction to the earlier result.

What am I missing? Why a 75% reduction in size in one case and double size
in other? Anyone with any explanation?

Sorry for the verbose email, just wanted to make my case clear.

Thanks in advance,
Regards
Shivam

? ? ? ?[[alternative HTML version deleted]]

I think you really have to show use your exact code that you did along
with an 'str' of each intermediate data structure since my quick test
does not bear out what you were saying:

test.df <- data.frame(a1= 1:6, a2= 1:6, a3 = 1:6, a4 = 1:6, a5 = 1:6, a6
= 1:6

+     , a7=1:6, a8 = 1:6, a9 = 1:6, a10 = 1:6, a11 = 1:6, a12 = 1:6)

test.df

  a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12
1  1  1  1  1  1  1  1  1  1   1   1   1
2  2  2  2  2  2  2  2  2  2   2   2   2
3  3  3  3  3  3  3  3  3  3   3   3   3
4  4  4  4  4  4  4  4  4  4   4   4   4
5  5  5  5  5  5  5  5  5  5   5   5   5
6  6  6  6  6  6  6  6  6  6   6   6   6

object.size(test.df)

2264 bytes

x <- 1:6
test.df1 <- data.frame(x,x,x,x,x,x,x,x,x,x,x,x)
object.size(test.df1)

2264 bytes

list.1 <- list(x,x,x,x,x,x,x,x,x,x,x,x)
object.size(list.1)

1032 bytes

list.2 <- NULL
for (i in 1:12) list.2[[i]] <- x
object.size(list.2)

1032 bytes

test.df3 <- as.data.frame(do.call(cbind, list.2))
object.size(test.df3)

2264 bytes

On Sat, May 5, 2012 at 5:50 PM, Shivam <shivamsingh@> wrote:

Hi,

I had a query regarding which object, a list or a dataframe, consumes
more
R memory. Let me clarify this:

For example, I have a df of 6 rows and 12 columns, say 'test'. I do
object.size() and find it uses 3.3 KB of memory.

I run a loop and make a list, say 'testlist', of 6 elements, each element
being the above mentioned df 'test'. The size of this list is 19.9 KB,
understandably.

Now I combine this list into a dataframe using rbind. The df formed has
12
cols and 36 rows. The size of this df is just 5.8 KB, almost a 75%
reduction in memory.

I had to work with a much larger list, and I thought of using the same
method to convert my bigger list (62 dataframes, each having 4 cols and
close to 200,000 rows) into a single dataframe. The big list, sat LIST A,
had a size of 571 MB. But when I convert it into a dataframe, say DF A,
using rbind, the object size increases to 1.35 GB. This was in
contradiction to the earlier result.

What am I missing? Why a 75% reduction in size in one case and double
size
in other? Anyone with any explanation?

Sorry for the verbose email, just wanted to make my case clear.

Thanks in advance,
Regards
Shivam

? ? ? ?[[alternative HTML version deleted]]

-- 
Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.