Skip to content

advice?

5 messages · Carson Baughman, David Winsemius, Daniel Malter +1 more

Original
David Winsemius
#
On May 3, 2010, at 6:16 PM, Carson Baughman wrote:

            

      
      
      
    

        
What's a step?

            

      
      
      
    

        
If you want to assign rownames that are different than the default  
sequential integers, you need to offer a vector of unique character  
values. Perhaps you wanted to specify row.names=NULL ?

            

      
      
      
    

        
Sounds like you may be in trouble with memory resources. As the  
message says, see :

help(memory.size)

            

      
      
      
    

        

      
      
    

        
David Winsemius, MD
West Hartford, CT

  
  


  


      

    

    

      
      

        


  

        

      Daniel Malter
    

    

      
      #
    

  


  

      
Hi, on the one hand, you write "fairly large," on the other hand, you write
"should be readable by anything." The warnings indicate that you are plain
out of memory at some point. Not too surprising, given that your dataset has
about 450000 rows and 720 columns. You may search the r-help files first for
how to allocate memory/how to read large files, since these questions are
asked frequently.

The error, however, seems to refer to the fact that there are columns with
identical column names, which is not allowed.

Daniel

-------------------------
cuncta stricte discussurus
-------------------------

-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On
Behalf Of Carson Baughman
Sent: Monday, May 03, 2010 6:17 PM
To: R-help at r-project.org
Subject: [R] advice?

All-
       Thank you in advance for any help you might be able to lend.  Here is
my issue.  I am trying to open a fairly large .dat file.  The file
originally was downloaded as a GZ file but I unzipped it (with 7-zip) into
it's current 1.86 gig .dat format.  I know that the data is "just a plain
ASCII file with 720 columns and 360 rows per time step (month). It should be
readable by anything!"  There are 1272 steps.  Here is what happens when I
try to assign the file to an object:

            

      
      
      
    

        
Data\\TS3.0\\Cloud\\cru_ts_3_00.1901.2006.cld.dat", header = TRUE, row.names
= 1)
       Error in read.table("C:\\CRU
Data\\TS3.0\\Cloud\\cru_ts_3_00.1901.2006.cld.dat",  :
       duplicate 'row.names' are not allowed
      In addition: There were 45 warnings (use warnings() to see them)

            

      
      
      
    

        
1: In scan(file, what, nmax, sep, dec, quote, skip, nlines,  ... :
       Reached total allocation of 1535Mb: see help(memory.size)
       X 25
       26: In type.convert(data[[i]], as.is = as.is[i], dec = dec,  ... :
       Reached total allocation of 1535Mb: see help(memory.size)

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      David Winsemius
    

    

      
      #
    

  


  

      
On May 3, 2010, at 9:48 PM, Daniel Malter wrote:

        

            

      
      
      
    

        
row.names

            

      
      
      
    

        

      
      
    

        
David Winsemius, MD
West Hartford, CT

  
  


  


      

    

    
1 day later

    

      
      

        


  

        

      eric
    

    

      
      #
    

  


  

      
Download the trial version of UltraEdit (windows only) to open, inspect
and edit the file. Rename columns as needed. Set a long Tab stop,
Find/replace your delimiters to tabs (^t) and use column mode to remove
unneeded columns. 

You can also split the file and check if loading in increments help.

        
"Daniel Malter" <daniel at umd.edu> wrote: