Skip to content

Handling large SAS file in R

4 messages · Dennis Fisher, Marc Schwartz, Milan Bouchet-Valat

Original
Marc Schwartz
# 
Dennis,

The key difference is that with R, you are, as always, dependent upon volunteers providing software at no charge to you, most of whom have full time (and then some) jobs. Those jobs (and in many cases, family) will be their priority, as I am sure is the case with Matt. 

Unless they are in a position where their employer specifically allows them to allocate a percentage of their work time to voluntary projects, like R, you are at the inevitable mercy of that volunteer's time and priorities.

In the case of Stat/Transfer, they are a profit motivated business with revenue tied directly to the sales of the application. Thus, they have a very different perspective on serving their paying customers and can allocate dedicated resources to the functionality in their application.

An alternative here would be for one of the for profit companies that sell and support R versions, to take on the task of providing some of these facilities and providing them back to the community as a service. But, that is up to them to consider in their overall business plan and the value that they perceive it brings to their products.

Regards,

Marc Schwartz
On Jan 28, 2014, at 9:59 AM, Dennis Fisher <fisher at plessthan.com> wrote:

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Dennis Fisher
    

    

      
      #
    

  


  

      
Marc

I understand that R is staffed entirely by volunteers and I appreciate the massive efforts that have created this awesome software. My suggestion was aiming to encourage the Core team that one weakness of the present structure is less-than-optimal interchangeability with SAS.  If this were improved, it would be a great service to the R community, particularly those of us in the pharmaceutical development world who must interface with SAS.

I just wrote to David Smith at Revolution Analytics, encouraging him to consider taking this on (as you suggested).

Dennis

Dennis Fisher MD
P < (The "P Less Than" Company)
Phone: 1-866-PLessThan (1-866-753-7784)
Fax: 1-866-PLessThan (1-866-753-7784)
www.PLessThan.com

        
On Jan 28, 2014, at 8:43 AM, Marc Schwartz <marc_schwartz at me.com> wrote:

        

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Milan Bouchet-Valat
    

    

      
      #
    

  


  

      
Le mardi 28 janvier 2014 ? 11:51 -0800, Fisher Dennis a ?crit :

            

      
      
      
    

        
FWIW, a new pure-C library to read SAS, SPSS and Stata files has been
released here:
https://github.com/WizardMac/ReadStat

I'm not sure whether it constitutes an improvement over the sas7bdat
package in terms of features, but in terms of speed it should perform
much better since it's written in C. With a reasonable amount of work it
could probably be adapted to work in R.


My two cents