Memory usage in R grows considerably while calculating word frequencies

I am working with some large text files (up to 16 GBytes).  I am interested in extracting the 
words and counting each time each word appears in the text. I have written a very simple R 
program by following some suggestions and examples I found online.

If my input file is 1 GByte, I see that R uses up to 11 GBytes of memory when executing the 
program on a 64-bit system running CentOS 6.3. Why is R using so much memory? Is there a
better way to do this that will minimize memory usage.

I am very new to R, so I would appreciate some tips on how to improve my program or a better 
way to do it.

R program: # Read in the entire file and convert all words in text to lower case 
words.txt<-tolower(scan("text_file","character",sep="\n"))

# Extract words pattern <- "(\\b[A-Za-z]+\\b)" match <- gregexpr(pattern,words.txt) words.txt 
<- regmatches(words.txt,match)

# Create a vector from the list of words words.txt<-unlist(words.txt)

# Calculate word frequencies words.txt<-table(words.txt,dnn="words")

# Sort by frequency, not alphabetically words.txt<-sort(words.txt,decreasing=TRUE)

# Put into some readable form, "Name of word" and "Number of times it occurs" 
words.txt<-paste(names(words.txt),words.txt,sep="\t")

# Results to a file cat("Word\tFREQ",words.txt,file="frequencies",sep="\n")



-- View this message in context: 
http://r.789695.n4.nabble.com/Memory-usage-in-R-grows-considerably-while-calculating-word-frequencies-tp4644053.html