read .csv from web from password protected site
Yes, I meant to reply to all (sorry still new at asking for help) 1. No, I am not able to open the file when I insert "userID:password@" between "http://" and "www" http://userid:password at www.frontierweather.com/degreedays/L15N15PowerRegionAverages_10weeks.txt (replacing userid and password with my actual information that is known to work) 2. Yes, the webpage where the data is stored does require me to my userid and password and gives me the option to remember password for future use which I have selected. Subsequent visits to the website do not require me to reenter info. -----Original Message----- From: Sarah Goslee [mailto:sarah.goslee at gmail.com] Sent: Monday, October 03, 2011 3:07 PM To: Mike Pfeiff; r-help Subject: Re: [R] read .csv from web from password protected site Hi, I've assumed that you meant to send this to the R-help list, and not just me.
On Mon, Oct 3, 2011 at 3:54 PM, Mike Pfeiff <MikeP at kfoc.net> wrote:
Sarah, ?Thanks for the suggestion. ?Although, read.table("http://userid:password at my.url/file.csv") did not work as it returned the following:
? ? ? ?Error in file(file, "rt") : cannot open the connection
? ? ? ?In addition: Warning message:
? ? ? ?In file(file, "rt") : unable to resolve 'userid'
(where 'usesid is my actual userid)
I've tried the following RCurl commands...
? ? ? ?myURL ? ="http://www.frontierweather.com/degreedays/L15N15PowerRegionAverages_10weeks.txt"
? ? ? ?h=getURL(myURL, userpw = "userid:passwod", followlocation=TRUE)
? ? ? ?test=read.table(h,header=TRUE,sep=",")
..and I can't get the data to read and get the following errors:
? ? ? ?Error in file(file, "rt") : cannot open the connection
? ? ? ?In addition: Warning message:
? ? ? ?In file(file, "rt") : unable to resolve 'userid'
I'm at a total loss. ?Any assistance anyone could provide would be greatly appreciated.
You can get to the file using your web browser and that userid/password combo, right? Do you have to go through a dialog box? Press a button to login? Any of those have the potential to complicate the task. If so, you'll need to work through the Forms section at http://www.omegahat.org/RCurl/philosophy.html Sarah
-----Original Message----- From: Sarah Goslee [mailto:sarah.goslee at gmail.com] Sent: Monday, October 03, 2011 2:26 PM To: Mike Pfeiff Cc: r-help at r-project.org Subject: Re: [R] read .csv from web from password protected site Hi Mike, On Mon, Oct 3, 2011 at 12:31 PM, Mike Pfeiff <MikeP at kfoc.net> wrote:
I am very new to R and have been struggling trying to read a basic ".csv" file from a password protected site with the following code: myURL ?="http://www.frontierweather.com/degreedays/L15N15PowerRegionAverages_10weeks.txt" test2=read.table(url(myURL),header=TRUE,sep=",")
A 'data.frame' is returned into the workspace, however it is not the data contained in the ".csv" file. ? I think this occurs because the website where I am trying to retrieve the data is password protected. Is there a way to specify the username and password?
I'd try first
read.table("http://userid:password at my.url/file.csv"), which is the standard way to do it (hint: try that form in your web browser and see whether you can access the data), and if that doesn't work look into the RCurl package. The list archives have a fair bit of information on this topic.
Sarah
Any guidance would be greatly appreciated. Sincerely, Mike
-- Sarah Goslee http://www.functionaldiversity.org