RCurl: authentication when posting forms

Valerie Obenchain wrote:
  

Duncan,

Thank you for the examples.  I had tried all of these different options 
for authentication but had no luck. I was getting a "100 continue" and 
then a "401 unauthorized" response.  This morning the owners of the 
server I was trying to access discovered a bug with their api when using 
basic authorization. Evidently the code on their side was explicitly 
checking that the request method was GET, hence why all of my POST 
attempts were failing. They are in the process of fixing that code and 
my guess is that when I am able to try again the RCurl post functions 
will work just fine.
    

Thanks for the update.  Good to know that there isn't a problem
with the libcurl/RCurl code.

  

I did have one other RCurl question I'd like to ask you about the 
parseHTTPHeader function.

The parser appears to parse on spaces, so when the error message is more 
than one word (eg, "not found"), the message returned is "not". I have 
modified the parseHTTPHeader function so that it works for me. I may not 
have done this in the most efficient way but at least you can see what I 
was trying to do.
    

Yes, this is a problem. Thanks for bringing it to my attention.
I did get your mail directly to me from about 2 weeks ago 
and I replied. Perhaps my reply got eaten by your filters.
In it, I said that it was a bug, and that it would be
very useful if you could send me an RCurl command
that illustrated the error so that I could add it to the RCurl
tests.
 

The only modification that can make your patch slightly
"better" is that the words in the statusMessage are essentially
all the words in the status variable, less the first two
(the status number and the protocol name). So

 header[["statusMessage"]] <- paste(els[-c(1,2)], collapse = " ")

is more convenient than

  hstring <- NULL
  for(i in 3:length(els)) hstring <- paste(hstring," ",els[i],sep="")
  hstring <- substr(hstring,2,nchar(hstring))
  header[["statusMessage"]] <- hstring


If you do have a URL that I can use to test the error handling code, I'd appreciate
if you could send it to me.  


 D.

  

Below I have pasted my modification for the parseHTTPHeader function 
which I am calling parseHeader. Please let me know if you think this is 
a bug or if I am just using the function incorrectly.

Thank you for the help.
Valerie

#---------------------------------------------------
#Sample code:
reader <- basicTextGatherer()
header <- basicTextGatherer()
handle <- getCurlHandle()
myopts <- curlOptions(  netrc=1, httpheader=c(Authorization=mypwd, 
Accept="test/xml",
                                      Accept="multipart/*", 
'Content-Type'="text/xml; charset=utf-8"),
                                      postfields=body, 
writefunction=reader$update, headerfunction=header$update,
                                      ssl.verifyhost=FALSE, 
ssl.verifypeer=FALSE, followlocation=TRUE)} else

curlPerform(url=myUrl, .opts=myopts, curl=handle)
h <- parseHeader(header$value())
status <- h$status
message <- h$statusMessage
#----------------------------------------------------

#Modified parse function:
parseHeader <- function (lines)
{
  if (length(lines) < 1)
      return(NULL)
  if (length(lines) == 1)
      lines = strsplit(lines, "\r\n")[[1]]
  status = lines[1]
  lines = lines[-c(1, length(lines))]
  lines = gsub("\r\n", "", lines)
  if (FALSE) {
      header = lines[-1]
      header <- read.dcf(textConnection(header))
  }
  else {
      els <- sapply(lines, function(x) strsplit(x, ":[ ]*"))
      header <- lapply(els, function(x) x[2])
      names(header) <- sapply(els, function(x) x[1])
  }
  els <- strsplit(status, " ")[[1]]
  header[["status"]] <- as.integer(els[2])
# new code below
  hstring <- NULL
  for(i in 3:length(els)) hstring <- paste(hstring," ",els[i],sep="")
  hstring <- substr(hstring,2,nchar(hstring))
  header[["statusMessage"]] <- hstring
  header
}












Duncan Temple Lang wrote:
    

Hi Valerie

Valerie Obenchain wrote:
      

Hi,

Has anyone successfully used RCurl for posting data to a 
password-protected site? 
        

Yes. I just set up a sample form to test with and the following
all work


# Perl script (and HTML form for testing in the browser) taken from
#   http://www.elated.com/articles/form-validation-with-perl-and-cgi/


# Provide the login & password directly
postForm("http://www.omegahat.org/RCurl/testPassword/form_validation.cgi", 

          your_name = "Duncan",
          your_age = "35-55",
          your_sex = "m",
          submit = "submit",
          .opts = list(userpwd = "bob:welcome"))

# Get the login & password in ~/.netrc
postForm("http://www.omegahat.org/RCurl/testPassword/form_validation.cgi", 

          your_name = "Duncan",
          your_age = "35-55",
          your_sex = "m",
          submit = "submit",
         .opts = list(netrc = TRUE))

# Get the login & password from a different netrc file

postForm("http://www.omegahat.org/RCurl/testPassword/form_validation.cgi", 

          your_name = "Duncan",
          your_age = "35-55",
          your_sex = "m",
          submit = "submit",
          .opts = list(netrc = TRUE,
                       netrc.file = 
"/Users/duncan/Projects/org/omegahat/R/RCurl/inst/examples/omg.netrc"))


So let me know what problems you are having and more details
about the OS, version of libcurl, and a sample URL to which
to post, etc.

 D.

      

I
have tired using option netrc=1 with both postForm and curlPerform 
(with postfields option) but can't authenticate.
I would happily provide more details if some one has had some 
experience with this.

Thanks very much.
Valerie