Error with installed.packages with R 3.4.0 on Windows - R-help

Mon, May 22, 2017 2:10 AM #

On Fri, 28-Apr-2017 at 07:04PM +0200, peter dalgaard wrote:

|>

|> > On 28 Apr 2017, at 12:08 , Duncan Murdoch <murdoch.duncan at gmail.com> wrote:

|> >

|> > On 28/04/2017 4:45 AM, Thierry Onkelinx wrote:

|> >> Dear Peter,
|> >> 
|> >> It actually breaks install.packages(). So it is not that innocent.
|> > 
|> > I don't think he meant that it is harmless, he meant that the fix is easy, and is in place in R-patched and R-devel.  You should use R-patched and you won't have the problem.
|> 
|> Read more carefully: I said that the _fix_ is harmless for this case, but might not be so in general.
|> 
|> -pd


Apparently it isn't harmless.

Error in readRDS(dest) : error reading from connection

R version 3.4.0 Patched (2017-05-19 r72713)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.5 LTS

Matrix products: default
BLAS: /home/hrapgc/local/R-patched/lib/libRblas.so
LAPACK: /home/hrapgc/local/R-patched/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_NZ.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_NZ.UTF-8        LC_COLLATE=en_NZ.UTF-8    
 [5] LC_MONETARY=en_NZ.UTF-8    LC_MESSAGES=en_NZ.UTF-8   
 [7] LC_PAPER=en_NZ.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_NZ.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] grDevices utils     stats     graphics  methods   base     

other attached packages:
[1] lattice_0.20-35

loaded via a namespace (and not attached):
[1] compiler_3.4.0 tools_3.4.0    grid_3.4.0

Has anyone a workaround?

~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.   
   ___    Patrick Connolly   
 {~._.~}                   Great minds discuss ideas    
 _( Y )_  	         Average minds discuss events 
(:_~*~_:)                  Small minds discuss people  
 (_)-(_)  	                      ..... Eleanor Roosevelt
	  
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.

Martin Morgan

Mon, May 22, 2017 2:43 AM #

On 05/22/2017 05:10 AM, Patrick Connolly wrote:

that seems like a plain-old network connectivity issue, or perhaps an 
issue with the CRAN mirror you're using. Can you debug on your end, e.g,.

   options(error=recover)
   install.packages("withr")
   ...

then select the 'frame' where the error occurs, look around

   ls()

find the value of 'dest', and e.g., try to open dest in your  browser.

Martin Morgan

This email message may contain legally privileged and/or...{{dropped:2}}

Patrick Connolly

Tue, May 23, 2017 1:47 AM #

On Mon, 22-May-2017 at 05:43AM -0400, Martin Morgan wrote:

|> On 05/22/2017 05:10 AM, Patrick Connolly wrote:

|> >Apparently it isn't harmless.
|> >
|> >>install.packages("withr")
|> >Error in readRDS(dest) : error reading from connection
|> 
|> that seems like a plain-old network connectivity issue, or perhaps
|> an issue with the CRAN mirror you're using. Can you debug on your
|> end, e.g,.
|> 
|>   options(error=recover)
|>   install.packages("withr")
|>   ...
|> 
|> then select the 'frame' where the error occurs, look around
|> 
|>   ls()
|> 
|> find the value of 'dest', and e.g., try to open dest in your  browser.

This is what I get

^C

Enter a frame number, or 0 to exit   

1: install.packages("withr")
2: available.packages(contriburl = contriburl, method = method)
3: tryCatch({
    download.file(url = paste0(repos, "/PACKAGES.rds"), destfile
4: tryCatchList(expr, classes, parentenv, handlers)
5: tryCatchOne(expr, names, parentenv, handlers[[1]])
6: doTryCatch(return(expr), name, parentenv, handler)
7: download.file(url = paste0(repos, "/PACKAGES.rds"), destfile = dest, method

Selection: 7
Called from: eval(substitute(browser(skipCalls = skip), list(skip = 7 - which)), 
    envir = sys.frame(which))
Browse[1]> dest
Error during wrapup: object 'dest' not found

That indicates to me that the problem is further back but I have no
idea how make use of that information.


Browse[1]> ls()
[1] "cacheOK"  "destfile" "extra"    "method"   "mode"     "quiet"    "url"     
Browse[1]> url
[1] "http://cran.stat.auckland.ac.nz/src/contrib/PACKAGES.rds"
Browse[1]> destfile
[1] "/tmp/RtmpplJSrB/repos_http%3A%2F%2Fcran.stat.auckland.ac.nz%2Fsrc%2Fcontrib.rds"
Browse[1]> 

The destfile above is zero-length and I suppose is where dest is
intended to end up.

Where else should I be looking?  Earlier installations never had this
issue so I don't have anything to compare.

TIA

~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.   
   ___    Patrick Connolly   
 {~._.~}                   Great minds discuss ideas    
 _( Y )_  	         Average minds discuss events 
(:_~*~_:)                  Small minds discuss people  
 (_)-(_)  	                      ..... Eleanor Roosevelt
	  
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.

Martin Maechler

Tue, May 23, 2017 3:20 AM #

> On Mon, 22-May-2017 at 05:43AM -0400, Martin Morgan wrote:

> |> On 05/22/2017 05:10 AM, Patrick Connolly wrote:

> |> >Apparently it isn't harmless.
    > |> >
    > |> >>install.packages("withr")
    > |> >Error in readRDS(dest) : error reading from connection
    > |> 
    > |> that seems like a plain-old network connectivity issue, or perhaps
    > |> an issue with the CRAN mirror you're using. Can you debug on your
    > |> end, e.g,.
    > |> 
    > |>   options(error=recover)
    > |>   install.packages("withr")
    > |>   ...
    > |> 
    > |> then select the 'frame' where the error occurs, look around
    > |> 
    > |>   ls()
    > |> 
    > |> find the value of 'dest', and e.g., try to open dest in your  browser.

    > This is what I get

    >> options(error=recover)
    >> install.packages("withr")
    > ^C

    > Enter a frame number, or 0 to exit   

    > 1: install.packages("withr")
    > 2: available.packages(contriburl = contriburl, method = method)
    > 3: tryCatch({
    > download.file(url = paste0(repos, "/PACKAGES.rds"), destfile
    > 4: tryCatchList(expr, classes, parentenv, handlers)
    > 5: tryCatchOne(expr, names, parentenv, handlers[[1]])
    > 6: doTryCatch(return(expr), name, parentenv, handler)
    > 7: download.file(url = paste0(repos, "/PACKAGES.rds"), destfile = dest, method

    > Selection: 7

'7'  was the wrong choice:  'dest' exists in the frame that
     *calls* download.file, in this case, it is frame 2, i.e.,
     inside available.packages(.)  where the  tryCatch() call to
     download.file() happens.

Given the above stack trace. 
It may be easier to just do

    debugonce(available.packages)
    install.packages("withr")

and then inside available.packages, (using 'n') step to the
point _before_ the tryCatch(...) call happens; there, e.g. use

      ls.str()

which gives an str() of all your local objects, notably 'dest'
and 'method'.
but you can also try other things once inside
available.packages().

Martin


    > Called from: eval(substitute(browser(skipCalls = skip), list(skip = 7 - which)), 
    > envir = sys.frame(which))
    > Browse[1]> dest
    > Error during wrapup: object 'dest' not found

    > That indicates to me that the problem is further back but I have no
    > idea how make use of that information.


    > Browse[1]> ls()
    > [1] "cacheOK"  "destfile" "extra"    "method"   "mode"     "quiet"    "url"     
    > Browse[1]> url
    > [1] "http://cran.stat.auckland.ac.nz/src/contrib/PACKAGES.rds"
    > Browse[1]> destfile
    > [1] "/tmp/RtmpplJSrB/repos_http%3A%2F%2Fcran.stat.auckland.ac.nz%2Fsrc%2Fcontrib.rds"
    > Browse[1]> 

    > The destfile above is zero-length and I suppose is where dest is
    > intended to end up.

    > Where else should I be looking?  Earlier installations never had this
    > issue so I don't have anything to compare.

    > TIA
    > --

Patrick Connolly

Wed, May 24, 2017 2:07 AM #

On Tue, 23-May-2017 at 12:20PM +0200, Martin Maechler wrote:

[...]

|> 
|> Given the above stack trace. 
|> It may be easier to just do
|> 
|>     debugonce(available.packages)
|>     install.packages("withr")
|> 
|> and then inside available.packages, (using 'n') step to the
|> point _before_ the tryCatch(...) call happens; there, e.g. use
|> 
|>       ls.str()

I got:

contriburl :  chr [1:2] "http://cran.stat.auckland.ac.nz/src/contrib" ...
dest :  chr "/tmp/Rtmp6D0KNY/repos_http%3A%2F%2Fcran.stat.auckland.ac.nz%2Fsrc%2Fcontrib.rds"
fields :  chr [1:16] "Package" "Version" "Priority" "Depends" "Imports" "LinkingTo" ...
filters :  NULL
localcran :  logi FALSE
method : <missing>
repos :  chr "http://cran.stat.auckland.ac.nz/src/contrib"
requiredFields :  chr [1:16] "Package" "Version" "Priority" "Depends" "Imports" "LinkingTo" ...
res :  chr[0 , 1:17] 
type :  chr "source"


So I thought the missing method was the problem.  However, it's
exactly the same with R-3.3.3.  So still no answwer.

Then I tried:

debugonce(readRDS)
install.packages("withr")

Browse[2]> con
A connection with    
description "/tmp/RtmpqzKzzK/repos_http%3A%2F%2Fcran.stat.auckland.ac.nz%2Fsrc%2Fcontrib.rds"
class       "gzfile"               
mode        "rb"                   
text        "binary"               
opened      "opened"               
can read    "yes"                  
can write   "no"

## With R-3.3.3

Browse[2]> con       
                                                description 
"/home/hrapgc/local/R-3.3.3/library/abind/Meta/package.rds" 
                                                      class 
                                                   "gzfile" 
                                                       mode 
                                                       "rb" 
                                                       text 
                                                   "binary" 
                                                     opened 
                                                   "opened" 
                                                   can read 
                                                      "yes" 
                                                  can write 
                                                       "no" 

So, in 3.4.0, con refers to a repo and is zero length whereas 3.3.3
refers to a library.  To me that would appear to be a major
difference.  That must have something to do with it.  But what?

Since the latter works and the former doesn't, it seems as though the
problem is with 3.4.0's readRDS().  But others seem not to have the
same problem.  I'm no closer to understanding what's happenning.

~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.   
   ___    Patrick Connolly   
 {~._.~}                   Great minds discuss ideas    
 _( Y )_  	         Average minds discuss events 
(:_~*~_:)                  Small minds discuss people  
 (_)-(_)  	                      ..... Eleanor Roosevelt
	  
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.

Patrick Connolly

Wed, May 31, 2017 1:38 AM #

On Tue, 23-May-2017 at 12:20PM +0200, Martin Maechler wrote:

[...]

|> 
|> Given the above stack trace. 
|> It may be easier to just do
|> 
|>     debugonce(available.packages)
|>     install.packages("withr")
|> 
|> and then inside available.packages, (using 'n') step to the
|> point _before_ the tryCatch(...) call happens; there, e.g. use
|> 
|>       ls.str()
|> 
|> which gives an str() of all your local objects, notably 'dest'
|> and 'method'.
|> but you can also try other things once inside
|> available.packages().

I couldn't see any differences between R-3.3.3 (which works) and
R-3.4.0 (which doesn't) until I got to here, a few lines before the
download.file line:

Browse[2]> 
debug: dest <- file.path(tempdir(), paste0("repos_", URLencode(repos, 
    TRUE), ".rds"))
Browse[2]> 

When I check out those directories in a terminal, there's a big diffrence:

With R-3.4.0
~ > ll /tmp/RtmpFUhtpY
total 4
drwxr-xr-x 2 hrapgc hrapgc 4096 May 31 10:45 downloaded_packages/
-rw-r--r-- 1 hrapgc hrapgc    0 May 31 10:56 repos_http%3A%2F%2Fcran.stat.auckland.ac.nz%2Fsrc%2Fcontrib.rds


With R-3.3.3
~ > ll /tmp/RtmpkPgL3A
total 380
drwxr-xr-x 2 hrapgc hrapgc   4096 May 31 11:01 downloaded_packages/
-rw-r--r-- 1 hrapgc hrapgc   8214 May 31 11:01 libloc_185_3165c7f52d5fdf96.rds
-rw-r--r-- 1 hrapgc hrapgc 372263 May 31 11:01 repos_http%3A%2F%2Fcran.stat.auckland.ac.nz%2Fsrc%2Fcontrib.rds

So, if I could figure out what makes *that* difference I could get
somewhere.  I see there's considerably extra code in the newer of the
two versions of available.packages() but being a bear with a small
brain, I can't figure out what differences should be expected.  I have
no idea what populates those 'dest' directories.

TIA

~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.   
   ___    Patrick Connolly   
 {~._.~}                   Great minds discuss ideas    
 _( Y )_  	         Average minds discuss events 
(:_~*~_:)                  Small minds discuss people  
 (_)-(_)  	                      ..... Eleanor Roosevelt
	  
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.

Martin Morgan

Wed, May 31, 2017 7:05 AM #

On 05/31/2017 04:38 AM, Patrick Connolly wrote:

The file repos_http%3A%2F%2Fcran.stat.auckland.ac.nz%2Fsrc%2Fcontrib.rds 
was likely created earlier in your R session. Likely the download a few 
lines down

                     download.file(url = paste0(repos, "/PACKAGES.rds"),
                                   destfile = dest, method = method,
                                   cacheOK = FALSE, quiet = TRUE, mode = 
"wb")

'succeeded' but created a zero-length file.

You could try to troubleshoot this with something like the following, 
downloading to a temporary location

   dest = tempfile()
   url = "http://cran.stat.auckland.ac.nz/src/contrib/PACKAGES.rds"
   download.file(url, dest)
   file.size(dest)

If this succeeds (it should download a file of several hundred KB), then 
try adding the options method, cacheOK, quiet, mode to the 
download.file() call. 'method' can be determined when you are in 
available.packages while debugging; if R says that it is missing, then 
it will be assigned, in download.file, to either 
getOption("download.file.method") or (if the option is NULL or "auto") 
"libcurl".

If the download 'succeeds' but the temporary file created is 0 bytes, 
then it would be good to share the problematic command with us.

Martin Morgan

This email message may contain legally privileged and/or...{{dropped:2}}