Skip to content

Problem understanding behaviour of versionCheck for loadNamespace (and when versions for Imports packages are checked)

9 messages · Geoff Lee, Duncan Murdoch

Original
Geoff Lee
# 
Hi

 

I'm still exploring the R programming universe, so if this is being asked in
the wrong place, or in the wrong way (e.g. too verbose or lacking in crucial
detail or in the wrong format) please let me know

 

I am trying to understand when the version constraints for packages which
appear in the Imports field of a DESCRIPTION file are checked.

 

Along the way I've hit a snag understanding what loadNamespace with the
versionCheck argument set does.  The core of my query is 'am I doing
something wrong, or have I stumbled across a bug?'.  Probably the former,
but after several days I still can't figure it out, so any guidance, hints
or outright help would be much appreciated.

 

Thanks in advance

 

Geoff

 

What I've tried so far.

 

"Writing R extensions" section 1.1.3 says 

 

"The 'Imports' field lists packages whose namespaces are imported from (as
specified in the NAMESPACE file) but which do not need to be attached.  ...
shortened for brevity ... Packages declared in the 'Depends' field should
not also be in the 'Imports' field. Version requirements can be specified
and are checked when the namespace is loaded (since R >= 3.0.0). "

 

It is slightly ambiguous, but seems to mean that the version dependencies
for both Depends packages and Imports packages are checked when the package
(namespace?) is loaded.

 

The release notes for R3.0.0 are more direct.  They say

 

"loadNamespace() allows a version specification to be given, and this is
used to check version specifications given in the Imports field when a
namespace is loaded"

 

But some toy (locally built and loaded) examples seem to show that while the
Depends versions are checked, the Imports version constraints are not (on
Windows 64, running R3.1.2, see full session_info later).

 

My tests (package2 imports package1) use implicit loading (via the
package1::fun1() idiom) so I have worked back to try get a minimal example
of what's causing me problems. I have tried

 

loadNamespace('package2', versionCheck = list (op = ">=", version =
package_version('3.0')))

 

This should fail (package2 has version 0.3, not 3.0) but instead it seems to
load package2, version 0.3 OK.

 

Reading the code of loadNamespace, there is some code which says

 

if (length(z <- versionCheck) == 3L && !do.call(z$op, 

    list(as.numeric_version(version), z$version))) 

    stop(gettextf("namespace %s %s is being loaded, but %s %s is required", 

      sQuote(package), version, z$op, z$version), 

       domain = NA)

 

I think it is the length(z <- versionCheck) == 3L part of the if test that
is allowing the incorrect version be loaded.

 

The documentation for loadNamespace says that "versionCheck" is

"NULL or a version specification (a list with components op and version))."

 

If I add a third (nonsense) component to the versionCheck argument list,
then loadNamespace does what I expect.  Is there supposed to be a third
component in the list, and if so what should it be? Or is this a bug?

 

I've got this far and am now stumped, hence my query.

 

Some output from my tests.

 

d> devtools::session_info()

Session
info--------------------------------------------------------------------

  setting  value                       

version  R version 3.1.2 (2014-10-31)

system   x86_64, mingw32             

ui       RStudio (0.98.953)          

language (EN)                        

collate  English_Australia.1252      

tz       Australia/Sydney            

 

Packages--------------------------------------------------------------------
----

  package    * version date       source        

devtools     1.6.1   2014-10-07 CRAN (R 3.1.2)

rstudioapi   0.1     2014-03-27 CRAN (R 3.1.1)

 

d> # I think this should fail but it does not

d> loadNamespace('package2', versionCheck = list(op='>=', version =
package_version('3.0')))

<environment: namespace:package2>

  

  

  d> devtools::session_info()

Session
info--------------------------------------------------------------------

  setting  value                       

version  R version 3.1.2 (2014-10-31)

system   x86_64, mingw32             

ui       RStudio (0.98.953)          

language (EN)                        

collate  English_Australia.1252      

tz       Australia/Sydney            

 

Packages--------------------------------------------------------------------
----

  package    * version date       source        

devtools     1.6.1   2014-10-07 CRAN (R 3.1.2)

package2     0.3     2014-11-24 local         

rstudioapi   0.1     2014-03-27 CRAN (R 3.1.1)

 

d> #Try again with a third (nonsense) entry in the versionCheck list

d> unloadNamespace('package2')

d> loadNamespace('package2', versionCheck = list(op='>=', version =
package_version('3.0'), please = 'check it'))

Error in loadNamespace("package2", versionCheck = list(op = ">=", version =
package_version("3.0"),  : 

 namespace 'package2' 0.3 is being loaded, but >= 3.0 is required

 

 

 

d> devtools::session_info()

Session
info--------------------------------------------------------------------

setting  value                       

version  R version 3.1.2 (2014-10-31)

system   x86_64, mingw32             

ui       RStudio (0.98.953)          

language (EN)                        

collate  English_Australia.1252      

tz       Australia/Sydney            

 

Packages--------------------------------------------------------------------
----

package    * version date       source        

devtools     1.6.1   2014-10-07 CRAN (R 3.1.2)

rstudioapi   0.1     2014-03-27 CRAN (R 3.1.1)

d> loadNamespace('package2', versionCheck = list(op='>=', version =
package_version('0.3'), please = 'check it'))

<environment: namespace:package2>

d>
Duncan Murdoch
# 
Summary:  I think you've found a bug.
On 26/11/2014, 5:39 PM, Geoff Lee wrote:
I would have assumed they are checked when you try to install the
package.  If you ask for version 3.0, and only have 0.3, the install
should fail.

But as you've found, the documents say something else...
According to what you have written, that should have failed.  So it
looks like a bug.
That does seem like a typo.

Duncan Murdoch
Geoff Lee
# 
Many thanks Duncan for the quick response. 

A bug is a relief in a way. I've been digging my way deeper into this (and
learning more as I go) for several days now - but it is a diversion from (a
diversion from) my main goal :-(

Is there somewhere specific I should report or log the bug or will that
happen from this mailing-list automatically? (I have seen the Bug Tracking
link on the r-project page and followed that, but don't yet have a Bugzilla
account, nor know the precise mechanics and protocols for notifying a bug)

Geoff

PS Building package2 - via devtools::build('package2', binary = TRUE) does
check and insist that I have an appropriate version of the 'to be imported'
package1 installed in my .libPath().

It was only when I simulated a user who has and older version of the
Imported package1 (by overwriting the later version of package 1 with an
earlier version), then trying to load or attach package2 that the (failure
of the version checking) at loadNamespace time for package1 became apparent
(well apparentish after quite a bit of digging)

-----Original Message-----
From: Duncan Murdoch [mailto:murdoch.duncan at gmail.com] 
Sent: Thursday, 27 November 2014 10:57 AM
To: Geoff Lee; r-devel at r-project.org
Subject: Re: [Rd] Problem understanding behaviour of versionCheck for
loadNamespace (and when versions for Imports packages are checked)

Summary:  I think you've found a bug.
On 26/11/2014, 5:39 PM, Geoff Lee wrote:
I would have assumed they are checked when you try to install the package.
If you ask for version 3.0, and only have 0.3, the install should fail.

But as you've found, the documents say something else...
...
"
later).
According to what you have written, that should have failed.  So it looks
like a bug.
That does seem like a typo.

Duncan Murdoch
version))."
bug?
Duncan Murdoch
#
On 26/11/2014, 5:39 PM, Geoff Lee wrote:
You need to look at how versionCheck is used in the existing code.  It
is used on items that were produced when the package was installed, and
in those, the length of the versionCheck entry really is 3.  For
example, the dplyr package imports "lazyeval (>= 0.1.8)".  That
specification becomes the list

 ..$ lazyeval  :List of 3
  .. ..$ name   : chr "lazyeval"
  .. ..$ op     : chr ">="
  .. ..$ version:Classes 'package_version', 'numeric_version'  hidden
list of 1
  .. .. ..$ : int [1:3] 0 1 8

So if you want to put one of these together, your versionCheck list
should be list(name = "package2", op = ">=", version =
package+version('3.0'))

Duncan Murdoch
Duncan Murdoch
#
On 26/11/2014, 7:38 PM, Geoff Lee wrote:
A little too quick, it seems...
If you can put together a simple set of steps to illustrate a bug, then
reporting on bugs.r-project.org is the way to get it recorded.
Reporting it on this list is really hit and miss.  But I think from your
previous description, something else is going on.  When I try to load a
namespace with a bad version number, I get an error:

loadNamespace("rgl", versionCheck=list(name="rgl", op=">",
version=package_version('3.0')))
Error in loadNamespace("rgl", versionCheck = list(name = "rgl", op =
">",  :
  namespace ?rgl? 0.95.1163 is being loaded, but > 3.0 is required

Duncan Murdoch
Geoff Lee
# 
Hi Duncan

The difference is that in your call to loadNamespace, the versionCheck list
has 3 components (name, op and version), whereas the documentation only
mentions 2 (op and version).
loadNamespace 'works' for me provided I add a third component to the list
(even a nonsense one).

What I haven't yet had the fortitude to do is track down through the code to
see what the arguments are (ie how many elements there are in the
versionCheck list) to any calls to loadNamespace that are generated
internally when the Imports to package2 are being loaded (either because
they are explicitly mentioned in the NAMESPACE file, or because they are
invoked by a "importedpackage1::fun_from_imported_package1" line of code).

I'll put a bug report together later today, and include my toy packages etc.

Once again thanks for looking at this for me

Geoff

-----Original Message-----
From: Duncan Murdoch [mailto:murdoch.duncan at gmail.com] 
Sent: Thursday, 27 November 2014 12:09 PM
To: Geoff Lee; r-devel at r-project.org
Subject: Re: [Rd] Problem understanding behaviour of versionCheck for
loadNamespace (and when versions for Imports packages are checked)
On 26/11/2014, 7:38 PM, Geoff Lee wrote:
A little too quick, it seems...
If you can put together a simple set of steps to illustrate a bug, then
reporting on bugs.r-project.org is the way to get it recorded.
Reporting it on this list is really hit and miss.  But I think from your
previous description, something else is going on.  When I try to load a
namespace with a bad version number, I get an error:

loadNamespace("rgl", versionCheck=list(name="rgl", op=">",
version=package_version('3.0')))
Error in loadNamespace("rgl", versionCheck = list(name = "rgl", op = ">",  :
  namespace 'rgl' 0.95.1163 is being loaded, but > 3.0 is required

Duncan Murdoch
imported'
attached.
3.0.0).
Duncan Murdoch
#
On 26/11/2014, 8:29 PM, Geoff Lee wrote:
As mentioned in a different message, the internal code creates a list
with members "name", "op", "version".  As you've noticed, the "name"
part is never used in the loadNamespace code.  I'm going to change the
code so that the documentation is correct.

But you said somewhere or other that you had an example where the
implicit load failed to do the check.  I don't see why it would.  So
that's what you should put in your bug report.

Duncan Murdoch
Geoff Lee
# 
Hi Duncan,

Many thanks (yet again).

With the hint given by your earlier email (viz that currently loadNamespace
expects a 3rd component called name in the list that is used for the
versionCheck argument) I had another look at what was going on with my toy
examples yesterday evening.  I'm still working on my issue, but thus far I
have:

1) Confirmed that internal calls to loadNamespace triggered by
import(somePackage) statements in the NAMESPACE file do create a
versionCheck argument that has 3 components in the list.  I deduce (but have
not worked my way through the specific lines of code insde loadNamespace)
that the name is needed while processing the 'import(somepackage)' NAMESPACE
directive at some stage (otherwise how do you know what installed
'somepackage' to import and find the version of!) but is *probably* not
needed in the versionCheck list argument itself when the loadNamespace call
is done (since by then the name 'somepackage' is used as the package
argument to the call).  I will (very very gratefully) leave to you the
resolution of the mismatch between the checkVersion documentation, and the
actual implementation - whether you change the documentation (easiest) or
the implementation (means tracking down and amending all the internally
generated calls) is not something I'm up to even contemplating at the
moment.

2) Discovered that the NAMESPACE file in my toy examples did not contain
correct import() or importfrom() statements. I'm still figuring out why (it
*appears* to be due to the way I have used devtools::create() and then
roxygen2 comments and then devtools::document() to generate my toy
examples.)  Once I figure out what is happening and have 'valid' toy
examples I will either a) 'shut up' (because I have found my own error) P
devtools itself  P ~ .3; or c)come back to this list since I am still
struggling P ~< .2

If the protocol is that I should record the complete resolution of my
question (for the benefit of future readers) pls let me know and I will do
that (once I get there!).

Geoff

PS Moral for me : Think twice before pulling on a loose piece of string.
I've learnt heaps - but I'm still not finished :-(  (At this stage the
benefits of the learning >> the pain of the journey, so I'll keep going)

-----Original Message-----
From: Duncan Murdoch [mailto:murdoch.duncan at gmail.com] 
Sent: Thursday, 27 November 2014 9:21 PM
To: Geoff Lee; r-devel at r-project.org
Subject: Re: [Rd] Problem understanding behaviour of versionCheck for
loadNamespace (and when versions for Imports packages are checked)
On 26/11/2014, 8:29 PM, Geoff Lee wrote:
"importedpackage1::fun_from_imported_package1" line of code).
etc.

As mentioned in a different message, the internal code creates a list with
members "name", "op", "version".  As you've noticed, the "name"
part is never used in the loadNamespace code.  I'm going to change the code
so that the documentation is correct.

But you said somewhere or other that you had an example where the implicit
load failed to do the check.  I don't see why it would.  So that's what you
should put in your bug report.

Duncan Murdoch
:
package.
1 day later
Geoff Lee
# 
Hi Duncan and others,

For the record I think I have pinned down what happens wrt checking the
version of an imported package.

In summary :

loadNamespace behaves as it ought to (once the discrepancy between the
documentation and reality of the versionCheck argument is taken into
account).

However (if packageA is the package to be imported) using the
"packageA::funA" idiom in packageB *instead* of using a NAMESPACE file with
either @import(package) or @importFrom(packageA, funA) seems to circumvent
the checking of whatever constraints on the version of packageA were given
in the packageB DESCRIPTION file.

I rather suspect this will fall into the category of "correct behaviour,
just not what I wanted) from the `::` function in this instance" rather than
a bug but I have recorded the behaviour, and made my toy examples available
as Bug 16094 on the R Bug reporting system.

Hoping this makes sense, and helps someone else later on down the track

Geoff

PS Along the way I had a problem with devtools and roxygen2 - it was my
error! I had a typo - a backtick ` when I should have had a single quote '
in a roxygen2 comment :-(

-----Original Message-----
From: Geoff Lee [mailto:geoff.lee99 at gmail.com] 
Sent: Friday, 28 November 2014 8:12 AM
To: 'Duncan Murdoch'; r-devel at r-project.org
Subject: RE: [Rd] Problem understanding behaviour of versionCheck for
loadNamespace (and when versions for Imports packages are checked)

Hi Duncan,

Many thanks (yet again).

With the hint given by your earlier email (viz that currently loadNamespace
expects a 3rd component called name in the list that is used for the
versionCheck argument) I had another look at what was going on with my toy
examples yesterday evening.  I'm still working on my issue, but thus far I
have:

1) Confirmed that internal calls to loadNamespace triggered by
import(somePackage) statements in the NAMESPACE file do create a
versionCheck argument that has 3 components in the list.  I deduce (but have
not worked my way through the specific lines of code insde loadNamespace)
that the name is needed while processing the 'import(somepackage)' NAMESPACE
directive at some stage (otherwise how do you know what installed
'somepackage' to import and find the version of!) but is *probably* not
needed in the versionCheck list argument itself when the loadNamespace call
is done (since by then the name 'somepackage' is used as the package
argument to the call).  I will (very very gratefully) leave to you the
resolution of the mismatch between the checkVersion documentation, and the
actual implementation - whether you change the documentation (easiest) or
the implementation (means tracking down and amending all the internally
generated calls) is not something I'm up to even contemplating at the
moment.

2) Discovered that the NAMESPACE file in my toy examples did not contain
correct import() or importfrom() statements. I'm still figuring out why (it
*appears* to be due to the way I have used devtools::create() and then
roxygen2 comments and then devtools::document() to generate my toy
examples.)  Once I figure out what is happening and have 'valid' toy
examples I will either a) 'shut up' (because I have found my own error) P
devtools itself  P ~ .3; or c)come back to this list since I am still
struggling P ~< .2

If the protocol is that I should record the complete resolution of my
question (for the benefit of future readers) pls let me know and I will do
that (once I get there!).

Geoff

PS Moral for me : Think twice before pulling on a loose piece of string.
I've learnt heaps - but I'm still not finished :-(  (At this stage the
benefits of the learning >> the pain of the journey, so I'll keep going)

-----Original Message-----
From: Duncan Murdoch [mailto:murdoch.duncan at gmail.com]
Sent: Thursday, 27 November 2014 9:21 PM
To: Geoff Lee; r-devel at r-project.org
Subject: Re: [Rd] Problem understanding behaviour of versionCheck for
loadNamespace (and when versions for Imports packages are checked)
On 26/11/2014, 8:29 PM, Geoff Lee wrote:
"importedpackage1::fun_from_imported_package1" line of code).
etc.

As mentioned in a different message, the internal code creates a list with
members "name", "op", "version".  As you've noticed, the "name"
part is never used in the loadNamespace code.  I'm going to change the code
so that the documentation is correct.

But you said somewhere or other that you had an example where the implicit
load failed to do the check.  I don't see why it would.  So that's what you
should put in your bug report.

Duncan Murdoch
:
package.