Skip to content

[R-pkg-devel] Strange error from CRAN on package submission

15 messages · Uwe Ligges, Kevin Coombes, Martin Morgan +1 more

#
Hi,

I am trying to figure out how to fix warnings from two of the CRAN 
machines on the submission of an update to a package. The only change to 
my package was to add a "show" method to one of the S4 classes, which 
was requested by a reviewer of the paper we submitted. The inability to 
get this updated package into CRAN? is the only thing holding up the 
revision (and probable acceptance) of the manuscript.

The same "warnings"s were found in the previous version. The package is 
called Mercator, and the CRAN check results from the? last version are here:
 ? https://cran.r-project.org/web/checks/check_results_Mercator.html

I get warnings from the two fedora machine instances (clang and gcc). 
They both report
>
 > Check: data for non-ASCII characters
The relationships in the DESCRIPTION files are:

1. Mercator depends on Thresher
2. Thresher imports moVMF
3. moMVF suggests flexmix

On my Windows machine, the package builds and installs with no errors or 
warnings even if flexmix is not available (which I believe to be the 
correct behavior). On R-Forge, both the Windows and LINUX versions build 
and install with no errors or warnings. On R-Hub, tested on multiple 
LINUX versions, the package builds and installs with no errors or warnings.

And flexmix is still clearly available from CRAN:
 ? https://cran.r-project.org/web/packages/flexmix/index.html

In the latest attempt to get things to work, I added
 ? Suggests: flexmix
into the DESCRIPTION file for Mercator, but this didn't help fix the 
problem on CRAN.

Is there anything I can do to fix this problem (other than moan here on 
this list and hope that CRAN can just install flexmix on those machines)?

Thanks in advance for your help,
 ? Kevin
#
You have to suggest flexmix.

Best,
Uwe Ligges
On 11.11.2020 14:44, Kevin R. Coombes wrote:
#
Uwe suggested you suggest flexmix, but I see below you already tried that.

I'd like to take a look, but I can't find your package.  The existing 
version on CRAN gives the URL as http://oompa.r-forge.r-project.org/, 
but I can't see it mentioned there.

Duncan Murdoch
On 11/11/2020 8:44 a.m., Kevin R. Coombes wrote:
#
Hi Uwe,

Thanks for the suggestion. I thought of that idea last night, and tried 
it with the latest submission of version 0.11.4 of Mercator. That 
version is still in the submission queue waiting for manual inspection, 
but it gave the same error message. (See 
https://win-builder.r-project.org/incoming_pretest/Mercator_0.11.4_20201111_014655/Debian/00check.log)

I think I may try to "Import" or "Depend" on flexmix to force it to be 
available.

Is there some reason why my program should even need to know that 
flexmix is suggested several layers down the dependency hierarchy? For 
building and installing the package (or running R CMD check ---as-cran), 
I tried on my machine when flexmix is not even installed, and got no 
errors or warnings.

There is clearly something I don't understand about namespaces, and I'd 
like to learn it to avoid this kind of situation in the future.

Best,
 ? Kevin
On 11/11/2020 11:14 AM, Uwe Ligges wrote:
#
Next guess is that you need more, because you may have an object that 
needs the flexmix, so likely something S4 related? I can take a closer look.

Best,
Uwe Ligges
On 11.11.2020 17:30, Duncan Murdoch wrote:
#
Hi Duncan,

Oops; I didn't realize I had forgotten to push updates to the OOMPA web 
site.

The code for Mercator is contained as part of the Thresher project in 
the subversion repository on R-Forge. 
(https://r-forge.r-project.org/projects/thresher/) It's under 
pkg/Mercator below that URL

Thanks,
 ? Kevin
On 11/11/2020 11:30 AM, Duncan Murdoch wrote:
#
Okay, I've tried testing on my Mac with R 4.0.3 and R-devel for the new 
one, 4.0.3 for the CRAN version.

I'm not seeing any check error with the CRAN version.  I get an error 
trying to check 0.11.4 from R-forge because I don't have flexmix 
installed.  If I take flexmix out of the Suggests list, it checks with 
no error on 4.0.3, but I get the error you saw on R-devel when checked 
with --as-cran.

I tried debugging this, and narrowed it down a bit.  It happens when 
your package is installed, in particular in the do_install_source() 
function in src/library/tools/R/install.R.  But that function runs a new 
R instance, and I didn't get to debugging that.  I'll try again later 
today if nobody else figures it out.

Duncan Murdoch
On 11/11/2020 12:03 p.m., Kevin R. Coombes wrote:
#
HI Uwe,

That might be the key observation. The change to Mercator in this 
package was to add a "show" method to an S4 class. In its NAMESPACE, the 
flexmix package also exports a "show" method.

Both "flexmix" and "Mercator" have an
 ? import("methods")
directive in their NAMESPACE files, which is where the base method for 
"show" comes from. Also, if you just type show at the command prompt to 
see the definition, you get the parenthetical note that
"(This generic function excludes non-simple inheritance' see ?setIs.)"
I don't completely understand what this parenthetical remark? means, but 
I have a suspicion that it is highly relevant to my problem.

If I run
 ? library("Mercator")
 ? showMethods("show")
on a system where flexmix is installed, then the "show" method for 
flexmix (and related objects) is listed, because the flexmix NAMESPACE 
has been attached. If I run the same code on an R system where flexmix 
is not installed, then, of course, neither the NAMESPACE nor the method 
is available.

I suspect (but am by no means certain) that the combination of that 
parenthetical remark above and the existence of the "show" method in the 
suggested package is why I am getting errors on some systems.

But I don't really understand why. My package doesn't need the flexmix 
version of show. And (as Uwe said in an earlier comment), I thought that 
"Suggests" isn't supposed to be inherited. I don't want the flexmix 
NAMESPACE attached, since nothing in my package nor in the packages I 
directly want (Thresher by "Depends" and then moVMF by "Imports") 
actually requires it.

Why does the flexmix NAMESPACE get attached if some other package down 
the line merely suggests it? Is that supposed to happen? To me, it feels 
like a bug in the sense that it surprises the user (i.e., the package 
developer). And I guess is potentially a bug for the ultimate user of 
the package, since it adds a NAMESPACE that was not specifically 
requested by the top level package being loaded.

In any event, what's the best advice now on how to proceed?

Thanks again,
 ? Kevin
On 11/11/2020 11:34 AM, Uwe Ligges wrote:
#
Hi Duncan,

I just sent a longer version of this message, but it looks to me like 
the underlying issue is the fact that flexmix and Mercator both define 
and export "show" methods for their S4 classes.? What confuses me is why 
the NAMESPACE of a package that is merely Suggest'ed by something 
several layers down the hierarchy should get attached and cause an issue 
like this one. (The attached NAMESPACE happens in current versions of R.)

Thanks,
 ? Kevin
On 11/11/2020 1:07 PM, Duncan Murdoch wrote:
#
Oh, I forgot to mention explicitly that checking (with --as-cran) on the 
development version of R on Windows also produces no errors or warnings.
On 11/11/2020 1:39 PM, Kevin R. Coombes wrote:
#
Here's what I think is happening.

In the movMF:::.onLoad function there's a test whether flexmix is 
installed.  If found, then it is loaded and some methods are set.  (I'm 
not sure what caused flexmix to be installed:  I didn't intentionally 
install it, but it ended up in there when I installed enough stuff to 
check Mercator.)

In the R-devel --as-cran checks, some checks are run with only strong 
dependencies of your package visible.  Somehow I think that .onLoad 
function sees flexmix and loads it, but then some other part of the 
check can't see it.

A workaround is to add flexmix to your Imports clause.  This is a strong 
enough dependency to make it visible, and the error goes away.

HOWEVER, to me this is pretty clearly an R-devel bug:  you have no 
control over methods set by packages that you don't even use, so you 
shouldn't have to change your dependency lists if one of them sets a 
method that you're using.

Duncan Murdoch
On 11/11/2020 3:31 p.m., Kevin R. Coombes wrote:
#
Hi Duncan,

Thanks for the help. For now (since I want my package to get into CRAN 
so I can resubmit my paper), I'll add the "Import" clause, and write 
myself a note to try removing it later.

Best,
 ? Kevin
On 11/11/2020 4:44 PM, Duncan Murdoch wrote:
#
This seems more like a problem with the CRAN test machine, with the movMF package installed with flexmix available but loaded with flexmix not available, maybe interacting with a caching mechanism used by the methods package to avoid re-computing methods tables? Otherwise how would movMF ever know to create the flexmix class / method?

It seems like this could cause problems for the user if they installed movMV with flexmix available, but removed flexmix. This seems like a subtler variation of 'I installed package A but then removed dependency B and now A doesn't work', which could be a bug in R's remove.packages() but....

I tried to emulate the scenario of installing movMF and then removing flexmix in an interactive session, and then looking for the warning reported below. I was not successful, but the build report with the error is no longer available so I don't know what I'm looking for...

Martin Morgan

?On 11/11/20, 4:44 PM, "R-package-devel on behalf of Duncan Murdoch" <r-package-devel-bounces at r-project.org on behalf of murdoch.duncan at gmail.com> wrote:

    Here's what I think is happening.

    In the movMF:::.onLoad function there's a test whether flexmix is 
    installed.  If found, then it is loaded and some methods are set.  (I'm 
    not sure what caused flexmix to be installed:  I didn't intentionally 
    install it, but it ended up in there when I installed enough stuff to 
    check Mercator.)

    In the R-devel --as-cran checks, some checks are run with only strong 
    dependencies of your package visible.  Somehow I think that .onLoad 
    function sees flexmix and loads it, but then some other part of the 
    check can't see it.

    A workaround is to add flexmix to your Imports clause.  This is a strong 
    enough dependency to make it visible, and the error goes away.

    HOWEVER, to me this is pretty clearly an R-devel bug:  you have no 
    control over methods set by packages that you don't even use, so you 
    shouldn't have to change your dependency lists if one of them sets a 
    method that you're using.

    Duncan Murdoch
On 11/11/2020 3:31 p.m., Kevin R. Coombes wrote:
> Oh, I forgot to mention explicitly that checking (with --as-cran) on the
    > development version of R on Windows also produces no errors or warnings.
    >
> On 11/11/2020 1:39 PM, Kevin R. Coombes wrote:
>> Hi Duncan,
    >>
    >> I just sent a longer version of this message, but it looks to me like
    >> the underlying issue is the fact that flexmix and Mercator both define
    >> and export "show" methods for their S4 classes.  What confuses me is
    >> why the NAMESPACE of a package that is merely Suggest'ed by something
    >> several layers down the hierarchy should get attached and cause an
    >> issue like this one. (The attached NAMESPACE happens in current
    >> versions of R.)
    >>
    >> Thanks,
    >>    Kevin
    >>
>> On 11/11/2020 1:07 PM, Duncan Murdoch wrote:
>>> Okay, I've tried testing on my Mac with R 4.0.3 and R-devel for the
    >>> new one, 4.0.3 for the CRAN version.
    >>>
    >>> I'm not seeing any check error with the CRAN version.  I get an error
    >>> trying to check 0.11.4 from R-forge because I don't have flexmix
    >>> installed.  If I take flexmix out of the Suggests list, it checks
    >>> with no error on 4.0.3, but I get the error you saw on R-devel when
    >>> checked with --as-cran.
    >>>
    >>> I tried debugging this, and narrowed it down a bit.  It happens when
    >>> your package is installed, in particular in the do_install_source()
    >>> function in src/library/tools/R/install.R. But that function runs a
    >>> new R instance, and I didn't get to debugging that.  I'll try again
    >>> later today if nobody else figures it out.
    >>>
    >>> Duncan Murdoch
    >>>
    >>>
    >>>
    >>>
>>> On 11/11/2020 12:03 p.m., Kevin R. Coombes wrote:
>>>> Hi Duncan,
    >>>>
    >>>> Oops; I didn't realize I had forgotten to push updates to the OOMPA web
    >>>> site.
    >>>>
    >>>> The code for Mercator is contained as part of the Thresher project in
    >>>> the subversion repository on R-Forge.
    >>>> (https://r-forge.r-project.org/projects/thresher/) It's under
    >>>> pkg/Mercator below that URL
    >>>>
    >>>> Thanks,
    >>>>      Kevin
    >>>>
>>>> On 11/11/2020 11:30 AM, Duncan Murdoch wrote:
>>>>> Uwe suggested you suggest flexmix, but I see below you already tried
    >>>>> that.
    >>>>>
    >>>>> I'd like to take a look, but I can't find your package.  The existing
    >>>>> version on CRAN gives the URL as http://oompa.r-forge.r-project.org/,
    >>>>> but I can't see it mentioned there.
    >>>>>
    >>>>> Duncan Murdoch
    >>>>>
>>>>> On 11/11/2020 8:44 a.m., Kevin R. Coombes wrote:
>>>>>> Hi,
    >>>>>>
    >>>>>> I am trying to figure out how to fix warnings from two of the CRAN
    >>>>>> machines on the submission of an update to a package. The only
    >>>>>> change to
    >>>>>> my package was to add a "show" method to one of the S4 classes, which
    >>>>>> was requested by a reviewer of the paper we submitted. The
    >>>>>> inability to
    >>>>>> get this updated package into CRAN  is the only thing holding up the
    >>>>>> revision (and probable acceptance) of the manuscript.
    >>>>>>
    >>>>>> The same "warnings"s were found in the previous version. The
    >>>>>> package is
    >>>>>> called Mercator, and the CRAN check results from the  last version
    >>>>>> are here:
    >>>>>> https://cran.r-project.org/web/checks/check_results_Mercator.html
    >>>>>>
    >>>>>> I get warnings from the two fedora machine instances (clang and gcc).
    >>>>>> They both report
    >>>>>>
    >>>>>>> Check: whether package can be installed.
    >>>>>>> Result: WARN
    >>>>>>>         Found the following significant  warnings:
    >>>>>>>         Warning: namespace ?flexmix? is  not available and has been
    >>>>>>> replaced
    >>>>>>     >
    >>>>>>     > Check: data for non-ASCII characters
    >>>>>>> Result: WARN
    >>>>>>>          Warning: namespace 'flexmix'  is not available and has been
    >>>>>>> replaced
    >>>>>>>          by .GlobalEnv when processing  object '<unknown>'
    >>>>>>
    >>>>>> The relationships in the DESCRIPTION files are:
    >>>>>>
    >>>>>> 1. Mercator depends on Thresher
    >>>>>> 2. Thresher imports moVMF
    >>>>>> 3. moMVF suggests flexmix
    >>>>>>
    >>>>>> On my Windows machine, the package builds and installs with no
    >>>>>> errors or
    >>>>>> warnings even if flexmix is not available (which I believe to be the
    >>>>>> correct behavior). On R-Forge, both the Windows and LINUX versions
    >>>>>> build
    >>>>>> and install with no errors or warnings. On R-Hub, tested on multiple
    >>>>>> LINUX versions, the package builds and installs with no errors or
    >>>>>> warnings.
    >>>>>>
    >>>>>> And flexmix is still clearly available from CRAN:
    >>>>>> https://cran.r-project.org/web/packages/flexmix/index.html
    >>>>>>
    >>>>>> In the latest attempt to get things to work, I added
    >>>>>>       Suggests: flexmix
    >>>>>> into the DESCRIPTION file for Mercator, but this didn't help fix the
    >>>>>> problem on CRAN.
    >>>>>>
    >>>>>> Is there anything I can do to fix this problem (other than moan
    >>>>>> here on
    >>>>>> this list and hope that CRAN can just install flexmix on those
    >>>>>> machines)?
    >>>>>>
    >>>>>> Thanks in advance for your help,
    >>>>>>       Kevin
    >>>>>>
    >>>>>>       [[alternative HTML version deleted]]
    >>>>>>
    >>>>>> ______________________________________________
    >>>>>> R-package-devel at r-project.org mailing list
    >>>>>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
    >>>>>>
    >>>>>
    >>>>
    >>>
    >>
    >

    ______________________________________________
    R-package-devel at r-project.org mailing list
    https://stat.ethz.ch/mailman/listinfo/r-package-devel
#
Hi Martin,

I think you may be right that it is something odd about the 
configuration on the test machine, since I haven't been able to 
reproduce it anywhere else. But Duncan did say he could reproduce it in 
R-devel but not R-4.0.3, so there's that.

The error message is buried deep in this thread, but it was:

Check: whether package can be installed.
Result: WARN
 ??? Found the following significant? warnings:
 ??? Warning: namespace ?flexmix? is? not available and has been replaced

Check: data for non-ASCII characters
Result: WARN
 ??? Warning: namespace 'flexmix'? is not available and has been replaced
 ??? by .GlobalEnv when processing? object '<unknown>'

Best,
 ? Kevin
On 11/12/2020 10:13 AM, Martin Morgan wrote:
#
Actually I think it is a bug in the check code.  I've just posted about 
this on the R-devel list.

Duncan Murdoch
On 12/11/2020 10:13 a.m., Martin Morgan wrote: