Skip to content

[Bioc-devel] Shouldn't we distinguish between package-specific and dependency errors?]

5 messages · Ludwig Geistlinger, Michael Lawrence, Gabriel Becker +2 more

Original
Ludwig Geistlinger
# 
Dan, thanks for clarifying.
With 'we can hardly do much about it', I meant that we cannot prevent that
for external dependencies in the way we can prevent it for dependendencies
within Bioc.

Question remains whether the landing page for the USER of the package is
the right place to alert the DEVELOPER of the package.

Best,
Ludwig


----- Original Message -----
package-specific and dependency errors?
The purpose of the build shield is to alert you to the fact that the build
is broken. If the build is broken due to a dependency, it's not true that
there is nothing you can do about it; as Michael points out, you can alert
the maintainer of the broken package or you can (as I did) contact KEGG
who promptly fixed their issue. This benefits the community as a whole.

There are other types of dependency-related errors, for example if a
package you depend on changes its API and you do not adapt to those
changes, your package will break, but YOU need to fix your package, nobody
else's package needs to change.

I think it is exceedingly difficult to determine programmatically whether
a given failure was caused by a dependency or by the package itself, and
I'm not sure it's a good idea to try.

I recognize that it can be bad for a reviewer to see the red build shield.
But the purpose is to alert the DEVELOPER to problems and I would
reiterate that there is always something you as the package author can do,
whether it's alerting the upstream developer to the problem, or if that
doesn't work, removing the dependency.

Dan

  
    

      
      
    

  


  


    

    

      
      

        


  

        

      Michael Lawrence
    

    

      
      #
    

  


  

      
The important question is whether the package actually works, as
distributed. if not, it's a user matter. If a build is failing because
there is a problem with the "next" version of the package, or something
specific to the build machine, it's a developer/admin matter. I'm guessing
we don't routinely test packages without version bumps, but perhaps we
should, at least when their deps change. Maybe certain packages that depend
on external resources could be tested on a regular but less frequent basis,
regardless.


On Thu, Sep 24, 2015 at 11:19 AM, Ludwig Geistlinger <

        
Ludwig.Geistlinger at bio.ifi.lmu.de> wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Gabriel Becker
    

    

      
      #
    

  


  

      
I agree with Michael. External dependencies are not avoidable in our field,
but they do put the user in a bad situation with respect to trusting their
software (and performing reproducible analyses).

If I have KEGGREST version x,y,z installed, and it was passing all it's
tests when it was deployed (and, we hope, later when I installed it), but
KEGG changes and KEGGREST no longer works, that is very much a problem for
me, as the user - it means the package is broken. Note that the package
being broken does not - directly - speak to the talent, dedication, etc of
the developer. Ludwig, you're correct in your (implied?) assertion that the
breakage isn't the *fault* of the package author, or any other package
developer, but I'm skeptical that that matters at all to the user.
Ultimately, the package is either working (and thus safe to use) or
"broken" (and thus in a use-at-your-own-risk sort of state).

As for reviewers, a good reviewer should assess the exact version of your
software that the paper is about (assuming you provided one). If *that
version* of the package works, the review shouldn't be negatively impacted.
If it doesn't, honestly I think the review *should* be negatively impacted,
even if the breakage is not really the author's fault. The reviewer's
allegiance should lie with the journal, and by extension the future
readers, who would expect the software to work as described.

~G

On Thu, Sep 24, 2015 at 11:50 AM, Michael Lawrence <

        
lawrence.michael at gene.com> wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Martin Morgan
    

    

      
      #
    

  


  

          

      
      
      
    

        
Packages are built and checked nightly, regardless of version bump. Only version bumps (and successful build / check) trigger a push to the public bioc repository. The build errors that Ludwig is concerned about typically are the result of these nightly builds catching incompatible changes in other packages.

In these cases the bioc packages that _are_ available via biocLite() (because they built before the incompatible change) are no longer valid; it seems it is particularly important to alert the user, including users who have already installed the bioc package, that there are problems. It is not possible to 'role back' the Bioc package (because there is no guarantee that the older version worked, and because R installs newer versions, not older versions). In terms of our hypothetical reviewer, the shield accurately conveys the situation they would experience if they were to install the software.

It might be helpful to remember that the shields on the release and devel pages are independent of one another -- the carnage of a bad check-in of a new feature (in devel, of course!) is not reflected on the release landing pages.

Roughly, I view the top line of shields as particularly useful to users; the second line is more developer oriented but still conveying relevant information to our more ardent users. In both cases I think the shields do a good job of making problems more apparent to the community in general, and hence contribute to better overall software.

There are 'best practices' that package developers can follow to mitigate the consequences of API changes in their package, especially following a strict deprecation cycle; the separation of 'release' and 'devel' versions of Bioconductor facilitate this. Likewise, package developers have a responsibility to their users to convey problems 'upstream' to be fixed at the source.

Bioconductor does have a more dense dependency graph than CRAN. Generally I think this is good, reflecting valuable software re-use rather than re-invention; the release / devel split also makes this approach viable when the dependencies are within Bioconductor. It is unfortunate when a domain specific package offers some functionality that is more generally useful, introducing a cascade of more-or-less irrelevant dependencies. In these cases it may well be worth-while to re-factor or identify the generally useful functionality into a new or different package, e.g., implementing or using rtracklayer::import(). If there are candidates for such re-factoring then the Bioc-devel mailing list is an appropriate venue for discussion.

Martin

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        
This email message may contain legally privileged and/or confidential information.  If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited.  If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.

  
  


  


      

    

    

      
      

        


  

        

      Kasper Daniel Hansen
    

    

      
      #
    

  


  

      
Two comments

1) I would not consider a package breaking in level a big deal.  This is
why we have devel.
2) Technically, Bioc packages should break extremely rarely in release,
because of the rules regarding committing changes to the release branch.
Unfortunately, there is often a lot of updates in the release branch of
various packages, which kind of goes against the general idea of having a
stable release in the first place.

Best,
Kasper

On Thu, Sep 24, 2015 at 4:57 PM, Morgan, Martin <

        
Martin.Morgan at roswellpark.org> wrote: