Skip to content

[Bioc-devel] HDF5Array failure on windows

5 messages · Martin Morgan, Kasper Daniel Hansen, Hervé Pagès

Original
Kasper Daniel Hansen
# 
We currently (and for a while) have had various errors in bsseq that seems
to have come and go. We now have a failure on Windows which is related to
HDF5. I see that HDF5Array also fails on Windows, which makes me believe
the error could be upstream. There is also a warning about hep page links
which HDF5Array has as well. Any comments on this is appreciated.

Perhaps relatedly: we are getting a timeout on linux. On my own OS X setup,
it completes in a timely fashion. Do we sometimes have unexplained timeouts
related to HDF5?
Martin Morgan
# 
Timeouts are often related to parallel evaluation, either competing for resources or underlying limitations in the robustness of the (parallel) code. Something close to best practice is to limit or eliminate parallel evaluation in examples, vignettes, and tests.

Documentation issues are usually related to the use of [] in \link[]{}, as described at https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Cross_002dreferences . The important point is that the [] information is the NAME OF THE HTML FILE of the documentation module, not the 'name' of the documentation module. The solution (from my perspective) is usually to simply omit the [] and let the R help system resolve the link dynamically (sometimes prompting the user to choose, if there multiple man pages).

Martin Morgan


?On 4/6/20, 9:55 AM, "Bioc-devel on behalf of Kasper Daniel Hansen" <bioc-devel-bounces at r-project.org on behalf of kasperdanielhansen at gmail.com> wrote:

    We currently (and for a while) have had various errors in bsseq that seems
    to have come and go. We now have a failure on Windows which is related to
    HDF5. I see that HDF5Array also fails on Windows, which makes me believe
    the error could be upstream. There is also a warning about hep page links
    which HDF5Array has as well. Any comments on this is appreciated.
    
    Perhaps relatedly: we are getting a timeout on linux. On my own OS X setup,
    it completes in a timely fashion. Do we sometimes have unexplained timeouts
    related to HDF5?
    
    -- 
    Best,
    Kasper
    
    
    _______________________________________________
    Bioc-devel at r-project.org mailing list
    https://stat.ethz.ch/mailman/listinfo/bioc-devel
Hervé Pagès
# 
Hi Kasper, Martin,

About bsseq's timeout: An important recent change to 
DelayedArray/HDF5Array is that, starting with DelayedArray 0.13.8, 
nothing in these packages uses parallel evaluation **by default**. 
Concretely this means that getAutoBPPARAM() now returns NULL on a fresh 
session instead of one of the parallelization backends defined in 
BiocParallel (e.g. MulticoreParam on Linux/Mac, SnowParam on Windows). 
This could slow down some code in bsseq or other packages that were 
benefiting from the automatic parallel evaluation. Now it's the 
responsibility of the user to set the parallelization backend (with 
setAutoBPPARAM) if they wish things like matrix multiplication, rowsum() 
or rowSums() use parallel evaluation again.

Also BiocParallel has been moved from Depends to Suggests.

About bsseq error on Windows: That seems indeed to be related to the 
HDF5Array error on the same platform. The unit tests in bsseq and 
HDF5Array both fail with the same error ("HDF5 dataset 
'./.HDF5ArrayAUTO00014_dimnames/2' does not exist"). I'll take a look. 
What's puzzling is that we see this error on both Windows archs for 
bsseq (i.e. on 64-bit and 32-bit) while we only see it on 64-bit Windows 
for HDF5Array. This suggests something nasty and hard to troubleshoot.. 
sigh!

Cheers,
H.
On 4/6/20 06:59, Martin Morgan wrote:

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Kasper Daniel Hansen
    

    

      
      #
    

  


  

      
Thanks for the two pointers.

Herve: we are using a number of unexpected functions from HDF5Array:
  ?HDF5Array:::.create_dir?, ?HDF5Array:::.replace_dir?,
?HDF5Array:::.shorten_assay2h5_links?
That might be relevant in light of that specific test error.

        
On Mon, Apr 6, 2020 at 1:13 PM Herv? Pag?s <hpages at fredhutch.org> wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    
2 days later

    

      
      

        


  

        

      Hervé Pagès
    

    

      
      #
    

  


  

      
On 4/7/20 06:15, Kasper Daniel Hansen wrote:

            

      
      
      
    

        
tss tss... naughty!

            

      
      
      
    

        
OK, always good to know, thanks! The error seems to be related to the 
code I added recently for handling on-disk dimnames. I only started to 
look at this and will report here when I find something.

H.