Skip to content

[Bioc-devel] Troubles with Windows build

3 messages · Hervé Pagès, Ulrich Bodenhofer

Original
Ulrich Bodenhofer
# 
Dear colleagues,

I have two issues with the Windows BUILD BIN of our 'msa' package. (to 
my horror, I figured out that the first problem has existed for quite a 
while; I am deeply sorry for that!)

The main problem can be seen on

    http://bioconductor.org/checkResults/3.10/bioc-LATEST/msa/tokay1-buildbin.html

It seems that something is wrong with the file 
src/ClustalOmega/src/exceptions4c/e4c_lite.o. Since it does not seem to 
me that e4c_lite.c is freshly compiled (I may be wrong in this point) 
and based on a StackOverflow post 
(https://stackoverflow.com/questions/54797615/trouble-downloading-r-package-o-file-format-not-recognized), 
I have the impression that the build system might still have an on old 
version of some .o files. I can hardly imagine this, but it would 
explain the problem. To be frank, I am quite lost and have no idea what 
else could be the problem. Any hint is gratefully appreciated!

The second issue: the 'msa' package integrates multiple other open 
source packages in one shared library. In order to to so, it uses the 
'ar' tool to compile static libraries of the individual packages and 
then combines them to a shared library. This has always worked fine, and 
on the Bioconductor build servers, too. Now it seems I can no longer run 
the binary build on my Windows system. My Windows makefiles refer to the 
'$(AR)' variable, but it seems this variable is not correctly expanded 
to the actual location of the 'ar' tool (in Rtools 35, it is not in 
Rtools\bin, but in Rtools\mingw_(32|64)\bin) . Any thoughts on this?

Thank you very much in advance and best regards,
Ulrich
Hervé Pagès
# 
Hi Ulrich,
On 9/11/19 07:38, Ulrich Bodenhofer wrote:
On our Windows servers the BUILD BIN stage uses the following compound 
command to produce the binary package:

   rm -rf msa.buildbin-libdir && mkdir msa.buildbin-libdir && 
C:\Users\biocbuild\bbs-3.10-bioc\R\bin\R.exe CMD INSTALL 
--merge-multiarch --build --library=msa.buildbin-libdir msa_1.17.0.tar.gz

(Note that you can see this command at the top of the details page for 
this stage).

As you can see, we use --library=msa.buildbin-libdir to install to a 
freshly created folder created at the beginning of the command. Command 
components are chained with && to make sure that it will not proceed 
further if one component fails so the 'R CMD INSTALL' component won't 
run if for example 'rm -rf msa.buildbin-libdir' fails which guarantees 
that we start with a fresh folder.

I think this looks indeed like a leftover/stale .o file but I think the 
file is in the source tarball produced by the BUILD stage 
(msa_1.17.0.tar.gz). If you look at your cleanup.win script, it doesn't 
remove the stuff in src/ClustalOmega/src/exceptions4c/ so I think what 
happens is that at the end of compilation the folder contains all the 
*.o files for the 64-bit arch and these files get included in the source 
tarball. Then the BUILD BIN stage choke on these files during the 
'install for i386' step.

'diff cleanup cleanup.win' actually shows many differences between the 2 
scripts. You might want to review them closely as it looks like some of 
them are not intended.
AFAIK, 'ar' stands for "archive". It doesn't compile anything, it only 
bundles things in one single *static* (not shared) library (.a file).
Don't know. You might want to try SO or R-devel for this one.

Cheers,
H.

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Ulrich Bodenhofer
    

    

      
      #
    

  


  

      
Hi Herv?,

Thanks a lot for your very helpful and insightful reply!

            

      
      
      
    

        
The hint to cleanup.win was great! I have no idea why the exceptions4c/ 
directory was missing in cleanup.win, but it seems that, according to my 
tests, that was actually the source of the problem. The other 
differences between cleanup and cleanup.win are either intended or 
benign (to be frank, I'd rather not touch this sensitive area as long as 
everything seems to work fine).

            

      
      
      
    

        
Sure! I meant 'compile' in the sense of 'putting together', not in the 
sense of 'translating source code into machine code'.

            

      
      
      
    

        
In the meantime, I have found a very slight and implicit hint to this in 
Section "1.2.1.3 Compiling in sub-directories" of "Writing R 
Extensions". The example at the bottom of this section explicitly 
forwards the 'AR' environment variable when running 'make' in a 
sub-directory. I have now included this in my Makevars.win and it seems 
that the second issue is now solved, too. However, I still have no idea 
why this has worked for so long and it still worked without this on the 
BioC build server. My only explanation is that the way 'R CMD SHLIB' 
handles environment variables has changed with a recent R version. I 
have not heard of this, but, to be frank, I do not have the time to 
follow all changes in detail.

Anyway, it seems everything is fine now. Thanks again!

Best regards,
Ulrich