Skip to content

Finding windows DLLs

14 messages · Duncan Murdoch, Oleg Sklyar, Brian Ripley +2 more

Original
Martin Morgan
# 
The XML package relies on libxml2.dll (e.g., bundled with the CRAN
binary) installed in library/XML/libs. Unfortunately,
c:/WINDOWS/system32/libxml2.dll will be found and loaded before
this.

Is there any programatic solution?

Thanks,

Martin
Duncan Murdoch
#
On 1/7/2008 2:51 PM, Martin Morgan wrote:
Search order for DLLs is version dependent on Windows.  The best way to 
be sure of what you're loading is to fully specify the path to the DLL, 
and this generally requires you to use API calls LoadLibrary() and 
GetProcAddress() rather than relying on automatic loading of dependencies.

I don't know how many entry points XML is importing from libxml2.dll; if 
there are a lot, LoadLibrary() will be a pain to use, and it might be 
easiest to rename libxml2.dll and hope to avoid a collision.

Duncan Murdoch
Oleg Sklyar
# 
Should adding PREFIX/library/XML/libs to PATH before system32 solve the 
issue as Windows relies on PATH when searching for libs as well?

Dr Oleg Sklyar | EBI-EMBL, Cambridge CB10 1SD, UK | +44-1223-494466
Martin Morgan wrote:
Duncan Murdoch
#
On 1/7/2008 3:06 PM, Oleg Sklyar wrote:
No, system32 is searched before the PATH.  See

http://msdn2.microsoft.com/en-us/library/ms682586.aspx

That page doesn't discuss Win9X, NT4, or Win2K; I don't recall for sure, 
but I suspect there were earlier changes to the search order as well.  R 
currently supports all of these, and 2.7 will continue to support Win2k 
or later.

Duncan Murdoch
Brian Ripley
#
On Mon, 7 Jan 2008, Oleg Sklyar wrote:

            

      
      
      
    

        
The Windows code for package XML says

            

      
      
      
    

        
function (libname, pkgname)
{
     if (.Platform$OS.type == "windows") {
         temp <- Sys.getenv("PATH")
         Sys.setenv(PATH = paste(utils::normalizePath(file.path(libname,
             pkgname, "libs")), temp, sep = ";"))
         on.exit(Sys.setenv(PATH = temp))
     }
     library.dynam("XML", pkgname, libname)
     if (exists("setMethod")) {
     }
     .C("RSXML_setErrorHandlers")
}
<environment: namespace:XML>

so it does already do that.

The order depends on the version of Windows *and* its settings: see

http://msdn2.microsoft.com/en-us/library/ms682586(VS.85).aspx

There is a way to change this:

http://msdn2.microsoft.com/en-us/library/ms686203(VS.85).aspx

but it would preclude Windows 2000.

Perhaps Martin can explain how libxml2.dll got into c:/WINDOWS/system32/?
My suggestion is that we rename the DLL when copied into library/XML/libs 
to something like libRxml2.dll.

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Brian Ripley
    

    

      
      #
    

  


  

      
On Mon, 7 Jan 2008, Duncan Murdoch wrote:

        

            

      
      
      
    

        
For the record, here is what the search order was at the time of VC++6 
(1998):

1. The directory from which the application loaded.

2. The current directory.

3. Windows 95 and Windows 98: The Windows system directory. Use 
the GetSystemDirectory function to get the path of this directory.

Windows NT: The 32-bit Windows system directory. Use the 
GetSystemDirectory function to get the path of this directory. The name of 
this directory is SYSTEM32.

4. Windows NT: The 16-bit Windows system directory. There is no function 
that obtains the path of this directory, but it is searched. The name of 
this directory is SYSTEM.

5. The Windows directory. Use theGetWindowsDirectory function to get the 
path of this directory.

6. The directories that are listed in the PATH environment variable.

The differences have been what is meant by the 'system directories', and 
the position of the current directory.

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Brian Ripley
    

    

      
      #
    

  


  

      
On Mon, 7 Jan 2008, Duncan Murdoch wrote:

        

            

      
      
      
    

        
About 85.  But it's worse than that, as libxml2.dll itself imports from 
zlib1.dll and iconv.dll, and there is one of the latter in the application 
directory in R >= 2.6.0.  So even if you find the right libxml2.dll, you 
may not find the right zlib1.dll and iconv.dll (and I already knew that 
you found R's iconv.dll, which fortunately works).

I think we do need to provide an interface to SetDllDirectory, probably as 
an additional argument to dyn.load.

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Martin Morgan
    

    

      
      #
    

  


  

      
Thanks Duncan for the hints.

libxml2.dll is not in my c:/WINDOWS/system32, but in that of a user of
a package of mine. I guess installed by some other application, though
I don't really know what it's doing there (or the consequences of
removing it).

The MSDN site seemed in the long term to point in the direction of
side-by-side assemblies, which would require cooperation /
infrastructure from R.

Martin

Prof Brian Ripley <ripley at stats.ox.ac.uk> writes:

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Duncan Temple Lang
    

    

      
      #
    

  


  

      
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

While I don't disagree with the general need to provide an interface
to SetDllDirectory, etc., I think the discussion about 3rd party
libraries has slightly missed the more obvious solution.
Instead of using a DLL, such packages can link against a _static_
version of the 3rd party and so not get into this dynamic search and
load but guarantee that the symbols are resolved to what they were
intended at build time.

There are lots of advantages to using DLLs, but in many cases
we are not exploiting these and a static link would make these
issues irrelevant and not require users to know about PATH settings, etc.

 D.

        
Prof Brian Ripley wrote:

            

      
      
      
    

        
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHgwIy9p/Jzwa2QP4RAr32AJ9tDRvunOM0RMeCnEPJIlY7s5NgGACfRmK5
yB/ZNnqkdqc2aeu4f6mxW1w=
=kytC
-----END PGP SIGNATURE-----

  
  


  


      

    

    

      
      

        


  

        

      Brian Ripley
    

    

      
      #
    

  


  

      
On Tue, 8 Jan 2008, Duncan Temple Lang wrote:

        

            

      
      
      
    

        
Only if there is one.  This has been my practice when building Windows 
binary packages, but as in the recent example of ncdf, the developers of 
the third-party (netcdf) set up their make system only to implement some 
Windows tweaks when a DLL was built.  More commonly, the third-party has 
to be built under a different compiler (typically VC++), and the static 
library is not compatible with MinGW.  (iconv and Tcl/Tk are two examples 
where at least at one time MinGW builds didn't work correctly.)

In the case of XML, we could revert to my practice before Uwe took over, 
as I did (after some struggle) build libxml2/zlib statically under older 
versions of MinGW and so would expect to be able to do so again.

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Duncan Temple Lang
    

    

      
      #
    

  


  

      
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

        
Prof Brian Ripley wrote:

            

      
      
      
    

        
I assume 'one' refers to DLLs.
Either way, of course you can link against static versions of all
libraries and not just one. It requires having static versions of each
library.

            

      
      
      
    

        
Well this is the issue - whether we want to build our own versions of
a static version of a DLL.  It would be a lot better if Windows users
were aware of the work done to support them and realized that it reduced
the resources available for more creative work.

            

      
      
      
    

        
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHgxrn9p/Jzwa2QP4RAnsVAJsEkOT7DEu3pIOkIiA23RZ9mACIfACeIqrG
L3YrWfoD916Vbzu1zy93KkU=
=6yOz
-----END PGP SIGNATURE-----

  
  


  


      

    

    

      
      

        


  

        

      Brian Ripley
    

    

      
      #
    

  


  

      
On Tue, 8 Jan 2008, Duncan Temple Lang wrote:

        

            

      
      
      
    

        
It refers to 'a _static_ version': normal English rules -- the immediate 
antecedent -- apply.

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Brian Ripley
    

    

      
      #
    

  


  

      
Let me try to pull the ends together on this.

- As DTL pointed out one way out is to use a static libxml2.  I tried
   that, and managed to  build and check it under MinGW but it did not work
   correctly with package XML.  That route used to be possible, but it is
   too much work to keep fixing it up, at least for me: package
   developers are strongly biased towards providing DLLs on Windows.

- Renaming the DLL is not an option, as libxml2.dll contains its name, and
   if you rename it you will not be able to link against it.  Perhaps there
   is a route through this involving import libs, but the standard
   mechanisms do not work.  It also does not solve the dependent DLL
   problem.

- Adding an interface to SetDLLDirectory is viable in Windows > 2000
   provided only one directory is involved.  (E.g. not if you want to link
   to DLLs provided in another package as well as your own.)  I've
   implemented that and am about to add it: there is a new argument
   'DLLpath' to dyn.load() and it is set by library.dynam().  This works in
   Windows 2000 by temporarily changing the current directory.  This
   provides a safer way for the Windows' tcltk package to link to its
   DLLs.

Note that the issues are not just confined to Windows: the search path for 
dlopen() is pretty much undocumented in most OSes (apart from that 
LD_LIBRARY_PATH is involved and it is usually read only at startup). It 
usually works because DSOs are versioned and so libxml2.so.2.6.30 is 
likely to be found only once (or once for each architecture).  Some other 
software makes use of ld -R or -rpath (and hence has relocation issues: it 
was decided not to consider that for R until libtool was fully usable 
which after several years it is not).

        
On Mon, 7 Jan 2008, Prof Brian Ripley wrote:

        

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Martin Morgan
    

    

      
      #
    

  


  

      
Thank you very much, Prof. Ripley, for the prompt solution, and to all
participants in the thread for their insights. Martin

Prof Brian Ripley <ripley at stats.ox.ac.uk> writes: