Skip to content

Building Packages on Windows using .Rbuildignore (PR#7379)

9 messages · Gabor Grothendieck, Duncan Murdoch

Original
Duncan Murdoch
# 
On Thu, 18 Nov 2004 16:36:03 +0100 (CET), murdoch@stats.uwo.ca wrote :
I've taken a look at the source now, and it appears that the intention
is that R CMD check should respect the .Rbuildignore file, but (on my
system) it's looking for it in the wrong place:  it looks in the
parent of the package directory, not in the main package directory.
That is, if I have a package in subdirectory test, then 

Rcmd build test

looks for test/.Rbuildignore as it should, but 

Rcmd check test

looks for .Rbuildignore.  The problem is in these lines of code:

    my $exclude_file = ".Rbuildignore";
    ## This is a bit tricky for bundles where the build ignore pattern
    ## file is in the top-level bundle dir.
    $exclude_file = &file_path(dirname($pkgdir), $exclude_file);

In my test, $pkgdir was "d:/stuff/R/test", and $exclude_file ended up
as "d:/stuff/R/.Rbuildignore".

Could someone on Unix confirm this behaviour?  I think the fix should
be to change the last line to 

    $exclude_file = &file_path($pkgdir, $exclude_file);

but I don't want to mess things up on other platforms.

Even with this change, Rcmd check is still going to install the files
it's supposed to ignore, because it uses Rcmd INSTALL, and there's no
.Rbuildignore support there.

Duncan Murdoch
Gabor Grothendieck
# 
<murdoch <at> stats.uwo.ca> writes:

: 
: On Thu, 18 Nov 2004 16:36:03 +0100 (CET), murdoch <at> stats.uwo.ca wrote :
: 
: >>The reason that the processing is different according to whether one
: >>uses .Rbuildignore or not is that 
: >>  R CMD build 
: >>takes the .Rbuildignore file into account but 
: >>  R CMD install
: >>  R CMD check
: >>  R CMD build --binary 
: >>do not take .Rbuildignore into account.  
: >
: >Okay, now I understand.  I think I'd call the last of those a bug, and
: >it would seem to me that the install and check scripts should also
: >respect this directive.   I've now copied this to the r-bugs list.
: >(This was reported for Windows; I don't know if it applies to other
: >platforms as well.)
: 
: I've taken a look at the source now, and it appears that the intention
: is that R CMD check should respect the .Rbuildignore file, but (on my
: system) it's looking for it in the wrong place:  it looks in the
: parent of the package directory, not in the main package directory.
: That is, if I have a package in subdirectory test, then 
: 
: Rcmd build test
: 
: looks for test/.Rbuildignore as it should, but 
: 
: Rcmd check test
: 
: looks for .Rbuildignore.  The problem is in these lines of code:
: 
:     my $exclude_file = ".Rbuildignore";
:     ## This is a bit tricky for bundles where the build ignore pattern
:     ## file is in the top-level bundle dir.
:     $exclude_file = &file_path(dirname($pkgdir), $exclude_file);
: 
: In my test, $pkgdir was "d:/stuff/R/test", and $exclude_file ended up
: as "d:/stuff/R/.Rbuildignore".
: 
: Could someone on Unix confirm this behaviour?  I think the fix should
: be to change the last line to 
: 
:     $exclude_file = &file_path($pkgdir, $exclude_file);
: 
: but I don't want to mess things up on other platforms.
: 
: Even with this change, Rcmd check is still going to install the files
: it's supposed to ignore, because it uses Rcmd INSTALL, and there's no
: .Rbuildignore support there.
: 

If the behaviour is suddenly changed then this is going to cause work
for people whose scripts depend on the current behavior.  In order to
minimize disruption I would ask that such change only be made at the
same time that a flag for turning on and off .Rbuildignore processing
is implemented on build, check, install and build --binary.  Even
with such a flag it may require revision to scripts but at least
any change with the flag will be minimal.  Even better, it may
mean some scripts can be eliminated.
Duncan Murdoch
# 
On Sat, 20 Nov 2004 00:39:17 +0000 (UTC), Gabor Grothendieck
<ggrothendieck@myway.com> wrote:

            

      
      
      
    

        
Yes, that's normal.   If you work around a bug and the bug gets fixed,
then you will need to change your code.  That's why the NEWS file
reports bug fixes and other changes.

            

      
      
      
    

        
There's a simple workaround to turn .Rbuildignore processing off: just
rename the file.  Adding a switch is *not* a prerequisite for the
other changes.

            

      
      
      
    

        
There are 3 changes that I would contemplate:

1.  Fix the bug that means "R CMD check" looks in the wrong place for
.Rbuildignore.

2.  Make "R CMD build --binary" consistent with "R CMD build" in its
handling of .Rbuildignore.

3.  Make "R CMD install" and "R CMD check" consistent with "R CMD
build" in their handling of .Rbuildignore.

Number 1 should definitely be fixed in the patches to 2.0.1.  I have a
feeling that both 2 and 3 should be done (and 2 would be an automatic
consequence of 3 unless we took action to stop it), but I'd put them
in 2.1.0, not 2.0.x.

Martin and you have also suggested 

4.  Add another flag to Rcmd build (and install and check?), to turn
.Rbuildignore processing on and off, for increased flexiblity or for
backward compatiblity.

My own feeling is that this doesn't increase flexibility enough, and
I'd like a better solution, but we've got lots of time before 2.1.0 is
released to discuss this.

Duncan Murdoch

  
  


  


      

    

    

      
      

        


  

        

      Gabor Grothendieck
    

    

      
      #
    

  


  

      
Duncan Murdoch <murdoch <at> stats.uwo.ca> writes:

            

      
      
      
    

        
I did not know it was a bug and even you did not realize it until you
looked at the code.

I do have one suggestion that might be trivial for you yet be beneficial
for everyone else, as an interim step, until a better solution comes about.

After fixing the scripts, could you leave the old scripts in bin 
with new names, e.g. oldbuild, oldcheck, etc. so that one can issue
the command:

   R CMD oldbuild ...

and get the old behavior.  That provides a simple way to get either
behavior while waiting for the ultimate solution and does not interfere
with the new scripts in any way.

  
  


  


      

    

    

      
      

        


  

        

      Duncan Murdoch
    

    

      
      #
    

  


  

      
On Sat, 20 Nov 2004 03:47:50 +0000 (UTC), Gabor Grothendieck

        
<ggrothendieck@myway.com> wrote:

        

            

      
      
      
    

        
I think you misunderstand the consequences of fixing the bug.  If I
fix #1, it should not break any scripts.  It will just stop "Rcmd
check" from giving a few false alarms about files that you didn't want
to distribute anyways.  Those files will still be installed in the
temporary directory for the checks to run.  

It is only changes #2 and #3 that would potentially break scripts,
which is why I'd save them for 2.1.0.

As to the suggestion of leaving both versions of the scripts in place:
no, I wouldn't do that.  There's nothing to stop you from copying the
script from your old version to the new one (or editing the new one to
do something completely different, for that matter).  But if I were to
add three new scripts to the collection, I'd have to document them,
and people would have to maintain them.  All in all, a big waste of
our time to save a little bit of yours?  No thanks.

Duncan Murdoch

  
  


  


      

    

    

      
      

        


  

        

      Gabor Grothendieck
    

    

      
      #
    

  


  

      
Duncan Murdoch <murdoch <at> stats.uwo.ca> writes:

: 
: On Sat, 20 Nov 2004 03:47:50 +0000 (UTC), Gabor Grothendieck

        
: <ggrothendieck <at> myway.com> wrote:

        
: 
: >Duncan Murdoch <murdoch <at> stats.uwo.ca> writes:
: >
: >> 
: >> On Sat, 20 Nov 2004 00:39:17 +0000 (UTC), Gabor Grothendieck

        
: >> <ggrothendieck <at> myway.com> wrote:

        
: >> 
: >> >: Even with this change, Rcmd check is still going to install the files
: >> >: it's supposed to ignore, because it uses Rcmd INSTALL, and there's no
: >> >: .Rbuildignore support there.
: >> >: 
: >> >
: >> >If the behaviour is suddenly changed then this is going to cause work
: >> >for people whose scripts depend on the current behavior. 
: >> 
: >> Yes, that's normal.   If you work around a bug and the bug gets fixed,
: >> then you will need to change your code.  That's why the NEWS file
: >> reports bug fixes and other changes.
: >> 
: >> > In order to
: >> >minimize disruption I would ask that such change only be made at the
: >> >same time that a flag for turning on and off .Rbuildignore processing
: >> >is implemented on build, check, install and build --binary.  
: >> 
: >> There's a simple workaround to turn .Rbuildignore processing off: just
: >> rename the file.  Adding a switch is *not* a prerequisite for the
: >> other changes.
: >> 
: >> >Even
: >> >with such a flag it may require revision to scripts but at least
: >> >any change with the flag will be minimal.  Even better, it may
: >> >mean some scripts can be eliminated.
: >> 
: >> There are 3 changes that I would contemplate:
: >> 
: >> 1.  Fix the bug that means "R CMD check" looks in the wrong place for
: >> .Rbuildignore.
: >> 
: >> 2.  Make "R CMD build --binary" consistent with "R CMD build" in its
: >> handling of .Rbuildignore.
: >> 
: >> 3.  Make "R CMD install" and "R CMD check" consistent with "R CMD
: >> build" in their handling of .Rbuildignore.
: >> 
: >> Number 1 should definitely be fixed in the patches to 2.0.1.  I have a
: >> feeling that both 2 and 3 should be done (and 2 would be an automatic
: >> consequence of 3 unless we took action to stop it), but I'd put them
: >> in 2.1.0, not 2.0.x.
: >> 
: >> Martin and you have also suggested 
: >> 
: >> 4.  Add another flag to Rcmd build (and install and check?), to turn
: >> .Rbuildignore processing on and off, for increased flexiblity or for
: >> backward compatiblity.
: >> 
: >> My own feeling is that this doesn't increase flexibility enough, and
: >> I'd like a better solution, but we've got lots of time before 2.1.0 is
: >> released to discuss this.
: >
: >I did not know it was a bug and even you did not realize it until you
: >looked at the code.
: >
: >I do have one suggestion that might be trivial for you yet be beneficial
: >for everyone else, as an interim step, until a better solution comes about.
: >
: >After fixing the scripts, could you leave the old scripts in bin 
: >with new names, e.g. oldbuild, oldcheck, etc. so that one can issue
: >the command:
: >
: >   R CMD oldbuild ...
: >
: >and get the old behavior.  That provides a simple way to get either
: >behavior while waiting for the ultimate solution and does not interfere
: >with the new scripts in any way.
: 
: I think you misunderstand the consequences of fixing the bug.  If I
: fix #1, it should not break any scripts.  It will just stop "Rcmd
: check" from giving a few false alarms about files that you didn't want
: to distribute anyways.  Those files will still be installed in the
: temporary directory for the checks to run.  

Perhaps I used the wrong words.  I do checks with and without 
.Rbuildignore processing so having both facilities is convenient.

: 
: It is only changes #2 and #3 that would potentially break scripts,
: which is why I'd save them for 2.1.0. 

I agree that that two stage approach would reduce the extent of the 
problem of change in behavior.

: 
: As to the suggestion of leaving both versions of the scripts in place:
: no, I wouldn't do that.  There's nothing to stop you from copying the
: script from your old version to the new one (or editing the new one to
: do something completely different, for that matter).  But if I were to
: add three new scripts to the collection, I'd have to document them,
: and people would have to maintain them.  All in all, a big waste of
: our time to save a little bit of yours?  No thanks.

You don't need to document deprecated code.  Its just there for the
convenience of the user base that has invested in it.

  
  


  


      

    

    
1 day later

    

      
      

        


  

        

      Gabor Grothendieck
    

    

      
      #
    

  


  

      
Gabor Grothendieck <ggrothendieck <at> myway.com> writes:

            

      
      
      
    

        
Assuming that the behavior of the R CMD commands is changed
so that they respect .Rbuildignore but that no flag or
switch is immediately implemented to turn on and off
.Rbuildignore processing I have the following workaround
to allow the user to implement either situation.
It is simple enough that it does not require a script.  
I have not actually tried it out yet since the changed
commands are not yet available.

It applies to Windows or UNIX but is described below relative
to Windows and to an assumed folder tree called mypkg
which is the folder that contains .Rbuildignore.

1. in Windows Explorer create a second folder called
mypkg-shortcuts

2. select all files and folders directly within mypkg except
.Rbuildignore and right-click drag them to mypkg-shortcuts
selecting Create ShortCut after letting up on the right
mouse button.  That will create a shortcut to each file and 
folder in mypkg.  (Note that the files and folders dragged
include DESCRIPTION, R, man, etc. directly in mypkg but 
not the .files another level deep such as the .R and .Rd
files within R and man.)

3. now run off mypkg if the .Rbuildignore version is wanted
and run off mypkg-shortcuts if not.  Any time a file
in mypkg is changed in mypkg it will automatically change in
mypkg-shortcuts since they are all links.  (Special care
should be taken if files are added or deleted immediately
within mypkg; however, this is relatively rare since no
source or documentation is there.)

Note that the R and man folders within mypkg-shortcuts are
themselves links to the corresponding folders in mypkg so
that any change that occurs in mypkg\R and mypkg\man will
automatically be reflected in the mypkg-shortcuts versions.

  
  


  


      

    

    

      
      

        


  

        

      Duncan Murdoch
    

    

      
      #
    

  


  

      
On Mon, 22 Nov 2004 01:54:53 +0000 (UTC), Gabor Grothendieck

        
<ggrothendieck@myway.com> wrote:

        

            

      
      
      
    

        
That's a premature assumption.  At the moment it seems there are two
other proposals on the table:

 - no change other than the bug fix.  Then if you want to use
.Rbuildignore, you use Rcmd build to make a tarball, and do everything
else (check, build binary, etc.) from the tarball

- some sort of change that allows you to turn .Rbuildignore on and
off.

            

      
      
      
    

        
I don't think it will work in Windows.  Windows shortcuts aren't soft
links, they're just files that the shell sometimes interprets that
way.  Most other programs just see them as regular files, and won't
know to use them as links.

Duncan Murdoch

  
  


  


      

    

    

      
      

        


  

        

      Gabor Grothendieck
    

    

      
      #
    

  


  

      
Duncan Murdoch <murdoch <at> stats.uwo.ca> writes:

: 
: On Mon, 22 Nov 2004 01:54:53 +0000 (UTC), Gabor Grothendieck

        
: <ggrothendieck <at> myway.com> wrote:

        
: 
: >Assuming that the behavior of the R CMD commands is changed
: >so that they respect .Rbuildignore but that no flag or
: >switch is immediately implemented to turn on and off
: >.Rbuildignore processing
: > I have the following workaround
: >to allow the user to implement either situation.
: >It is simple enough that it does not require a script.  
: >I have not actually tried it out yet since the changed
: >commands are not yet available.
: 
: >It applies to Windows or UNIX but is described below relative
: >to Windows and to an assumed folder tree called mypkg
: >which is the folder that contains .Rbuildignore.
: >
: >1. in Windows Explorer create a second folder called
: >mypkg-shortcuts
: >
: >2. select all files and folders directly within mypkg except
: >.Rbuildignore and right-click drag them to mypkg-shortcuts
: >selecting Create ShortCut after letting up on the right
: >mouse button.  That will create a shortcut to each file and 
: >folder in mypkg.  (Note that the files and folders dragged
: >include DESCRIPTION, R, man, etc. directly in mypkg but 
: >not the .files another level deep such as the .R and .Rd
: >files within R and man.)
: 
: I don't think it will work in Windows.  Windows shortcuts aren't soft
: links, they're just files that the shell sometimes interprets that
: way.  Most other programs just see them as regular files, and won't
: know to use them as links.


Good point although I think it could still be made to work with
Folder Shortcuts (not to be confused with Shortcuts to Folders)
which are real folders that point to other folders and are
described at the thread at:

	http://www.annoyances.org/exec/forum/winxp/1076436311

Folder Shortcuts are not the same as Shortcuts to
Folders.  For example, if you delete a Folder Shortcut it 
deletes the target folder so you have to undo it in by
reversing the creation procedure.

What one would do is to copy the files directly under mypkg,
i.e. DESCRIPTION, etc.  to a new folder mypkg2.  These
files don't normally change much anyways as they do not include
either the source or the documentation so it should be ok in
most situations that they are copied rather than linked. Then 
create folder shortcuts as described in the link in mypkg2 which 
point to the corresponding folders in mypkg.

In a small test I found that the Folder Shortcut worked with a 
small gawk program while the Shortcut to the Folder did not.

Unfortunately this all increases complexity so one might
have to go back to creating a script after all.    Thus it
would be nice if there were some built in way to turn it on
and off.

Of course the best solution would be to be able to avoid this
all in the first place and if it came to this I would probably
just make copies since I already have scripts that do that.