[R-pkg-devel] About a need for hooks from R CMD ... commands, and a question

For a few years now I had mused about how nice it would be to have other
scripts triggered like `cleanup`.  For Rcpp, it would be nice to run
compileAttributes(). For roxygen2, it would be nice to run roxygenize()
(especially if that continued to work the way it did, but I digress).
That was mostly a "wouldn't it be nice" question and not that urgent as I
wrote myself littler scripts for most tasks anyway.

But I now have a related problem at work. We are authoring C++ libraries and
R packages 'mono-repo' style. And I need to reference builds of the R
packages back to the repo branches / repo directories used where R CMD build
ran in a branch -- in order to at R CMD INSTALL time access the matching
header files and libraries from that branch.  I do not see an obvious way of
encoding this in the .tar.gz created by the R CMD build step. The best I can
think of is creating binaries via R CMD INSTALL --build instead of creating
source tarballs.

Is there something better I am missing? Any hacks, hints, or ideas?

I'm not sure whether I understand correctly what you are trying to
achieve. Isn't generating a custom configure script enough for your
needs?

I?aki

| >
| >
| > For a few years now I had mused about how nice it would be to have other
| > scripts triggered like `cleanup`.  For Rcpp, it would be nice to run
| > compileAttributes(). For roxygen2, it would be nice to run roxygenize()
| > (especially if that continued to work the way it did, but I digress).
| > That was mostly a "wouldn't it be nice" question and not that urgent as I
| > wrote myself littler scripts for most tasks anyway.
| >
| > But I now have a related problem at work. We are authoring C++ libraries and
| > R packages 'mono-repo' style. And I need to reference builds of the R
| > packages back to the repo branches / repo directories used where R CMD build
| > ran in a branch -- in order to at R CMD INSTALL time access the matching
| > header files and libraries from that branch.  I do not see an obvious way of
| > encoding this in the .tar.gz created by the R CMD build step. The best I can
| > think of is creating binaries via R CMD INSTALL --build instead of creating
| > source tarballs.
| >
| > Is there something better I am missing? Any hacks, hints, or ideas?
| 
| I'm not sure whether I understand correctly what you are trying to
| achieve. Isn't generating a custom configure script enough for your
| needs?

That is what I thought yesterday when I wrote / updated and tested those.
They work great when you install from source as they have access to shell's
${PWD} and R's getwd().

But when you run R CMD build, (essentially) just cleanup and a tar are
running. No other code. Hence my Subject: here -- a need for a hook.

Once you have built a .tar.gz and run R CMD INSTALL on it, R goes off to
_another temporary directory_ and while configure may be running ... you are
no longer in the directory you called it from. So I have now lost the
connection to the particular variant of headers and libraries I may have had
in that branch.

Doing R CMD INSTALL --build ... is a workaround.  But it still would be nice
to have an explicit hook.

Hope this explains it better, and thanks for the follow-up.

Dirk

http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

On 4 January 2019 at 19:24, I?aki Ucar wrote:
| On Fri, 4 Jan 2019 at 16:43, Dirk Eddelbuettel <edd at debian.org> wrote:
| >
| >
| > For a few years now I had mused about how nice it would be to have other
| > scripts triggered like `cleanup`.  For Rcpp, it would be nice to run
| > compileAttributes(). For roxygen2, it would be nice to run roxygenize()
| > (especially if that continued to work the way it did, but I digress).
| > That was mostly a "wouldn't it be nice" question and not that urgent as I
| > wrote myself littler scripts for most tasks anyway.
| >
| > But I now have a related problem at work. We are authoring C++ libraries and
| > R packages 'mono-repo' style. And I need to reference builds of the R
| > packages back to the repo branches / repo directories used where R CMD build
| > ran in a branch -- in order to at R CMD INSTALL time access the matching
| > header files and libraries from that branch.  I do not see an obvious way of
| > encoding this in the .tar.gz created by the R CMD build step. The best I can
| > think of is creating binaries via R CMD INSTALL --build instead of creating
| > source tarballs.
| >
| > Is there something better I am missing? Any hacks, hints, or ideas?
|
| I'm not sure whether I understand correctly what you are trying to
| achieve. Isn't generating a custom configure script enough for your
| needs?

That is what I thought yesterday when I wrote / updated and tested those.
They work great when you install from source as they have access to shell's
${PWD} and R's getwd().

But when you run R CMD build, (essentially) just cleanup and a tar are
running. No other code. Hence my Subject: here -- a need for a hook.

Once you have built a .tar.gz and run R CMD INSTALL on it, R goes off to
_another temporary directory_ and while configure may be running ... you are
no longer in the directory you called it from. So I have now lost the
connection to the particular variant of headers and libraries I may have had
in that branch.

Doing R CMD INSTALL --build ... is a workaround.  But it still would be nice
to have an explicit hook.

Hope this explains it better, and thanks for the follow-up.

I see. Then I suggest the following hack (not tested). If the package
has a vignette, put there a hidden chunk that generates a file with
info about the branch and the path. This file will be available for
your configure script, which runs at INSTALL time. There you can,
e.g., retrieve that branch from a remote repo.

I?aki

On 4 January 2019 at 19:24, I?aki Ucar wrote:
| On Fri, 4 Jan 2019 at 16:43, Dirk Eddelbuettel <edd at debian.org> wrote:
| >
| >
| > For a few years now I had mused about how nice it would be to have other
| > scripts triggered like `cleanup`.  For Rcpp, it would be nice to run
| > compileAttributes(). For roxygen2, it would be nice to run roxygenize()
| > (especially if that continued to work the way it did, but I digress).
| > That was mostly a "wouldn't it be nice" question and not that urgent as I
| > wrote myself littler scripts for most tasks anyway.
| >
| > But I now have a related problem at work. We are authoring C++ libraries and
| > R packages 'mono-repo' style. And I need to reference builds of the R
| > packages back to the repo branches / repo directories used where R CMD build
| > ran in a branch -- in order to at R CMD INSTALL time access the matching
| > header files and libraries from that branch.  I do not see an obvious way of
| > encoding this in the .tar.gz created by the R CMD build step. The best I can
| > think of is creating binaries via R CMD INSTALL --build instead of creating
| > source tarballs.
| >
| > Is there something better I am missing? Any hacks, hints, or ideas?
|
| I'm not sure whether I understand correctly what you are trying to
| achieve. Isn't generating a custom configure script enough for your
| needs?

That is what I thought yesterday when I wrote / updated and tested those.
They work great when you install from source as they have access to shell's
${PWD} and R's getwd().

But when you run R CMD build, (essentially) just cleanup and a tar are
running. No other code. Hence my Subject: here -- a need for a hook.

Once you have built a .tar.gz and run R CMD INSTALL on it, R goes off to
_another temporary directory_ and while configure may be running ... you are
no longer in the directory you called it from. So I have now lost the
connection to the particular variant of headers and libraries I may have had
in that branch.

Doing R CMD INSTALL --build ... is a workaround.  But it still would be nice
to have an explicit hook.

Hope this explains it better, and thanks for the follow-up.

If you want a total hack, the help system can run R code during a build, 
e.g. \Sexpr[stage=build]{paste("Built at", Sys.time())}.  Certainly it 
could embed some information in a help page; perhaps it could do more. 
Similarly, vignettes are typically built during R CMD build, so they 
might be able to have useful side effects.

But it seems likely that something less of a kludge would be desirable.

Duncan Murdoch

I?aki, Duncan,

Fabulous -- that is just the type of 'hack' I was hoping for in the short
term.

Longer term, it would indeed be nice to do something about this and create
something better.  I had looked once or twice at packages utils and tools but
didn't quite find the right spot to connect it to. Pointers appreciated.

Best, Dirk

http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

Add a tag to the DESCRIPTION file or configure.ac-like indicating the git (presumably) revision, perhaps like https://stackoverflow.com/a/11535358/547331 ?

?On 1/4/19, 1:42 PM, "R-package-devel on behalf of Dirk Eddelbuettel" <r-package-devel-bounces at r-project.org on behalf of edd at debian.org> wrote:

| >
    | >
    | > For a few years now I had mused about how nice it would be to have other
    | > scripts triggered like `cleanup`.  For Rcpp, it would be nice to run
    | > compileAttributes(). For roxygen2, it would be nice to run roxygenize()
    | > (especially if that continued to work the way it did, but I digress).
    | > That was mostly a "wouldn't it be nice" question and not that urgent as I
    | > wrote myself littler scripts for most tasks anyway.
    | >
    | > But I now have a related problem at work. We are authoring C++ libraries and
    | > R packages 'mono-repo' style. And I need to reference builds of the R
    | > packages back to the repo branches / repo directories used where R CMD build
    | > ran in a branch -- in order to at R CMD INSTALL time access the matching
    | > header files and libraries from that branch.  I do not see an obvious way of
    | > encoding this in the .tar.gz created by the R CMD build step. The best I can
    | > think of is creating binaries via R CMD INSTALL --build instead of creating
    | > source tarballs.
    | >
    | > Is there something better I am missing? Any hacks, hints, or ideas?
    | 
    | I'm not sure whether I understand correctly what you are trying to
    | achieve. Isn't generating a custom configure script enough for your
    | needs?
    
    That is what I thought yesterday when I wrote / updated and tested those.
    They work great when you install from source as they have access to shell's
    ${PWD} and R's getwd().
    
    But when you run R CMD build, (essentially) just cleanup and a tar are
    running. No other code. Hence my Subject: here -- a need for a hook.
    
    Once you have built a .tar.gz and run R CMD INSTALL on it, R goes off to
    _another temporary directory_ and while configure may be running ... you are
    no longer in the directory you called it from. So I have now lost the
    connection to the particular variant of headers and libraries I may have had
    in that branch.
    
    Doing R CMD INSTALL --build ... is a workaround.  But it still would be nice
    to have an explicit hook.
    
    Hope this explains it better, and thanks for the follow-up.
    
    Dirk
    
    -- 
    http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
    
    ______________________________________________
    R-package-devel at r-project.org mailing list
    https://stat.ethz.ch/mailman/listinfo/r-package-devel

| Add a tag to the DESCRIPTION file or configure.ac-like indicating the git (presumably) revision, perhaps like https://stackoverflow.com/a/11535358/547331 ?

I need the path. I need to find the particular build / checkout of headers
and libaries.

Dirk

http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

I see. Then I suggest the following hack (not tested). If the package
has a vignette, put there a hidden chunk that generates a file with
info about the branch and the path. This file will be available for
your configure script, which runs at INSTALL time. There you can,
e.g., retrieve that branch from a remote repo.

Actually, if there is any vignette, the configure script is executed
during the build process, because R needs to install the package to
build the vignettes. There, you can rely on $OLDPWD to retrieve the
path to the package.

I?aki

| >
| > I see. Then I suggest the following hack (not tested). If the package
| > has a vignette, put there a hidden chunk that generates a file with
| > info about the branch and the path. This file will be available for
| > your configure script, which runs at INSTALL time. There you can,
| > e.g., retrieve that branch from a remote repo.
| 
| Actually, if there is any vignette, the configure script is executed
| during the build process, because R needs to install the package to
| build the vignettes. There, you can rely on $OLDPWD to retrieve the
| path to the package.

Not really. Consider

  ~/git/repo/proj-brnchA
  ~/git/repo/proj-brnchB
  ~/git/repo/proj-brnchC

where each branch may have a in-repo build with static libraries and
header. And I build Rpkg_0.1.0.tar.gz in each with R CMD build. I need inside
of Rpkg_0.1.0.tar.gz a reference to whether we need headers from A, B or C.

$OLDPWD does not (generally) give me that.  Only if I invoked R CMD INSTALL
in the path.  My preference would be to do some "not unlink configure" which
write down the path in src/Makevars.  I think the vignette trick will let me
do that but I have not had time to try.

Dirk

http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

Hi Duncan and I?aki,

| If you want a total hack, the help system can run R code during a build, 
| e.g. \Sexpr[stage=build]{paste("Built at", Sys.time())}.  Certainly it 
| could embed some information in a help page; perhaps it could do more. 
| Similarly, vignettes are typically built during R CMD build, so they 
| might be able to have useful side effects.
| 
| But it seems likely that something less of a kludge would be desirable.

A first simple

  [...]
  
  <<preliminaries,echo=FALSE,results=hide>>=
  user <- Sys.getenv("USER")
  dir <- getwd()
  @

  [...]

  Built by '\Sexpr{user}' in directory '\Sexpr{dir}'.

did not work. By the time that chunk is evaluate, I am already in TEMPDIR.

Any clever ideas about I could inject code into the part that creates the tarball?

Dirk

http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

Hi Duncan and I?aki,

On 4 January 2019 at 14:13, Duncan Murdoch wrote:
| If you want a total hack, the help system can run R code during a build,
| e.g. \Sexpr[stage=build]{paste("Built at", Sys.time())}.  Certainly it
| could embed some information in a help page; perhaps it could do more.
| Similarly, vignettes are typically built during R CMD build, so they
| might be able to have useful side effects.
|
| But it seems likely that something less of a kludge would be desirable.

A first simple

  [...]

  <<preliminaries,echo=FALSE,results=hide>>=
  user <- Sys.getenv("USER")
  dir <- getwd()
  @

  [...]

  Built by '\Sexpr{user}' in directory '\Sexpr{dir}'.

did not work. By the time that chunk is evaluate, I am already in TEMPDIR.

Any clever ideas about I could inject code into the part that creates the tarball?

Did you check Sys.getenv("OLDPWD")? I checked with a knitr vignette
and the correct path was there.

I?aki

Hi Duncan and I?aki,

On 4 January 2019 at 14:13, Duncan Murdoch wrote:
| If you want a total hack, the help system can run R code during a build,
| e.g. \Sexpr[stage=build]{paste("Built at", Sys.time())}.  Certainly it
| could embed some information in a help page; perhaps it could do more.
| Similarly, vignettes are typically built during R CMD build, so they
| might be able to have useful side effects.
|
| But it seems likely that something less of a kludge would be desirable.

A first simple

   [...]
   
   <<preliminaries,echo=FALSE,results=hide>>=
   user <- Sys.getenv("USER")
   dir <- getwd()
   @

   [...]

   Built by '\Sexpr{user}' in directory '\Sexpr{dir}'.

did not work. By the time that chunk is evaluate, I am already in TEMPDIR.

Any clever ideas about I could inject code into the part that creates the tarball?

Same seems to be true for macros in help pages.

Duncan

It just ocurred to me the following. Instead of

R CMD build pkg

I've tried this:

cd pkg; R CMD build --log `pwd`

and voil?: the tarball includes a log file, and inside there's the
complete path to the DESCRIPTION.

I?aki

On 09/01/2019 11:39 a.m., Dirk Eddelbuettel wrote:

Hi Duncan and I?aki,

On 4 January 2019 at 14:13, Duncan Murdoch wrote:
| If you want a total hack, the help system can run R code during a build,
| e.g. \Sexpr[stage=build]{paste("Built at", Sys.time())}.  Certainly it
| could embed some information in a help page; perhaps it could do more.
| Similarly, vignettes are typically built during R CMD build, so they
| might be able to have useful side effects.
|
| But it seems likely that something less of a kludge would be desirable.

A first simple

   [...]

   <<preliminaries,echo=FALSE,results=hide>>=
   user <- Sys.getenv("USER")
   dir <- getwd()
   @

   [...]

   Built by '\Sexpr{user}' in directory '\Sexpr{dir}'.

did not work. By the time that chunk is evaluate, I am already in TEMPDIR.

Any clever ideas about I could inject code into the part that creates the tarball?

Same seems to be true for macros in help pages.

Duncan

I?aki ?car

| Did you check Sys.getenv("OLDPWD")? I checked with a knitr vignette
| and the correct path was there.

Code:

[...]

  <<preliminaries,echo=FALSE,results=hide>>=
  prettyVersion <- packageVersion("someNameHere")
  prettyDate <- format(Sys.Date(), "%B %e, %Y")
  user <- Sys.getenv("USER")
  dir <- getwd()
  olddir <- Sys.getenv("OLDPWD")
  @

  [...]

  \date{Built on \Sexpr{prettyDate} using version \Sexpr{prettyVersion}}

  \maketitle

  Built by '\Sexpr{user}' in directory '\Sexpr{dir}' coming from directory
  '\Sexpr{olddir}'.

Comes up with an empty / unset olddir when I do

  R CMD build someNameHere; R CMD INSTALL someNameHere_1.2.3.tar.gz

No mas :-/

Your idea about the logfile is fair, but I would *much* prefer to have
something unconditional.

Dirk

http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

On 9 January 2019 at 17:57, I?aki Ucar wrote:
| Did you check Sys.getenv("OLDPWD")? I checked with a knitr vignette
| and the correct path was there.

Code:

[...]

  <<preliminaries,echo=FALSE,results=hide>>=
  prettyVersion <- packageVersion("someNameHere")
  prettyDate <- format(Sys.Date(), "%B %e, %Y")
  user <- Sys.getenv("USER")
  dir <- getwd()
  olddir <- Sys.getenv("OLDPWD")
  @

  [...]

  \date{Built on \Sexpr{prettyDate} using version \Sexpr{prettyVersion}}

  \maketitle

  Built by '\Sexpr{user}' in directory '\Sexpr{dir}' coming from directory
  '\Sexpr{olddir}'.

Comes up with an empty / unset olddir when I do

  R CMD build someNameHere; R CMD INSTALL someNameHere_1.2.3.tar.gz

This is quite interesting. Because if you run

R CMD build someNameHere/; R CMD INSTALL someNameHere_1.2.3.tar.gz

(note the "/" after the package name) then the olddir is there.

I?aki

| > Comes up with an empty / unset olddir when I do
| >
| >   R CMD build someNameHere; R CMD INSTALL someNameHere_1.2.3.tar.gz
| 
| This is quite interesting. Because if you run
| 
| R CMD build someNameHere/; R CMD INSTALL someNameHere_1.2.3.tar.gz
| 
| (note the "/" after the package name) then the olddir is there.

Not for me. Same result as before.  (That is as work on CentOS 7.5 with R 3.5.1).

Dirk

http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

Will this work for you?

I had to stash the working directory (when R CMD build is run) in an
environment variable via ~/.R/build.Renviron`

$ cat ~/.R/build.Renviron
OLDWD=${PWD}

But a Makefile tucked in vignettes directory will run and the two
files will be there in the built package.  For me, I couldn't get
OLDPWD so I saved it as OLDWD instead

$ tree vignettes/
vignettes/
??? Makefile
??? Untitled.Rmd

$ more vignettes/Makefile

all:
        echo ${OLDWD} > ../inst/old.txt
        echo ${PWD} > ../inst/current.txt

Kyle

On 10 January 2019 at 22:33, I?aki Ucar wrote:
| On Thu, 10 Jan 2019 at 21:55, Dirk Eddelbuettel <edd at debian.org> wrote:
| > Comes up with an empty / unset olddir when I do
| >
| >   R CMD build someNameHere; R CMD INSTALL someNameHere_1.2.3.tar.gz
|
| This is quite interesting. Because if you run
|
| R CMD build someNameHere/; R CMD INSTALL someNameHere_1.2.3.tar.gz
|
| (note the "/" after the package name) then the olddir is there.

Not for me. Same result as before.  (That is as work on CentOS 7.5 with R 3.5.1).

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

______________________________________________
R-package-devel at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Kyle Baron | Principal Scientist I

Metrum Research Group

On 10 January 2019 at 22:33, I?aki Ucar wrote:
| On Thu, 10 Jan 2019 at 21:55, Dirk Eddelbuettel <edd at debian.org> wrote:
| > Comes up with an empty / unset olddir when I do
| >
| >   R CMD build someNameHere; R CMD INSTALL someNameHere_1.2.3.tar.gz
|
| This is quite interesting. Because if you run
|
| R CMD build someNameHere/; R CMD INSTALL someNameHere_1.2.3.tar.gz
|
| (note the "/" after the package name) then the olddir is there.

Not for me. Same result as before.  (That is as work on CentOS 7.5 with R 3.5.1).

Silly me... It was showing my last cd! Of course! So what about the
very hacky...

cd ./anRpackage; cd ..; R CMD build anRpackage; R CMD INSTALL anRpackage_*.gz

I?aki

We have a winner! (By bending the rules.)

| Will this work for you?
| 
| I had to stash the working directory (when R CMD build is run) in an
| environment variable via ~/.R/build.Renviron`
| 
| $ cat ~/.R/build.Renviron
| OLDWD=${PWD}
 
That is less unconditional that I desired, but it works.  

| But a Makefile tucked in vignettes directory will run and the two
| files will be there in the built package.  For me, I couldn't get
| OLDPWD so I saved it as OLDWD instead
| 
| $ tree vignettes/
| vignettes/
| ??? Makefile
| ??? Untitled.Rmd
| 
| $ more vignettes/Makefile
| 
| all:
|         echo ${OLDWD} > ../inst/old.txt
|         echo ${PWD} > ../inst/current.txt
 
Then 'current.txt' is as before a temporary directory per R's build
conventions, but old.txt does indeed contain the source directory.

Close enought :)

Thanks very much for passing that creative hack along!

Dirk

http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org