build, data and vignettes

(forwarding to R-devel)

Prof. Ripley

Thanks a lot for the help.

I greatly appreciate the clarification that R does indeed install the
package during the build process.  I must have been confused.  The
error is now fixed and was not related to my (false) claim that R
could not locate a dataset in /data.

You were not able to replicate the error because you were using the
_build_ tarballs available from (I guess)
  http://bioconductor.org/packages/2.6/bioc/src/contrib/Genominator_1.1.3.tar.gz
These tarballs are only put up, when the package passes R CMD build
(naturally enough).  The version of Genominator I had problems with
was 1.1.4 which is only available from subversion (since it did not
build when I wrote the email).  1.1.4 ought to be available in a few
days when the build system picks up on the changes I just committed.

I did mention that I saw that the package did not complete R CMD
build, but (semi-passed)
  R CMD build --no-vignettes Genominator
  R CMD check Genominator_XXX.tar.gz

I have no double checked this, see output below.  I am using an
R-devel that is a week old or so.  What I do below is
1) check out the package from svn
2) R CMD build --no-vignettes
3) R CMD check on the tarball.  Looks like it "passes" R CMD check
with a warning about missing PDFs
4) verify that the source directory is empty by running svn status
5) R CMD build  which fails (as it should)
(Note the package - in this svn version - has errors in the vignette)

I would have expected that R CMD check failed or at least told me that
the vignettes do not build.

Of course, given my earlier mistakes with this, I could be messing
something up.

Output below.

Thanks as always,
Kasper

compute-0-19:~/Work/packages/temp/>  echo $BIOCSVN
https://hedgehog.fhcrc.org/gentleman/bioconductor/trunk/madman/Rpacks
compute-0-19:~/Work/packages/temp/>  svn co -r 44843 $BIOCSVN/Genominator
  <OUTOUR REMOVED>
Checked out revision 44843.
compute-0-19:~/Work/packages/temp/>  R-devel CMD build --no-vignettes
Genominator
* checking for file 'Genominator/DESCRIPTION' ... OK
* preparing 'Genominator':
* checking DESCRIPTION meta-information ... OK
* removing junk files
* checking for LF line-endings in source and make files
* checking for empty or unneeded directories
* building 'Genominator_1.1.4.tar.gz'

compute-0-19:~/Work/packages/temp/>  R-devel CMD check Genominator_1.1.4.tar.gz
* checking for working pdflatex ... OK
* using log directory
'/home/bst/student/khansen/Work/packages/temp/Genominator.Rcheck'
* using R version 2.11.0 Under development (unstable) (2010-02-15 r51142)
* using session charset: ISO8859-15
* checking for file 'Genominator/DESCRIPTION' ... OK
* this is package 'Genominator' version '1.1.4'
* checking package name space information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking for executable files ... OK
* checking whether package 'Genominator' can be installed ... OK
* checking package directory ... OK
* checking for portable file names ... OK
* checking for sufficient/correct file permissions ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking R files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the name space can be loaded with stated dependencies ... OK
* checking whether the name space can be unloaded cleanly ... OK
* checking for unstated dependencies in R code ... OK
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... OK
* checking Rd files ... OK
* checking Rd metadata ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... OK
* checking Rd contents ... OK
* checking data for non-ASCII characters ... OK
* checking examples ... OK
* checking tests ...
  OK
* checking package vignettes in 'inst/doc' ... WARNING
Package vignettes without corresponding PDF:
  /home/bst/student/khansen/Work/packages/temp/Genominator.Rcheck/00_pkg_src/Genominator/inst/doc/Genominator.Rnw
  /home/bst/student/khansen/Work/packages/temp/Genominator.Rcheck/00_pkg_src/Genominator/inst/doc/plotting.Rnw
  /home/bst/student/khansen/Work/packages/temp/Genominator.Rcheck/00_pkg_src/Genominator/inst/doc/withShortRead.Rnw
* checking PDF version of manual ... OK

WARNING: There was 1 warning, see
  /home/bst/student/khansen/Work/packages/temp/Genominator.Rcheck/00check.log
for details

compute-0-19:~/Work/packages/temp/>  svn status Genominator
compute-0-19:~/Work/packages/temp/>  R-devel CMD build Genominator
* checking for file 'Genominator/DESCRIPTION' ... OK
* preparing 'Genominator':
* checking DESCRIPTION meta-information ... OK
* installing the package to re-build vignettes
* installing *source* package 'Genominator' ...
** R
** data
** inst
** preparing package for lazy loading
Loading required package: DBI

Attaching package: 'IRanges'

The following object(s) are masked from 'package:base':

    Map, cbind, mapply, order, pmax, pmax.int, pmin, pmin.int, rbind,
    rep.int, table

** help
*** installing help indices
** building package indices ...
* DONE (Genominator)
* creating vignettes ... ERROR
Loading required package: RSQLite
Loading required package: DBI
Loading required package: IRanges

Attaching package: 'IRanges'

The following object(s) are masked from 'package:base':

    cbind, Map, mapply, order, pmax, pmax.int, pmin, pmin.int,
    rbind, rep.int, table

writing regions table: 0.026 sec
SQL query: SELECT counts,feature FROM counts_tbl INNER JOIN
__regions__ ON __regions__.chr = counts_tbl.chr AND
counts_tbl.location BETWEEN __regions__.start AND __regions__.end AND
(counts_tbl.strand = __regions__.strand OR __regions__.strand = 0 OR
counts_tbl.strand = 0)

fetching merge table: 0.027 sec
splitting by: feature: 0.002 sec
matplot: doing 1 plots with  col= ("1") pch= ("1" "2" "3" "4" "5" "6"
"7" "8" "9" "0" "a" "b" "c" "d" "e" "f" "g" "h" "i" "j" "k" "l" "m"
"n" "o" "p" "q" "r" "s" "t" "u" "v" "w" "x" "y" "z" "A" "B" "C" "D"
"E" "F" "G" "H" "I" "J" "K" "L" "M" "N" "O" "P" "Q" "R" "S" "T" "U"
"V" "W" "X" "Y" "Z") ...

writing regions table: 0.025 sec
SQL query: SELECT __regions__.id, TOTAL(counts) FROM __regions__ LEFT
OUTER JOIN counts_tbl ON __regions__.chr = counts_tbl.chr AND
counts_tbl.location BETWEEN __regions__.start AND __regions__.end AND
(counts_tbl.strand = __regions__.strand OR __regions__.strand = 0 OR
counts_tbl.strand = 0) GROUP BY __regions__.id ORDER BY __regions__.id

fetching summary table: 0.044 sec
fetching summary: 0.016 sec
matplot: doing 1 plots with  col= ("1") pch= ("1" "2" "3" "4" "5" "6"
"7" "8" "9" "0" "a" "b" "c" "d" "e" "f" "g" "h" "i" "j" "k" "l" "m"
"n" "o" "p" "q" "r" "s" "t" "u" "v" "w" "x" "y" "z" "A" "B" "C" "D"
"E" "F" "G" "H" "I" "J" "K" "L" "M" "N" "O" "P" "Q" "R" "S" "T" "U"
"V" "W" "X" "Y" "Z") ...

writing regions table: 0.026 sec
SQL query: SELECT __regions__.id, TOTAL(counts_1), TOTAL(counts_2)
FROM __regions__ LEFT OUTER JOIN allcounts ON __regions__.chr =
allcounts.chr AND allcounts.location BETWEEN __regions__.start AND
__regions__.end AND (allcounts.strand = __regions__.strand OR
__regions__.strand = 0 OR allcounts.strand = 0) GROUP BY
__regions__.id ORDER BY __regions__.id

fetching summary table: 0.048 sec
Loading required package: GenomeGraphs
Loading required package: biomaRt
Loading required package: grid
Loading required package: ShortRead
Loading required package: Biostrings
Loading required package: BSgenome
Loading required package: lattice
Loading required package: yeastRNASeq
Error: no function to return from, jumping to top level
Execution halted


Thanks again,
Kasper

2010/2/26 Prof Brian Ripley<ripley at stats.ox.ac.uk>:

On Wed, 24 Feb 2010, Kasper Daniel Hansen wrote:

Based on some testing it seems to me that if I have a package with
  a dataset in /data
  a Sweave vignette in inst/doc (but no associated pdf file)
  the vignette loads the data in /data through
    data(dataset)
and I do a
  R CMD build
R will try to build the pdf version of the vignette, but will be
unable to find the dataset in data because the package is not yet
installed.  However, if I do

But R CMD build *does* install the package to build vignettes.  If I take
the current BioC-2.6 version of Genominator (1.1.3), remove the
inst/doc/*.pdf and any installed versions I see

gannet% Rdev CMD build Genominator
* checking for file 'Genominator/DESCRIPTION' ... OK
* preparing 'Genominator':
* checking DESCRIPTION meta-information ... OK
* installing the package to re-build vignettes
* installing *source* package ?Genominator? ...
** R
** data
** inst
** preparing package for lazy loading
Loading required package: DBI
Creating a new generic function for "head" in "Genominator"
** help
*** installing help indices
** building package indices ...
* DONE (Genominator)
* creating vignettes ... OK
* removing junk files
* checking for LF line-endings in source and make files
* checking for empty or unneeded directories
* building 'Genominator_1.1.3.tar.gz'

(Well, once I had installed the 307Mb dependency yeastRNASeq.)

Now, you may be using an unreleased version of Genominator, but your
hypothesis (that the package needs to be installed beforehand) is not
correct.

  R CMD build --no-vignettes PKGNAME
  R CMD check PKGNAME_VERSION.tar.gz
the package passes R CMD check! Presumably because R CMD check
installs the package first.

Yes, to a temporary library, just as R CMD build does if there are
vignettes.

This took me a long time to track down (inspired by a recent addition
of dataset to the Bioconductor package Genominator and a subsequent
failed build - specifically the dataset yeastAnno.sources and the
vignette withShortRead.Rnw).  I am using lazy loading (in case it
matters, which I don't think it does).

It seems like the relevant fix is to include pdf versions of the
vignette(s) in inst/doc.

On one hand I can see why the build fails.

So given the evidence that R CMD build does install the package, please
explain.

And the fix is easy.  But
just thought I would mention it.

Kasper

--
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595