R package development in windows

On 5/4/2007 11:31 AM, Gabor Grothendieck wrote:

On 5/4/07, Duncan Murdoch <murdoch at stats.uwo.ca> wrote:

On 5/4/2007 9:30 AM, Doran, Harold wrote:

OK, so I just want to go on record as noting that following the
instructions exactly (exactly, exactly, exactly, emphatically) on the
web given by Duncan, Gabor, and Tony at the page below, and following
the instructions in Writing R Extensions results in a successful Windows
build of a package.

Should this email turn up in someones search using RSiteSearch() or
elsewhere, my best advice is to ignore all other documents on the web
attempting to explain this. I personally went through 4 or 5 different
personal web sites looking for a straightforward guide. After a week or
so of late night reading and experimentation, all of those documents led
me to a path of failure. After reading the info on the link below, and
then referring back to Writing R Extensions, I was able to build the
package in only a few hours of time. It took significantly less time to
read the manual and information on the link below than it did to google
help pages on the topic, read those, and end up even more confused.

I'm not in the best position to offer specific recommendations on how to
improve this process. The reality is that after collecting all programs
required, making the required changes to the path, the Rcmd commands for
the build are very simple. So, the biggest task is the collection of
essential programs and modification to the path.

Obviously, Rtools is designed to move forward in this direction. Its
current phase still doesn't automate that collection and modification
process entirely, and the reasons are given on Duncan's web site.
However, I suppose the only good recommendation I can make at this point
is to still move forward with Rtools such that it installs all required
programs and automatically updates the path accordingly. Maybe this can
even become part of the base installation. Since the open source nature
of R encourages development, the tools needed for development should be
readily available.

I am certain this recommendation is not novel, and Duncan et al have
already thought of this. But, it's my attempt to do a little better than
complain.

The only other piece of advice I can offer is to keep this stable. That
is, when R 2.5.1 et seq are released, I beg of the developers not to
require a new collection of tools or a different set up than what is
already required. It is a bit of work, and if changes are required at
each R release, this may be too disheartening.

Thanks for your comments and suggestions.  I've thought of a few changes
we can make; let me know if you see any problems with these:

1.  I will simplify the Rtools site.  Rather than duplicating the
information from the R-admin manual, I will just include the
#The-Windows-toolset link as given below, together with post-release
updates.  This should help with the "too many places to look" problem.
Now, if other sites did the same...

2.  You suggest that Rtools.exe make automatic changes to the user's
PATH.  I'm quite reluctant to do that, because what R needs may conflict
with what a user needs for other work.  However, something that I think
would be fairly easy to do would be to install a shortcut to start a
CMD.EXE or Cygwin shell with the path set appropriately.  I'll look into
that.

I agree that setting the path permanently would be bad but
the shortcut could check to see if they are in the path already
and if not it could find them in the registry and just change the
session path rather than permanently set the path.

Do you have a batch file already to do this?  If I were writing it, I'd
do something simpler:  I'd just hard code the paths, and expect the user
to edit the file if I didn't get them right.

3.  You also suggested adding more tools to Rtools.exe.  I think the key
tool to add would be Perl; it's the only essential tool that's currently
missing.  Unfortunately, there are licensing issues with that for
ActiveState Perl, the version we currently recommend.  I'll  try to get
specific permission from ActiveState to include a copy.

The best, of course, would be to get rid of Perl altogether.

I agree, but there is a lot of Perl code in the R scripts now, and I
think the maintainers of those scripts are comfortable with it.  I think
R code for string handling is even harder to read than Perl, so it's not
likely to change.

4.  We could include the tools in the base installation of R; that's
what the MacOSX release has done.  However, that would at least double
the size of the download, even with just the minimal additions, and
under the GPL would require CRAN and its mirrors to host source for most
of the tools.  What I plan to do instead is to put a link to the Rtools
page on the CRAN page for the Windows R binary.

Some other possiblities are:
- use a different toolset.  I seem to remember that there were a few very
compact toolsets for use with minimal systems.  One basically put all
the UNIX tools in one executable and you could just rename it successively
to each tool you wanted.  Another one used lua for implementation which
is embeddable and very small.

This is not really relevant.  The Unix command line tools are fine:
Rtools.exe installs them, and they just work, if you get the PATH right.
 The current "problem children" are Perl, LaTeX, and HHW. (Also Inno
Setup, but it's not for package builders).

- rewrite as many tools as possible in R.  The full power of the tool might
not be needed making it easier.  If 80% of the tools could be easily eliminated
it might not be a problem to include the remaining 20%.  Or if 20% of the
tools take up 80% of the space this might be sufficient.  Of course getting
them all in R would be nice.

Rewriting the Perl code as R scripts is imaginable, but rewriting LaTeX
and HHW in R is not.

5.  We could modify the MkRules file that we distribute with R to assume
the minimal toolset, i.e. no LaTeX or CHM help.  This would mean you'd
get fewer "not found" messages from a basic install, but might be a
little inconvenient for people who have the tools.  Perhaps we can
detect whether those tools have been installed and act accordingly.

Not sure I understand this but could this be decided upon automatically
by checking the user's registry for the existence of the tools?

It might be possible to automate the tests by looking at the PATH.
Neither LaTeX nor HHW is necessarily going to show up at a predictable
location in the registry.

Duncan Murdoch

6.  You suggested avoiding changes to the toolset.  This is hard,
because the updates often fix bugs in older versions.  Bugs in the tools
lead to bugs in R.  So I think we don't have much choice but to continue
to make necessary updates.

Duncan Murdoch

Harold

-----Original Message-----
From: Duncan Murdoch [mailto:murdoch at stats.uwo.ca]
Sent: Thursday, May 03, 2007 3:51 PM
To: Doran, Harold
Cc: Gabor Grothendieck; r-help at stat.math.ethz.ch
Subject: [SPAM] - Re: [SPAM] - Re: [R] R package development
in windows - Bayesian Filter detected spam - Bayesian Filter
detected spam

On 5/3/2007 3:33 PM, Doran, Harold wrote:

Thanks, Duncan. I'll look into that. Is there an authoritative
document that codifies the new package development procedures for
2.5.0 (windows-specific), or is that Writing R Extensions? In this
thread alone I've received multiple emails pointing to multiple web
sites with instructions for windows. Inasmuch as its

appreciated, I'm

a bit confused as to which I should consider authoritative.

I do hope I can resolve this and appreciate the help I've received.
However, I feel a bit compelled to note how very difficult this
process is.

The manuals are the authoritative reference.  Writing R
Extensions tells you what to do to set up a package.  The R
Administration and Installation manual tells you how to set
up your system so those instructions will work.  On Windows,
you want to pay particular attention to the appendix of the
Admin manual called "The Windows Toolset". This is Appendix E
in the distributed .pdf; an HTML version is available here:


http://cran.r-project.org/doc/manuals/R-admin.html#The-Windows-toolset

We realize that these instructions are not easy to follow,
there's no need to tell us that.  However, specific
corrections or improvements would be welcome.

Duncan Murdoch

Harold

-----Original Message-----
From: Duncan Murdoch [mailto:murdoch at stats.uwo.ca]
Sent: Thursday, May 03, 2007 3:24 PM
To: Doran, Harold
Cc: Gabor Grothendieck; r-help at stat.math.ethz.ch
Subject: [SPAM] - Re: [R] R package development in windows

- Bayesian

Filter detected spam

On 5/3/2007 3:04 PM, Doran, Harold wrote:

Thanks Gabor, Sundar, and Tony. Indeed, Rtools was

missing from the

path. With that resolved, and another 10 minute windows

restart, I get

the following below. The log suggests that hhc is not

installed. It

is, and, according to the directions I am following, I have

placed it

in the c:\cygwin directory.

I think the problem is that you are following a real mix of
instructions, and they don't make sense.

It would be nice if folks would submit patches to the R

Admin manual

(or to the Rtools web site) rather than putting together web sites
with advice that is bad from day one, and quickly gets

worse when it

is not updated.

BTW, package.skeleton() doesn't seem to create the correct

DESCRIPTION

template. I had to add the DEPENDS line. Without this, I

get another

error.


C:\Program Files\R\R-2.4.1\bin>Rcmd build --force --binary g:\foo

R 2.4.1 is no longer current; the package building

instructions in R

2.5.0 have been simplified a bit.  You might want to try those.

Duncan Murdoch

* checking for file 'g:\foo/DESCRIPTION' ... OK
* preparing 'g:\foo':
* checking DESCRIPTION meta-information ... OK
* removing junk files
* checking for LF line-endings in source files
* checking for empty or unneeded directories
* building binary distribution
 WARNING
* some HTML links may not be found
installing R.css in c:/TEMP/Rinst40061099

Using auto-selected zip options ''
latex: not found
latex: not found
latex: not found

---------- Making package foo ------------
latex: not found
  adding build stamp to DESCRIPTION
latex: not found
latex: not found
latex: not found
  installing R files
latex: not found
  installing data files
latex: not found
  installing man source files
  installing indices
latex: not found
  not zipping data
  installing help
Warning: \alias{foo} already in foo-package.Rd -- skipping

the one in

foo.Rd  >>> Building/Updating help pages for package 'foo'
     Formats: text html latex example chm
  foo-package                       text    html    latex

example chm

  foo                               text    html    latex

example chm

  mydata                            text    html    latex

example chm

hhc: not found
cp: cannot stat `c:/TEMP/Rbuild40048815/foo/chm/foo.chm':

No such file

or direct ory
make[1]: *** [chm-foo] Error 1
make: *** [pkg-foo] Error 2
*** Installation of foo failed ***

Removing 'c:/TEMP/Rinst40061099/foo'
 ERROR
* installation failed


C:\Program Files\R\R-2.4.1\bin>

-----Original Message-----
From: Gabor Grothendieck [mailto:ggrothendieck at gmail.com]
Sent: Thursday, May 03, 2007 2:50 PM
To: Doran, Harold
Cc: r-help at stat.math.ethz.ch
Subject: Re: [R] R package development in windows

It can find sh.exe so you haven't installed Rtools.

There are several HowTo's listed in the links section here that
include pointers to R manuals and other step by step
instructions:

http://code.google.com/p/batchfiles/

On 5/3/07, Doran, Harold <HDoran at air.org> wrote:

I'm attempting to build an R package for distribution and

am working

from the directions found at

http://www.maths.bris.ac.uk/~maman/computerstuff/Rhelp/Rpackages.html

#

Wi
n-Win

I've read through Writing R Extensions and various other

"helpful" web

sites. I've installed all relevant software (perl,

cygwin, mingwin,

hhc.exe). Here is what I have done so far:

1) Sourced data and functions into R
2) Used package.skeleton
3) Edited Description file
4) Edited my windows path to ensure the new programs (e.g.,

perl) are

in the path
5) Then, I open DOS and do the following in the script below

C:\Program Files\R\R-2.4.1\bin>Rcmd build  --force

--binary g:\foo

* checking for file 'g:\foo/DESCRIPTION' ... OK
* preparing 'g:\foo':
* checking DESCRIPTION meta-information ...'sh' is not

recognized as

an internal  or external command, operable program or

batch file.

 OK
* removing junk files
'sh' is not recognized as an internal or external

command, operable

program or batch file.
'sh' is not recognized as an internal or external

command, operable

program or batch file.
'sh' is not recognized as an internal or external

command, operable

program or batch file.
Error: cannot open file 'foo/DESCRIPTION' for reading

I'ver read through about as much documentation as I can

find, and I'm

just not sure what I should do from here. I admit that I

have reached

a point of frustration and must apologize if the problem

would be

evident if I read documentation further, but I'm about

tapped out

after a few days of experimentation.

Can anyone suggest how I could resolve this and what the

next steps

would be?

Thanks:

I'm using Windows XP and R 2.4.1

Harold

       [[alternative HTML version deleted]]

reproducible code.

reproducible code.