Comments disappearing from local functions (R 2.14.0)

I've installed R-2.14.0 from source on CentOS and on Kubuntu and in
both cases, I see something I've never seen before.  Comments in
locally written functions disappear.  I put comments there for a
purpose and I'd like to keep them.  I can still use older versions of
R without that happening.  Nothing I noticed in the NEWS file seems to
indicate a change that could be related to that phenomenon.

(Using ESS, it's more complicated but I figure if I get it working from
the terminal the rest will work.)

Since nobody else has reported such a problem, it might be something
in my local settings that's behind what I'm seeing but I can't guess
where I should look.

Suggestions welcome.

It *doesn't* happen to me.  Ergo it would appear to be peculiar
to your particular set-up.

What does options()$keep.source say?  (Mine says "TRUE".)

          cheers,

             Rolf

P. S.:

 > sessionInfo()
R version 2.14.0 (2011-10-31)
Platform: i686-pc-linux-gnu (32-bit)

locale:
  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
  [7] LC_PAPER=C                 LC_NAME=C
  [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] misc_0.0-15

[....]

|> It *doesn't* happen to me.  Ergo it would appear to be peculiar
|> to your particular set-up.
|> 
|> What does options()$keep.source say?  (Mine says "TRUE".)
|> 

Same here in both cases.


|> 
|> P. S.:
|> 
|> > sessionInfo()
|> R version 2.14.0 (2011-10-31)
|> Platform: i686-pc-linux-gnu (32-bit)
|> 
|> locale:
|>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
|>  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
|>  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
|>  [7] LC_PAPER=C                 LC_NAME=C
|>  [9] LC_ADDRESS=C               LC_TELEPHONE=C
|> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

Both the same except the Kubuntu one is 
Platform: x86_64-unknown-linux-gnu (64-bit)


|> 
|> attached base packages:
|> [1] stats     graphics  grDevices utils     datasets  methods   base
|> 
|> other attached packages:
|> [1] misc_0.0-15

attached base packages:
[1] splines   grDevices utils     stats     graphics  methods   base     

other attached packages:
[1] multicore_0.1-7  gbm_1.6-3.1      survival_2.36-10 lattice_0.20-0  

loaded via a namespace (and not attached):
[1] grid_2.14.0  tools_2.14.0

With the one using multicore, I get a strange error that didn't occur
with R-2.13.1.  A function which runs on 7 cores produces a swag of
PDF files and then returns a list of information.  New to the 2.14.0
is an error like this:

"Error in dev.off() : internal error in PDF_endpage\n"

From what I can work out, that is a C message and tricky to trace.

However, if I use a postscript device instead of PDF, it works fine.
It's a simple matter to convert a postscript file to pdf, so that's
easy to deal with.  It's impossible to make a toy example of the
problem which doesn't appear when run on a single core.  Maybe if I
knew exactly where the problem arises I could make a toy example but
it's rather a lot of code I'm dealing with.  The fact that the two
problems I'm encountering did not occur with earlier versions makes me
less suspicious of my own code.


My main question is: Can I redo the updating of the packages with
checkBuilt to downgrade them back to the previous version?  Something
is not right with either of my installations and I'd like to revert.

The problem with the disappearing comments occurs when I wish to
attach to another directory.  I commonly reuse functions from previous
years and modify them for the current year.  

If my pwd is, say ~/thisYear, I'll do something like 
attach("~/lastYear/.RData")

If I then edit the function of interest, when it's saved I have the
appropriate version in ~/thisYear/.RData which I find a convenient way
to work.  If the comments aren't there, it's far less usable.  If I
can go back to R-2.13.1 I can still work that way.


TIA

~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.   
   ___    Patrick Connolly   
 {~._.~}                   Great minds discuss ideas    
 _( Y )_  	         Average minds discuss events 
(:_~*~_:)                  Small minds discuss people  
 (_)-(_)  	                      ..... Eleanor Roosevelt
	  
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.

On Mon, 21-Nov-2011 at 09:46PM +1300, Rolf Turner wrote:

[....]

Please don't excise content, nor mix up topics.  I am going to reply 
here just to the subject line, and reply to your use of pdf() under 
multicore separately.

|> It *doesn't* happen to me.  Ergo it would appear to be peculiar
|> to your particular set-up.
|>
|> What does options()$keep.source say?  (Mine says "TRUE".)
|>

Same here in both cases.

[...]

My main question is: Can I redo the updating of the packages with
checkBuilt to downgrade them back to the previous version?  Something
is not right with either of my installations and I'd like to revert.

The problem with the disappearing comments occurs when I wish to
attach to another directory.  I commonly reuse functions from previous
years and modify them for the current year.

If my pwd is, say ~/thisYear, I'll do something like
attach("~/lastYear/.RData")

If I then edit the function of interest, when it's saved I have the
appropriate version in ~/thisYear/.RData which I find a convenient way
to work.  If the comments aren't there, it's far less usable.  If I
can go back to R-2.13.1 I can still work that way.

So you missed the NEWS item:

     ? The "source" attribute on functions created with keep.source=TRUE
       has been replaced with a "srcref" attribute.  The "srcref"
       attribute references an in-memory copy of the source file using
       the "srcfilecopy" class or the new "srcfilealias" class.

This means that if you save a function under R 2.13.1 and load/attach 
it under R 2.14.0, its kept source (with the comments) is no longer 
available.

However your original posting did not mention save/attach, nor that 
you saved under one version and attached under another.  That is hte 
only circumstance under which I can make it do what you claim.

And now you know not to do that.  Dump your functions in 2.13.1, and 
source/save in 2.14.0.

Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

[... about a completely different topic]

With the one using multicore, I get a strange error that didn't occur
with R-2.13.1.  A function which runs on 7 cores produces a swag of
PDF files and then returns a list of information.  New to the 2.14.0
is an error like this:

"Error in dev.off() : internal error in PDF_endpage\n"

From what I can work out, that is a C message and tricky to trace.

Easy to trace; you look for it in the code! It indicates that you used 
PDF compression (new to 2.14.0) and that the page-stream file got 
corrupted.  So simply use compress=FALSE to get the previous 
behaviour.

However, mixing graphics devices with multicore is probably asking for 
trouble.  If your fork with an open graphics device you certainly are 
since the parent and child will share file handles.
But even if each child opens a separate pdf() device and closes it 
before it returns, there still are many pitfalls as the parent and 
child share a session temporary directory and all assume they have 
exclusive access to it.  My guess is that two pdf() devices in 
different processes are using the same temporary file, or the same 
FILE stream associated with a temporary file.  And only onefile=FALSE 
or compress=TRUE use such temporary files.

However, if I use a postscript device instead of PDF, it works fine.
It's a simple matter to convert a postscript file to pdf, so that's
easy to deal with.  It's impossible to make a toy example of the
problem which doesn't appear when run on a single core.  Maybe if I
knew exactly where the problem arises I could make a toy example but
it's rather a lot of code I'm dealing with.  The fact that the two
problems I'm encountering did not occur with earlier versions makes me
less suspicious of my own code.

You could at least have outlined what you are doing.

Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

|>

|> >
|> >[....]
|> 
|> Please don't excise content, nor mix up topics.  I am going to

I was indicating two reasons why I wanted to be able to downgrade to
2.13.1 but I should have modified the subject.  This is what I'd
stated:

|> >My main question is: Can I redo the updating of the packages with
|> >checkBuilt to downgrade them back to the previous version?
|> >Something is not right with either of my installations and I'd
|> >like to revert.


However, that will be unnecessary since I now have perfectly good
explanations for both issues, neither of which I would have figured
out for myself.  


[...]

|> So you missed the NEWS item:
|> 
|>     ? The "source" attribute on functions created with keep.source=TRUE
|>       has been replaced with a "srcref" attribute.  The "srcref"
|>       attribute references an in-memory copy of the source file using
|>       the "srcfilecopy" class or the new "srcfilealias" class.

Yes.  I missed that on line 270odd.  It's so long since I set up the
keep.source option I'd forgotten that's what I should have been
searching for. 

|> 
|> This means that if you save a function under R 2.13.1 and
|> load/attach it under R 2.14.0, its kept source (with the comments)
|> is no longer available.

Thanks for that explanation.  I'm still struggling to see what the
benefit of such a change would be.  It didn't look broke to me.


|>  However your original posting did not mention save/attach, nor
|> that you saved under one version and attached under another.  That
|> is hte only circumstance under which I can make it do what you
|> claim.

I was even more ignorant at the time of my first post.  Later
experimentation narrowed down the occasions that produced the
phenomenon which I hoped might help indicate what was behind it.


|> 
|> And now you know not to do that.  Dump your functions in 2.13.1,
|> and source/save in 2.14.0.
|> 
|> 
|> -- 
|> Brian D. Ripley,                  ripley at stats.ox.ac.uk
|> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
|> University of Oxford,             Tel:  +44 1865 272861 (self)
|> 1 South Parks Road,                     +44 1865 272866 (PA)
|> Oxford OX1 3TG, UK                Fax:  +44 1865 272595

~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.   
   ___    Patrick Connolly   
 {~._.~}                   Great minds discuss ideas    
 _( Y )_  	         Average minds discuss events 
(:_~*~_:)                  Small minds discuss people  
 (_)-(_)  	                      ..... Eleanor Roosevelt
	  
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.

|> 
|> [... about a completely different topic]
|> 
|> >With the one using multicore, I get a strange error that didn't occur
|> >with R-2.13.1.  A function which runs on 7 cores produces a swag of
|> >PDF files and then returns a list of information.  New to the 2.14.0
|> >is an error like this:
|> >
|> >"Error in dev.off() : internal error in PDF_endpage\n"
|> >
|> >>From what I can work out, that is a C message and tricky to trace.
|> 
|> Easy to trace; you look for it in the code! It indicates that you

I did find it at about line 7050 in
RHOME/src/library/grDevices/src/devPS.c

But being a bear with such a small brain (and even smaller when it
comes to C), it was rather tricky for me to trace that it had to do
with the 'if (pd->useCompression) {' line about 10 before that one.

|> used PDF compression (new to 2.14.0) and that the page-stream file
|> got corrupted.  So simply use compress=FALSE to get the previous
|> behaviour.

Wow! simple once you know.  That would certainly explain why it's not
a problem with the postscript device.


|> 
|> However, mixing graphics devices with multicore is probably asking
|> for trouble.  If your fork with an open graphics device you
|> certainly are since the parent and child will share file handles.
|> But even if each child opens a separate pdf() device and closes it
|> before it returns, there still are many pitfalls as the parent and
|> child share a session temporary directory and all assume they have
|> exclusive access to it.  My guess is that two pdf() devices in
|> different processes are using the same temporary file, or the same
|> FILE stream associated with a temporary file.  And only
|> onefile=FALSE or compress=TRUE use such temporary files.

That sounds like a pretty good guess.


|> 
|> >However, if I use a postscript device instead of PDF, it works
|> >fine.  It's a simple matter to convert a postscript file to pdf,
|> >so that's easy to deal with.  It's impossible to make a toy
|> >example of the problem which doesn't appear when run on a single
|> >core.  Maybe if I knew exactly where the problem arises I could
|> >make a toy example but it's rather a lot of code I'm dealing with.
|> >The fact that the two problems I'm encountering did not occur with
|> >earlier versions makes me less suspicious of my own code.
|> 
|> You could at least have outlined what you are doing.

In my defence, I did indicate that I was making a bunch of PDF files
in parallel and getting a list together at the end.  The inclusion of
gbm in the attached packages would give a hint at what was happening
in the parallel processes.  Would have knowing more about what was
happening with gbm have made it any clearer?  


As usual, Brian's insights were spot on and very much appreciated.



|> 
|> -- 
|> Brian D. Ripley,                  ripley at stats.ox.ac.uk
|> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
|> University of Oxford,             Tel:  +44 1865 272861 (self)
|> 1 South Parks Road,                     +44 1865 272866 (PA)
|> Oxford OX1 3TG, UK                Fax:  +44 1865 272595

~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.   
   ___    Patrick Connolly   
 {~._.~}                   Great minds discuss ideas    
 _( Y )_  	         Average minds discuss events 
(:_~*~_:)                  Small minds discuss people  
 (_)-(_)  	                      ..... Eleanor Roosevelt
	  
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.