R-devel Digest, Vol 229, Issue 21

On Wed, 30 Mar 2022 17:45:15 +0000
Georgi Boshnakov <georgi.boshnakov at manchester.ac.uk> wrote:

I am surprised that the above macro ever worked as intended.

I'm sorry, somehow I didn't pay enough attention when preparing that
part of the e-mail. The actual macro uses tools::toRd(bibentry(...)).

I don't know if something was fixed between the R-devel you used and
probably the most recent one I did but reproducible example and
system information would be helpful.

I'll remember to provide more reproducible examples next time.

c81965 was reverted, and "take two" of the improvement in c82029
doesn't crash my macros (thanks, Kurt Hornik!), but I'm still willing
to rewrite my macros to give R maintainers more freedom to evolve Rd in
the future. Maybe it's better to use a per-package database of
bibentries and reference them with much simpler macro arguments, but
that would be better fit for R-pkg-devel :)

The problem here is the Rd format, and the way user macros are handled.

The Rd format is unreasonably complicated, with several different modes 
of parsing (LaTeX-like, R-like and verbatim), and context dependent 
rules on what needs escaping.

The user macros, on the other hand, are very simple.  After expanding a 
macro, the parser just pushes the characters onto the input stream. 
This makes it pretty much impossible for automatic escaping to work, 
since the context can change in the middle of the new characters, but 
they haven't been parsed yet.

The motivation for r81965 was that some DOIs contain characters that 
need escaping in the context where the  \doi{} macro is being used. 
Kurt and I attempted to fix this automatically, rather than burdening 
the user with doing the escaping by hand.  That's hard for users to do, 
and if they get it wrong, the parser is likely to get confused and not 
give a helpful error message.

Unfortunately, our first attempt was a little too eager, and it caused 
the problems some of you saw.

It might be possible to do things differently, but really the only real 
solution would be to change the help file markup definition to something 
a little less insane.  When I wrote the Rd parser I was partly hoping 
that it would eventually be used to convert legacy help pages to a 
better format.  Roxygen allowing the use of Markdown instead of Rd 
markup is a step in the right direction, but (as I've said before) I 
believe forcing help text to be embedded in code is harmful to the goal 
of writing good documentation.  It addresses a real problem (writing .Rd 
documentation is hard), but I think it creates new problems, making both 
the code and the documentation harder to read and to edit than if they 
were separate.

Duncan Murdoch