R CMD Rdconv drops sections: arguments, seealso, examples (PR#9649)

On Tue, 10 Apr 2007 timh at insightful.com wrote:

I've created a .Rd file (below), then converted that to .sgml using
	R CMD Rdconv --type=Ssgm combn.Rd  > combn.sgml
The output (shown below) is missing some of the sections:
	arguments
	seealso
	examples
If instead I convert to .d (below), the same sections are missing,
and the "note" section is included but without the necessary newline.
The underlying problem was that there were some unmatched open-braces.
(or 'brackets'?) The attached patch to share/perl/R/Rdconv.pm:mark_brackets()
seems to detect unmatched open and close braces and throw an error:

   % R CMD INSTALL -l /tmp/Rlib timhPackage
   * Installing *source* package 'timhPackage' ...
   ** help
    >>> Building/Updating help pages for package 'timhPackage'
        Formats: text html latex example
   Rdconv(): mismatched braces in help file timh.Rd on or after line 12
   ERROR: building help failed for package 'timhPackage'
   ** Removing '/tmp/Rlib/timhPackage'
   ** Restoring previous '/tmp/Rlib/timhPackage'

The code to report the line number doesn't work quite right
since any "% comment" lines in the Rd file have been removed
from $complete_text by the time we get here.  I fudged by
putting 'on or after line XXX' in the error message.
The following patch adds a little more information (the
name of the \tag or whether it is an unmatched { or })
to the error message, so Tim's example results in

    % R CMD INSTALL -l /tmp/Rlib timhPackage
    * Installing *source* package 'timhPackage' ...
    ** help
     >>> Building/Updating help pages for package 'timhPackage'
         Formats: text html latex example
    Rdconv(): mismatched braces ('\arguments{') in help file timh.Rd on or after line 12
    ERROR: building help failed for package 'timhPackage'
    ** Removing '/tmp/Rlib/timhPackage'
    ** Restoring previous '/tmp/Rlib/timhPackage'


*** Rdconv.pm~	2007-03-29 19:05:08.000000000 -0700

--- Rdconv.pm	2007-05-01 10:28:55.000000000 -0700
***************
*** 254,259 ****
--- 254,275 ----
  	$complete_text =~ s/{([^{}]*)}/$id$1$id/s;
  	print STDERR "." if $debug;
      }
+     # Any remaining brackets must be unmatched ones, hence report error.
+     if ($complete_text =~ /([{}])/s) {
+         # Would like to tell which which line has unmatched { or },
+         # but lines starting with % have already been removed.
+         # Hence the 'on or after' in the message.
+         my $badlineno=0 ;
+ 	my $extra_info = "(\'$1\')" ;
+         if ($complete_text =~ /(\\\w+{)/) {
+ 		$extra_info = "(\'$1\')" ;
+ 	}
+         foreach my $line (split /\n/, $complete_text) {
+             $badlineno++;
+             last if ($line =~ /[{}]/) ;
+         }
+         die "Rdconv(): mismatched braces $extra_info in help file $Rdname on or after line $badlineno\n" ;
+     }
  }

  sub unmark_brackets {
Bill,

Now we have access again to the R-bugs repository (the database has been 
offline for about a week) I can try to understand this.

This is a follow up to PR#9645 which was itself a follow up to PR#9606, 
which was a report that Rdconv silently drops sections it makes no sense 
of.  (The latter is well known, and that's why R CMD check has an 
independent check of Rd syntax using R code.)

And this supersedes the suggestion in PR#9645, both of which are that 
Rdconv should detect unclosed sections and throw an error.

It is not clear to me that throwing an error is helpful as it would stop 
the package installation process when all but one section in one .Rd file 
would be useful.  But it would seem good to give a warning, and so I 
propose that we adapt your code to do so.

Does that reasonably reflect the various PRs?

Brian

On Mon, 30 Apr 2007 bill at insightful.com wrote:

On Tue, 10 Apr 2007 timh at insightful.com wrote:

I've created a .Rd file (below), then converted that to .sgml using
	R CMD Rdconv --type=Ssgm combn.Rd  > combn.sgml
The output (shown below) is missing some of the sections:
	arguments
	seealso
	examples
If instead I convert to .d (below), the same sections are missing,
and the "note" section is included but without the necessary newline.
The underlying problem was that there were some unmatched open-braces.
(or 'brackets'?) The attached patch to share/perl/R/Rdconv.pm:mark_brackets()
seems to detect unmatched open and close braces and throw an error:

   % R CMD INSTALL -l /tmp/Rlib timhPackage
   * Installing *source* package 'timhPackage' ...
   ** help
   >>> Building/Updating help pages for package 'timhPackage'
        Formats: text html latex example
   Rdconv(): mismatched braces in help file timh.Rd on or after line 12
   ERROR: building help failed for package 'timhPackage'
   ** Removing '/tmp/Rlib/timhPackage'
   ** Restoring previous '/tmp/Rlib/timhPackage'

The code to report the line number doesn't work quite right
since any "% comment" lines in the Rd file have been removed
from $complete_text by the time we get here.  I fudged by
putting 'on or after line XXX' in the error message.
The following patch adds a little more information (the
name of the \tag or whether it is an unmatched { or })
to the error message, so Tim's example results in

   % R CMD INSTALL -l /tmp/Rlib timhPackage
   * Installing *source* package 'timhPackage' ...
   ** help
    >>> Building/Updating help pages for package 'timhPackage'
        Formats: text html latex example
   Rdconv(): mismatched braces ('\arguments{') in help file timh.Rd on or after line 12
   ERROR: building help failed for package 'timhPackage'
   ** Removing '/tmp/Rlib/timhPackage'
   ** Restoring previous '/tmp/Rlib/timhPackage'


*** Rdconv.pm~	2007-03-29 19:05:08.000000000 -0700
--- Rdconv.pm	2007-05-01 10:28:55.000000000 -0700
***************
*** 254,259 ****
--- 254,275 ----
 	$complete_text =~ s/{([^{}]*)}/$id$1$id/s;
 	print STDERR "." if $debug;
     }
+     # Any remaining brackets must be unmatched ones, hence report error.
+     if ($complete_text =~ /([{}])/s) {
+         # Would like to tell which which line has unmatched { or },
+         # but lines starting with % have already been removed.
+         # Hence the 'on or after' in the message.
+         my $badlineno=0 ;
+ 	my $extra_info = "(\'$1\')" ;
+         if ($complete_text =~ /(\\\w+{)/) {
+ 		$extra_info = "(\'$1\')" ;
+ 	}
+         foreach my $line (split /\n/, $complete_text) {
+             $badlineno++;
+             last if ($line =~ /[{}]/) ;
+         }
+         die "Rdconv(): mismatched braces $extra_info in help file $Rdname on or after line $badlineno\n" ;
+     }
 }

 sub unmark_brackets {

______________________________________________
R-devel at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel


Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

Bill,

Now we have access again to the R-bugs repository (the database has been
offline for about a week) I can try to understand this.

This is a follow up to PR#9645 which was itself a follow up to PR#9606,
which was a report that Rdconv silently drops sections it makes no sense
of.  (The latter is well known, and that's why R CMD check has an
independent check of Rd syntax using R code.)

And this supersedes the suggestion in PR#9645, both of which are that
Rdconv should detect unclosed sections and throw an error.

It is not clear to me that throwing an error is helpful as it would stop
the package installation process when all but one section in one .Rd file
would be useful.  But it would seem good to give a warning, and so I
propose that we adapt your code to do so.
Doing so (now in R-devel) shows a couple of false positives (\align{{} in 
Paren.Rd is one), and rather a lot of correct warnings about excess }s 
that are not doing any harm.  So I have tuned the warnings to be less 
strident in the latter case.
Does that reasonably reflect the various PRs?

Brian


On Tue, 1 May 2007, bill at insightful.com wrote:

On Mon, 30 Apr 2007 bill at insightful.com wrote:

On Tue, 10 Apr 2007 timh at insightful.com wrote:

I've created a .Rd file (below), then converted that to .sgml using
	R CMD Rdconv --type=Ssgm combn.Rd  > combn.sgml
The output (shown below) is missing some of the sections:
	arguments
	seealso
	examples
If instead I convert to .d (below), the same sections are missing,
and the "note" section is included but without the necessary newline.
The underlying problem was that there were some unmatched open-braces.
(or 'brackets'?) The attached patch to share/perl/R/Rdconv.pm:mark_brackets()
seems to detect unmatched open and close braces and throw an error:

   % R CMD INSTALL -l /tmp/Rlib timhPackage
   * Installing *source* package 'timhPackage' ...
   ** help
  >>> Building/Updating help pages for package 'timhPackage'
        Formats: text html latex example
   Rdconv(): mismatched braces in help file timh.Rd on or after line 12
   ERROR: building help failed for package 'timhPackage'
   ** Removing '/tmp/Rlib/timhPackage'
   ** Restoring previous '/tmp/Rlib/timhPackage'

The code to report the line number doesn't work quite right
since any "% comment" lines in the Rd file have been removed
from $complete_text by the time we get here.  I fudged by
putting 'on or after line XXX' in the error message.
The following patch adds a little more information (the
name of the \tag or whether it is an unmatched { or })
to the error message, so Tim's example results in

   % R CMD INSTALL -l /tmp/Rlib timhPackage
   * Installing *source* package 'timhPackage' ...
   ** help
   >>> Building/Updating help pages for package 'timhPackage'
        Formats: text html latex example
   Rdconv(): mismatched braces ('\arguments{') in help file timh.Rd on or after line 12
   ERROR: building help failed for package 'timhPackage'
   ** Removing '/tmp/Rlib/timhPackage'
   ** Restoring previous '/tmp/Rlib/timhPackage'


*** Rdconv.pm~	2007-03-29 19:05:08.000000000 -0700
--- Rdconv.pm	2007-05-01 10:28:55.000000000 -0700
***************
*** 254,259 ****
--- 254,275 ----
 	$complete_text =~ s/{([^{}]*)}/$id$1$id/s;
 	print STDERR "." if $debug;
     }
+     # Any remaining brackets must be unmatched ones, hence report error.
+     if ($complete_text =~ /([{}])/s) {
+         # Would like to tell which which line has unmatched { or },
+         # but lines starting with % have already been removed.
+         # Hence the 'on or after' in the message.
+         my $badlineno=0 ;
+ 	my $extra_info = "(\'$1\')" ;
+         if ($complete_text =~ /(\\\w+{)/) {
+ 		$extra_info = "(\'$1\')" ;
+ 	}
+         foreach my $line (split /\n/, $complete_text) {
+             $badlineno++;
+             last if ($line =~ /[{}]/) ;
+         }
+         die "Rdconv(): mismatched braces $extra_info in help file $Rdname on or after line $badlineno\n" ;
+     }
 }

 sub unmark_brackets {

______________________________________________
R-devel at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel



Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

On Thu, 3 May 2007, Prof Brian Ripley wrote:

It is not clear to me that throwing an error is helpful as it would stop
the package installation process when all but one section in one .Rd file
would be useful.  But it would seem good to give a warning, and so I
propose that we adapt your code to do so.
Doing so (now in R-devel) shows a couple of false positives (\align{{} in
Paren.Rd is one), and rather a lot of correct warnings about excess }s
that are not doing any harm.  So I have tuned the warnings to be less
strident in the latter case.
That sounds good.

Here is a modification of your fix that prints a message for
problem in the Rd file, not just the first.   E.g., in an old
version of the msm package it gives
  Note: unmatched right brace in './msm/man/deltamethod.Rd' on or after line 32
  Note: unmatched right brace in './msm/man/deltamethod.Rd' on or after line 54
  Note: unmatched right brace in './msm/man/deltamethod.Rd' on or after line 55
instead of just complaining about the first one.

Index: Rdconv.pm
===================================================================
--- Rdconv.pm	(revision 41470)
+++ Rdconv.pm	(working copy)
@@ -258,24 +258,25 @@
     }
     # Any remaining brackets must be unmatched ones.
     # However, unmatched brackets are sometimes legal,
-    # (e.g. \alias{{}), so only warn.
+    # (e.g. \alias{{}), so only warn. # }match brace in comment
     if ($complete_text =~ /([{}])/s) {
         # Would like to tell which which line has unmatched { or },
         # but lines starting with % have already been removed.
         # Hence the 'on or after' in the message.
         my $badlineno = 0 ;
-	my $extra_info = "\'$1\'" ;
-        $extra_info = "\'$1\'" if $complete_text =~ /(\\\w+{)/ ;
 	foreach my $line (split /\n/, $complete_text) {
 	    $badlineno++;
-	    last if ($line =~ /[{}]/) ;
+	    if ($line =~ /([{}])/) {
+	        my $extra_info = "\'$1\'" ;
+            	$extra_info = "\'$1\'" if $line =~ /(\\\w+{)/ ; # }match brace in pattern
+	        if( $extra_info =~ /^'}'$/ ) {
+	            warn "Note: unmatched right brace in '$Rdname'".
+		        " on or after line $badlineno\n";
+	        } elsif(! ($extra_info =~ /\\alias{/) )  # }match brace in pattern
+	            { warn "Warning: unmatched brace ($extra_info) in '$Rdname'".
+		           " on or after line $badlineno\n"; }
+           }
 	}
-	if( $extra_info =~ /^'}'$/ ) {
-	    warn "Note: unmatched right brace in '$Rdname'".
-		" on or after line $badlineno\n";
-	} elsif(! ($extra_info =~ /\\alias{/) )
-	    { warn "Warning: unmatched brace ($extra_info) in '$Rdname'".
-		   " on or after line $badlineno\n"; }
     }
 }


----------------------------------------------------------------------------
Bill Dunlap
Insightful Corporation
bill at insightful dot com
360-428-8146

 "All statements in this message represent the opinions of the author and do
 not necessarily reflect Insightful Corporation policy or position."