Core dump with a 64-bit system (PR#2165)

Full_Name: Kai Mäkisara
Version: 1.6.0
OS: Tru64
Submission from: (NULL) (128.214.53.92)


Any command given in terminal window dumps core if readline is compiled in.
The reason is a bug in pushReadline: the first call pushes the function below
the stack. The following patch solves the problem:

--- R-1.6.0/src/unix/sys-std.c~ Sun Aug 25 12:51:20 2002
+++ R-1.6.0/src/unix/sys-std.c  Tue Oct 15 14:43:00 2002
@@ -389,10 +389,10 @@
 void
 pushReadline(char *prompt, rl_vcpfunc_t f)
 {
-   if(ReadlineStack.current >= ReadlineStack.max) {
+   if(ReadlineStack.current >= ReadlineStack.max - 1) {
      warning("An unusual circumstance has arisen in the nesting of readline
input. Please report using bug.report()");
    } else
-     ReadlineStack.fun[ReadlineStack.current++] = f; 
+     ReadlineStack.fun[++ReadlineStack.current] = f; 
 
    rl_callback_handler_install(prompt, f);
 }

In a 32-bit system the first call overwrites only ReadlineStack.max and this is
probably never noticed. In a 64-bit system the results are more interesting :-)

             Kai



-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-devel-request@stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._