Embedded R, last errormessage, and stack smashing

The only way to overcome the problem I can find is to tweak the
R_CStackLimit with:

R_CStackLimit = (uintptr_t) -1;

The question I am having now is: what are the implications of doing
so, that is what are the potential problems ?

The R-extensions manual says:
" Stack-checking can be disabled by seting R_CStackLimit =
(uintptr_t)-1, but it is better to
if possible set appropriate values. (What these are and how to
determine them are OS-specific,
and the stack size limit may differ for secondary threads. If you have
a choice of stack size, at
least 8Mb is recommended.)"
I am not very sure of how an appropriate value can be determined.

I looked around, and the JRI (Java/R Interface) is just disabling
stack checking for example.


Thanks,


Laurent


2008/6/30 Laurent Gautier <lgautier at gmail.com>:

Dear list,

I am having an embedded R, dying with
*** stack smashing detected *** in one specific case.

My code is such as I evaluate R expression with C code like

res = R_tryEval(expr, env, &error);

and in case of error, get the error message (usually sucessfully) with
code like below:

SEXP geterrmessage = findVar(install("geterrmessage"), R_BaseNamespace);
PROTECT(expr = allocVector(LANGSXP, 1));
SETCAR(expr, geterrmessage);
PROTECT(res = Rf_eval(expr, R_GlobalEnv));
// ---> *** stack smashing detected ***

The call to Rf_eval does not return as the stack smashing error stops the show.

Tracing with gdb where the problem occurs, I follow the path:
eval -> R_CheckStack in main/error.c -> errorcall in main/error.c


In errorcall, I narrow down the origin of the problem around the lines:
658    va_start(ap, format);
659    verrorcall_dflt(call, format, ap);
660    va_end(ap);
and add a breakpoint on each one of those lines.


My gdb session till the stack smashing crashing is then looking like:

Breakpoint 7, Rf_errorcall (call=0x828ef8, format=0x7f4dbdc87698 "C
stack usage is too close to the limit")
   at errors.c:658
658         va_start(ap, format);
(gdb) p ap
$2 = {{gp_offset = 16, fp_offset = 48, overflow_arg_area = 0x41c22f60,
reg_save_area = 0x41c22e70}}
(gdb) p format
$3 = 0x7f4dbdc87698 "C stack usage is too close to the limit"
(gdb) continue
Continuing.

Breakpoint 8, Rf_errorcall (call=0x828ef8, format=0x7f4dbdc87698 "C
stack usage is too close to the limit")
   at errors.c:659
659         verrorcall_dflt(call, format, ap);
(gdb) continue
Continuing.

Breakpoint 6, Rf_errorcall (call=0x828ef8, format=0x7f4dbdc87406 "%s")
at errors.c:648
648         if (R_ErrorHook != NULL) {
(gdb) continue
Continuing.

Breakpoint 7, Rf_errorcall (call=0x828ef8, format=0x7f4dbdc87406 "%s")
at errors.c:658
658         va_start(ap, format);
(gdb) continue
Continuing.

Breakpoint 8, Rf_errorcall (call=0x828ef8, format=0x7f4dbdc87406 "%s")
at errors.c:659
659         verrorcall_dflt(call, format, ap);
(gdb) continue
Continuing.
*** stack smashing detected ***: /usr/bin/python terminated


My understanding is that there is a C stack problem about to happen R
tries to report, but in the process of reporting it
causes a C stack-related crash. Is there something odd with R handling
the situation, or should I look for the cause
of the problem elsewhere ?

Thanks,


L.