[Rcpp-devel] "Error during wrapup"

Hi everyone,

Thank for you sticking with the problem even though it is so hard to
recreate. I believe I have solved it, but the solution is based on a
workaround, and I believe there is still a problem in Rcpp. First, the
problem is not limited to wrapping map objects. I changed the code to return
a NumericVector and had the same issue, although perhaps it's funneled
through the same wrap code down the line -- again I'm not really familiar
with Rcpp internals. Second, I noticed the problem occurred only when I used
R's random number generator and RNGScope; code using C++'s RNG worked fine.
Third, with that information in hand, I ran across this thread:
http://lists.r-forge.r-project.org/pipermail/rcpp-devel/2013-May/005838.html,
which suggests that the order in which the destructor of the RNGScope object
and the returned object fire can muck things up. Based on this idea, I
changed things so the returned NumericVector is created before the RNGScope
object, and the already existing object is populated before being returned.
This appears to have fixed it.

This solution is good enough for my immediate problem. Unfortunately for
helping you fix this in Rcpp, all of this was done in the original code. I
still can't create a small reproducible example. The problem is really hard
to recreate -- the code in question is used for numerical simulation and the
function runs hundreds of thousands of times before failing. But perhaps
this new information can help inspire someone. I will be away for a few
weeks, but am happy to test patches on the original code (if someone is
inspired) if it's helpful once I return.




On Thu, May 29, 2014 at 1:10 AM, Kevin Ushey <kevinushey at gmail.com> wrote:

What OS are you running on (Mac, Linux, Windows)? I really can't tease
this out (with Mac + clang) thus far.

On Wed, May 28, 2014 at 9:56 PM, Romain Francois
<romain at r-enthusiasts.com> wrote:

Le 29 mai 2014 ? 00:35, John Mous <john.mous0 at gmail.com> a ?crit :

No luck with gctorture on yet, but I haven't run it for very long. It
slowed
things down to a complete crawl, but maybe in the long run it can
recreate
the problem faster than running the original code (about 12 hours, the
previous estimate of a day was too high), so this may still be
worthwhile.

On a whim I did try changing the two instances of Shield to Armor this
morning (I was able to locate the code in wrap.h by digging around
before I
saw your e-mail) and ran the original full code without gctorture on,
and
the problem still persists.


Would not be enough, you'd also have to do:

x = ::Rf_setAttrib( x, R_NamesSymbol, names ) ;

so that the result of setAttrib gets protected even if it is a new SEXP,
which AFAIU can happen with setAttrib.

But I'm not sure this is the problem.

Romain


On Wed, May 28, 2014 at 3:44 PM, Romain Francois
<romain at r-enthusiasts.com>
wrote:

(now with some links):

Le 28 mai 2014 ? 16:31, John Mous <john.mous0 at gmail.com> a ?crit :

The object really is just built as part of the return statement. i.e.
the
lines from my prior e-mail exist as-is in the full code,


Sure. What happens is that Rcpp::export generates something that calls
wrap( std::map<std::string,int> ).

there's just more that actually builds the variables X1-X4 beforehand.


It should not be relevant.

So I'm not sure where to debug from the client side. I'm a C/C++
developer, but have no experience with Rcpp internals or the general
interface between R and C. I can insert some debug statements on the
Rcpp
side if you can guide me to where wrap is defined.


well wrap is defined here:


https://github.com/RcppCore/Rcpp/blob/master/inst/include/Rcpp/internal/wrap_end.h#L28

It then performs a series of dispatch, that eventually lead to
something
that handles std::map<std::string,int>:


https://github.com/RcppCore/Rcpp/blob/master/inst/include/Rcpp/internal/wrap.h#L255


template <typename InputIterator, typename T>
inline SEXP range_wrap_dispatch___impl__cast( InputIterator first,
InputIterator last, ::Rcpp::traits::false_type ){
     size_t size = std::distance( first, last ) ;
     const int RTYPE = ::Rcpp::traits::r_sexptype_traits<typename
T::second_type>::rtype ;
     Shield<SEXP> x( Rf_allocVector( RTYPE, size ) );
     Shield<SEXP> names( Rf_allocVector( STRSXP, size ) ) ;
     typedef typename ::Rcpp::traits::storage_type<RTYPE>::type CTYPE ;
     CTYPE* start = r_vector_start<RTYPE>(x) ;
     size_t i =0;
     std::string buf ;
     for( ; i<size; i++, ++first){
             start[i] = (*first).second ;
             buf = (*first).first ;
             SET_STRING_ELT( names, i, Rf_mkChar(buf.c_str()) ) ;
     }
     ::Rf_setAttrib( x, R_NamesSymbol, names ) ;
     return wrap_extra_steps<T>( x ) ;
}


what I suspect to be the problem is this line:

 ::Rf_setAttrib( x, R_NamesSymbol, names ) ;


What Romain says makes sense to me, despite my lack of expertise in
this
area.. the really intermittent nature of the problem and the fact that
I
can't recreate it in a small / fast running example suggests that
perhaps
this manifests when R happens to garbage collect at an unfortunate time
(if
I understood correctly). Thanks again.


Something like that. as hadley hinted, please try this under gc
torture,
or preferably through a debugger.

On Wed, May 28, 2014 at 10:12 AM, Dirk Eddelbuettel <edd at debian.org>
wrote:

On 28 May 2014 at 10:02, John Mous wrote:
| Hmm, unfortunately the GitHub version failed also.

Darn.

| The attributes on the failed
| object are a little different though, here's what they look like:
|
| Browse[1]> str(results)
|  atomic [1:4] 1 1 2270 0
|  - attr(*, "")= symbol sim
|  - attr(*, "value")= promise to  NULL

I am not sure what we can do without a reproducible example. :-/
The code just got a review / refreshment over the last few months.

You best bet may the slow and tedious insertion of debug statements to
see
when / if the object changes.

Dirk

--
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com