Serializing many small objects efficiently

An embedded and charset-unspecified text was scrubbed...
Name: not available
URL: <https://stat.ethz.ch/pipermail/r-devel/attachments/20120322/e52c82d4/attachment.pl>
Here's a snip from r-hcp. You can probably find it in the archive:

From: Michael Spiegel
Date: Thu, Sep 29, 2011 at 11:38 AM
Subject: RE: [R-sig-hpc] [zeromq-dev] rzmq package

Calling serialize/serialize from c/c++ is not too convoluted. You can
find a good example in
https://github.com/mspiegel/PiebaldMPI/blob/master/src/lapply_workers_helpers.c,
look for the function "generateReturnList". I'm doing both
serialization and unserialization in that function, but you'll be able
to tease apart the two calls.

On Thu, Mar 22, 2012 at 12:34 PM, Antonio Piccolboni
Hi,
sorry if this question is trivial or unclear, this is my first venture into
mixed C/R programming (I am reasonably experienced in each separately).
I am trying to write a serialization function for a format called
typedbytes, which is used as an interchange format in Hadoop circles. Since
I would need to serialize according to the internal R format many small R
objects I looked at the c interface

void R_Serialize(SEXP s, R_outpstream_t ops);
SEXP R_Unserialize(R_inpstream_t ips);

If I look at the source for e.g. unserialize is see a

?.Call("R_unserialize", connection, refhook, PACKAGE = "base")

which, despite the name of the second argument, accepts as 'connection' a
raw vector. Is there any way to call that function from C -- without
calling the R function? Failing that, from what I've read I gather that it
is not possible to get a C stream from a connection, so unless I am wrong
using R_serialize directly is not possible. If all else fails I would have
probably to use a hack requiring knowledge of the serialization format,
which I'd much rather avoid. Suggestions? Thanks

Antonio

? ? ? ?[[alternative HTML version deleted]]

______________________________________________
R-devel at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel