problem of the very large memory require by the Sign extension.
--- R-2.5.0.orig/src/main/serialize.c 2007-03-27 01:42:08.000000000 +0900
+++ R-2.5.0/src/main/serialize.c 2007-06-25 00:48:58.000000000 +0900
@@ -1866,7 +1866,7 @@
static void resize_buffer(membuf_t mb, int needed)
{
- int newsize = 2 * needed;
+ size_t newsize = 2 * needed;
mb->buf = realloc(mb->buf, newsize);
if (mb->buf == NULL)
error(_("cannot allocate buffer"));
Yes, thanks, but the structure also needs to be changed as the next line
is
mb->size = newsize;
and so this would set mb->size to a negative value.
Could you please tell us where you encountered this?
As far as I can see, this code is only used via R_serialize for
serializing to a raw vector, in which case 'size' can safely be 'int' and
the re-allocation should be up to 2^31-1 bytes at most (and allocating
twice what you are asked for seems undesirable). But there is potential
overflow at
if (mb->count + length > mb->size)
and possibly elsewhere.
The time-out of read and write was not set.
51:sendData.SOCKnode <- function(node, data) {
52: timeout <- getClusterOption("timeout")
53: old <- options(timeout = timeout);
54: on.exit(options(old))
55: serialize(data, node$con)
56: }
57:
58:recvData.SOCKnode <- function(node) {
59: timeout <- getClusterOption("timeout")
60: old <- options(timeout = timeout);
61: on.exit(options(old))
62: unserialize(node$con)
63: }
I don't think sock_read/sock_write is the right place to make that
setting. Ideally it would be set when the option is set, but as this is
in a module that needs an extension to the interface.
Looking at the code, we read from a socket in blocks of 4096, but we write
in a single block. The latter is likely to be the problem, and I think
some redesigning is necessary here.
Perhaps you and Luke Tierney can comment on exactly what the problem is
and how best to work around it.
--
Brian D. Ripley, ripley at stats.ox.ac.uk
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UK Fax: +44 1865 272595
EI-JI Nakama <nakama at ki.rim.or.jp>
"\u4e2d\u9593\u6804\u6cbb" <nakama at ki.rim.or.jp>