malloc/calloc/strdup and R's aequivalents

| Hello,
| 
| when looking at "Writing R Extensions"
| with mem-allocation in mind, I wondered,
| which functions to use to substitute
| malloc(), calloc(), realloc() and strdup() and free().
| 
| It looked like Calloc() or R_Calloc() might be useful for
| some of my tasks, but when trying to use R_Calloc() for example,
| I got some error messages which I don't see where they are coming from.
| 
| Maybe I just have forgotten to includ ethe right header file?

Maybe, but we can't tell as you didn't post a reproducible example. Here is
one, and I turned verbose on to give you the (default) headers:

R> library(inline)
R> 
R> f <- cfunction(signature(), verbose=TRUE, body='
+    double *p = Calloc(5, double);
+    Free(p);
+    return R_NilValue;
+ ')
Compilation argument:
 /usr/lib/R/bin/R CMD SHLIB file25df49ab1ccf.cpp 2> file25df49ab1ccf.cpp.err.txt 
ccache g++-4.6 -I/usr/share/R/include      -fpic  -g0 -O3 -Wall -pipe -Wno-unused -pedantic -c file25df49ab1ccf.cpp -o file25df49ab1ccf.o
g++ -shared -o file25df49ab1ccf.so file25df49ab1ccf.o -L/usr/lib/R/lib -lR
Program source:
  1: #include <R.h>
  2: #include <Rdefines.h>
  3: #include <R_ext/Error.h>
  4: 
  5: 
  6: extern "C" {
  7:   SEXP file25df49ab1ccf (  );
  8: }
  9: 
 10: SEXP file25df49ab1ccf (  ) {
 11: 
 12:    double *p = Calloc(5, double);
 13:    Free(p);
 14:    return R_NilValue;
 15: 
 16:   warning("your C program does not return anything!");
 17:   return R_NilValue;
 18: }
R> 
R> str(f())
 NULL
R> 


Dirk

"Outside of a dog, a book is a man's best friend. Inside of a dog, it is too
dark to read." -- Groucho Marx

Hello,

when looking at "Writing R Extensions"
with mem-allocation in mind, I wondered,
which functions to use to substitute
malloc(), calloc(), realloc() and strdup() and free().

It looked like Calloc() or R_Calloc() might be useful for
some of my tasks, but when trying to use R_Calloc() for example,
I got some error messages which I don't see where they are coming from.

Maybe I just have forgotten to includ ethe right header file?

Very likely.

#include <R.h>

... you'll need that in any case, obviously.

So, how to use Calloc() / R_Calloc()?
What is the prototype of this function/macro?

See R-ext 6.1.2 (or the headers - whichever you prefer).

And: what if I just want to easily substitute strdup()/free()?

You can always define Strdup() since strdup() is just a shorthand for malloc()+strcpy() -- in fact in R it's easier since Calloc will never return NULL so trivially
#define Strdup(X) strcpy(Calloc(strlen(X)+1, char), X)

But when using Calloc be aware that you have to clean up -- even if there is an error (which is why it is generally not a good idea to use Calloc for anything but memory you want to keep beyond your call).

There are two reasons for removing malloc/calloc: a) R uses custom allocators on systems with poor OS allocators (like Windows) so R's allocation is more efficient [and combining different allocators even worsens the performance] and b) you should think twice about the life span of your objects under error conditions. It is quite challenging to keep that straight, so in most cases it is better to use managed memory instead (but there is a performance trade-off). There are still valid reasons for not using R's allocators, but those are special cases that the author must be conscious about (after weighing the options).

mkChar seems not to be the right thing, because, what I found in the
above mentioned documentation, says,m that a C-string is fed in, but
CHARSXP is what I get as result.

What I try to do, is, to pick some code and port it to R.
If I need to rewrite a lot of stuff, just because I need
to adapt to R, this would be more effort than if it would be possible
just to have some functions, that could make porting easier.

For example, if I want to resuse a function, that opens a file
and needs a   char* stringname
then it would add more effort if I need to go via CHARSXP or so.

I'm not sure where you are going with this, since all strings will already come as CHARSXPs from R so typically you don't need to allocate anything. The only reason to allocate is to create persistent objects (for which you'd use Calloc) - your example doesn't fit here...

I would need to rewrite many pieces of the code.

(Should I implement my own kind of R_str_alloc() ?)


My main problem at the moemnt is, that I can use R_alloc()
but when I try to use Calloc() or R_Calloc() I can't compile.

So, what header-files are necessary, and what are the prototypes
of these functions?

See above.

If nothing of that stuff works, I would need to use the original
calloc() / free() functions, which are deprecated in the above
mentioned manual.


Ciao,
  Oliver

P.S.: Also I wonder... R-alloc() does not need a free()-like call?
     Is it freed automaticlaly?
     And at which time?
     In the docs is mentioned, at the end of the .C/.Call/.External call.
     Does this mean, this is freed/reclaimed automatically, when returning
     from the C-world to the R-world?
     At (after) return of the extension's code?

Yes. It simply uses an R object so it is subject to regular R object rules and thus gets cleaned up on error as well (unlike anything you use Calloc for).

Cheers,
Simon

On 18 March 2012 at 03:30, oliver wrote:
| Hello,
| 
| when looking at "Writing R Extensions"
| with mem-allocation in mind, I wondered,
| which functions to use to substitute
| malloc(), calloc(), realloc() and strdup() and free().
| 
| It looked like Calloc() or R_Calloc() might be useful for
| some of my tasks, but when trying to use R_Calloc() for example,
| I got some error messages which I don't see where they are coming from.
| 
| Maybe I just have forgotten to includ ethe right header file?

Maybe, but we can't tell as you didn't post a reproducible example. Here is
one, and I turned verbose on to give you the (default) headers:

[...]

It was not a missing header-file.

R> library(inline)
R> 
R> f <- cfunction(signature(), verbose=TRUE, body='
+    double *p = Calloc(5, double);

[...]

That line cleared the issue.
Thank you for providing an example.

When reading in the documentation I was not sure,
how to interpret "type" in setion 6.1.2.

It was meant as the name of the type, and that's, why my
sizeof() stuff created the problem.

I tried around and then saw, that the name of the type
is accepted, not only with typical base-types of C, but
it also eats the names of my structs/typedefed structs.

OK, so this is the solution... Calloc()'s snd arg is
a name of a type. So I assume it will internally use sizeof.

As this is rather untypical to the C-ish thinking,
and also not aequivalent to the calloc() which is
substituted by calloc(),
I think, it would be good, if this could be explained
clearer / more precise in the Writing Extensions document.

For example the R_alloc() prototype is like the one from calloc(). (*)
But Calloc() is different here.
My first impression was, that it's rather an accident / typo,
what was described in the docs, because it was a bit unusual.


The usage with names of types is nice; but explaining,
that this is not a typo, would be good too.


Ciao,
   Oliver

P.S.: (*) Not quite right: calloc() returns void*,
          but R_alloc() is mentioned to return char*.
          Here I'm also not sure if this is a typo,
          and one reason, why I thought the "type"
          in the following section might also be one.

| > R> library(inline)
| > R> 
| > R> f <- cfunction(signature(), verbose=TRUE, body='
| > +    double *p = Calloc(5, double);
| [...]
| 
| That line cleared the issue.
| Thank you for providing an example.
| 
| When reading in the documentation I was not sure,
| how to interpret "type" in setion 6.1.2.
| 
| It was meant as the name of the type, and that's, why my
| sizeof() stuff created the problem.
| 
| I tried around and then saw, that the name of the type
| is accepted, not only with typical base-types of C, but
| it also eats the names of my structs/typedefed structs.

Understandable, the manual is not toi clear there.  The easiest answer, in
such cases, is IMHO provided by other source packages.

Too bad the Google Code Search is now defunct.  But besides a recursive grep
in R's src/ directory, a simple Google search for "site:r-project.org Calloc"
also brings the solution up rightaway.

Dirk

"Outside of a dog, a book is a man's best friend. Inside of a dog, it is too
dark to read." -- Groucho Marx

[...]

You can always define Strdup() since strdup() is just a shorthand for
malloc()+strcpy() -- in fact in R it's easier since Calloc will never return
NULL so trivially
#define Strdup(X) strcpy(Calloc(strlen(X)+1, char), X)

[...]

Yes, something like that I had in mind (and already implemented,
but nur used so far).
I thought maybe something might be also somehoew available as a
R-internal adaption.

But when using Calloc be aware that you have to clean up -- even if there is
an error (which is why it is generally not a good idea to use Calloc for
anything but memory you want to keep beyond your call).

Yes.

This seems not to be the case for R_alloc() ?

There are two reasons for removing malloc/calloc: a) R uses custom allocators
on systems with poor OS allocators (like Windows) so R's allocation is more
efficient [and combining different allocators even worsens the performance] and
b) you should think twice about the life span of your objects under error
conditions. It is quite challenging to keep that straight, so in most cases it
is better to use managed memory instead (but there is a performance trade-off).

Yes, good reasons.

There are still valid reasons for not using R's allocators, but those are
special cases that the author must be conscious about (after weighing the
options).

Calloc() is provided by R, so it apears as being an R-allocator,
but rather seems to be a wrapper around calloc().
With R allocators you mean functions like allocVector()?
And R_alloc()?
The behaviour of the later one is not clear to me.
(A reason to prefer Calloc() here, then the behaviour
 is under my control, even of it's more "risky". That's the C-ish-way.)

mkChar seems not to be the right thing, because, what I found in the
above mentioned documentation, says,m that a C-string is fed in, but
CHARSXP is what I get as result.

What I try to do, is, to pick some code and port it to R.
If I need to rewrite a lot of stuff, just because I need
to adapt to R, this would be more effort than if it would be possible
just to have some functions, that could make porting easier.

For example, if I want to resuse a function, that opens a file
and needs a   char* stringname
then it would add more effort if I need to go via CHARSXP or so.

I'm not sure where you are going with this, since all strings will already
come as CHARSXPs from R so typically you don't need to allocate anything. The
only reason to allocate is to create persistent objects (for which you'd use
Calloc) - your example doesn't fit here...

If I want to copy the filename into a C-struct and then pass (a pointer to)
that struct around, then using something like strdup() is a good idea.
Therwise the filename-pointer might have vanished and the pointer
points into nirvana.
That was, where it came from. I used some already existing code,
where this all made sense.

And extracting char* from the CARSXP once, instead of
using something like CHAR( STRING_ELT(filename_sexp, 0) )
at every place, where I may want to have access to that filename
makes the code clearer.

As this I reused existing code, I tried to change it
as less as possible (to make it easy).

But as I now see, for the special case here I can forget the
strdup() and maybe make my struct smaller (throwing out the filename),
because here I call the reader-function with filename as argument,
and do not pass around the structure-ptr too much.
And I store the filename in the list I give back to the user.
(I can do this even in the R-code which calls my C-code.)

So in this certain case the problem vanishes.
Nevertheless, in other cases it might be a problem.

But then maybe never touching the code and just live with the
already existing strdup()/malloc/calloc()/free()
would also work.

I can see the advantages of using R-provided functions.
But if they do not substitute all these deprecated functions,
then either porting the stand-alone-code to R is made
much effort, or the deprecated functions might just stay inside.

Do you see what I mean?

If nothing of that stuff works, I would need to use the original
calloc() / free() functions, which are deprecated in the above
mentioned manual.


Ciao,
  Oliver

P.S.: Also I wonder... R-alloc() does not need a free()-like call?
     Is it freed automaticlaly?
     And at which time?
     In the docs is mentioned, at the end of the .C/.Call/.External call.
     Does this mean, this is freed/reclaimed automatically, when returning
     from the C-world to the R-world?
     At (after) return of the extension's code?

Yes. It simply uses an R object so it is subject to regular R object rules
and thus gets cleaned up on error as well (unlike anything you use Calloc for).

This point is not so clear to me.

The prototype of R_alloc() looks like as if R_alloc() is just the same as calloc().
But now you say, it's an R-object that will be handled like any R-object.

But then... why is PROTECT not mentioned there in section 6.1.1?
Would a GC-run possibly vanish that memory?

What rules are there for it?

Is it save to use it from entry-point to my code to the return-point of my code?

So, does it behave like an environment in functional languages?

Can I allocate memory with R_alloc() somewehere  deep in my code
and pass that memory until the return of my .Call-entry-return-point?
Without GC claiming it until returning?


Can you please clarify theese points?


Ciao,
   Oliver