Skip to content

[R-pkg-devel] [FWD] [CRAN-pretest-archived] CRAN submission JuliaConnectoR 0.5.0

18 messages · Dirk Eddelbuettel, Ben Bolker, Stefan Lenz IMBI +3 more

#
Dear all,

I just submitted the new package "JuliaConnectoR" to CRAN.
(See https://github.com/stefan-m-lenz/JuliaConnectoR )
The submission does not pass the automatic checks because of two notes (see below).
One note is simply that I am a new submitter.
The other note is because of the the use of "attach".
But I am certain that I am using "attach" in the right way. In its documentation, under "Good practice" , it says,
"In programming, functions should not change the search path unless that is their purpose."
Here this is exactly the purpose because one of the main functionalities of the package is to add Julia functions to the search path.
Can anybody help me with this?
Best wishes
Stefan

----------------urspr?ngliche Nachricht-----------------
Von: Stefan Lenz [Stefan-M-Lenz at web.de ]
An: lenz at imbi.uni-freiburg.de 
Datum: Mon, 6 Apr 2020 14:04:21 +0200
----------------------------------------------------------
__________________________________________________
Stefan Lenz

Institut f?r Medizinische Biometrie und Statistik
Medizinische Fakult?t / Universit?tsklinikum Freiburg

Postadresse: Stefan-Meier-Str. 26, 79104 Freiburg

Tel.: 0761/203-6670
E-Mail: lenz at imbi.uni-freiburg.de
#
Just reply to the CRAN maintainers and explain this situation.  It?s
slightly buried, but the e-mail you received does say:
cheers
    Ben Bolker

On Mon, Apr 6, 2020 at 8:28 AM Stefan Lenz IMBI
<lenz at imbi.uni-freiburg.de> wrote:
#
On 6 April 2020 at 08:38, Ben Bolker wrote:
|   Just reply to the CRAN maintainers and explain this situation.  It?s
| slightly buried, but the e-mail you received does say:
| 
| > If you are fairly certain the rejection is a false positive, please reply-all to this
| > message and explain.

True, but this misses the "Letter of the law" versus the "Spirit of the law".

It might be worth mentioning that use of attach() is seen, to find one poor
analogy, pretty much like use of global variables these days. "Just because
you could does not mean you should".

See e.g. one of the first google hits for 'r do not use attach' here:
https://stackoverflow.com/questions/10067680/why-is-it-not-advisable-to-use-attach-in-r-and-what-should-i-use-instead

Dirk
#
Fair enough.  I'm going to derail/repurpose this thread to ask a
couple of questions about attach().

   I have often used with() for situations where I want to evaluate a
bunch of expressions with the elements of a parameter vector, e.g.
inside a gradient function for deSolve::ode().  However, I have found
this very hard to debug because (AFAIK) you can't debug-step through the
components of a with() expression.
  Potential solutions for this include

 1. attach(params); on.exit(detach(params))

  * this will be flagged by CRAN
  * I have found some really surprising (to me) precedence issues with
this _when used in a package context_ - it looks like the elements in
'params' are found _after_ built-in objects in R?? (I have to take some
time to make a MRE of this

  2. The zeallot package does 'unpacking' as with Python tuples.  I was
worried about dragging in tidyverse dependencies, but it looks like it
doesn't actually Import: anything.
   This doesn't quite do what I want, as I want to unpack using the
names in the object (which makes it look perfect for the attach() solution)


  Thoughts?
On 2020-04-06 10:49 a.m., Dirk Eddelbuettel wrote:
#
On 06/04/2020 10:49 a.m., Dirk Eddelbuettel wrote:
I don't have a strong opinion on this:  the proposed use seems to be no 
worse than library() or require().  Those are fine for users to use, but 
are discouraged in a package.  If the attach() never happens without an 
explicit request from the user (and that's what it sounds like), I'd say 
it's probably okay.

However, there is an easy workaround:  just return the environment 
without attaching it.  Then the user has the choice of attaching it, or 
using it as a prefix when they call the functions in it.  So it's not as 
though this will destroy the utility of the package if Stefan isn't 
allowed to call attach().

Duncan Murdoch
#
My case is a little bit different from the use of global variables for data described in the Stackoverflow post.
I import a data base which contains function names of Julia functions from a Julia package.
This is analogous to loading a package in R.
The goal is to make it possible to call Julia functions like R functions.
If you call library(...) in R then the functions contained in the package are also available in the global search path.
In this case it doesn't make sense to use "with".
I am open to suggestions how I can change it to make it better, but the goal is really to be able to write

    juliaUsing("SomeJuliaPackage") # exports myJuliaFunction
    myJuliaFunction()

same as you would write

    library("MyRPackage") # exports myRFunction
    myRFunction()

Best wishes

Stefan
 
----------------urspr?ngliche Nachricht-----------------
Von: Dirk Eddelbuettel [edd at debian.org]
An: Ben Bolker [bbolker at gmail.com]
Kopie: Stefan Lenz IMBI [lenz at imbi.uni-freiburg.de], List r-package-devel [r-package-devel at r-project.org]
Datum: Mon, 6 Apr 2020 09:49:49 -0500
-------------------------------------------------

  
    
#
Yes, as I wrote earlier, I would like to imitate the behaviour of loading an R package

 juliaUsing("SomeJuliaPackage") # exports myJuliaFunction
 myJuliaFunction()

like R:

 library("MyRPackage") # exports myRFunction
 myRFunction()

I could return an environment, such that the call becomes

 attach(juliaUsing("SomeJuliaPackage"))
 myJuliaFunction()

But calling juliaUsing(), as it is now, takes care that if a package is imported a second time,
the first data base is removed via detach().
This way, users do not have to worry about calling juliaUsing() mutliple times in a script, same
as R users don't have to worry about calling library() multiple times.
If you call the code with attach() multiple times and do not detach, you get your screen cluttered with
warnings "xxx is masked by xxx".
So I would say it would decrease user-friendliness to return an environment.
I also want to make explicit, that the call to attach
occurs only once in my code, after creating the environment:

 envName <- paste0("JuliaConnectoR:", absoluteModulePath)
 if (envName %in% search()) {
 detach(envName, character.only = TRUE)
 }
 attach(funenv, name = envName)

This code is only called by juliaImport() and juliaUsing(), which aren't called by any other function of 
the package
and are supposed to be called directly by the user.

Stefan 
 
----------------urspr?ngliche Nachricht-----------------
Von: Duncan Murdoch [murdoch.duncan at gmail.com]
An: Dirk Eddelbuettel [edd at debian.org], Ben Bolker [bbolker at gmail.com]
Kopie: List r-package-devel [r-package-devel at r-project.org]
Datum: Mon, 6 Apr 2020 11:00:09 -0400
-------------------------------------------------

  
    
#
On 06/04/2020 10:59 a.m., Ben Bolker wrote:
I suspect what happened is that you had a copy of a built-in function in 
the global environment.  The global environment always comes ahead of 
attach()'d environments.  On the other hand, with() gets it right.

You can see this:

x <- 123
df <- data.frame(x = 456)
with(df, x)
#> [1] 456
attach(df)
#> The following object is masked _by_ .GlobalEnv:
#>
#>     x
x
#> [1] 123
detach(df)

As for debugging, the setBreakpoint() function can set a breakpoint in 
the code block in with().  RStudio uses that or some equivalent for 
setting breakpoints, so it can set breakpoints within the block.

If you call debug() on a function and get to a with() statement, you can 
eventually get to the code within it by hitting "s" (step in) several 
times.  It's fairly scary stepping into .Internal(), but it does 
eventually get you there.

Duncan Murdoch
#
On 06/04/2020 11:25 a.m., Stefan Lenz IMBI wrote:
I wouldn't use it that way.  I'd write it as

     sjp <- juliaUsing("SomeJuliaPackage")
     sjp$myJuliaFunction()

This is similar to the advice to use pkg::foo() rather than library(pkg) 
followed by plain foo() in the code.

Duncan Murdoch
#
One could take the position that the library and require functions were a mistake to begin with and that all contributed packages should be accessed using ::...  or one could recognize that these functions are an expected feature of R at this point and then it is not defensible to ban the proposed approach of importing names as Stefan wants to. I don't think it is fair to require this higher level of specificity just because it involves use of attach.

That said, another feature of R packages is the integrated help system... importing Julia functions wholesale may lead to problems with consistency in navigating the help files. IMO it may be justifiable to ban this particular syntactic convenience to maintain some separation in the minds of users looking for help on these new functions, since the syntactic and semantic structure of functions from another language may not align well with normal R functions. But I am not familiar with Julia or Stefan's package or the support it brings in this area... I just disagree with banning a "library" lookalike function "just because" it happens to involve attach.
On April 6, 2020 8:40:12 AM PDT, Duncan Murdoch <murdoch.duncan at gmail.com> wrote:

  
    
#
Thank you very much for your comment. 
Could you elaborate how you think that it could interfere with the help system?
I haven't yet connected the Julia help with the R help, as the R help system is quite complex and RStudio handles it again differently. So it's simply like the functions were declared on the command line. They have no help.
A simply way to print the help for a Julia function, e.g. the function "first", is to call
 juliaEval("@doc first")
This then simply prints the output on the command line.

 
----------------urspr?ngliche Nachricht-----------------
Von: Jeff Newmiller [jdnewmil at dcn.davis.ca.us]
An: r-package-devel at r-project.org, Duncan Murdoch [murdoch.duncan at gmail.com], Stefan Lenz IMBI [lenz at imbi.uni-freiburg.de]
Datum: Mon, 06 Apr 2020 10:51:53 -0700
-------------------------------------------------

  
    
#
I did not say "interfere"... I said "problems with consistency". I don't think your wholesale import of functions without corresponding help pages is consistent with the normal use of R, which will make reading R code written with this mechanism in place a painful source of trouble for help forums.

On the other hand, if the code is prefaced with an environment name then there will be no expectation that a help page should exist on the part of someone reading that code for the first time. (It will be no less obscure, but at least it won't be misleading.)
On April 7, 2020 9:40:02 AM PDT, Stefan Lenz IMBI <lenz at imbi.uni-freiburg.de> wrote:

  
    
1 day later
#
It is important for me to get my package into CRAN.
I haven't gotten any answer from the CRAN maintainers. I already tried to submit a version of the package before some months and haven't gotten any reply then.
The replies you gave me (thanks to all engaged in the discussion) were mixed.
I still think that the library-like import is a good feature, but if it is not possible to get this package on CRAN with it, I will change it with returning an environment like reticulate.
My question now is:
So what would be the process how to proceed?
Is there anybody who can make a decision in such cases?
Or do I make this decision myself after I lost hope that it will be accepted the way it is now?

Stefan

 
----------------urspr?ngliche Nachricht-----------------
Von: Jeff Newmiller [jdnewmil at dcn.davis.ca.us]
An: Stefan Lenz IMBI [lenz at imbi.uni-freiburg.de], r-package-devel at r-project.org, Duncan Murdoch [murdoch.duncan at gmail.com]
Datum: Tue, 07 Apr 2020 09:52:52 -0700
-------------------------------------------------

  
    
#
Hi Stefan,

first of all, thank you for taking the time and effort to link Julia and R. That said, I would strongly encourage you to go for a reticulate-like approach. There's a good reason for that. Once you start mixing languages, you really want to have some indication what code is ran in what language. Especially since I can imagine at one point there'll be a Julia function masking an R function of the same name, and that is going to cause a world of hurt and confusion. 

If you don't like environments, you can try to use namespace functionality. There's the "namespace" package that allows you to create one, and some internal namespace functions are visible (eg namespaceExport() from base). That's going to be the closest to mimicking R packages, but it's beyond my knowledge to know how feasible this approach actually is. I hope someone else chimes in, especially if this is dangerous for other reasons I overlooked.

Kind regards
Joris

Joris Meys
Statistician
T +32 9 264 61 79

Department Data Analysis and Mathematical Modelling

Campus Coupure, Block A 1st floor 110.050, Coupure links 653, B-9000 Ghent, Belgium
T administration office +32 9 264 59 32

www.ugent.be

e-maildisclaimer
#
Hi!
Thanks for pointing to the package "namespace". 
I experimented with it but I couldn't really figure out how it is supposed to work:

The following code leads to strange errors that I don't understand:

 > nse <- list2env(list(data="x", asdf = 2))
 > namespace::registerNamespace(name = "NewNamespace", env = nse)
 > NewNamespace::asdf
 Error in asNamespace(ns) : not a namespace

It would really be interesting if someone could comment on whether the namespace approach is feasible, how it is supposed to work, and whether/how it would be better than using attach.
Greetings!
Stefan

 
----------------urspr?ngliche Nachricht-----------------
Von: Joris Meys [Joris.Meys at ugent.be]
An: Stefan Lenz IMBI [lenz at imbi.uni-freiburg.de], Jeff Newmiller [jdnewmil at dcn.davis.ca.us], r-package-devel at r-project.org, Duncan Murdoch [murdoch.duncan at gmail.com]
Datum: Thu, 9 Apr 2020 12:59:23 +0000
-------------------------------------------------

  
    
#
Hi Stefan,

you need the function makeNamespace() to create the environment as a namespace. Look at the examples in the help files, I found them rather instructive. I also hope someone else chimes in to shed some light on whether or not that approach would be acceptable. 


Joris Meys
Statistician
T +32 9 264 61 79

Department Data Analysis and Mathematical Modelling

Campus Coupure, Block A 1st floor 110.050, Coupure links 653, B-9000 Ghent, Belgium
T administration office +32 9 264 59 32

www.ugent.be

e-maildisclaimer
#
When I execute the example, I also get a strange behaviour:

 library(namespace)
 ns <- makeNamespace("myNamespace")
 assign("test",7, env=ns)
 ls(env=ns) 

When I now want to invoke any help for a function, I get a warning message:

 > ?loadNamespace
 Warning message:
 In find.package(if (is.null(package)) loadedNamespaces() else package, :
 there is no package called ?myNamespace?

Also sessionInfo doesn't work any more:

 > sessionInfo()
 Error in if (pkgpath == "") { : argument is of length zero

I don't think this is a good sign....
Stefan

 
----------------urspr?ngliche Nachricht-----------------
Von: Joris Meys [Joris.Meys at ugent.be]
An: Stefan Lenz IMBI [lenz at imbi.uni-freiburg.de], r-package-devel at r-project.org
Datum: Thu, 9 Apr 2020 15:37:09 +0000
-------------------------------------------------

  
    
#
You're right, that's enough to not dig any further. I remembered it from the earlier days of devtools, but they removed that dependency ages ago. I also believe now the namespace package predates the later changes to the namespace mechanism. Forget what I said, I should have spent a bit more time testing it myself first.

Kind regards
Joris


Joris Meys
Statistician
T +32 9 264 61 79

Department Data Analysis and Mathematical Modelling

Campus Coupure, Block A 1st floor 110.050, Coupure links 653, B-9000 Ghent, Belgium
T administration office +32 9 264 59 32

www.ugent.be

e-maildisclaimer