Wish there were a "strict mode" for R interpreter. What about You?

Years ago, I did lots of Perl programming. Perl will let you be lazy
and write functions that refer to undefined variables (like R does),
but there is also a strict mode so the interpreter will block anything
when a variable is mentioned that has not been defined. I wish there
were a strict mode for checking R functions.

Here's why. We have a lot of students writing R functions around here
and they run into trouble because they use the same name for things
inside and outside of functions. When they call functions that have
mistaken or undefined references to names that they use elsewhere,
then variables that are in the environment are accidentally used. Know
what I mean?

dat<- whatever

someNewFunction<- function(z, w){
    #do something with z and w and create a new "dat"
    # but forget to name it "dat"
     lm (y, x, data=dat)
    # lm just used wrong data
}

I wish R had a strict mode to return an error in that case. Users
don't realize they are getting nonsense because R finds things to fill
in for their mistakes.

Is this possible?  Does anybody agree it would be good?

On 11-04-09 3:51 PM, Paul Johnson wrote:

Years ago, I did lots of Perl programming. Perl will let you be lazy
and write functions that refer to undefined variables (like R does),
but there is also a strict mode so the interpreter will block anything
when a variable is mentioned that has not been defined. I wish there
were a strict mode for checking R functions.

Here's why. We have a lot of students writing R functions around here
and they run into trouble because they use the same name for things
inside and outside of functions. When they call functions that have
mistaken or undefined references to names that they use elsewhere,
then variables that are in the environment are accidentally used. Know
what I mean?

dat<- whatever

someNewFunction<- function(z, w){
    #do something with z and w and create a new "dat"
    # but forget to name it "dat"
     lm (y, x, data=dat)
    # lm just used wrong data
}

I wish R had a strict mode to return an error in that case. Users
don't realize they are getting nonsense because R finds things to fill
in for their mistakes.

Is this possible?  Does anybody agree it would be good?

It would be really bad, unless done carefully.

In your function the free (undefined) variables are dat and lm.  You 
want to be warned about dat, but you don't want to be warned about lm. 
What rule should R use to determine that?

(One possible rule would work in a package with a namespace.  In that 
case, all variables must be found in declared dependencies, the search 
could stop before it got to globalenv().  But it seems unlikely that 
your students are writing packages with namespaces.)

Duncan Murdoch

Years ago, I did lots of Perl programming. Perl will let you be lazy
and write functions that refer to undefined variables (like R does),
but there is also a strict mode so the interpreter will block anything
when a variable is mentioned that has not been defined. I wish there
were a strict mode for checking R functions.

Here's why. We have a lot of students writing R functions around here
and they run into trouble because they use the same name for things
inside and outside of functions. When they call functions that have
mistaken or undefined references to names that they use elsewhere,
then variables that are in the environment are accidentally used. Know
what I mean?

dat <- whatever

someNewFunction <- function(z, w){
? #do something with z and w and create a new "dat"
? # but forget to name it "dat"
? ?lm (y, x, data=dat)
? # lm just used wrong data
}

I wish R had a strict mode to return an error in that case. Users
don't realize they are getting nonsense because R finds things to fill
in for their mistakes.

Is this possible? ?Does anybody agree it would be good?

library(codetools)
checkUsage(someNewFunction)

On Sat, Apr 9, 2011 at 2:51 PM, Paul Johnson<pauljohn32 at gmail.com>  wrote:

Years ago, I did lots of Perl programming. Perl will let you be lazy
and write functions that refer to undefined variables (like R does),
but there is also a strict mode so the interpreter will block anything
when a variable is mentioned that has not been defined. I wish there
were a strict mode for checking R functions.

Here's why. We have a lot of students writing R functions around here
and they run into trouble because they use the same name for things
inside and outside of functions. When they call functions that have
mistaken or undefined references to names that they use elsewhere,
then variables that are in the environment are accidentally used. Know
what I mean?

dat<- whatever

someNewFunction<- function(z, w){
   #do something with z and w and create a new "dat"
   # but forget to name it "dat"
    lm (y, x, data=dat)
   # lm just used wrong data
}

I wish R had a strict mode to return an error in that case. Users
don't realize they are getting nonsense because R finds things to fill
in for their mistakes.

Is this possible?  Does anybody agree it would be good?

library(codetools)
checkUsage(someNewFunction)

<anonymous>: no visible binding for global variable ?y?
<anonymous>: no visible binding for global variable ?x?
<anonymous>: no visible binding for global variable ?dat?

Which also picks up another bug in your function ;)

Hadley

On 4/9/2011 2:31 PM, Hadley Wickham wrote:

On Sat, Apr 9, 2011 at 2:51 PM, Paul Johnson<pauljohn32 at gmail.com>   wrote:

Years ago, I did lots of Perl programming. Perl will let you be lazy
and write functions that refer to undefined variables (like R does),
but there is also a strict mode so the interpreter will block anything
when a variable is mentioned that has not been defined. I wish there
were a strict mode for checking R functions.

Here's why. We have a lot of students writing R functions around here
and they run into trouble because they use the same name for things
inside and outside of functions. When they call functions that have
mistaken or undefined references to names that they use elsewhere,
then variables that are in the environment are accidentally used. Know
what I mean?

dat<- whatever

someNewFunction<- function(z, w){
    #do something with z and w and create a new "dat"
    # but forget to name it "dat"
     lm (y, x, data=dat)
    # lm just used wrong data
}

I wish R had a strict mode to return an error in that case. Users
don't realize they are getting nonsense because R finds things to fill
in for their mistakes.

Is this possible?  Does anybody agree it would be good?

library(codetools)
checkUsage(someNewFunction)

<anonymous>: no visible binding for global variable ?y?
<anonymous>: no visible binding for global variable ?x?
<anonymous>: no visible binding for global variable ?dat?

Which also picks up another bug in your function ;)

        Is this run by "R CMD check"?  I've seen this message.


        "R CMD check" will give this message sometimes when I don't feel
it's appropriate.  For example, I define a data object ETB in a package,
then give that as the default in a function call like
f(data.=ETB){if(missing(data.))data(ETB);  data.}.  When I run "R CMD
check", I get "no visible binding for global variable 'ETB'", even
though the function is tested and works during R CMD check.

On 11-04-09 7:02 PM, Spencer Graves wrote:

On 4/9/2011 2:31 PM, Hadley Wickham wrote:

On Sat, Apr 9, 2011 at 2:51 PM, Paul Johnson<pauljohn32 at gmail.com>   
wrote:

Years ago, I did lots of Perl programming. Perl will let you be lazy
and write functions that refer to undefined variables (like R does),
but there is also a strict mode so the interpreter will block anything
when a variable is mentioned that has not been defined. I wish there
were a strict mode for checking R functions.

Here's why. We have a lot of students writing R functions around here
and they run into trouble because they use the same name for things
inside and outside of functions. When they call functions that have
mistaken or undefined references to names that they use elsewhere,
then variables that are in the environment are accidentally used. Know
what I mean?

dat<- whatever

someNewFunction<- function(z, w){
    #do something with z and w and create a new "dat"
    # but forget to name it "dat"
     lm (y, x, data=dat)
    # lm just used wrong data
}

I wish R had a strict mode to return an error in that case. Users
don't realize they are getting nonsense because R finds things to fill
in for their mistakes.

Is this possible?  Does anybody agree it would be good?

library(codetools)
checkUsage(someNewFunction)

<anonymous>: no visible binding for global variable ?y?
<anonymous>: no visible binding for global variable ?x?
<anonymous>: no visible binding for global variable ?dat?

Which also picks up another bug in your function ;)

        Is this run by "R CMD check"?  I've seen this message.


        "R CMD check" will give this message sometimes when I don't feel
it's appropriate.  For example, I define a data object ETB in a package,
then give that as the default in a function call like
f(data.=ETB){if(missing(data.))data(ETB);  data.}.  When I run "R CMD
check", I get "no visible binding for global variable 'ETB'", even
though the function is tested and works during R CMD check.

What is ETB?  Your code is looking for a global variable by that name, 
and that's what codetools is telling you.

Duncan Murdoch

On 11-04-09 7:02 PM, Spencer Graves wrote:

On 4/9/2011 2:31 PM, Hadley Wickham wrote:

On Sat, Apr 9, 2011 at 2:51 PM, Paul Johnson<pauljohn32 at gmail.com>
wrote:

Years ago, I did lots of Perl programming. Perl will let you be lazy
and write functions that refer to undefined variables (like R does),
but there is also a strict mode so the interpreter will block anything
when a variable is mentioned that has not been defined. I wish there
were a strict mode for checking R functions.

Here's why. We have a lot of students writing R functions around here
and they run into trouble because they use the same name for things
inside and outside of functions. When they call functions that have
mistaken or undefined references to names that they use elsewhere,
then variables that are in the environment are accidentally used. Know
what I mean?

dat<- whatever

someNewFunction<- function(z, w){
    #do something with z and w and create a new "dat"
    # but forget to name it "dat"
     lm (y, x, data=dat)
    # lm just used wrong data
}

I wish R had a strict mode to return an error in that case. Users
don't realize they are getting nonsense because R finds things to fill
in for their mistakes.

Is this possible?  Does anybody agree it would be good?

library(codetools)
checkUsage(someNewFunction)

<anonymous>: no visible binding for global variable ?y?
<anonymous>: no visible binding for global variable ?x?
<anonymous>: no visible binding for global variable ?dat?

Which also picks up another bug in your function ;)

        Is this run by "R CMD check"?  I've seen this message.


        "R CMD check" will give this message sometimes when I don't feel
it's appropriate.  For example, I define a data object ETB in a package,
then give that as the default in a function call like
f(data.=ETB){if(missing(data.))data(ETB);  data.}.  When I run "R CMD
check", I get "no visible binding for global variable 'ETB'", even
though the function is tested and works during R CMD check.

What is ETB?  Your code is looking for a global variable by that name,
and that's what codetools is telling you.

Duncan Murdoch

On 4/9/2011 6:12 PM, Duncan Murdoch wrote:

On 11-04-09 7:02 PM, Spencer Graves wrote:

On 4/9/2011 2:31 PM, Hadley Wickham wrote:

On Sat, Apr 9, 2011 at 2:51 PM, Paul Johnson<pauljohn32 at gmail.com>
wrote:

Years ago, I did lots of Perl programming. Perl will let you be lazy
and write functions that refer to undefined variables (like R does),
but there is also a strict mode so the interpreter will block anything
when a variable is mentioned that has not been defined. I wish there
were a strict mode for checking R functions.

Here's why. We have a lot of students writing R functions around here
and they run into trouble because they use the same name for things
inside and outside of functions. When they call functions that have
mistaken or undefined references to names that they use elsewhere,
then variables that are in the environment are accidentally used. Know
what I mean?

dat<- whatever

someNewFunction<- function(z, w){
     #do something with z and w and create a new "dat"
     # but forget to name it "dat"
      lm (y, x, data=dat)
     # lm just used wrong data
}

I wish R had a strict mode to return an error in that case. Users
don't realize they are getting nonsense because R finds things to fill
in for their mistakes.

Is this possible?  Does anybody agree it would be good?

library(codetools)
checkUsage(someNewFunction)

<anonymous>: no visible binding for global variable ?y?
<anonymous>: no visible binding for global variable ?x?
<anonymous>: no visible binding for global variable ?dat?

Which also picks up another bug in your function ;)

         Is this run by "R CMD check"?  I've seen this message.


         "R CMD check" will give this message sometimes when I don't feel
it's appropriate.  For example, I define a data object ETB in a package,
then give that as the default in a function call like
f(data.=ETB){if(missing(data.))data(ETB);  data.}.  When I run "R CMD
check", I get "no visible binding for global variable 'ETB'", even
though the function is tested and works during R CMD check.

What is ETB?  Your code is looking for a global variable by that name,
and that's what codetools is telling you.

Duncan:  Thanks for the question.


ETB is a data object in my package.  codetools can't find it because
data(ETB) is needed before ETB becomes available.  codetools is not
smart enough to check to see if ETB is a data object in the package.

On 11-04-09 9:22 PM, Spencer Graves wrote:

On 4/9/2011 6:12 PM, Duncan Murdoch wrote:

On 11-04-09 7:02 PM, Spencer Graves wrote:

On 4/9/2011 2:31 PM, Hadley Wickham wrote:

On Sat, Apr 9, 2011 at 2:51 PM, Paul Johnson<pauljohn32 at gmail.com>
wrote:

Years ago, I did lots of Perl programming. Perl will let you be lazy
and write functions that refer to undefined variables (like R does),
but there is also a strict mode so the interpreter will block 
anything
when a variable is mentioned that has not been defined. I wish there
were a strict mode for checking R functions.

Here's why. We have a lot of students writing R functions around 
here
and they run into trouble because they use the same name for things
inside and outside of functions. When they call functions that have
mistaken or undefined references to names that they use elsewhere,
then variables that are in the environment are accidentally used. 
Know
what I mean?

dat<- whatever

someNewFunction<- function(z, w){
     #do something with z and w and create a new "dat"
     # but forget to name it "dat"
      lm (y, x, data=dat)
     # lm just used wrong data
}

I wish R had a strict mode to return an error in that case. Users
don't realize they are getting nonsense because R finds things to 
fill
in for their mistakes.

Is this possible?  Does anybody agree it would be good?

library(codetools)
checkUsage(someNewFunction)

<anonymous>: no visible binding for global variable ?y?
<anonymous>: no visible binding for global variable ?x?
<anonymous>: no visible binding for global variable ?dat?

Which also picks up another bug in your function ;)

         Is this run by "R CMD check"?  I've seen this message.


         "R CMD check" will give this message sometimes when I 
don't feel
it's appropriate.  For example, I define a data object ETB in a 
package,
then give that as the default in a function call like
f(data.=ETB){if(missing(data.))data(ETB);  data.}.  When I run "R CMD
check", I get "no visible binding for global variable 'ETB'", even
though the function is tested and works during R CMD check.

What is ETB?  Your code is looking for a global variable by that name,
and that's what codetools is telling you.

Duncan:  Thanks for the question.


ETB is a data object in my package.  codetools can't find it because
data(ETB) is needed before ETB becomes available.  codetools is not
smart enough to check to see if ETB is a data object in the package.

Okay, I understand what you are trying to do.  Yes, you have fooled 
codetools in this instance.

Duncan Murdoch

On 11-04-09 9:22 PM, Spencer Graves wrote:

On 4/9/2011 6:12 PM, Duncan Murdoch wrote:

On 11-04-09 7:02 PM, Spencer Graves wrote:

        "R CMD check" will give this message sometimes when I don't feel
it's appropriate.  For example, I define a data object ETB in a package,
then give that as the default in a function call like
f(data.=ETB){if(missing(data.))data(ETB);  data.}.  When I run "R CMD
check", I get "no visible binding for global variable 'ETB'", even
though the function is tested and works during R CMD check.

What is ETB?  Your code is looking for a global variable by that name,
and that's what codetools is telling you.

Duncan:  Thanks for the question.


ETB is a data object in my package.  codetools can't find it because
data(ETB) is needed before ETB becomes available.  codetools is not
smart enough to check to see if ETB is a data object in the package.

Okay, I understand what you are trying to do.  Yes, you have fooled codetools in this instance.

Okay, I understand what you are trying to do. ?Yes, you have fooled codetools in this instance.

...but notice that the codetools warning is just that: It _is_ acknowledged that these things occasionally happen by design. There are a couple of cases in base R too:

* checking R code for possible problems ... NOTE
glm.fit: no visible binding for global variable ?n?

quantile.ecdf: no visible binding for global variable ?y?

Okay, I understand what you are trying to do. ?Yes, you have fooled codetools in this instance.

...but notice that the codetools warning is just that: It _is_ acknowledged that these things occasionally happen by design. There are a couple of cases in base R too:

* checking R code for possible problems ... NOTE
glm.fit: no visible binding for global variable ?n?

Are you sure that's not a bug?  There's:

aic.model <- aic(y, n, mu, weights, dev) + 2 * rank

and n.ok is defined, but n isn't defined anywhere.

quantile.ecdf: no visible binding for global variable ?y?

I wonder why it warns on y, but not nobs.

codetools::checkUsage(stats:::quantile.ecdf)

Hadley

On Sun, 10 Apr 2011, Hadley Wickham wrote:

Okay, I understand what you are trying to do.  Yes, you have fooled codetools in this instance.

...but notice that the codetools warning is just that: It _is_ acknowledged that these things occasionally happen by design. There are a couple of cases in base R too:

* checking R code for possible problems ... NOTE
glm.fit: no visible binding for global variable ?n?

Are you sure that's not a bug?  There's:

aic.model <- aic(y, n, mu, weights, dev) + 2 * rank

and n.ok is defined, but n isn't defined anywhere.

It is (or should be) defined by the call to

       eval(family$initialize)

quantile.ecdf: no visible binding for global variable ?y?

I wonder why it warns on y, but not nobs.

It does when run on stats:::quantile.ecdf directly:

codetools::checkUsage(stats:::quantile.ecdf)

<anonymous>: no visible binding for global variable ?nobs?
<anonymous>: no visible binding for global variable ?y?

Maybe in the context where you saw this nobs is defined in an
enclosing environment.

luke

Hadley

I'm with Duncan on this one! On the other hand, I can understand the
issues that Paul's students might encounter.

I think the right thing to so is to introduce the students to the
basics of scoping, early in the process of learning R.

I think the right thing to so is to introduce the students to the
basics of scoping, early in the process of learning R.

On 2011-04-09, at 2:08 PM, Ted Harding wrote:

I think the right thing to so is to introduce the students to the
basics of scoping, early in the process of learning R.

I know the basics of scoping perfectly well, but that doesn't stop me from occasionally misspelling a variable name that only causes an error much later.

OTOH, I think with Perl you can start declaring your variables "local" and keep the interpreter happy. But in R's context you then have to also start declaring what you expect to inherit from parent environments, and pretty soon the code is so encrusted with annotation barnacles that it loses the simplicity that  makes R so nice in the interactive mode. 

What would be really nice is if we had a smart R editor/IDE that would "DWIM" and put a red underline under a misspelled name, but leave it alone when, as Duncan said, it's in the environment. 

On Apr 11, 2011, at 11:28 AM, Davor Cubranic wrote:

On 2011-04-09, at 2:08 PM, Ted Harding wrote:

I think the right thing to so is to introduce the students to the
basics of scoping, early in the process of learning R.

I know the basics of scoping perfectly well, but that doesn't stop me from occasionally misspelling a variable name that only causes an error much later.

OTOH, I think with Perl you can start declaring your variables "local" and keep the interpreter happy. But in R's context you then have to also start declaring what you expect to inherit from parent environments, and pretty soon the code is so encrusted with annotation barnacles that it loses the simplicity that  makes R so nice in the interactive mode.

What would be really nice is if we had a smart R editor/IDE that would "DWIM" and put a red underline under a misspelled name, but leave it alone when, as Duncan said, it's in the environment.

... which is, of course, impossible since the editor has no idea what environment you will evaluate the function in ... It can make assumptions but they may as wrong as the spurious warnings discussed so people will complain either way ;)

Cheers,
Simon

On 4/11/2011 8:46 AM, Simon Urbanek wrote:

On Apr 11, 2011, at 11:28 AM, Davor Cubranic wrote:

On 2011-04-09, at 2:08 PM, Ted Harding wrote:

I think the right thing to so is to introduce the students to the
basics of scoping, early in the process of learning R.

I know the basics of scoping perfectly well, but that doesn't stop 
me from occasionally misspelling a variable name that only causes an 
error much later.

OTOH, I think with Perl you can start declaring your variables 
"local" and keep the interpreter happy. But in R's context you then 
have to also start declaring what you expect to inherit from parent 
environments, and pretty soon the code is so encrusted with 
annotation barnacles that it loses the simplicity that  makes R so 
nice in the interactive mode.

What would be really nice is if we had a smart R editor/IDE that 
would "DWIM" and put a red underline under a misspelled name, but 
leave it alone when, as Duncan said, it's in the environment.

... which is, of course, impossible since the editor has no idea what 
environment you will evaluate the function in ... It can make 
assumptions but they may as wrong as the spurious warnings discussed 
so people will complain either way ;)

      For the record, my "complaint" stemmed from my inability to see 
a way to get rid of that message in that context.  In most cases, I've 
found that message to be very valuable in identifying latent bugs in 
code.  In that context, however, the message seemed inappropriate.  
Duncan privately suggested I add "LazyData:  yes" to the package 
DESCRIPTION file.  I did that, and the offending message disappeared!


      Thanks again to Duncan.


      Best Wishes,
      Spencer

Cheers,
Simon

On 4/11/2011 8:46 AM, Simon Urbanek wrote:

On Apr 11, 2011, at 11:28 AM, Davor Cubranic wrote:

On 2011-04-09, at 2:08 PM, Ted Harding wrote:

I think the right thing to so is to introduce the students to the
basics of scoping, early in the process of learning R.

I know the basics of scoping perfectly well, but that doesn't stop me from
occasionally misspelling a variable name that only causes an error much
later.

OTOH, I think with Perl you can start declaring your variables "local" and
keep the interpreter happy. But in R's context you then have to also start
declaring what you expect to inherit from parent environments, and pretty
soon the code is so encrusted with annotation barnacles that it loses the
simplicity that  makes R so nice in the interactive mode.

What would be really nice is if we had a smart R editor/IDE that would
"DWIM" and put a red underline under a misspelled name, but leave it alone
when, as Duncan said, it's in the environment.

... which is, of course, impossible since the editor has no idea what
environment you will evaluate the function in ... It can make assumptions but
they may as wrong as the spurious warnings discussed so people will complain
either way ;)

       For the record, my "complaint" stemmed from my inability to see a
way to get rid of that message in that context.  In most cases, I've
found that message to be very valuable in identifying latent bugs in
code.  In that context, however, the message seemed inappropriate.
Duncan privately suggested I add "LazyData:  yes" to the package
DESCRIPTION file.  I did that, and the offending message disappeared!

On 4/11/11 11:04 AM, "Spencer Graves"<spencer.graves at prodsyse.com>  wrote:

 On 4/11/2011 8:46 AM, Simon Urbanek wrote:

 On Apr 11, 2011, at 11:28 AM, Davor Cubranic wrote:

 On 2011-04-09, at 2:08 PM, Ted Harding wrote:

 I think the right thing to so is to introduce the students to the
 basics of scoping, early in the process of learning R.

 I know the basics of scoping perfectly well, but that doesn't stop me from
 occasionally misspelling a variable name that only causes an error much
 later.

 OTOH, I think with Perl you can start declaring your variables "local" and
 keep the interpreter happy. But in R's context you then have to also start
 declaring what you expect to inherit from parent environments, and pretty
 soon the code is so encrusted with annotation barnacles that it loses the
 simplicity that  makes R so nice in the interactive mode.

 What would be really nice is if we had a smart R editor/IDE that would
 "DWIM" and put a red underline under a misspelled name, but leave it alone
 when, as Duncan said, it's in the environment.

 ... which is, of course, impossible since the editor has no idea what
 environment you will evaluate the function in ... It can make assumptions but
 they may as wrong as the spurious warnings discussed so people will complain
 either way ;)

        For the record, my "complaint" stemmed from my inability to see a
 way to get rid of that message in that context.  In most cases, I've
 found that message to be very valuable in identifying latent bugs in
 code.  In that context, however, the message seemed inappropriate.
 Duncan privately suggested I add "LazyData:  yes" to the package
 DESCRIPTION file.  I did that, and the offending message disappeared!

So would it be possible to have something akin to lint comment directives
to allow specific "errors" to be ignored by codetools?