Hi, I have a problem with a package I have developed in that functions do not get loaded due to older versions of the functions being in the .GlobalEnv? fetched from .Rdata files stored from previous saved workspaces. I need to be able to fix this somehow when I load the package. I do not want to mess up the search order to fix the problem. How I got myself into this mess is that I started developing using a bunch of R files and a dynamic library and then loaded these from an R function rather than a package. This is great when developing as it allows quick reloading. After doing this for about a year now I have (unknowingly) populated all my favourite working directories with .Rdata files that contain various versions of my functions. And I shared this collection of code, with its flakey loading mechanism, with PhD students and colleagues who have used it and nicely populated their working directories with it as well?. So now I have learnt how to turn it into a package as it is in a more stable state. But when I load it of course the .GlobalEnv with the definition from old saved .Rdata files is first in the search order. I say "of course" but tt has taken me a while to find this out. At first I thought I was going mad as the old functions seemed to magically turn up even though I had updated the package and checked that the new versions of the functions were the only ones in the system (or so I thought)? My question is: how can I force a package to overwrite the "old" definitions in the .GlobalEnv. If it cannot be done automatically it could be a user triggered function. While developing I still want newer versions of functions to take precedence in the .GlobalEnv over the package versions. i.e. I do not want the package to be earlier in the search order than .GlobalEnv (if this is even possible), I just want to be able to delete the old definitions. I do not want to delete the .Rdata files as there is useful state in them... Any help getting me out of this mess would be appreciated! Martin
problem with package development and older defs earlier in search order
6 messages · Rolf Turner, Duncan Murdoch, Martin J Reed
Suggestions:
(1) Work in a different directory (so that you will have a different ---
initially
empty) .RData file.
(2) Or: Clean up the .Rdata file in the directory that you are
currently using;
rm(list=ls()) does this for you. Every so slightly dangerous! :-)
(3) Possibly: Before doing (2), save stuff that you actually *want*,
e.g. via
save(mung,gorp,clyde,irving,melvin,file="savedStuff.rda")
Then you can recover mung, gorp, etc. via
load("savedStuff.rda")
cheers,
Rolf Turner
On 08/11/12 13:11, Martin J Reed wrote:
Hi, I have a problem with a package I have developed in that functions do not get loaded due to older versions of the functions being in the .GlobalEnv? fetched from .Rdata files stored from previous saved workspaces. I need to be able to fix this somehow when I load the package. I do not want to mess up the search order to fix the problem. How I got myself into this mess is that I started developing using a bunch of R files and a dynamic library and then loaded these from an R function rather than a package. This is great when developing as it allows quick reloading. After doing this for about a year now I have (unknowingly) populated all my favourite working directories with .Rdata files that contain various versions of my functions. And I shared this collection of code, with its flakey loading mechanism, with PhD students and colleagues who have used it and nicely populated their working directories with it as well?. So now I have learnt how to turn it into a package as it is in a more stable state. But when I load it of course the .GlobalEnv with the definition from old saved .Rdata files is first in the search order. I say "of course" but tt has taken me a while to find this out. At first I thought I was going mad as the old functions seemed to magically turn up even though I had updated the package and checked that the new versions of the functions were the only ones in the system (or so I thought)? My question is: how can I force a package to overwrite the "old" definitions in the .GlobalEnv. If it cannot be done automatically it could be a user triggered function. While developing I still want newer versions of functions to take precedence in the .GlobalEnv over the package versions. i.e. I do not want the package to be earlier in the search order than .GlobalEnv (if this is even possible), I just want to be able to delete the old definitions. I do not want to delete the .Rdata files as there is useful state in them... Any help getting me out of this mess would be appreciated!
On 12-11-07 7:11 PM, Martin J Reed wrote:
Hi, I have a problem with a package I have developed in that functions do not get loaded due to older versions of the functions being in the .GlobalEnv? fetched from .Rdata files stored from previous saved workspaces. I need to be able to fix this somehow when I load the package. I do not want to mess up the search order to fix the problem. How I got myself into this mess is that I started developing using a bunch of R files and a dynamic library and then loaded these from an R function rather than a package. This is great when developing as it allows quick reloading. After doing this for about a year now I have (unknowingly) populated all my favourite working directories with .Rdata files that contain various versions of my functions. And I shared this collection of code, with its flakey loading mechanism, with PhD students and colleagues who have used it and nicely populated their working directories with it as well?. So now I have learnt how to turn it into a package as it is in a more stable state. But when I load it of course the .GlobalEnv with the definition from old saved .Rdata files is first in the search order. I say "of course" but tt has taken me a while to find this out. At first I thought I was going mad as the old functions seemed to magically turn up even though I had updated the package and checked that the new versions of the functions were the only ones in the system (or so I thought)? My question is: how can I force a package to overwrite the "old" definitions in the .GlobalEnv. If it cannot be done automatically it could be a user triggered function. While developing I still want newer versions of functions to take precedence in the .GlobalEnv over the package versions. i.e. I do not want the package to be earlier in the search order than .GlobalEnv (if this is even possible), I just want to be able to delete the old definitions. I do not want to delete the .Rdata files as there is useful state in them... Any help getting me out of this mess would be appreciated!
Rolf's advice is good: delete the copies from your global env, and the ones in your package won't be hidden. Here's some more long term advice: keep your global environment empty at the start of each session, and thereby avoid this problem. If you want to temporarily edit a function from your package, you can still make a copy in the global environment, and it will take precedence (when called from the top level; the package functions that call each other won't see it). At the end of your session, when offered the chance to save your workspace, just say no. Regarding your specific question: a package could overwrite user objects in the global environment, but that's very bad practice. The user owns those, and packages should avoid messing with them. Let the user say foo <- foo to make a copy of the foo function in the global environment, don't force your own copy there. (The user will still be able to get back to the original one using the pkg::foo notation, and as I mentioned, packages will generally look internally first, so they'll see the original one.) Duncan Murdoch
1 day later
Rolf and Duncan
Many thanks. Your answers pointed me to a refinement that is closer to what I want:
rm(list=intersect(ls(".GlobalEnv"),ls("package:reedgraph")),
pos=".GlobalEnv")
This only removes items that are "masked" by GlobalEnv from my package.
As this is a bit long for some of the people that need to update their workspaces I have created a function to fix it:
<packagename>.update2package <- function() {
rm(list=intersect(ls(".GlobalEnv"),ls("package:<packagename>")),
pos=".GlobalEnv")
}
Just for completeness (if anyone else reads this). It is possible to make this happen automatically at package load using
.onAttach <- function(libname, pkgname) { <packagename>.update2package() }
However, as Duncan says this is REALLY bad practice, but is useful to me while debugging?.
Duncan: you say avoid saving the workspace. While I can understand why you are saying this, one of the really powerful features of R is the ability to start R in a working directory and have the state saved so you can come back to it. While this may be bad for the GUI versions of R as the workspace would just keep growing, if you start it from a particular directory this is not a problem and allows coming back to where you left off?
Regards,
Martin
On 8 Nov 2012, at 02:38, Rolf Turner <rolf.turner at xtra.co.nz> wrote:
Suggestions:
(1) Work in a different directory (so that you will have a different --- initially
empty) .RData file.
(2) Or: Clean up the .Rdata file in the directory that you are currently using;
rm(list=ls()) does this for you. Every so slightly dangerous! :-)
(3) Possibly: Before doing (2), save stuff that you actually *want*, e.g. via
save(mung,gorp,clyde,irving,melvin,file="savedStuff.rda")
Then you can recover mung, gorp, etc. via
load("savedStuff.rda")
cheers,
Rolf Turner
On 08/11/12 13:11, Martin J Reed wrote:
Hi, I have a problem with a package I have developed in that functions do not get loaded due to older versions of the functions being in the .GlobalEnv? fetched from .Rdata files stored from previous saved workspaces. I need to be able to fix this somehow when I load the package. I do not want to mess up the search order to fix the problem. How I got myself into this mess is that I started developing using a bunch of R files and a dynamic library and then loaded these from an R function rather than a package. This is great when developing as it allows quick reloading. After doing this for about a year now I have (unknowingly) populated all my favourite working directories with .Rdata files that contain various versions of my functions. And I shared this collection of code, with its flakey loading mechanism, with PhD students and colleagues who have used it and nicely populated their working directories with it as well?. So now I have learnt how to turn it into a package as it is in a more stable state. But when I load it of course the .GlobalEnv with the definition from old saved .Rdata files is first in the search order. I say "of course" but tt has taken me a while to find this out. At first I thought I was going mad as the old functions seemed to magically turn up even though I had updated the package and checked that the new versions of the functions were the only ones in the system (or so I thought)? My question is: how can I force a package to overwrite the "old" definitions in the .GlobalEnv. If it cannot be done automatically it could be a user triggered function. While developing I still want newer versions of functions to take precedence in the .GlobalEnv over the package versions. i.e. I do not want the package to be earlier in the search order than .GlobalEnv (if this is even possible), I just want to be able to delete the old definitions. I do not want to delete the .Rdata files as there is useful state in them... Any help getting me out of this mess would be appreciated!
On 10/11/12 12:08, Martin J Reed wrote:
Rolf and Duncan
Many thanks. Your answers pointed me to a refinement that is closer to what I want:
rm(list=intersect(ls(".GlobalEnv"),ls("package:reedgraph")),
pos=".GlobalEnv")
This only removes items that are "masked" by GlobalEnv from my package.
As this is a bit long for some of the people that need to update their workspaces I have created a function to fix it:
<packagename>.update2package <- function() {
rm(list=intersect(ls(".GlobalEnv"),ls("package:<packagename>")),
pos=".GlobalEnv")
}
Just for completeness (if anyone else reads this). It is possible to make this happen automatically at package load using
.onAttach <- function(libname, pkgname) { <packagename>.update2package() }
However, as Duncan says this is REALLY bad practice, but is useful to me while debugging?.
I really don't see it as being at all useful. What is the point, in
terms of package
development, of keeping copies of those functions in the global
environment if
you are going to remove them whenever you load the package?
It sounds to me like you need to implement some system of version control,
such as subversion (svn). Personally I use rcs --- simple enough for
the simple
minded such as my very good self to use, and amply adequate for my needs.
A version control system allows you to "backtrack" if a revision that
you make
to a bit of software turns out to be undesirable. It will do that much more
effectively IMHO than your current shaganappi strategy.
Finally you should note that CRAN policies expressly forbid the sort of
thing that you
propose doing with your .onAttach() function, should you ever be
inclined to submit
your package to CRAN.
cheers,
Rolf
Rolf, Re version control: I use SVN and Git depending on the project I am working on and what others are using. Years ago I used RCS, as you say its great for a local repository (as is Git). The point I was making was not about version control but that others like me might get caught out by saving environments on session quit and those environment having older versions of a function definition in it. I appreciate that you can get around this by never saving a session (but I suspect I am not the only one that finds saving a session useful). The point of my "shaganappi" (great, and very apt word!) strategy was: if you are developing code and are debugging with fast change/recompile/test cycles then you probably to not want to rebuild the package and load it into R every time, it is much quicker to just source the R files and dynload the compiled library. It took me a few hours of testing to realise that doing this is a problem if you save the session as then when you do get round to producing stable code and testing the loading of the package it does not overwrite these older versions and is later in the search order. The functions I listed are just useful as part of the debugging of package loading for me. Also some of the people I shared my earlier (non-packaged) version of the code need to clean their saved sessions of the old versions - hence a utility function to do this. If you never save a session then clearly this is completely unnecessary, but I have a number of users that have saved sessions and need this fix. I quite understand that doing what I suggested in .onAttach would not be acceptable for any production code and I should have made it clearer in my email that this was the case - thanks for doing this for other readers. If I do submit this to CRAN I would of course not include these functions. I still have quite a lot to learn before I would feel confident of doing this anyway? Thanks again for the helpful comments and guidance on good practice. Regards, Martin
On 10 Nov 2012, at 02:50, Rolf Turner <rolf.turner at xtra.co.nz> wrote:
On 10/11/12 12:08, Martin J Reed wrote:
Rolf and Duncan
Many thanks. Your answers pointed me to a refinement that is closer to what I want:
rm(list=intersect(ls(".GlobalEnv"),ls("package:reedgraph")),
pos=".GlobalEnv")
This only removes items that are "masked" by GlobalEnv from my package.
As this is a bit long for some of the people that need to update their workspaces I have created a function to fix it:
<packagename>.update2package <- function() {
rm(list=intersect(ls(".GlobalEnv"),ls("package:<packagename>")),
pos=".GlobalEnv")
}
Just for completeness (if anyone else reads this). It is possible to make this happen automatically at package load using
.onAttach <- function(libname, pkgname) { <packagename>.update2package() }
However, as Duncan says this is REALLY bad practice, but is useful to me while debugging?.
I really don't see it as being at all useful. What is the point, in terms of package
development, of keeping copies of those functions in the global environment if
you are going to remove them whenever you load the package?
It sounds to me like you need to implement some system of version control,
such as subversion (svn). Personally I use rcs --- simple enough for the simple
minded such as my very good self to use, and amply adequate for my needs.
A version control system allows you to "backtrack" if a revision that you make
to a bit of software turns out to be undesirable. It will do that much more
effectively IMHO than your current shaganappi strategy.
Finally you should note that CRAN policies expressly forbid the sort of thing that you
propose doing with your .onAttach() function, should you ever be inclined to submit
your package to CRAN.
cheers,
Rolf