[Bioc-devel] shiny, Bioconductor, and reproducible research
Hi Winston,
On Mon, Jul 29, 2013 at 2:39 PM, Winston Chang <winston at stdout.org> wrote:
Hi everyone - Great to hear from you all on your thoughts about Shiny. I've tried to answer some of Dan's questions below...
1) Testing shiny apps Typically, bioconductor packages have man page examples, vignettes, and (sometimes) unit tests, so when we build the packages every day on our build system, the code in all of these is evaluated and we have some idea of whether the package is working as it's supposed to. I'm not clear on how to do similar testing of a shiny application. Since launching a shiny app takes away the R console (until the app is closed), shiny apps should probably not be launched in example or test code (unless interactive() is TRUE). Do the shiny folks (or anyone else) have thoughts on testing shiny apps?
With regard to testing, we have unit tests for various components, and we're working on end-to-end tests using Selenium. We do have some unit tests in inst/tests/, but they are entirely on the R side - they don't test interaction with the browser. We have some other tests in inst/tests-js/, which exercise the client (Javascript) side, but they don't test interaction with the server. Of course it's important to have tests for client-server communication. We're also in the process of setting up some end-to-end tests using Selenium, but that's just in the beginning stages right now. Hopefully in the future we'll have more to say about these kinds of tests.
My feeling (and I could be wrong) is that Bioconductor package developers will be more concerned with testing the basic logic of shiny apps, and that therefore testing only in R would be sufficient for the most part. For example, if you could simulate a slider in a unit test by just changing a reactive value, it would not be necessary to make sure it works in the deployed context (with browser and server). Of course those kinds of tests are also important, but to an extent they just ensure that shiny itself works, and so they are more the province of the shiny developers.
2) Reproducible research Reproducible research is really important in our community, but shiny apps are sort of a black box as far as reproducibility is concerned. If shiny apps are "read-only" (that is, if they are just used to view an object) then this is not really a problem. But shiny has the ability now to change objects back in your R session, so we need to be able to track what is done inside the shiny app. shiny apps can return some sort of object that tells you what was done in the shiny session (insofar as it modifies any objects in the users's session) and this object can provide a way to reproduce those steps without the shiny app. So maybe the object would consist of lines of code. This means that if there is a shiny app, the package author must also provide ways to do the same transformations on objects without shiny.
If the goal is to allow the same transformations in a Shiny app and in a regular R session, this certainly is possible. You could, for example, have a bunch of inputs in a Shiny app that map directly to arguments in a function call, and save those arguments so that you can later call the function with the same settings. This is something that app authors will have to keep in mind when they write their apps.
There are probably other approaches as well. But the end goal would be a scriptable, reproducible shiny session. Another thought was that a shiny app could emit some sort of state object, and then be restarted with that object.
Capturing and restoring the full state of a Shiny session probably isn't possible, since the full state includes the client, server, and interaction between the two. An app author could save/restore the state of particular objects of interest in a Shiny app, but as of now, this would have to be implemented for each app. I hope this helps!
That's very helpful! Thanks, Dan
-Winston