[Bioc-devel] shiny, Bioconductor, and reproducible research

Hi Winston,



On Mon, Jul 29, 2013 at 2:39 PM, Winston Chang <winston at stdout.org> wrote:

Hi everyone -

Great to hear from you all on your thoughts about Shiny. I've tried to
answer some of Dan's questions below...

1) Testing shiny apps

Typically, bioconductor packages have man page examples, vignettes,
and (sometimes) unit tests, so when we build the packages every day on
our build system, the code in all of these is evaluated and we have
some idea of whether the package is working as it's supposed to.

I'm not clear on how to do similar testing of a shiny application.
Since launching a shiny app takes away the R console (until the app is
closed), shiny apps should probably not be launched in example or test
code (unless interactive() is TRUE).

Do the shiny folks (or anyone else) have thoughts on testing shiny apps?

With regard to testing, we have unit tests for various components, and we're
working on end-to-end tests using Selenium.

We do have some unit tests in inst/tests/, but they are entirely on the R
side - they don't test interaction with the browser. We have some other
tests in inst/tests-js/, which exercise the client (Javascript) side, but
they don't test interaction with the server.

Of course it's important to have tests for client-server communication.
We're also in the process of setting up some end-to-end tests using
Selenium, but that's just in the beginning stages right now. Hopefully in
the future we'll have more to say about these kinds of tests.

My feeling (and I could be wrong) is that Bioconductor package
developers will be more concerned with testing the basic logic of
shiny apps, and that therefore testing only in R would be sufficient
for the most part. For example, if you could simulate a slider in a
unit test by just changing a reactive value, it would not be necessary
to make sure it works in the deployed context (with browser and
server).

Of course those kinds of tests are also important, but to an extent
they just ensure that shiny itself works, and so they are more the
province of the shiny developers.

2) Reproducible research

Reproducible research is really important in our community, but shiny
apps are sort of a black box as far as reproducibility is concerned.

If shiny apps are "read-only" (that is, if they are just used to view
an object) then this is not really a problem. But shiny has the
ability now to change objects back in your R session, so we need to be
able to track what is done inside the shiny app.

shiny apps can return some sort of object that tells you what was done
in the shiny session (insofar as it modifies any objects in the
users's session) and this object can provide a way to reproduce those
steps without the shiny app. So maybe the object would consist of
lines of code. This means that if there is a shiny app, the package
author must also provide ways to do the same transformations on
objects without shiny.

If the goal is to allow the same transformations in a Shiny app and in a
regular R session, this certainly is possible. You could, for example, have
a bunch of inputs in a Shiny app that map directly to arguments in a
function call, and save those arguments so that you can later call the
function with the same settings. This is something that app authors will
have to keep in mind when they write their apps.

There are probably other approaches as well. But the end goal would be
a scriptable, reproducible shiny session.

Another thought was that a shiny app could emit some sort of state
object, and then be restarted with that object.

Capturing and restoring the full state of a Shiny session probably isn't
possible, since the full state includes the client, server, and interaction
between the two. An app author could save/restore the state of particular
objects of interest in a Shiny app, but as of now, this would have to be
implemented for each app.


I hope this helps!

That's very helpful!
Thanks,
Dan

-Winston