[Bioc-devel] BrowserViz and sub-protocols

Hi Michael,

Thanks for the link to the subprotocol negotiaion discussion in "High
Performance Browser Networking".  There's a number of things in that
discussion of which I was not aware.  Very helpful.

There may be real merit -- that is, a good trade-off between some added
complexity and extra clarity -- to adding a mandatory content-type subfield
to the payload or to the message.  Or adding full sub-protocol negotiation
as the reference you supplied discusses.

For the first release of BrowserViz (and its demo subclass,
BrowserVizDemo, along with RCyjs) I would like to keep things as simple as
possible.  By which I mean:  provide (as we do now) the mechanism by which
content-type can be specified, without promulgating the policy that every
message must have that payload subfield.

In practice I have found it quick and easy in both Javascript and R to
examine the incoming 4-field message (cmd, status, callback, payload):

1) Is the cmd field something I can respond to?  (Do I have a handler
registered for that cmd?)
2) Does the status indicate trouble? A request?
3) If 1 & 2 suggest proceeding, then  names(msg$payload) can help me
figure out if the incoming data is structured as I expect.

One could reasonably object that this is all "programming by (ad hoc)
agreement", rather than by transparent, self-describing, clearly
negotiatied data and exchange standards.

In order to arrive at such standards, to figure out how formal we should
go, let's experiment for a release cycle.  This could be accomplished
without any changes to BrowserViz by using the status field
(status="summarizedExperimentRequest"), or by always using two fields in
every message payload (payload$contentType, payload$content).  If a few
people collaborated on (let's say) a SummarizedExperiment browser viz
webapp, those people could agree on how to do this, try it out, perhaps
discover a few sub-varieties of content-type are actually needed.  Then,
when things have settled down, we could explore adding some formality to
the process.

How does that strike you?

Thanks for the discussion!

- Paul



On Apr 3, 2015, at 3:11 PM, Michael Lawrence <lawrence.michael at gene.com>
wrote:

The high-level type issue is sort of discussed here:

http://chimera.labs.oreilly.com/books/1230000000545/ch17.html#_subprotocol_negotiation

What about extending your protocol so that the payload consists of two

fields:

content-type and content, where the content-type adheres to the media

type specification? This is analogous to how every S4 object has a class
attribute.

Thanks for the discussion!

On Fri, Apr 3, 2015 at 2:57 PM, Paul Shannon <

paul.thurmond.shannon at gmail.com> wrote:

Hi Michael,

Thanks for the clarification.   You make a good point about caching:

nobody wants to have to reinvent and re-engineer that!   If too many
features like that become important, then the simplicity of websockets will
have an attendant cost.

With regard to more formalism around the payload:  I can imagine that

there will be circumstances in which that is essential.  A
SummarizedExperiment browser would be very useful, and would clearly
benefit from a standard payload structure.

I figure, however, that that's above my pay-grade ;).   I leave that up

to people like you.  And I stand ready to add any features such formalized
structures might require.  I'd like to think (I may be naive) that the
architects of SummarizedExperiment and heavy users of it could devise and
negotiate some standard JSON representation of the class, translators to
and from, which all could then be used by BrowserViz without BrowserViz
needing to know it's there.

The "binary JSON" format may be useful in some circumstances:

http://bjson.org

- Paul

On Apr 3, 2015, at 2:43 PM, Michael Lawrence <lawrence.michael at gene.com>

wrote:

On Fri, Apr 3, 2015 at 2:00 PM, Paul Shannon <

paul.thurmond.shannon at gmail.com> wrote:

Hi Michael,

Great to get your response, comments and questions.  Answers attempted

below.

I think our overriding difference lies in our contrasting experience of

complexity.  I have come to see websockets as minimal, simple, fast and
flexible, whereas you see them quite differently.  I would like to
understand your views on this; I could be overlooking somce important and
maybe costly complicating features, perhaps because of my fondness for
other simplifying features.

I was just referring to potential complexity. If one attempted to

reimplement the useful features of HTTP (like caching), complexity would be
introduced into application code, while you really want that complexity in
the protocol implementation. HTTP seems most appropriate for when the
server is acting as the data model. Outside of that, I see the benefits of
web sockets.

I think you missed my question about RPC. Also, any thoughts as to more

formalism around the payload? We obviously have complex data structures,
and it would be nice to communicate semantics somehow to the web browser.
For example, could there be some convention for representing a
SummarizedExperiment? Could a payload contain the equivalent of a media
type that an R/JS library could understand to marshal objects? Could there
be some way to query for the types of payload a command supports?

I've seen stuff like WAMP, but they seem to lack the ability to declare

high-level types. Maybe that's just out of style?

More below...

I look forward to hearing back from you.

- Paul

On Apr 3, 2015, at 1:45 PM, Michael Lawrence <lawrence.michael at gene.com>

wrote:

Thanks to Val's excellent newsletter, I've had my first glance at
BrowserViz. I'm glad to see something that is more flexible and

low-level

than e.g. shiny.

I'm curious about the motivation behind web sockets. I guess any
application with an R-driven web UI actually has two UIs: the R

console and

the browser. But what if the R session is headless, or if there is no

need

for commands in R to affect the browser? Then the web socket layer

brings

mostly unneeded complexity.

I see websockets (like TCP sockets) as musch simpler than HTTP.  No

headers,

no explicit server and explicit client (once the connection is open).
Could you explain the complexity you see?

An interesting comparison to BrowserViz is not
shiny but OpenCPU. It's purely HTTP-based and still manages to maintain
state (not sure how efficiently). I guess one advantage of web sockets

is

that one can program imperatively instead of declaratively on the

server,

i.e., the server can send a command to show a popup in response to some
event, instead of returning a "declaration" that the popup should be

shown.

Exactly!

So essentially web sockets are more natural for implementing

server-side

controllers (think MVC), instead of just the data model, but man, it's

a

shame to lose the features of HTTP.

I'll confess:  I set out to -shed- the features of HTTP.  Isn't it a

protocol designed for serving up web pages on demand?  Not for fast,
lighweight peer-to-peer communications?

Ultimately, I think we want web apps that are easy to develop and

maintain,

and run equally well from either a useR's session or a remote client
communicating to a dedicated, headless server. Is the generality of
websockets worth the complexity?

As an aside, it would seem relatively straight-forward to implement a
simple bi-directional RPC mechanism between R and JS using the standard
protocol (i.e., hide details like the callback). Does that sound

reasonable?

I was also a bit surprised about the need to copy/paste the JS

boilerplate.

Certainly there must be javascript frameworks with a more elegant

solution

to extensibility.

You are correct on this, and I only need to correct the vignette,

written many weeks ago.

I now provide "BrowserViz.jx", a simple Javascript module.  It is used

by both

BrowserVizDemo and RCyjs.  Sorry to have sent out an obsolete vignette.

     [[alternative HTML version deleted]]