Skip to content

Question on multiple sessions...

10 messages · Vince Fulco, Ana Nelson, Gabor Grothendieck +2 more

Original
Vince Fulco
# 
Assuming one wants to use an open session to collect real time data a
la IBrokers or similar, what would be the best way to maintain another
session for calculations?  Is RServe the answer or is there a lighter
solution for semi-auto analytical script running?  Either Windows or
Linux solutions would be appreciated.  TIA


Vince Fulco, CFA, CAIA
612.424.5477 (universal)
vfulco1 at gmail.com
Gabor Grothendieck
# 
One possibility is to write the real time acquisition in tcl
and then the actual data processing in R.  The tcltk package
(which is included in the Windows distribution of R but is available
from CRAN on all distributions) can handle tasks running concurrently
with R.  See the svSocket and Rpad packages as examples
of this approach.
On Wed, Feb 11, 2009 at 6:17 PM, Vince Fulco <vfulco1 at gmail.com> wrote:
Vince Fulco
# 
Yes to both question sbut it could be as simple as taking a copy of
the real time data and calculating on that.  I've noted Gabor's answer
(thanks!) but would welcome yours as well.


Vince Fulco, CFA, CAIA
612.424.5477 (universal)
vfulco1 at gmail.com
On Thu, Feb 12, 2009 at 5:53 AM, Ana Nelson <nelson.ana at gmail.com> wrote:
Jeff Ryan
# 
Within the context of IBrokers, the entire process of collecting data and
acting upon it can take place within one session.

The notion of needing this to be multi-threaded isn't quite accurate, at
least by my figuring.

I am working on a paper/tutorial to outline this whole process, but here is
what I have found to work.

R is single threaded.  The approach I initially envisioned did involve some
sort of dual R-process, where communication was via a file or sockets.  This
obviously wasn't ideal, but seemed workable.

Turns out that with sockets, *blocking* sockets in R are the only
reliable/efficient cross-platform mechanism to communicate that I have found
while keeping everything in R.  This causes some interesting problems if you
are waiting for messages from another process, and those messages are
irregular in time.  Essentially you are just shifting the issue from one
single-threaded process into two.  Same problems persist.

So, a quick primer on how IBrokers can manage this.

A request for data is sent via reqMktData.  The connection then blocks until
a message is received back from Interactive Brokers.  This is in the form of
a continuous stream of messages.  At each iteration/call of readBin [where
new messages are taken from the socket] the proper handling via callbacks
take place.

In the CRAN version I have two methods available to the user. One is where
custom handlers for each _type_ of data can be passed into the original
reqMktData call, e.g. on each new bid message that arrives, do *something*. 
The other original approach was to use a custom CALLBACK argument to manage
the entire message stream.

The CALLBACK basically handles _all_ possible messages, and with custom
action based on what is happening.

The newest version at  http://ibrokers.googlecode.com
http://ibrokers.googlecode.com  has a much better interface.  This new
version allows for a user defined actions to take place at each new message
(essentially a global level of processing), and within each different type
of message received.  By using special eWrapper closures you can enable data
to persist from one message to another, all in memory, all within one
session.

The obvious caveat here is that everything is non-interactive.  Which at
first glance seems a bit strange given most interaction with R is
interactive, but from an automated strategy perspective is just fine.

Again, I am working on some solid documentation related to IBrokers
specifically, but will also be publishing a white-paper on the general idea
of real-time processing in the near future at  http://www.insightalgo.com
http://www.insightalgo.com .

Jeff


I will be making a paper/presentation available with more details at some
point in the near future
Vince Fulco wrote:

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Daniel Cegiełka
    

    

      
      #
    

  


  

      
Jeff Ryan pisze:

            

      
      
      
    

        
Hi Jeff ;)

Twisted (python framework) include non-blocking function...
It can be ported to R.

http://twistedmatrix.com/trac/wiki/FrequentlyAskedQuestions#HowdoIuseDeferredstomakemyblockingcodenon-blocking


daniel

  
  


  


      

    

    

      
      

        


  

        

      Jeff Ryan
    

    

      
      #
    

  


  

      
Daniel,

The issue isn't blocking vs. non-blocking, as R technically has that
as well, the problem is in reliability and performance, at least in my
experience.

readBin on *nix when non-blocking will return a character(0) (or
whatever you request) when _nothing_ is available to read.  That is
testable and logical.  On Windows (this is the point related to
cross-platform) it returns garbage.  No way to test/etc.  Additionally
you are limited by nul terminated strings in R's processing.  There
are workarounds, but they are just that.

This is all within the context of IBrokers mind you, other connections
will be different in terms of requirements.  IB uses simple
nul-terminated character messages.

You can of course write your own C code, use Java, or Python etc. But
that then has the multiprocess issue, and all its complexity/ugliness.

Given the correct logic I contend most everything is doable in R.  And
that is a very good thing.

Jeff

On Thu, Feb 12, 2009 at 10:47 AM, Daniel Cegielka

        
<daniel.cegielka at gmail.com> wrote:

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Daniel Cegiełka
    

    

      
      #
    

  


  

      
Jeff Ryan pisze:

            

      
      
      
    

        
http://code.google.com/p/ibrokers/source/browse/trunk/R/processMsg.R

So if I understand this code...

daniel

  
  


  


      

    

    

      
      

        


  

        

      Jeff Ryan
    

    

      
      #
    

  


  

      
Again, in the IBrokers case:

twsCALLBACK is the main callback to the reqMktData request.

*processMsg* is the main routine to process each individual message
that comes in.

Within there are eWrapper$calls that handle the actual message
details, e.g. print to a file, do some processing, etc.

http://code.google.com/p/ibrokers/source/browse/trunk/R/eWrapper.R
http://code.google.com/p/ibrokers/source/browse/trunk/R/eWrapper.MktData.CSV.R

reqMktData => twsCALLBACK => processMsg => eWrapper$method [repeat in
while loop]

Using closures (as in the MktData.CSV version), or global variables,
you can keep track of last data or details that can be used for
real-time processing.

placeOrder is also available to _act_ on the processing as well.

Jeff


On Thu, Feb 12, 2009 at 11:07 AM, Daniel Cegielka

        
<daniel.cegielka at gmail.com> wrote: