Skip to content

runtime on ising model

9 messages · Michael D, Jim Lemon, Mike Marchywka +2 more

Original
Jim Lemon
#
On 10/26/2010 04:50 PM, Michael D wrote:
Hi Michael,
One bottleneck is probably the sampling. If it doesn't grab too much 
memory, setting up a vector of the samples (maybe a million at a time if 
10 million is too big - might be able to rewrite your sample vector when 
you store the state) and using k (and an offset if you don't have one 
big vector) to index it will give you some speed.

Jim
Mike Marchywka
# 
----------------------------------------
I haven't looked at this since some passing interest in magnetics
decades ago, something about 8-tracks and cassettes, but you have
to be careful with conclusions like " I removed foo and problem
went away therefore problem was foo." Performance issues are often
caused by memory, not CPU limitations. Removing anything with a big
memory footprint could speed things up. IO can be a real bottleneck.
If you are talking about things on minute timescales, look at task
manager and see if you are even CPU limited. Look for page faults
or IO etc. If you really need performance and have a task which
is relatively simple, don't ignore c++ as a way to generate data
points and then import these into R for analysis. 

In short, just because you are focusing on math it doesn't mean
the computer is limited by that.
1 day later
David Winsemius
#
On Oct 28, 2010, at 11:52 AM, Michael D wrote:

            

      
      
      
    

        
Have you tried replacing all of those 10^x operations with their  
integer equivalents, c(10000L, 100000L, 1000000L)? Each time through  
the loop you are unnecessarily calling the "^" function 4 times. You  
could also omit the last one. 10^7,  during testing since M at the  
last iteration (k=10^7) would be the final value and you could just  
assign the state of M at the end. So we have eliminated 4*10^7  
unnecessary "^" calls and 10^7 unnecessary comparisons. (The CS  
professor is perhaps used to having the C compiler do all thinking of  
this sort for him.)

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      William Dunlap
    

    

      
      #
    

  


  

          

      
      
      
    

        
Did that one change really make a difference?
R does not evaluate anything in the if or else
clauses of an if statement before evaluating
the condition.

            

      
      
      
    

        
%in% is a relatively expensive function.  Use == if you
can.  E.g., compare the following 2 ways of stashing
something at times 1e4, 1e5, and 1e6:

            

      
      
      
    

        
for(k in seq_len(1e6))
                   if(k %in% set) z[length(z)+1]<-k
                print(z)})
[1]   10000  100000 1000000
   user  system elapsed
 46.790   0.023  46.844

            

      
      
      
    

        
nextCheckPoint <- 10^4
               for(k in seq_len(1e6))
                   if( k == nextCheckPoint ) {
                       nextCheckPoint <- nextCheckPoint * 10
                       z[length(z)+1]<-k
                   }
               print(z)})
[1]   10000  100000 1000000
   user  system elapsed
  4.529   0.013   4.545

With such a large number of iterations it pays to
remove unneeded function calls in arithmetic expressions.
R does not optimize them out - it is up to you to
do that.  E.g.,

  > system.time(for(i in seq_len(1e6)) sign(pi)*(-1))
     user  system elapsed
    6.802   0.014   6.818
  > system.time(for(i in seq_len(1e6)) -sign(pi))
     user  system elapsed
    3.896   0.011   3.911

Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com

            

      
      
      
    

        

      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      David Winsemius
    

    

      
      #
    

  


  

      
On Oct 28, 2010, at 12:20 PM, David Winsemius wrote:

        

            

      
      
      
    

        
Bill Dunlap's suggestion to use "==" instead of %in% cut the time to  
1/3 of what it had been even after the pre-calculation of the integer  
values( which only improved the looping times by 30%). The combination  
of the two with:
  if (k ==10000L|k==100000L|k==1000000L ) { ... }

... resulted in an improvement by a factor or 12.006/2.523 or 475% for  
the interim checking and printing operation using Bill's test suite.

            

      
      
      
    

        

      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        
David Winsemius, MD
West Hartford, CT

  
  


  


      

    

    

      
      

        


  

        

      Mike Marchywka
    

    

      
      #
    

  


  

      
----------------------------------------

            

      
      
      
    

        
Agreed on first part but not second- empirical debugging rarely 
produces compelling results in isolation. As a collection
of symptons fine but not conclusive- if you learn c++ you will
find out about all kinds of things like memory corruption that
never make sense :) Here, the big concern is issues with memory
as you never determined to be CPU limited although based on
others comments you likely are in any case.

            

      
      
      
    

        
What is at issue here? That is, the OP claimed inverting polarity
sped things up, suggesting that the branch mattered. AFAIK he
never actually proved which branch was taken. This could
imply many things or nothing: one branch may be slow, or cause
a page fault, or the test may fail fast but succed slowly( testing
huge array for equality for example) .