Skip to content

byte coding compiling.....

11 messages · akshay kulkarni, Bert Gunter, Jeff Newmiller +2 more

akshay kulkarni
# 
Dear members,
                          I am using pbmclapply, the progress bar version of mclapply, from the parallel package. The point is, pbmclapply is three times faster than mclapply, and I think the most probable reason would be that pbmclapply is byte code compiled (I can think of no other reason).

I know the cmpfun function from compiler package. If I do:
will the job be done? The point is mclapply may look for other functions in the parallel package. So I have to compile the whole package right? How do you do that? or in general, how do you byte code compile a whole package?

Thanking you,
Yours sincerely,
AKSHAY M KULKARNI
Bert Gunter
# 
Unlikely

See here:

https://www.r-bloggers.com/2017/08/how-to-make-best-use-of-the-byte-compiler-in-r/

Byte code compilation should be automatic in both cases, as I understand
it. Of course, I could be wrong due to special features of parallel
programming, etc.

A reprex might be helpful here.

Cheers,
Bert
On Wed, Jul 6, 2022, 7:29 PM akshay kulkarni <akshay_e4 at hotmail.com> wrote:

            

      
      
      
    

        

      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      akshay kulkarni
    

    

      
      #
    

  


  

      
Dear Bert,
                 Thanks for your reply...

So

            

      
      
      
    

        
should  do the job right?

By the by, how can I give a reprex? Reprex of the code that I am giving to mclapply (as FUN argument)?

Yours sincerely,
AKSHAY M KULKARNI

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Jeff Newmiller
    

    

      
      #
    

  


  

      
A reprex is a minimal example that performs as you described. It is up to you to trim your code down to eliminate irrelevant computation while not eliminating the described behavior. You may discover something about your own code in the process that answers your question, or you will be able to demonstrate the issue, or we may still not be able to reproduce if it is related to your system.

        
On July 6, 2022 10:54:18 AM PDT, akshay kulkarni <akshay_e4 at hotmail.com> wrote:

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Uwe Ligges
    

    

      
      #
    

  


  

      
On 06.07.2022 19:54, akshay kulkarni wrote:

            

      
      
      
    

        
mclapply is already byte compiled as it is in a package.

You may want to
cmpfun(yourFunction)
the function that you use in the mclapply call.

Best,
Uwe Ligges

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      akshay kulkarni
    

    

      
      #
    

  


  

      
Dear Uwe,
                  I have attached the info from the parallel package description from my Rstudio IDE:

Package: parallel
Version: 4.1.2
Priority: base
Title: Support for Parallel computation in R
Author: R Core Team
Maintainer: R Core Team <do-use-Contact-address at r-project.org>
Contact: R-help mailing list <r-help at r-project.org>
Description: Support for parallel computation, including by forking
   (taken from package multicore), by sockets (taken from package snow)
   and random-number generation.
License: Part of R 4.1.2
Imports: tools, compiler
Suggests: methods
Enhances: snow, nws, Rmpi
NeedsCompilation: yes
Built: R 4.1.2; x86_64-w64-mingw32; 2021-11-01 18:38:05 UTC; windows

It says: NeedsCompilation: yes

How about it?

Yours sincerely,
AKSHAY M KULKARNI

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Rui Barradas
    

    

      
      #
    

  


  

      
Hello,

Sorry to intrude in this thread but what NeedsCompilation means is that 
the package contains C, C++ or Fortran native code in a directory named src.

 From the documentation, Writing R Extensions, section 1.1.1 The 
DESCRIPTION file:


The ?NeedsCompilation? field should be set to "yes" if the package 
contains native code which needs to be compiled, otherwise "no" (when 
the package could be installed from source on any platform without 
additional tools). This is used by install.packages(type = "both") in R 
 >= 2.15.2 on platforms where binary packages are the norm: it is 
normally set by R CMD build or the repository assuming compilation is 
required if and only if the package has a src directory.


And package parallel does have a src directory with C code.


[1] 
https://cran.r-project.org/doc/manuals/r-release/R-exts.html#The-DESCRIPTION-file


Hope this helps,

Rui Barradas

?s 16:38 de 07/07/2022, akshay kulkarni escreveu:

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Jeff Newmiller
    

    

      
      #
    

  


  

      
That item refers to the package having some compiled language (e.g. C, C++, Fortran, etc) components. The very fact that it got installed confirms that compilation occurred... it would not be usable otherwise.

        
On July 7, 2022 8:38:49 AM PDT, akshay kulkarni <akshay_e4 at hotmail.com> wrote:

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      akshay kulkarni
    

    

      
      #
    

  


  

      
Dear Jeff,
                THen why is pbmclapply 3 time faster than mclappy? In the package description of pbmclapply it says: "Needs compilation no". When i ran my code iwith pbmclapply, I did not compile my code. So I do presume that the speed up must in some way connected to this "NeedsCompilation" field. Any thoughts on that?

Yours sincerely
AKSHAY M KULKARNI

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Jeff Newmiller
    

    

      
      #
    

  


  

      
I don't know why one is faster. You have not provided a reprex (the "reprex" package may be of help to you with that). However, as Rui confirmed, you are completely wrong as to the NeedsCompilation setting in the package... that is a wild goose chase.

        
On July 7, 2022 9:48:22 AM PDT, akshay kulkarni <akshay_e4 at hotmail.com> wrote:

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      akshay kulkarni
    

    

      
      #
    

  


  

      
Dear Jeff,
               Thanks for the reply....can you please refer some online resources for  getting the reason why pbmclapply is more faster? There should be some reason right?  If a reprex is essential, I will provide when the code is ready(I am still working on it). Thanks in advance...

Yours sincerely,
AKSHAY M KULKARNI