Skip to content

[Rcpp-devel] Troubleshooting dependency issues

7 messages · Dirk Eddelbuettel, Simon Urbanek, Naeem Khoshnevis

Original
Naeem Khoshnevis
# 
Hi,

I hope this email finds you well.
I wondered what the best approaches to troubleshooting C++ dependencies
issues in general and specifically in R are; specifically, those packages
which are using OpenMP and we want to install on macOS.
So far, I am just googling the error and finding out what the problem is.
However, it is hard to address issues and come up with long-term solutions
without internalizing the concepts.
One example is creating a conda environment for each R package project and
installing llvm, clang, etc into each environment.
I talked to several software engineers; however, I could not find a
convincing answer. I think I do not know what I do not know about this
topic, which prevents me from asking good questions. I would be grateful if
you could let me know your thoughts about these questions.

Where is a good place to start learning about this topic?
Is a professional C++ developer necessarily good at troubleshooting
problems with compilers and linkers? Or these are two different topics.

Best regards,
Naeem Khoshnevis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20210714/39987440/attachment.html>
Dirk Eddelbuettel
# 
Hi Naeem,
On 14 July 2021 at 09:41, Naeem Khoshnevis wrote:
| Hi,
| 
| I hope this email finds you well.
| I wondered what the best approaches to troubleshooting C++ dependencies
| issues in general and specifically in R are; specifically, those packages
| which are using OpenMP and we want to install on macOS.

Is this dependencies (plural) or a single core dependency (singular) on a
single OS (macOS) ?  Those are different issues. (And Rcpp has no incluence
over what Cupertino (macOS) or Auckland (macOS deployment for R does) -- we
have to play the card we are dealt, and that have always been "issues" and
some HOWTOs and duct tape.  On the margin, if this is your question, it may
also be one for r-sig-mac.

Also, and to make this concrete: Do you have a minimally reproducible issue?

| So far, I am just googling the error and finding out what the problem is.
| However, it is hard to address issues and come up with long-term solutions
| without internalizing the concepts.
| One example is creating a conda environment for each R package project and
| installing llvm, clang, etc into each environment.

As an R developer, I have a simple rule: Just say No to conda. Mixing and
matching between standard / core builds (using the recommended R toolchain)
and conda appears to mostly end in tears.

| I talked to several software engineers; however, I could not find a
| convincing answer. I think I do not know what I do not know about this
| topic, which prevents me from asking good questions. I would be grateful if
| you could let me know your thoughts about these questions.

Again, a reproducible example would help.

Rcpp is used by over 2300 packages on CRAN. Many use OpenMP. You can search
at GitHub within the mirroring 'cran' organisation to see what other packages
do. Given that you can also see the outcome (the package 'status' pages at
CRAN) that is not a bad approach.

Dirk

| Where is a good place to start learning about this topic?
| Is a professional C++ developer necessarily good at troubleshooting
| problems with compilers and linkers? Or these are two different topics.
| 
| Best regards,
| Naeem Khoshnevis
| _______________________________________________
| Rcpp-devel mailing list
| Rcpp-devel at lists.r-forge.r-project.org
| https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Naeem Khoshnevis
# 
Hi Dirk,

Thank you so much for your response. Rcpp is excellent, and I am using it
(with OpenMP), and it significantly improved the package (Thank you!). I
have tried it on macOS and centos. I was trying to set up a conda
environment. There are many HPC clusters that are using old versions of R
as well as rstudio. In the past, using conda env helped me to install an
updated version of R., But I understand why you do not recommend it. I will
prepare a reproducible example and will let you know. Thank you so much.

Best regards,
Naeem
On Wed, Jul 14, 2021 at 9:55 AM Dirk Eddelbuettel <edd at debian.org> wrote:

            

      
      
      
    

        
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20210714/12ee23b4/attachment.html>

  
  


  


      

    

    

      
      

        


  

        

      Dirk Eddelbuettel
    

    

      
      #
    

  


  

      
Hi Naeem,

        
On 14 July 2021 at 11:03, Naeem Khoshnevis wrote:

        
| Thank you so much for your response. Rcpp is excellent, and I am using it
| (with OpenMP), and it significantly improved the package (Thank you!). I
| have tried it on macOS and centos.

Ok, as stated, I do not work on or know macOS. Others may be of help, there
are tutorials (often by @coatless) which got a lot people going. Give that a
shot, ask here when you get stuck.

CentOS can be iffy. When I had to use in the past the setup in the distro was
so old that we went the "devtoolset" (IIRC) route: a semi- or fully official
newer compiler set a few versions ahead of what is in the distro. That should
include OpenMP.

| I was trying to set up a conda
| environment. There are many HPC clusters that are using old versions of R
| as well as rstudio. In the past, using conda env helped me to install an

HPC can indeed be tricky. Best to liaise with your local sysadmins. One
difficulty once you load the 'modules' (or however the scheme is called at
your end) you have to align it with R. Use 'R CMD config --all' to see what R
uses, you be able to tweak via your ~/.R/Makevars

| updated version of R., But I understand why you do not recommend it. I will
| prepare a reproducible example and will let you know. Thank you so much.

Sounds good. Hopefully we get that squared away in a few controlled steps.

Dirk

| Best regards,
| Naeem
|

        
| On Wed, Jul 14, 2021 at 9:55 AM Dirk Eddelbuettel <edd at debian.org> wrote:

        
| 
| >
| > Hi Naeem,
| >

        
| > On 14 July 2021 at 09:41, Naeem Khoshnevis wrote:

        
| > | Hi,
| > |
| > | I hope this email finds you well.
| > | I wondered what the best approaches to troubleshooting C++ dependencies
| > | issues in general and specifically in R are; specifically, those packages
| > | which are using OpenMP and we want to install on macOS.
| >
| > Is this dependencies (plural) or a single core dependency (singular) on a
| > single OS (macOS) ?  Those are different issues. (And Rcpp has no incluence
| > over what Cupertino (macOS) or Auckland (macOS deployment for R does) -- we
| > have to play the card we are dealt, and that have always been "issues" and
| > some HOWTOs and duct tape.  On the margin, if this is your question, it may
| > also be one for r-sig-mac.
| >
| > Also, and to make this concrete: Do you have a minimally reproducible
| > issue?
| >
| > | So far, I am just googling the error and finding out what the problem is.
| > | However, it is hard to address issues and come up with long-term
| > solutions
| > | without internalizing the concepts.
| > | One example is creating a conda environment for each R package project
| > and
| > | installing llvm, clang, etc into each environment.
| >
| > As an R developer, I have a simple rule: Just say No to conda. Mixing and
| > matching between standard / core builds (using the recommended R toolchain)
| > and conda appears to mostly end in tears.
| >
| > | I talked to several software engineers; however, I could not find a
| > | convincing answer. I think I do not know what I do not know about this
| > | topic, which prevents me from asking good questions. I would be grateful
| > if
| > | you could let me know your thoughts about these questions.
| >
| > Again, a reproducible example would help.
| >
| > Rcpp is used by over 2300 packages on CRAN. Many use OpenMP. You can search
| > at GitHub within the mirroring 'cran' organisation to see what other
| > packages
| > do. Given that you can also see the outcome (the package 'status' pages at
| > CRAN) that is not a bad approach.
| >
| > Dirk
| >
| > | Where is a good place to start learning about this topic?
| > | Is a professional C++ developer necessarily good at troubleshooting
| > | problems with compilers and linkers? Or these are two different topics.
| > |
| > | Best regards,
| > | Naeem Khoshnevis
| > | _______________________________________________
| > | Rcpp-devel mailing list
| > | Rcpp-devel at lists.r-forge.r-project.org
| > | https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
| > --
| > https://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
| >

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Simon Urbanek
    

    

      
      #
    

  


  

      
Naeem,

I'm not sure which issues are you talking about specifically. If your package is well written, the instructions at https://mac.r-project.org/openmp/ should be sufficient with regular CRAN installation of R and standard tools.

Cheers,
Simon

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Naeem Khoshnevis
    

    

      
      #
    

  


  

      
Thanks, Simon and Dirk,

I can install the packages normally on macOS. However, I want to install it
in a conda environment. Thanks for sharing the link. I really appreciate
it.

Here is a reproducible example, in case you have time to try.

On macOS:

Step 1: create a conda environment and install R

conda create -n r-environment r-essentials r-base

Step 2: activate the environment

conda activate r-environment

Step 3: Install rstudio

conda install -c r rstudio

Step 4: Install some required packages

conda install -c r r-devtools
conda install -c r r-wcorr
conda install -c r r-ranger
conda install -c conda-forge r-rcpparmadillo
conda install -c r r-testthat
conda install -c conda-forge r-superlearner
conda install -c conda-forge r-polycor
conda install -c conda-forge r-logger

Step 6:
According to this page, download dependencies:
https://mac.r-project.org/openmp/

And copied them into the environment include and lib directory.


Step 7:
According to this page, I exported variables in step 6.
https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#set-env-vars

#!/bin/sh

export PKG_CPPFLAGS='-Xclang -fopenmp'
export PKG_LIBS='-lomp'

Step 8:
Reactivated the conda environment

Step 9:
run: rstudio

Step 10:
inside rstudio
library(devtools)
install_github("fasrc/CausalGPS")

I get the following error:

Downloading GitHub repo fasrc/CausalGPS at master
Error: Could not find tools necessary to compile a package

I think something is wrong with Step 7. Because apparently, I lost the path
to Xcode. Without Step 7, I get the following error:

In file included from ColorSpace.cpp:1:
In file included from ./ColorSpace.h:4:
In file included from env/bin/../include/c++/v1/typeinfo:60:
In file included from env/bin/../include/c++/v1/exception:81:
In file included from env/bin/../include/c++/v1/cstdlib:85:
In file included from env/bin/../include/c++/v1/stdlib.h:100:
env/bin/../include/c++/v1/math.h:773:12: error: no member named 'labs' in
the global namespace; did you mean 'abs'?
  return ::labs(__x);
         ~~^
~/env/bin/../include/c++/v1/math.h:772:39: note: 'abs' declared here
inline _LIBCPP_INLINE_VISIBILITY long abs(long __x) _NOEXCEPT {
                                      ^
~/env/bin/../include/c++/v1/math.h:777:12: error: no member named 'llabs'
in the global namespace
  return ::llabs(__x);
         ~~^
~/env/bin/../include/c++/v1/math.h:785:12: error: no member named 'fabsf'
in the global namespace
  return ::fabsf(__lcpp_x);
         ~~^
~/env/bin/../include/c++/v1/math.h:789:12: error: no member named 'fabs' in
the global namespace; did you mean 'abs'?
  return ::fabs(__lcpp_x);
         ~~^
~/env/bin/../include/c++/v1/math.h:772:39: note: 'abs' declared here
inline _LIBCPP_INLINE_VISIBILITY long abs(long __x) _NOEXCEPT {
                                      ^
~/env/bin/../include/c++/v1/math.h:794:12: error: no member named 'fabsl'
in the global namespace
  return ::fabsl(__lcpp_x);
         ~~^

Best regards,
Naeem

On Thu, Jul 15, 2021 at 3:25 AM Simon Urbanek <simon.urbanek at r-project.org>
wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20210715/aef00e0d/attachment-0001.html>

  
  


  


      

    

    

      
      

        


  

        

      Dirk Eddelbuettel
    

    

      
      #
    

  


  

      
On 15 July 2021 at 15:44, Naeem Khoshnevis wrote:

        
| Thanks, Simon and Dirk,
| 
| I can install the packages normally on macOS. However, I want to install it
| in a conda environment.

That is (for all intents and purposes) a different use case than what we
support (for free) so I suggest you rely on the Conda volunteers (or
commercial supporters) to help you.

We support software on CRAN, which tends to "just work" if used as intended.

But we are volunteers in this, so there are limits to what we can cover.

Dirk

| Thanks for sharing the link. I really appreciate
| it.
| 
| Here is a reproducible example, in case you have time to try.
| 
| On macOS:
| 
| Step 1: create a conda environment and install R
| 
| conda create -n r-environment r-essentials r-base
| 
| Step 2: activate the environment
| 
| conda activate r-environment
| 
| Step 3: Install rstudio
| 
| conda install -c r rstudio
| 
| Step 4: Install some required packages
| 
| conda install -c r r-devtools
| conda install -c r r-wcorr
| conda install -c r r-ranger
| conda install -c conda-forge r-rcpparmadillo
| conda install -c r r-testthat
| conda install -c conda-forge r-superlearner
| conda install -c conda-forge r-polycor
| conda install -c conda-forge r-logger
| 
| Step 6:
| According to this page, download dependencies:
| https://mac.r-project.org/openmp/
| 
| And copied them into the environment include and lib directory.
| 
| 
| Step 7:
| According to this page, I exported variables in step 6.
| https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#set-env-vars
| 
| #!/bin/sh
| 
| export PKG_CPPFLAGS='-Xclang -fopenmp'
| export PKG_LIBS='-lomp'
| 
| Step 8:
| Reactivated the conda environment
| 
| Step 9:
| run: rstudio
| 
| Step 10:
| inside rstudio
| library(devtools)
| install_github("fasrc/CausalGPS")
| 
| I get the following error:
| 
| Downloading GitHub repo fasrc/CausalGPS at master
| Error: Could not find tools necessary to compile a package
| 
| I think something is wrong with Step 7. Because apparently, I lost the path
| to Xcode. Without Step 7, I get the following error:
| 
| In file included from ColorSpace.cpp:1:
| In file included from ./ColorSpace.h:4:
| In file included from env/bin/../include/c++/v1/typeinfo:60:
| In file included from env/bin/../include/c++/v1/exception:81:
| In file included from env/bin/../include/c++/v1/cstdlib:85:
| In file included from env/bin/../include/c++/v1/stdlib.h:100:
| env/bin/../include/c++/v1/math.h:773:12: error: no member named 'labs' in
| the global namespace; did you mean 'abs'?
|   return ::labs(__x);
|          ~~^
| ~/env/bin/../include/c++/v1/math.h:772:39: note: 'abs' declared here
| inline _LIBCPP_INLINE_VISIBILITY long abs(long __x) _NOEXCEPT {
|                                       ^
| ~/env/bin/../include/c++/v1/math.h:777:12: error: no member named 'llabs'
| in the global namespace
|   return ::llabs(__x);
|          ~~^
| ~/env/bin/../include/c++/v1/math.h:785:12: error: no member named 'fabsf'
| in the global namespace
|   return ::fabsf(__lcpp_x);
|          ~~^
| ~/env/bin/../include/c++/v1/math.h:789:12: error: no member named 'fabs' in
| the global namespace; did you mean 'abs'?
|   return ::fabs(__lcpp_x);
|          ~~^
| ~/env/bin/../include/c++/v1/math.h:772:39: note: 'abs' declared here
| inline _LIBCPP_INLINE_VISIBILITY long abs(long __x) _NOEXCEPT {
|                                       ^
| ~/env/bin/../include/c++/v1/math.h:794:12: error: no member named 'fabsl'
| in the global namespace
|   return ::fabsl(__lcpp_x);
|          ~~^
| 
| Best regards,
| Naeem
| 
| On Thu, Jul 15, 2021 at 3:25 AM Simon Urbanek <simon.urbanek at r-project.org>

        
| wrote:

        
| 
| > Naeem,
| >
| > I'm not sure which issues are you talking about specifically. If your
| > package is well written, the instructions at
| > https://mac.r-project.org/openmp/ should be sufficient with regular CRAN
| > installation of R and standard tools.
| >
| > Cheers,
| > Simon
| >
| >
| >
| > > On Jul 15, 2021, at 02:41, Naeem Khoshnevis <khoshnevis.naeem at gmail.com>

        
| > wrote:

        
| > >
| > > Hi,
| > >
| > > I hope this email finds you well.
| > > I wondered what the best approaches to troubleshooting C++ dependencies
| > issues in general and specifically in R are; specifically, those packages
| > which are using OpenMP and we want to install on macOS.
| > > So far, I am just googling the error and finding out what the problem
| > is. However, it is hard to address issues and come up with long-term
| > solutions without internalizing the concepts.
| > > One example is creating a conda environment for each R package project
| > and installing llvm, clang, etc into each environment.
| > > I talked to several software engineers; however, I could not find a
| > convincing answer. I think I do not know what I do not know about this
| > topic, which prevents me from asking good questions. I would be grateful if
| > you could let me know your thoughts about these questions.
| > >
| > > Where is a good place to start learning about this topic?
| > > Is a professional C++ developer necessarily good at troubleshooting
| > problems with compilers and linkers? Or these are two different topics.
| > >
| > > Best regards,
| > > Naeem Khoshnevis
| > > _______________________________________________
| > > Rcpp-devel mailing list
| > > Rcpp-devel at lists.r-forge.r-project.org
| > > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
| >
| >
| _______________________________________________
| Rcpp-devel mailing list
| Rcpp-devel at lists.r-forge.r-project.org
| https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel