Skip to content

[Bioc-devel] Build error for CaDrA package on kunpeng2 Linux machine

10 messages · Hervé Pagès, Martin Grigorov, Chau, Reina

Original
Chau, Reina
# 
Hi Bioconductor Core Team,

I?m the maintainer of CaDrA package, and recently, I notice that my package built successfully for all platforms except on Kunpeng2 Linux machine (see https://bioconductor.org/checkResults/devel/bioc-LATEST/CaDrA/ and https://bioconductor.org/checkResults/devel/bioc-LATEST/CaDrA/kunpeng2-checksrc.html)

I did try to figure out what is causing the different results using this particular platform compare with others but with no resolves.

Hope you can give me some guidance or feedback of why the build is failing for this particular case.

Thanks so much!

Best,

Reina C.
Hervé Pagès
# 
Hi Reina,

Note that CaDrA results on Mac ARM64 are also affected: 
https://bioconductor.org/checkResults/devel/bioc-mac-arm64-LATEST/CaDrA/

See Martin Grigorov's blog post here 
https://blog.bioconductor.org/posts/2023-06-09-debug-linux-arm64-on-docker/ 
for how to debug Linux ARM64 related issues on a x86_64 host.

Note that different architectures use slightly different floating point 
arithmetic with slightly different precision. This can affect the 
results of your numerical calculations. The degree to which they are 
affected will vary greatly depending on what calculations you perform 
and how you perform them. So it's important to implement things in a way 
that tries to minimize the impact of these variations as much as possible.

Best,

H.
On 10/18/23 10:46, Chau, Reina wrote:

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Martin Grigorov
    

    

      
      #
    

  


  

      
Hi,

On Wed, Oct 18, 2023 at 10:48?PM Herv? Pag?s <hpages.on.github at gmail.com>
wrote:

            

      
      
      
    

        
See https://go.dev/ref/spec#Floating_point_operators for some further
explanation of the problem and possible fix by explicit rounding (in
Golang, but I guess something similar happens in R)

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Chau, Reina
    

    

      
      #
    

  


  

      
Thank you both! I will look thru the links and see if I can implement a fix.

Best,

Reina C.

        
On Oct 18, 2023, at 4:03 PM, Martin Grigorov <martin.grigorov at gmail.com> wrote:

        
Hi,

        
On Wed, Oct 18, 2023 at 10:48?PM Herv? Pag?s <hpages.on.github at gmail.com<mailto:hpages.on.github at gmail.com>> wrote:

        
Hi Reina,

Note that CaDrA results on Mac ARM64 are also affected:
https://bioconductor.org/checkResults/devel/bioc-mac-arm64-LATEST/CaDrA/

See Martin Grigorov's blog post here
https://blog.bioconductor.org/posts/2023-06-09-debug-linux-arm64-on-docker/
for how to debug Linux ARM64 related issues on a x86_64 host.

Note that different architectures use slightly different floating point
arithmetic with slightly different precision. This can affect the
results of your numerical calculations. The degree to which they are
affected will vary greatly depending on what calculations you perform
and how you perform them. So it's important to implement things in a way
that tries to minimize the impact of these variations as much as possible.

See https://go.dev/ref/spec#Floating_point_operators for some further explanation of the problem and possible fix by explicit rounding (in Golang, but I guess something similar happens in R)


Best,

H.

        
On 10/18/23 10:46, Chau, Reina wrote:

            

      
      
      
    

        

      
      
    

        
--
Herv? Pag?s

Bioconductor Core Team
hpages.on.github at gmail.com<mailto:hpages.on.github at gmail.com>


_______________________________________________
Bioc-devel at r-project.org<mailto:Bioc-devel at r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

  
  


  


      

    

    

      
      

        


  

        

      Martin Grigorov
    

    

      
      #
    

  


  

      
On Wed, Oct 18, 2023 at 11:11?PM Chau, Reina <rchau88 at bu.edu> wrote:

        

            

      
      
      
    

        
I'd be happy to test any patches if it will be easier for you !

Martin

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Chau, Reina
    

    

      
      #
    

  


  

      
Hi Martin,

That would be very helpful. Thanks so much! Right now, it is failing in one of the unit test cases. In particular, our core function, CaDrA::CaDrA, which is used to run permutation-based tests to determine whether the strength of the association between the set of features and the observed input scores (e.g., pathway activity, drug sensitivity, etc.) is greater than it would be expected by chance.

Here is a snippet of where the test failed in Mac and Linux ARM64 machine:

# R library
library(CaDrA)
library(testthat)

data("sim_FS?)
data("sim_Scores")

# Set seed
set.seed(21)

# ks_score
result <- CaDrA::CaDrA(
    FS = sim_FS,
    input_score = sim_Scores,
    method = "ks_score",
    weights = NULL,
    alternative = "less",
    top_N = 1,
    search_start = NULL,
    search_method = "both",
    max_size = 7,
    n_perm = 10,
    plot = FALSE,
    smooth = TRUE,
    obs_best_score = NULL,
    ncores = 1,
    cache = FALSE,
    cache_path = NULL
)

testthat::expect_length(result, 4L)
testthat::expect_type(result, "list")
testthat::expect_named(result,
                             c("key","perm_best_scores","obs_best_score","perm_pval"))
testthat::expect_type(result$key, "list")
testthat::expect_length(result$key, 11L)
testthat::expect_named(result$key,
                             c("FS", "input_score", "method", "custom_function",
                               "custom_parameters", "alternative", "weights", "top_N",
                               "search_start", "search_method", "max_size"))
testthat::expect_type(result$key$FS, "double")

testthat::expect_equal(round(result$perm_best_scores[1:10], 5),
                         c('TP_8'=0.34,
                           'TP_10'=0.54,
                           'TP_9'=0.37,
                           'TP_6'=0.40,
                           'TP_9'=0.38,
                           'TP_9'=0.52,
                           'TP_2'=0.44,
                           'TP_2'=0.40,
                           'TP_4'=0.44,
                           'TP_9'=0.49))

testthat::expect_equal(round(result$obs_best_score, 2), c("TP_9"=0.66))

# A smooth factor of 1
c <- 1

# Add a smoothing factor of 1
# This is just to not return a p-value of 0
testthat::expect_equal(
    round((sum(result$perm_best_scores[1:10] > result$obs_best_score)+c)/(10+c), 6),
    c(0.090909)
)

Any helps or directions on how to resolve this problem would be be greatly appreciated!

Best,

Reina C.

        
On Oct 18, 2023, at 4:35 PM, Martin Grigorov <martin.grigorov at gmail.com> wrote:

        

        
On Wed, Oct 18, 2023 at 11:11?PM Chau, Reina <rchau88 at bu.edu<mailto:rchau88 at bu.edu>> wrote:

        
Thank you both! I will look thru the links and see if I can implement a fix.

I'd be happy to test any patches if it will be easier for you !

Martin



Best,

Reina C.

        
On Oct 18, 2023, at 4:03 PM, Martin Grigorov <martin.grigorov at gmail.com<mailto:martin.grigorov at gmail.com>> wrote:

        
Hi,

        
On Wed, Oct 18, 2023 at 10:48?PM Herv? Pag?s <hpages.on.github at gmail.com<mailto:hpages.on.github at gmail.com>> wrote:

        
Hi Reina,

Note that CaDrA results on Mac ARM64 are also affected:
https://bioconductor.org/checkResults/devel/bioc-mac-arm64-LATEST/CaDrA/

See Martin Grigorov's blog post here
https://blog.bioconductor.org/posts/2023-06-09-debug-linux-arm64-on-docker/
for how to debug Linux ARM64 related issues on a x86_64 host.

Note that different architectures use slightly different floating point
arithmetic with slightly different precision. This can affect the
results of your numerical calculations. The degree to which they are
affected will vary greatly depending on what calculations you perform
and how you perform them. So it's important to implement things in a way
that tries to minimize the impact of these variations as much as possible.

See https://go.dev/ref/spec#Floating_point_operators for some further explanation of the problem and possible fix by explicit rounding (in Golang, but I guess something similar happens in R)


Best,

H.

        
On 10/18/23 10:46, Chau, Reina wrote:

            

      
      
      
    

        

      
      
    

        
--
Herv? Pag?s

Bioconductor Core Team
hpages.on.github at gmail.com<mailto:hpages.on.github at gmail.com>


_______________________________________________
Bioc-devel at r-project.org<mailto:Bioc-devel at r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

  
  


  


      

    

    

      
      

        


  

        

      Martin Grigorov
    

    

      
      #
    

  


  

      
Hello Reina,

The checks pass now after your last commit:

```
...
* checking for unstated dependencies in ?tests? ... OK
* checking tests ...
  Running ?testthat.R?
 OK
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes in ?inst/doc? ... OK
* checking running R code from vignettes ...
  ?permutation_based_testing.Rmd? using ?UTF-8?... OK
  ?scoring_functions.Rmd? using ?UTF-8?... OK
 NONE
* checking re-building of vignette outputs ... OK
* checking PDF version of manual ... OK
* DONE

Status: OK
```

biocbuild at kunpeng2 ~/g/CaDrA (devel)> git log -1
commit ae6513e0877d5866fc8240eb123b8bc83af73e19 (HEAD -> devel,
origin/master, origin/devel, origin/HEAD)
Author: RC-88 <lilychau999 at gmail.com>
Date:   Wed Oct 18 16:14:42 2023 -0400

    remove testing for ks_score method due to inconsistent results on linux
and macOS amd64


I will try to debug the disabled code later !

Martin

        
On Thu, Oct 19, 2023 at 12:21?AM Chau, Reina <rchau88 at bu.edu> wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Chau, Reina
    

    

      
      #
    

  


  

      
Hi Martin,

Yes, I removed that specified test case that failed the check in order to meet Bioconductor 3.18 release deadline.

Thanks again for looking into this!

Best,

Reina C.

        
On Oct 19, 2023, at 4:20 AM, Martin Grigorov <martin.grigorov at gmail.com> wrote:

        
Hello Reina,

The checks pass now after your last commit:

```
...
* checking for unstated dependencies in ?tests? ... OK
* checking tests ...
  Running ?testthat.R?
 OK
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes in ?inst/doc? ... OK
* checking running R code from vignettes ...
  ?permutation_based_testing.Rmd? using ?UTF-8?... OK
  ?scoring_functions.Rmd? using ?UTF-8?... OK
 NONE
* checking re-building of vignette outputs ... OK
* checking PDF version of manual ... OK
* DONE

Status: OK
```

biocbuild at kunpeng2 ~/g/CaDrA (devel)> git log -1
commit ae6513e0877d5866fc8240eb123b8bc83af73e19 (HEAD -> devel, origin/master, origin/devel, origin/HEAD)
Author: RC-88 <lilychau999 at gmail.com<mailto:lilychau999 at gmail.com>>
Date:   Wed Oct 18 16:14:42 2023 -0400

    remove testing for ks_score method due to inconsistent results on linux and macOS amd64


I will try to debug the disabled code later !

Martin

        
On Thu, Oct 19, 2023 at 12:21?AM Chau, Reina <rchau88 at bu.edu<mailto:rchau88 at bu.edu>> wrote:

        
Hi Martin,

That would be very helpful. Thanks so much! Right now, it is failing in one of the unit test cases. In particular, our core function, CaDrA::CaDrA, which is used to run permutation-based tests to determine whether the strength of the association between the set of features and the observed input scores (e.g., pathway activity, drug sensitivity, etc.) is greater than it would be expected by chance.

Here is a snippet of where the test failed in Mac and Linux ARM64 machine:

# R library
library(CaDrA)
library(testthat)

data("sim_FS?)
data("sim_Scores")

# Set seed
set.seed(21)

# ks_score
result <- CaDrA::CaDrA(
    FS = sim_FS,
    input_score = sim_Scores,
    method = "ks_score",
    weights = NULL,
    alternative = "less",
    top_N = 1,
    search_start = NULL,
    search_method = "both",
    max_size = 7,
    n_perm = 10,
    plot = FALSE,
    smooth = TRUE,
    obs_best_score = NULL,
    ncores = 1,
    cache = FALSE,
    cache_path = NULL
)

testthat::expect_length(result, 4L)
testthat::expect_type(result, "list")
testthat::expect_named(result,
                             c("key","perm_best_scores","obs_best_score","perm_pval"))
testthat::expect_type(result$key, "list")
testthat::expect_length(result$key, 11L)
testthat::expect_named(result$key,
                             c("FS", "input_score", "method", "custom_function",
                               "custom_parameters", "alternative", "weights", "top_N",
                               "search_start", "search_method", "max_size"))
testthat::expect_type(result$key$FS, "double")

testthat::expect_equal(round(result$perm_best_scores[1:10], 5),
                         c('TP_8'=0.34,
                           'TP_10'=0.54,
                           'TP_9'=0.37,
                           'TP_6'=0.40,
                           'TP_9'=0.38,
                           'TP_9'=0.52,
                           'TP_2'=0.44,
                           'TP_2'=0.40,
                           'TP_4'=0.44,
                           'TP_9'=0.49))

testthat::expect_equal(round(result$obs_best_score, 2), c("TP_9"=0.66))

# A smooth factor of 1
c <- 1

# Add a smoothing factor of 1
# This is just to not return a p-value of 0
testthat::expect_equal(
    round((sum(result$perm_best_scores[1:10] > result$obs_best_score)+c)/(10+c), 6),
    c(0.090909)
)

Any helps or directions on how to resolve this problem would be be greatly appreciated!

Best,

Reina C.

        
On Oct 18, 2023, at 4:35 PM, Martin Grigorov <martin.grigorov at gmail.com<mailto:martin.grigorov at gmail.com>> wrote:

        

        
On Wed, Oct 18, 2023 at 11:11?PM Chau, Reina <rchau88 at bu.edu<mailto:rchau88 at bu.edu>> wrote:

        
Thank you both! I will look thru the links and see if I can implement a fix.

I'd be happy to test any patches if it will be easier for you !

Martin



Best,

Reina C.

        
On Oct 18, 2023, at 4:03 PM, Martin Grigorov <martin.grigorov at gmail.com<mailto:martin.grigorov at gmail.com>> wrote:

        
Hi,

        
On Wed, Oct 18, 2023 at 10:48?PM Herv? Pag?s <hpages.on.github at gmail.com<mailto:hpages.on.github at gmail.com>> wrote:

        
Hi Reina,

Note that CaDrA results on Mac ARM64 are also affected:
https://bioconductor.org/checkResults/devel/bioc-mac-arm64-LATEST/CaDrA/

See Martin Grigorov's blog post here
https://blog.bioconductor.org/posts/2023-06-09-debug-linux-arm64-on-docker/
for how to debug Linux ARM64 related issues on a x86_64 host.

Note that different architectures use slightly different floating point
arithmetic with slightly different precision. This can affect the
results of your numerical calculations. The degree to which they are
affected will vary greatly depending on what calculations you perform
and how you perform them. So it's important to implement things in a way
that tries to minimize the impact of these variations as much as possible.

See https://go.dev/ref/spec#Floating_point_operators for some further explanation of the problem and possible fix by explicit rounding (in Golang, but I guess something similar happens in R)


Best,

H.

        
On 10/18/23 10:46, Chau, Reina wrote:

            

      
      
      
    

        

      
      
    

        
--
Herv? Pag?s

Bioconductor Core Team
hpages.on.github at gmail.com<mailto:hpages.on.github at gmail.com>


_______________________________________________
Bioc-devel at r-project.org<mailto:Bioc-devel at r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

  
  


  


      

    

    

      
      

        


  

        

      Martin Grigorov
    

    

      
      #
    

  


  

      
Hi Reina,

I tried to debug it but without success!
I think the problem is at
https://code.bioconductor.org/browse/CaDrA/blob/devel/src/score_fun.c#L613
because it fails only with method = "ks_score"
It does a lot of calculations with doubles and I am almost sure FMA [1] is
to blame.
You could try to replace the "manual" calculations with fma() [2] or
disable it at compile time

1. https://accurate-algorithms.readthedocs.io/en/latest/ch03fma.html
2. https://www.oreilly.com/library/view/c-in-a/0596006977/re80.html

Martin

        
On Thu, Oct 19, 2023 at 4:49?PM Chau, Reina <rchau88 at bu.edu> wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Chau, Reina
    

    

      
      #
    

  


  

      
Hi Martin,

Agreed that FMA [1] could be the cause. Anyhow, I had disabled that specific unit-test. That is the quickest fix right now to meet the Bioconductor 3.18 release date. I will try to revisit the C code that you mentioned and see if I can fix the problem.

Thanks so much for your help!

Best,

Reina C.

        
On Oct 20, 2023, at 8:51 AM, Martin Grigorov <martin.grigorov at gmail.com> wrote:

        
Hi Reina,

I tried to debug it but without success!
I think the problem is at https://code.bioconductor.org/browse/CaDrA/blob/devel/src/score_fun.c#L613 because it fails only with method = "ks_score"
It does a lot of calculations with doubles and I am almost sure FMA [1] is to blame.
You could try to replace the "manual" calculations with fma() [2] or disable it at compile time

1. https://accurate-algorithms.readthedocs.io/en/latest/ch03fma.html
2. https://www.oreilly.com/library/view/c-in-a/0596006977/re80.html

Martin

        
On Thu, Oct 19, 2023 at 4:49?PM Chau, Reina <rchau88 at bu.edu<mailto:rchau88 at bu.edu>> wrote:

        
Hi Martin,

Yes, I removed that specified test case that failed the check in order to meet Bioconductor 3.18 release deadline.

Thanks again for looking into this!

Best,

Reina C.

        
On Oct 19, 2023, at 4:20 AM, Martin Grigorov <martin.grigorov at gmail.com<mailto:martin.grigorov at gmail.com>> wrote:

        
Hello Reina,

The checks pass now after your last commit:

```
...
* checking for unstated dependencies in ?tests? ... OK
* checking tests ...
  Running ?testthat.R?
 OK
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes in ?inst/doc? ... OK
* checking running R code from vignettes ...
  ?permutation_based_testing.Rmd? using ?UTF-8?... OK
  ?scoring_functions.Rmd? using ?UTF-8?... OK
 NONE
* checking re-building of vignette outputs ... OK
* checking PDF version of manual ... OK
* DONE

Status: OK
```

biocbuild at kunpeng2 ~/g/CaDrA (devel)> git log -1
commit ae6513e0877d5866fc8240eb123b8bc83af73e19 (HEAD -> devel, origin/master, origin/devel, origin/HEAD)
Author: RC-88 <lilychau999 at gmail.com<mailto:lilychau999 at gmail.com>>
Date:   Wed Oct 18 16:14:42 2023 -0400

    remove testing for ks_score method due to inconsistent results on linux and macOS amd64


I will try to debug the disabled code later !

Martin

        
On Thu, Oct 19, 2023 at 12:21?AM Chau, Reina <rchau88 at bu.edu<mailto:rchau88 at bu.edu>> wrote:

        
Hi Martin,

That would be very helpful. Thanks so much! Right now, it is failing in one of the unit test cases. In particular, our core function, CaDrA::CaDrA, which is used to run permutation-based tests to determine whether the strength of the association between the set of features and the observed input scores (e.g., pathway activity, drug sensitivity, etc.) is greater than it would be expected by chance.

Here is a snippet of where the test failed in Mac and Linux ARM64 machine:

# R library
library(CaDrA)
library(testthat)

data("sim_FS?)
data("sim_Scores")

# Set seed
set.seed(21)

# ks_score
result <- CaDrA::CaDrA(
    FS = sim_FS,
    input_score = sim_Scores,
    method = "ks_score",
    weights = NULL,
    alternative = "less",
    top_N = 1,
    search_start = NULL,
    search_method = "both",
    max_size = 7,
    n_perm = 10,
    plot = FALSE,
    smooth = TRUE,
    obs_best_score = NULL,
    ncores = 1,
    cache = FALSE,
    cache_path = NULL
)

testthat::expect_length(result, 4L)
testthat::expect_type(result, "list")
testthat::expect_named(result,
                             c("key","perm_best_scores","obs_best_score","perm_pval"))
testthat::expect_type(result$key, "list")
testthat::expect_length(result$key, 11L)
testthat::expect_named(result$key,
                             c("FS", "input_score", "method", "custom_function",
                               "custom_parameters", "alternative", "weights", "top_N",
                               "search_start", "search_method", "max_size"))
testthat::expect_type(result$key$FS, "double")

testthat::expect_equal(round(result$perm_best_scores[1:10], 5),
                         c('TP_8'=0.34,
                           'TP_10'=0.54,
                           'TP_9'=0.37,
                           'TP_6'=0.40,
                           'TP_9'=0.38,
                           'TP_9'=0.52,
                           'TP_2'=0.44,
                           'TP_2'=0.40,
                           'TP_4'=0.44,
                           'TP_9'=0.49))

testthat::expect_equal(round(result$obs_best_score, 2), c("TP_9"=0.66))

# A smooth factor of 1
c <- 1

# Add a smoothing factor of 1
# This is just to not return a p-value of 0
testthat::expect_equal(
    round((sum(result$perm_best_scores[1:10] > result$obs_best_score)+c)/(10+c), 6),
    c(0.090909)
)

Any helps or directions on how to resolve this problem would be be greatly appreciated!

Best,

Reina C.

        
On Oct 18, 2023, at 4:35 PM, Martin Grigorov <martin.grigorov at gmail.com<mailto:martin.grigorov at gmail.com>> wrote:

        

        
On Wed, Oct 18, 2023 at 11:11?PM Chau, Reina <rchau88 at bu.edu<mailto:rchau88 at bu.edu>> wrote:

        
Thank you both! I will look thru the links and see if I can implement a fix.

I'd be happy to test any patches if it will be easier for you !

Martin



Best,

Reina C.

        
On Oct 18, 2023, at 4:03 PM, Martin Grigorov <martin.grigorov at gmail.com<mailto:martin.grigorov at gmail.com>> wrote:

        
Hi,

        
On Wed, Oct 18, 2023 at 10:48?PM Herv? Pag?s <hpages.on.github at gmail.com<mailto:hpages.on.github at gmail.com>> wrote:

        
Hi Reina,

Note that CaDrA results on Mac ARM64 are also affected:
https://bioconductor.org/checkResults/devel/bioc-mac-arm64-LATEST/CaDrA/

See Martin Grigorov's blog post here
https://blog.bioconductor.org/posts/2023-06-09-debug-linux-arm64-on-docker/
for how to debug Linux ARM64 related issues on a x86_64 host.

Note that different architectures use slightly different floating point
arithmetic with slightly different precision. This can affect the
results of your numerical calculations. The degree to which they are
affected will vary greatly depending on what calculations you perform
and how you perform them. So it's important to implement things in a way
that tries to minimize the impact of these variations as much as possible.

See https://go.dev/ref/spec#Floating_point_operators for some further explanation of the problem and possible fix by explicit rounding (in Golang, but I guess something similar happens in R)


Best,

H.

        
On 10/18/23 10:46, Chau, Reina wrote:

            

      
      
      
    

        

      
      
    

        
--
Herv? Pag?s

Bioconductor Core Team
hpages.on.github at gmail.com<mailto:hpages.on.github at gmail.com>


_______________________________________________
Bioc-devel at r-project.org<mailto:Bioc-devel at r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel