[Bioc-devel] Cannot reproduce "TeX capacity exceeded" error on build hosts (package messina)

2 messages · Mark Pinese, Hervé Pagès

Thu, May 13, 2021 4:15 PM #

Thank you both, with your help I managed to reproduce the error, and it's now fixed.

If anyone else runs into a similar issue:

  1.  The core issue was due to a deprecation warning (specifically regarding survival::survConcordance.fit). During vignette generation a very large number of deprecation warnings were thrown, which overloaded TeX. Changing the survival::survConcordance.fit call to the updated survival::concordancefit fixed the problem.
  2.  The cause of the reproducibility issue is less clear. I initially tested using Ubuntu 20.04 LTS running on Windows through WSL, which seemed to work. However, subsequent tests on a true Ubuntu 20.04 Linux machine failed. My tentative conclusion is that Ubuntu 20.04 LTS running through WSL is not the most reliable build host for BioC packages.

Best wishes,

Mark

From: Kern, Lori <Lori.Shepherd at RoswellPark.org>
Sent: Wednesday, 21 April 2021 9:31 PM
To: Herv? Pag?s <hpages.on.github at gmail.com>; Mark Pinese <MPinese at ccia.org.au>; bioc-devel at r-project.org <bioc-devel at r-project.org>
Subject: Re: [Bioc-devel] Cannot reproduce "TeX capacity exceeded" error on build hosts (package messina)

EXTERNAL EMAIL: Think before you click. If suspicious send to CyberReport at ccia.org.au.

I can also reproduce the error locally on my laptop Ubuntu 18.04.5,  R version 4.0.5 Patched (2021-03-31 r80152),   Bioc 3.12.

Lori Shepherd

Bioconductor Core Team

Roswell Park Comprehensive Cancer Center

Department of Biostatistics & Bioinformatics

Elm & Carlton Streets

Buffalo, New York 14263

________________________________
From: Bioc-devel <bioc-devel-bounces at r-project.org> on behalf of Herv? Pag?s <hpages.on.github at gmail.com>
Sent: Wednesday, April 21, 2021 2:00 AM
To: Mark Pinese <MPinese at ccia.org.au>; bioc-devel at r-project.org <bioc-devel at r-project.org>
Subject: Re: [Bioc-devel] Cannot reproduce "TeX capacity exceeded" error on build hosts (package messina)

Hi Mark,

I can reproduce this on my laptop (Ubuntu 20.10, R 4.0.3, BioC 3.12):

hpages at spectre:~/git.bioconductor.org/software$ time brelbuild messina
* checking for file ?messina/DESCRIPTION? ... OK
* preparing ?messina?:
* checking DESCRIPTION meta-information ... OK
* cleaning src
* installing the package to build vignettes
* creating vignettes ... ERROR
--- re-building ?messina.Rnw? using knitr
Loading required package: survival
Loading required package: Biobase
Loading required package: BiocGenerics
Loading required package: parallel

Attaching package: 'BiocGenerics'

The following objects are masked from 'package:parallel':

     clusterApply, clusterApplyLB, clusterCall, clusterEvalQ,
     clusterExport, clusterMap, parApply, parCapply, parLapply,
     parLapplyLB, parRapply, parSapply, parSapplyLB

The following objects are masked from 'package:stats':

     IQR, mad, sd, var, xtabs

The following objects are masked from 'package:base':

     Filter, Find, Map, Position, Reduce, anyDuplicated, append,
     as.data.frame, basename, cbind, colnames, dirname, do.call,
     duplicated, eval, evalq, get, grep, grepl, intersect,
     is.unsorted, lapply, mapply, match, mget, order, paste, pmax,
     pmax.int, pmin, pmin.int, rank, rbind, rownames, sapply,
     setdiff, sort, table, tapply, union, unique, unsplit, which.max,
     which.min

Welcome to Bioconductor

     Vignettes contain introductory material; view with
     'browseVignettes()'. To cite Bioconductor, see
     'citation("Biobase")', and for packages 'citation("pkgname")'.

Error: processing vignette 'messina.Rnw' failed with diagnostics:
Running 'texi2dvi' on 'messina.tex' failed.
LaTeX errors:
! TeX capacity exceeded, sorry [main memory size=5000000].
l.32169

If you really absolutely need more capacity,
you can ask a wizard to enlarge me.
!  ==> Fatal error occurred, no output PDF file produced!
--- failed re-building ?messina.Rnw?

SUMMARY: processing the following file failed:
   ?messina.Rnw?

Error: Vignette re-building failed.
Execution halted

real    2m40.464s
user    2m38.072s
sys     0m1.776s

I'm surprised you can't reproduce this. What platform are you on?

My understanding is that the Docker images were intentionally kept light
so this is why they don't have TeX/LaTeX and other things needed to run
'R CMD build' on a package.

Since we see the error in release and devel, I suspected a change in a
CRAN package involved in the vignette knitting process. I tried to
downgrade knitr from 1.32 to 1.31 but that didn't solve the problem.
Then I downgraded xfun from 0.22 to 0.21 but that didn't help either.
So I gave up.

Sorry for not being able to help more.

Best,
H.

On 4/20/21 6:17 PM, Mark Pinese wrote:
> Hi all,
>
> My package messina has recently started failing on all build platforms, eg https://secure-web.cisco.com/1iNAEmxW_NMxwnEIs_OdNqw1U28YG6RTA_d4Qj80AB8xPF2WAj4JoGuuPYrFJsE5BRToE2jh0rPYFN3EPb7NkzShrBmIn6skNzneNzD1t63RBm10h01pMeahxJeDOXmWqmbDHabJeSFu5yOPzAiUeH75sAzyo1Sy9L1Zc5fRhsDmxP1ZBn0nmV4QJRWV2wsxqN_B7HXaghk_26sDQGBvy3JvXscFsr3sNFAnUZwdohPI4pP1XyzHmLxK2Oi_g0nLd70J_KnlyS9wz6hSgAYY1VSEiZgTV_FKqnYIkoJYfAfUPNnL8vktDPqLr3dCwMtOv/https%3A%2F%2Fmaster.bioconductor.org%2FcheckResults%2F3.12%2Fbioc-LATEST%2Fmessina%2Fmalbec1-buildsrc.html This seems due to a "TeX capacity exceeded" error, though I suspect that something upstream is the true culprit.
>
> To debug I have tried to reproduce this locally with BioC 3.12 on R 4.0.3 (both clean installs), but I cannot reproduce the problem. Locally the build works without issues for me:
>
> source ~/.Renv_biocbuild
> R CMD build ~/repos/messina/
> * checking for file ?~repos/messina/DESCRIPTION? ... OK
> * preparing ?messina?:
> * checking DESCRIPTION meta-information ... OK
> * cleaning src
> * installing the package to build vignettes
> * creating vignettes ... OK
> * cleaning src
> * checking for LF line-endings in source and make files and shell scripts
> * checking for empty or unneeded directories
> * looking to see if a ?data/datalist? file should be added
> * building ?messina_1.23.1.tar.gz?
>
> Can anyone offer suggestions on how to reproduce the error as seen on the build environments? Something like a docker image matching the build environments would be ideal. I tried the image bioconductor/bioconductor_docker:devel, but I needed to install quite a few build dependencies, so I suspect this image doesn't match the build envs.
>
>
> Many thanks,
>
> Mark
>
>
> Mark Pinese
>
> Senior Bioinformatics Research Officer
> Conjoint Senior Lecturer, School of Women?s and Children?s Health, UNSW Medicine
> NHMRC Emerging Leadership Fellow
>
> Children's Cancer Institute
> Lowy Cancer Research Centre, UNSW Australia
> PO Box 81 Randwick 2031 Australia
>
> E: MPinese at ccia.org.au  |  W: www.ccia.org.au<http://www.ccia.org.au>
>
>
>
>
>
>
>
>
>
>
>
> _______________________________________________
> Bioc-devel at r-project.org mailing list
> https://secure-web.cisco.com/1onVAfZj3SbghVX3o7wQGJzOm2gIC_Gs1FDTfjh3-4tftOPw5cILEty4iUtnyKOMBvluYEs4qiAhILr9L4mmGEtDZS9FF40TuMEjt_uNQOxn84klDiOlmj2S72kDhZlQiOMClwdIKGCKzIGJyAmXfhAONjqxwlcX2qu-qJPPY87qACPWj0er2Z3cxugIteYyHRU-XRH_W44HLH6MhJ9eXPlbMAh-22grL5IR0F60rQTnr7kqSaWrGn1bEbEDPgUm8a6GvCSEEYY2ua5tcjYEzZWYneN2F1NwW1_7bKl2dA58toq3G28yM7_-swFvkhgPE/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel
>

--
Herv? Pag?s

Bioconductor Core Team
hpages.on.github at gmail.com

_______________________________________________
Bioc-devel at r-project.org mailing list
https://secure-web.cisco.com/1onVAfZj3SbghVX3o7wQGJzOm2gIC_Gs1FDTfjh3-4tftOPw5cILEty4iUtnyKOMBvluYEs4qiAhILr9L4mmGEtDZS9FF40TuMEjt_uNQOxn84klDiOlmj2S72kDhZlQiOMClwdIKGCKzIGJyAmXfhAONjqxwlcX2qu-qJPPY87qACPWj0er2Z3cxugIteYyHRU-XRH_W44HLH6MhJ9eXPlbMAh-22grL5IR0F60rQTnr7kqSaWrGn1bEbEDPgUm8a6GvCSEEYY2ua5tcjYEzZWYneN2F1NwW1_7bKl2dA58toq3G28yM7_-swFvkhgPE/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel

This email message may contain legally privileged and/or confidential information. If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited. If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.

Hervé Pagès

Thu, May 13, 2021 9:30 PM #

Thanks Mark for the update and for the detailed explanations.

Best,
H.

On 13/05/2021 16:15, Mark Pinese wrote:

Thank you both, with your help I managed to reproduce the error, and 
it's now fixed.

If anyone else runs into a similar issue:

 1. The core issue was due to a deprecation warning (specifically
    regarding survival::survConcordance.fit). During vignette generation
    a very large number of deprecation warnings were thrown, which
    overloaded TeX. Changing the survival::survConcordance.fit call to
    the updated survival::concordancefit fixed the problem.
 2. The cause of the reproducibility issue is less clear. I initially
    tested using Ubuntu 20.04 LTS running on Windows through WSL, which
    seemed to work. However, subsequent tests on a true Ubuntu 20.04
    Linux machine failed. My tentative conclusion is that Ubuntu 20.04
    LTS running through WSL is not the most reliable build host for BioC
    packages.


Best wishes,

Mark
------------------------------------------------------------------------
*From:* Kern, Lori <Lori.Shepherd at RoswellPark.org>
*Sent:* Wednesday, 21 April 2021 9:31 PM
*To:* Herv? Pag?s <hpages.on.github at gmail.com>; Mark Pinese 
<MPinese at ccia.org.au>; bioc-devel at r-project.org <bioc-devel at r-project.org>
*Subject:* Re: [Bioc-devel] Cannot reproduce "TeX capacity exceeded" 
error on build hosts (package messina)

EXTERNAL EMAIL: Think before you click. If suspicious send to 
CyberReport at ccia.org.au.

I can also reproduce the error locally on my laptop?Ubuntu 18.04.5,??R 
version 4.0.5 Patched (2021-03-31 r80152),? ?Bioc 3.12.



Lori Shepherd

Bioconductor Core Team

Roswell Park Comprehensive Cancer Center

Department of Biostatistics & Bioinformatics

Elm & Carlton Streets

Buffalo, New York 14263

------------------------------------------------------------------------
*From:* Bioc-devel <bioc-devel-bounces at r-project.org> on behalf of Herv? 
Pag?s <hpages.on.github at gmail.com>
*Sent:* Wednesday, April 21, 2021 2:00 AM
*To:* Mark Pinese <MPinese at ccia.org.au>; bioc-devel at r-project.org 
<bioc-devel at r-project.org>
*Subject:* Re: [Bioc-devel] Cannot reproduce "TeX capacity exceeded" 
error on build hosts (package messina)
Hi Mark,

I can reproduce this on my laptop (Ubuntu 20.10, R 4.0.3, BioC 3.12):

hpages at spectre:~/git.bioconductor.org/software$ time brelbuild messina
* checking for file ?messina/DESCRIPTION? ... OK
* preparing ?messina?:
* checking DESCRIPTION meta-information ... OK
* cleaning src
* installing the package to build vignettes
* creating vignettes ... ERROR
--- re-building ?messina.Rnw? using knitr
Loading required package: survival
Loading required package: Biobase
Loading required package: BiocGenerics
Loading required package: parallel

Attaching package: 'BiocGenerics'

The following objects are masked from 'package:parallel':

 ???? clusterApply, clusterApplyLB, clusterCall, clusterEvalQ,
 ???? clusterExport, clusterMap, parApply, parCapply, parLapply,
 ???? parLapplyLB, parRapply, parSapply, parSapplyLB

The following objects are masked from 'package:stats':

 ???? IQR, mad, sd, var, xtabs

The following objects are masked from 'package:base':

 ???? Filter, Find, Map, Position, Reduce, anyDuplicated, append,
 ???? as.data.frame, basename, cbind, colnames, dirname, do.call,
 ???? duplicated, eval, evalq, get, grep, grepl, intersect,
 ???? is.unsorted, lapply, mapply, match, mget, order, paste, pmax,
 ???? pmax.int, pmin, pmin.int, rank, rbind, rownames, sapply,
 ???? setdiff, sort, table, tapply, union, unique, unsplit, which.max,
 ???? which.min

Welcome to Bioconductor

 ???? Vignettes contain introductory material; view with
 ???? 'browseVignettes()'. To cite Bioconductor, see
 ???? 'citation("Biobase")', and for packages 'citation("pkgname")'.

Error: processing vignette 'messina.Rnw' failed with diagnostics:
Running 'texi2dvi' on 'messina.tex' failed.
LaTeX errors:
! TeX capacity exceeded, sorry [main memory size=5000000].
l.32169

If you really absolutely need more capacity,
you can ask a wizard to enlarge me.
!? ==> Fatal error occurred, no output PDF file produced!
--- failed re-building ?messina.Rnw?

SUMMARY: processing the following file failed:
 ?? ?messina.Rnw?

Error: Vignette re-building failed.
Execution halted

real??? 2m40.464s
user??? 2m38.072s
sys???? 0m1.776s

I'm surprised you can't reproduce this. What platform are you on?

My understanding is that the Docker images were intentionally kept light
so this is why they don't have TeX/LaTeX and other things needed to run
'R CMD build' on a package.

Since we see the error in release and devel, I suspected a change in a
CRAN package involved in the vignette knitting process. I tried to
downgrade knitr from 1.32 to 1.31 but that didn't solve the problem.
Then I downgraded xfun from 0.22 to 0.21 but that didn't help either.
So I gave up.

Sorry for not being able to help more.

Best,
H.

On 4/20/21 6:17 PM, Mark Pinese wrote:

Hi all,

My package messina has recently started failing on all build platforms, eg https://secure-web.cisco.com/1iNAEmxW_NMxwnEIs_OdNqw1U28YG6RTA_d4Qj80AB8xPF2WAj4JoGuuPYrFJsE5BRToE2jh0rPYFN3EPb7NkzShrBmIn6skNzneNzD1t63RBm10h01pMeahxJeDOXmWqmbDHabJeSFu5yOPzAiUeH75sAzyo1Sy9L1Zc5fRhsDmxP1ZBn0nmV4QJRWV2wsxqN_B7HXaghk_26sDQGBvy3JvXscFsr3sNFAnUZwdohPI4pP1XyzHmLxK2Oi_g0nLd70J_KnlyS9wz6hSgAYY1VSEiZgTV_FKqnYIkoJYfAfUPNnL8vktDPqLr3dCwMtOv/https%3A%2F%2Fmaster.bioconductor.org%2FcheckResults%2F3.12%2Fbioc-LATEST%2Fmessina%2Fmalbec1-buildsrc.html

<https://secure-web.cisco.com/1iNAEmxW_NMxwnEIs_OdNqw1U28YG6RTA_d4Qj80AB8xPF2WAj4JoGuuPYrFJsE5BRToE2jh0rPYFN3EPb7NkzShrBmIn6skNzneNzD1t63RBm10h01pMeahxJeDOXmWqmbDHabJeSFu5yOPzAiUeH75sAzyo1Sy9L1Zc5fRhsDmxP1ZBn0nmV4QJRWV2wsxqN_B7HXaghk_26sDQGBvy3JvXscFsr3sNFAnUZwdohPI4pP1XyzHmLxK2Oi_g0nLd70J_KnlyS9wz6hSgAYY1VSEiZgTV_FKqnYIkoJYfAfUPNnL8vktDPqLr3dCwMtOv/https%3A%2F%2Fmaster.bioconductor.org%2FcheckResults%2F3.12%2Fbioc-LATEST%2Fmessina%2Fmalbec1-buildsrc.html> 
This seems due to a "TeX capacity exceeded" error, though I suspect that 
something upstream is the true culprit.

To debug I have tried to reproduce this locally with BioC 3.12 on R 4.0.3 (both clean installs), but I cannot reproduce the problem. Locally the build works without issues for me:

source ~/.Renv_biocbuild
R CMD build ~/repos/messina/
* checking for file ?~repos/messina/DESCRIPTION? ... OK
* preparing ?messina?:
* checking DESCRIPTION meta-information ... OK
* cleaning src
* installing the package to build vignettes
* creating vignettes ... OK
* cleaning src
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
* looking to see if a ?data/datalist? file should be added
* building ?messina_1.23.1.tar.gz?

Can anyone offer suggestions on how to reproduce the error as seen on the build environments? Something like a docker image matching the build environments would be ideal. I tried the image bioconductor/bioconductor_docker:devel, but I needed to install quite  a few build dependencies, so I suspect this image doesn't match the

build envs.


Many thanks,

Mark


Mark Pinese

Senior Bioinformatics Research Officer
Conjoint Senior Lecturer, School of Women?s and Children?s Health, UNSW Medicine
NHMRC Emerging Leadership Fellow

Children's Cancer Institute
Lowy Cancer Research Centre, UNSW Australia
PO Box 81 Randwick 2031 Australia

E: MPinese at ccia.org.au? | ?W: www.ccia.org.au <http://www.ccia.org.au>

_______________________________________________
Bioc-devel at r-project.org mailing list
https://secure-web.cisco.com/1onVAfZj3SbghVX3o7wQGJzOm2gIC_Gs1FDTfjh3-4tftOPw5cILEty4iUtnyKOMBvluYEs4qiAhILr9L4mmGEtDZS9FF40TuMEjt_uNQOxn84klDiOlmj2S72kDhZlQiOMClwdIKGCKzIGJyAmXfhAONjqxwlcX2qu-qJPPY87qACPWj0er2Z3cxugIteYyHRU-XRH_W44HLH6MhJ9eXPlbMAh-22grL5IR0F60rQTnr7kqSaWrGn1bEbEDPgUm8a6GvCSEEYY2ua5tcjYEzZWYneN2F1NwW1_7bKl2dA58toq3G28yM7_-swFvkhgPE/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel

_______________________________________________
Bioc-devel at r-project.org mailing list
https://secure-web.cisco.com/1onVAfZj3SbghVX3o7wQGJzOm2gIC_Gs1FDTfjh3-4tftOPw5cILEty4iUtnyKOMBvluYEs4qiAhILr9L4mmGEtDZS9FF40TuMEjt_uNQOxn84klDiOlmj2S72kDhZlQiOMClwdIKGCKzIGJyAmXfhAONjqxwlcX2qu-qJPPY87qACPWj0er2Z3cxugIteYyHRU-XRH_W44HLH6MhJ9eXPlbMAh-22grL5IR0F60rQTnr7kqSaWrGn1bEbEDPgUm8a6GvCSEEYY2ua5tcjYEzZWYneN2F1NwW1_7bKl2dA58toq3G28yM7_-swFvkhgPE/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel 
<https://secure-web.cisco.com/1onVAfZj3SbghVX3o7wQGJzOm2gIC_Gs1FDTfjh3-4tftOPw5cILEty4iUtnyKOMBvluYEs4qiAhILr9L4mmGEtDZS9FF40TuMEjt_uNQOxn84klDiOlmj2S72kDhZlQiOMClwdIKGCKzIGJyAmXfhAONjqxwlcX2qu-qJPPY87qACPWj0er2Z3cxugIteYyHRU-XRH_W44HLH6MhJ9eXPlbMAh-22grL5IR0F60rQTnr7kqSaWrGn1bEbEDPgUm8a6GvCSEEYY2ua5tcjYEzZWYneN2F1NwW1_7bKl2dA58toq3G28yM7_-swFvkhgPE/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel>


This email message may contain legally privileged and/or confidential 
information. If you are not the intended recipient(s), or the employee 
or agent responsible for the delivery of this message to the intended 
recipient(s), you are hereby notified that any disclosure, copying, 
distribution, or use of this email message is prohibited. If you have 
received this message in error, please notify the sender immediately by 
e-mail and delete this email message from your computer. Thank you.

Herv? Pag?s

Bioconductor Core Team
hpages.on.github at gmail.com