Dear developers, I am struggling to reduce R CMD check times for my package, MIRit, on macOS. The check on linux takes 4 minutes, while on macOS it needs 9/10 minutes. I have severely reduced the size of examples and tests, which now use smaller datasets and cached resources. On linux, unit tests require 30 seconds more or less, and examples 40 seconds. On the other hand, macOS tests take 100 seconds, and 75 for the examples. The issue is that sometimes the check runs in less than 10 minutes, sometimes it needs a little bit more. However I am out of ideas on how to further reduce examples and tests... The vignette is really extensive, but since SPB uses the "--no-vignette" option this should not influence heavily on this behavior, at least to my understanding. Sorry for bothering you, Best regards, Jacopo
[Bioc-devel] Check time on macOS (MIRit)
8 messages · Jacopo Ronchi, Vincent Carey, Kasper Daniel Hansen +2 more
Thanks Jacopo. You shouldn't have to struggle. We are aware of disparities of platform performance and are working to improve the build process. It will take time. I believe that achievement of acceptable timings on the linux platform should be the key goal at this time. On Wed, Feb 7, 2024 at 5:18?AM Jacopo Ronchi <jacopo.ronchi at unimib.it> wrote:
Dear developers,
I am struggling to reduce R CMD check times for my package, MIRit, on
macOS. The check on linux takes 4 minutes, while on macOS it needs 9/10
minutes. I have severely reduced the size of examples and tests, which now
use smaller datasets and cached resources.
On linux, unit tests require 30 seconds more or less, and examples 40
seconds. On the other hand, macOS tests take 100 seconds, and 75 for the
examples. The issue is that sometimes the check runs in less than 10
minutes, sometimes it needs a little bit more.
However I am out of ideas on how to further reduce examples and tests...
The vignette is really extensive, but since SPB uses the "--no-vignette"
option this should not influence heavily on this behavior, at least to my
understanding.
Sorry for bothering you,
Best regards,
Jacopo
[[alternative HTML version deleted]]
_______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
The information in this e-mail is intended only for the ...{{dropped:18}}
Dear Vincent, thanks for your answer. So, at least for now, can I ignore the warning on macOS for the submission of MIRit? Il mer 7 feb 2024, 12:26 Vincent Carey <stvjc at channing.harvard.edu> ha scritto:
Thanks Jacopo. You shouldn't have to struggle. We are aware of disparities of platform performance and are working to improve the build process. It will take time. I believe that achievement of acceptable timings on the linux platform should be the key goal at this time. On Wed, Feb 7, 2024 at 5:18?AM Jacopo Ronchi <jacopo.ronchi at unimib.it> wrote:
Dear developers,
I am struggling to reduce R CMD check times for my package, MIRit, on
macOS. The check on linux takes 4 minutes, while on macOS it needs 9/10
minutes. I have severely reduced the size of examples and tests, which now
use smaller datasets and cached resources.
On linux, unit tests require 30 seconds more or less, and examples 40
seconds. On the other hand, macOS tests take 100 seconds, and 75 for the
examples. The issue is that sometimes the check runs in less than 10
minutes, sometimes it needs a little bit more.
However I am out of ideas on how to further reduce examples and tests...
The vignette is really extensive, but since SPB uses the "--no-vignette"
option this should not influence heavily on this behavior, at least to my
understanding.
Sorry for bothering you,
Best regards,
Jacopo
[[alternative HTML version deleted]]
_______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
The information in this e-mail is intended only for th...{{dropped:16}}
We need a bit of internal discussion before committing to this notion. I will get back to the list on this. On Wed, Feb 7, 2024 at 6:34?AM Jacopo Ronchi <jacopo.ronchi at unimib.it> wrote:
Dear Vincent, thanks for your answer. So, at least for now, can I ignore the warning on macOS for the submission of MIRit? Il mer 7 feb 2024, 12:26 Vincent Carey <stvjc at channing.harvard.edu> ha scritto:
Thanks Jacopo. You shouldn't have to struggle. We are aware of disparities of platform performance and are working to improve the build process. It will take time. I believe that achievement of acceptable timings on the linux platform should be the key goal at this time. On Wed, Feb 7, 2024 at 5:18?AM Jacopo Ronchi <jacopo.ronchi at unimib.it> wrote:
Dear developers,
I am struggling to reduce R CMD check times for my package, MIRit, on
macOS. The check on linux takes 4 minutes, while on macOS it needs 9/10
minutes. I have severely reduced the size of examples and tests, which
now
use smaller datasets and cached resources.
On linux, unit tests require 30 seconds more or less, and examples 40
seconds. On the other hand, macOS tests take 100 seconds, and 75 for the
examples. The issue is that sometimes the check runs in less than 10
minutes, sometimes it needs a little bit more.
However I am out of ideas on how to further reduce examples and tests...
The vignette is really extensive, but since SPB uses the "--no-vignette"
option this should not influence heavily on this behavior, at least to my
understanding.
Sorry for bothering you,
Best regards,
Jacopo
[[alternative HTML version deleted]]
_______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners?Compliance? HelpLine at http://www.partners.org/complianceline <http://www.partners.org/complianceline>?. If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail. [[alternative HTML version deleted]]
Ok, understood. Thanks Il mer 7 feb 2024, 12:46 Vincent Carey <stvjc at channing.harvard.edu> ha scritto:
We need a bit of internal discussion before committing to this notion. I will get back to the list on this. On Wed, Feb 7, 2024 at 6:34?AM Jacopo Ronchi <jacopo.ronchi at unimib.it> wrote:
Dear Vincent, thanks for your answer. So, at least for now, can I ignore the warning on macOS for the submission of MIRit? Il mer 7 feb 2024, 12:26 Vincent Carey <stvjc at channing.harvard.edu> ha scritto:
Thanks Jacopo. You shouldn't have to struggle. We are aware of disparities of platform performance and are working to improve the build process. It will take time. I believe that achievement of acceptable timings on the linux platform should be the key goal at this time. On Wed, Feb 7, 2024 at 5:18?AM Jacopo Ronchi <jacopo.ronchi at unimib.it> wrote:
Dear developers,
I am struggling to reduce R CMD check times for my package, MIRit, on
macOS. The check on linux takes 4 minutes, while on macOS it needs 9/10
minutes. I have severely reduced the size of examples and tests, which
now
use smaller datasets and cached resources.
On linux, unit tests require 30 seconds more or less, and examples 40
seconds. On the other hand, macOS tests take 100 seconds, and 75 for the
examples. The issue is that sometimes the check runs in less than 10
minutes, sometimes it needs a little bit more.
However I am out of ideas on how to further reduce examples and tests...
The vignette is really extensive, but since SPB uses the "--no-vignette"
option this should not influence heavily on this behavior, at least to
my
understanding.
Sorry for bothering you,
Best regards,
Jacopo
[[alternative HTML version deleted]]
_______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
The information in this e-mail is intended only for th...{{dropped:16}}
To be a bit more clear and vague at the same time: currently, some of our macOS builders exhibit weird slowdowns when the build system is run automatically compared to running builds "by hand". This is pretty weird, and hopefully we can figure out the cause and fix it. Specifically, builds on kjohnson3 is (at the moment) weirdly slow. Best, Kasper On Wed, Feb 7, 2024 at 7:24?AM Jacopo Ronchi <jacopo.ronchi at unimib.it> wrote:
Ok, understood. Thanks Il mer 7 feb 2024, 12:46 Vincent Carey <stvjc at channing.harvard.edu> ha scritto:
We need a bit of internal discussion before committing to this notion. I will get back to the list on this. On Wed, Feb 7, 2024 at 6:34?AM Jacopo Ronchi <jacopo.ronchi at unimib.it> wrote:
Dear Vincent, thanks for your answer. So, at least for now, can I ignore the warning on macOS for the submission of MIRit? Il mer 7 feb 2024, 12:26 Vincent Carey <stvjc at channing.harvard.edu> ha scritto:
Thanks Jacopo. You shouldn't have to struggle. We are aware of disparities of platform performance and are working to improve the build process. It will take time. I believe that achievement of acceptable timings on the linux platform should be the key goal at this time. On Wed, Feb 7, 2024 at 5:18?AM Jacopo Ronchi <jacopo.ronchi at unimib.it> wrote:
Dear developers, I am struggling to reduce R CMD check times for my package, MIRit, on macOS. The check on linux takes 4 minutes, while on macOS it needs
9/10
minutes. I have severely reduced the size of examples and tests, which now use smaller datasets and cached resources. On linux, unit tests require 30 seconds more or less, and examples 40 seconds. On the other hand, macOS tests take 100 seconds, and 75 for
the
examples. The issue is that sometimes the check runs in less than 10 minutes, sometimes it needs a little bit more. However I am out of ideas on how to further reduce examples and
tests...
The vignette is really extensive, but since SPB uses the
"--no-vignette"
option this should not influence heavily on this behavior, at least to
my
understanding.
Sorry for bothering you,
Best regards,
Jacopo
[[alternative HTML version deleted]]
_______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance
HelpLine
at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
The information in this e-mail is intended only for th...{{dropped:16}}
_______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Best, Kasper [[alternative HTML version deleted]]
Hi Kasper, According to https://github.com/Bioconductor/BBS/blob/9ebb2cf20e70e4bebf24a2a824a67193de821b08/3.19/bioc-mac-arm64/kjohnson3/config.sh#L17-L20 the machine has 24 vCPUs. The automatic/weekly build utilizes 12-16 out of those at any given time! With the respective RAM and disk usage. By "by hand" do you mean to run the build/check of a given package on a "calm" machine ? It would be expected that it will be faster if it has all/most resources just for it. Martin On Wed, Feb 7, 2024 at 5:12?PM Kasper Daniel Hansen <
kasperdanielhansen at gmail.com> wrote:
To be a bit more clear and vague at the same time: currently, some of our macOS builders exhibit weird slowdowns when the build system is run automatically compared to running builds "by hand". This is pretty weird, and hopefully we can figure out the cause and fix it. Specifically, builds on kjohnson3 is (at the moment) weirdly slow. Best, Kasper On Wed, Feb 7, 2024 at 7:24?AM Jacopo Ronchi <jacopo.ronchi at unimib.it> wrote:
Ok, understood. Thanks Il mer 7 feb 2024, 12:46 Vincent Carey <stvjc at channing.harvard.edu> ha scritto:
We need a bit of internal discussion before committing to this
notion. I
will get back to the list on this. On Wed, Feb 7, 2024 at 6:34?AM Jacopo Ronchi <jacopo.ronchi at unimib.it> wrote:
Dear Vincent, thanks for your answer. So, at least for now, can I
ignore
the warning on macOS for the submission of MIRit? Il mer 7 feb 2024, 12:26 Vincent Carey <stvjc at channing.harvard.edu>
ha
scritto:
Thanks Jacopo. You shouldn't have to struggle. We are aware of disparities of platform performance and are working to improve the build process. It will
take
time. I believe that achievement of acceptable timings on the linux platform should be the key goal at this time. On Wed, Feb 7, 2024 at 5:18?AM Jacopo Ronchi <
jacopo.ronchi at unimib.it>
wrote:
Dear developers, I am struggling to reduce R CMD check times for my package, MIRit,
on
macOS. The check on linux takes 4 minutes, while on macOS it needs
9/10
minutes. I have severely reduced the size of examples and tests,
which
now use smaller datasets and cached resources. On linux, unit tests require 30 seconds more or less, and examples
40
seconds. On the other hand, macOS tests take 100 seconds, and 75 for
the
examples. The issue is that sometimes the check runs in less than 10 minutes, sometimes it needs a little bit more. However I am out of ideas on how to further reduce examples and
tests...
The vignette is really extensive, but since SPB uses the
"--no-vignette"
option this should not influence heavily on this behavior, at least
to
my
understanding.
Sorry for bothering you,
Best regards,
Jacopo
[[alternative HTML version deleted]]
_______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
The information in this e-mail is intended only for the person to
whom
it is addressed. If you believe this e-mail was sent to you in error and
the
e-mail contains patient information, please contact the Partners Compliance
HelpLine
at http://www.partners.org/complianceline . If the e-mail was sent to
you
in error but does not contain patient information, please contact the sender
and
properly dispose of the e-mail.
The information in this e-mail is intended only for th...{{dropped:16}}
_______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
--
Best,
Kasper
[[alternative HTML version deleted]]
_______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Hi Martin, A very short summary is that on kjohnson3 a build/check on package launched by cron takes significantly longer than if one runs the same command manually in a terminal. There's some further discussion on this at https://community-bioc.slack.com/archives/CEQ04GKEC/p1706816310942079 and and Jen has some nice documentation of their investigation at https://github.com/Bioconductor/BBS/issues/387 However since the original inquiry in this thread is about a package in submission, and so the relevant tests are run by the single package builder on merida1, I'm not sure it's pertinent to this particular issue. Cheers, Mike On Wed, 7 Feb 2024 at 16:34, Martin Grigorov <martin.grigorov at gmail.com> wrote:
Hi Kasper, According to https://github.com/Bioconductor/BBS/blob/9ebb2cf20e70e4bebf24a2a824a67193de821b08/3.19/bioc-mac-arm64/kjohnson3/config.sh#L17-L20 the machine has 24 vCPUs. The automatic/weekly build utilizes 12-16 out of those at any given time! With the respective RAM and disk usage. By "by hand" do you mean to run the build/check of a given package on a "calm" machine ? It would be expected that it will be faster if it has all/most resources just for it. Martin On Wed, Feb 7, 2024 at 5:12?PM Kasper Daniel Hansen < kasperdanielhansen at gmail.com> wrote:
To be a bit more clear and vague at the same time: currently, some of our macOS builders exhibit weird slowdowns when the build system is run automatically compared to running builds "by hand". This is pretty weird, and hopefully we can figure out the cause and fix it. Specifically,
builds
on kjohnson3 is (at the moment) weirdly slow. Best, Kasper On Wed, Feb 7, 2024 at 7:24?AM Jacopo Ronchi <jacopo.ronchi at unimib.it> wrote:
Ok, understood. Thanks Il mer 7 feb 2024, 12:46 Vincent Carey <stvjc at channing.harvard.edu> ha scritto:
We need a bit of internal discussion before committing to this
notion. I
will get back to the list on this. On Wed, Feb 7, 2024 at 6:34?AM Jacopo Ronchi <
jacopo.ronchi at unimib.it>
wrote:
Dear Vincent, thanks for your answer. So, at least for now, can I
ignore
the warning on macOS for the submission of MIRit? Il mer 7 feb 2024, 12:26 Vincent Carey <stvjc at channing.harvard.edu>
ha
scritto:
Thanks Jacopo. You shouldn't have to struggle. We are aware of disparities of platform performance and are working to improve the build process. It will
take
time. I believe that achievement of acceptable timings on the linux platform should be
the
key goal at this time. On Wed, Feb 7, 2024 at 5:18?AM Jacopo Ronchi <
jacopo.ronchi at unimib.it>
wrote:
Dear developers, I am struggling to reduce R CMD check times for my package, MIRit,
on
macOS. The check on linux takes 4 minutes, while on macOS it needs
9/10
minutes. I have severely reduced the size of examples and tests,
which
now use smaller datasets and cached resources. On linux, unit tests require 30 seconds more or less, and examples
40
seconds. On the other hand, macOS tests take 100 seconds, and 75
for
the
examples. The issue is that sometimes the check runs in less than
10
minutes, sometimes it needs a little bit more. However I am out of ideas on how to further reduce examples and
tests...
The vignette is really extensive, but since SPB uses the
"--no-vignette"
option this should not influence heavily on this behavior, at
least
to
my
understanding.
Sorry for bothering you,
Best regards,
Jacopo
[[alternative HTML version deleted]]
_______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
The information in this e-mail is intended only for the person to
whom
it is addressed. If you believe this e-mail was sent to you in error and
the
e-mail contains patient information, please contact the Partners
Compliance
HelpLine
at http://www.partners.org/complianceline . If the e-mail was sent to
you
in error but does not contain patient information, please contact the sender
and
properly dispose of the e-mail.
The information in this e-mail is intended only for
th...{{dropped:16}}
_______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
--
Best,
Kasper
[[alternative HTML version deleted]]
_______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
[[alternative HTML version deleted]]
_______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel