Skip to content

[R-pkg-devel] Two (newbie/luser) questions about adding CI to github hosted package

9 messages · Dirk Eddelbuettel, Chris Evans, Stefan McKinnon Høj-Edwards +2 more

#
I would be very grateful of help as I seem to be making very heavy weather of all the intricacies of creating a 
robust R package that might be useful (mainly to R newbies/lusers of even lower R/programming skills than mine). 

I am making a lot of progress and am awed by the work so many have put in to make things possible: huge gratitude 
to too many to name. However, I seem to be hitting a wall on adding CI to my package. 

1) I am sure that linting my package will help me but there are some defaults in lintr/superlinter that I am want
to take a personal stand and not use: allowing myself not always to use perfect camelCase because I think it will
confuse my end users and the line length limit which at the default of 80 seems to me, on a modern set up, to make
my code far less readable than if I allow some long lines. My problem is that I am failing to work out how to override
the defaults. I thought that putting in /TEMPLATES/.lintr in my package directory and having it contain:

# linters: with_defaults(object_usage_linter = NULL)
linters: with_defaults(
  line_length_linter(120),
  cyclocomp_linter(25),
  object_name_linter = NULL
  )
would override the defaults (I'm working on the cyclocomp complaint!)  Can anyone, perhaps looking at my 
https://github.com/cpsyctc/CECPfuns/ package, tell me what I'm doing wrong and how to get it right?!

2) I work on a machine running Ubuntu 20.04 in Rstudio and everything checks and tests fine there but I wanted
to use the github actions to check the package out on Windows and Mac and, rather more by luck I think than really
understanding how I got there, I seem to have achieved that.  Rather bizarrely, the code (it's only R source)
passes on Windows and Mac but fails in the Ubuntu test systems with errors like:

?? R CMD build ?????????????????????????????????????????????????????????????????
* checking for file ?.../DESCRIPTION? ... OK
* preparing ?CECPfuns?:
* checking DESCRIPTION meta-information ... OK
* installing the package to process help pages
      -----------------------------------
* installing *source* package ?CECPfuns? ...
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
Error in dyn.load(file, DLLpath = DLLpath, ...) : 
  unable to load shared object '/home/runner/work/_temp/Library/stringi/libs/stringi.so':
  libicui18n.so.55: cannot open shared object file: No such file or directory
Calls: <Anonymous> ... namespaceImport -> loadNamespace -> library.dynam -> dyn.load
Execution halted
ERROR: lazy loading failed for package ?CECPfuns?

I _think_ these are just changes in the stirngi package that seem to failing on the test machine
and OK on my machine. That made me think that I can see a lot of logic in my checking things on
the various default versions of R in Ubuntu 16.04, Windows and MacOS but perhaps it is unnecessary
and wasting resources to check the release version of R on Ubuntu 20.04 as it seems safe enough to
assume that the package built on my local machine in the release version of R and on Ubuntu 20.04
that fails like this will only be transient (or that successes on my machine will only be ... but
at least I'll see them and not waste github resources!)

The catch is that I am baffled by how to configure github actions to change the lint defaults and
to change the selection of releases and platforms.  I think that may because I may have created
a mess in my github actions workflows.

Would anyone be willing to help or even to look at it and point me to the explanations I am failing
to find?! 

TIA,

Chris
#
On 16/03/2021 5:58 a.m., Chris Evans wrote:
I use lintr, and put my .lintr file in the main directory of the package.
You have 5 workflows, and their current content doesn't appear to match 
the results on your actions page.  Pick one, run it, and I'll see if I 
can spot the reason for a failure.

Duncan Murdoch
#
On 16 March 2021 at 06:48, Duncan Murdoch wrote:
| On 16/03/2021 5:58 a.m., Chris Evans wrote:
| You have 5 workflows, and their current content doesn't appear to match 
| the results on your actions page.  Pick one, run it, and I'll see if I 
| can spot the reason for a failure.

More granularly, one of these five workflow spawns nine builds of which the
ones with older R version (3.3, 3.4, 3.5) fail as seen eg for run #3:

https://github.com/cpsyctc/CECPfuns/actions/runs/657501549

Maybe you want to comment out runs for R versions that old, or debug why they
fail. The error shown in R 3.5 (a version soon three years out of date you
are not obliged to care for: CRAN uses only current, previous and next
release) is not something I have seen before:

  ?? R CMD build ?????????????????????????????????????????????????????????????????
  * checking for file ?.../DESCRIPTION? ... OK
  * preparing ?CECPfuns?:
  * checking DESCRIPTION meta-information ... OK
  Error in parse_Rd("/tmp/RtmpydYbB0/Rbuild4d3e1cce5e37/CECPfuns/man/getCSC.Rd",  : 
    Value of SET_STRING_ELT() must be a 'CHARSXP' not a 'NULL'
  Execution halted
  Error: Error in proc$get_built_file() : Build process failed
  Calls: <Anonymous> ... build_package -> with_envvar -> force -> <Anonymous>
  Execution halted
  Error: Process completed with exit code 1.

Here proce$get_built_file points to `pkgbuild` which is part of a non-CRAN
toolchain.  So this may not be caused by your package -- I can't tell. My
preference is to keep it simpler: I usually get by with one workflow and
generally at most two builds.

Dirk
#
Hugely appreciated Duncan.

----- Original Message -----
OK.  I have removed the TEMPLATES directory and moved .lintr to the package root.

I think I am better keeping lintr local and having moved .lintr using

lintr::lint_package()

locally is working fine, flagging up cosmetics I can see are all improvements so I
can sort those out without wasting github power.  Which takes to the next stage ...
That is so appreciated Duncan. One stupid question: is there a way to run a
workflow other than by making an upload/push (or a pull request)?  

I think I have discovered that the workflows are controlled by the yaml files
in .github/workflows so I have deleted all except R-CMD-check.yaml and I think
that when I now commit and push this that should delete the other workflows 
on github and trigger this one to run.  If you could look at that one then
that would be hugely appreciated.  I see it says:

    strategy:
      fail-fast: false
      matrix:
        config:
          - {os: macOS-latest,   r: 'release'}
          - {os: windows-latest, r: 'release'}
          - {os: windows-latest, r: '3.6'}
          - {os: ubuntu-18.04,   r: 'devel', rspm: "https://packagemanager.rstudio.com/cran/__linux__/bionic/latest", http-user-agent: "R/4.0.0 (ubuntu-18.04) R (4.0.0 x86_64-pc-linux-gnu x86_64 linux-gnu) on GitHub Actions" }
          - {os: ubuntu-18.04,   r: 'release', rspm: "https://packagemanager.rstudio.com/cran/__linux__/bionic/latest"}
          - {os: ubuntu-18.04,   r: 'oldrel',  rspm: "https://packagemanager.rstudio.com/cran/__linux__/bionic/latest"}
          - {os: ubuntu-18.04,   r: '3.5',     rspm: "https://packagemanager.rstudio.com/cran/__linux__/bionic/latest"}
          - {os: ubuntu-18.04,   r: '3.4',     rspm: "https://packagemanager.rstudio.com/cran/__linux__/bionic/latest"}
          - {os: ubuntu-18.04,   r: '3.3',     rspm: "https://packagemanager.rstudio.com/cran/__linux__/bionic/latest"}


Some questions as I, I think, make some progress:

1) Why is windows-latest pointing at 3.6?  I am on 4.0.4 locally for windows.  I assume this is because I have copied or pulled in
a template that is out of date.  Can I just replace that '3.6' with 'release'

2) My guts say that at this point it is sensible for me to test against devel, release and oldrel but that after the next
release I could stop testing against oldrel?  Is that sensible?

3) I see tests against 3.3, 3.4 and 3.5 there for ubuntu 18.04 and no 20.04.  Am I correct that I could replace the above
with this:

    strategy:
      fail-fast: false
      matrix:
        config:
          - {os: macOS-latest,   r: 'release'}
          - {os: windows-latest, r: 'release'}
          - {os: windows-latest, r: 'release'}
          - {os: ubuntu-18.04,   r: 'devel', rspm: "https://packagemanager.rstudio.com/cran/__linux__/bionic/latest", http-user-agent: "R/4.0.0 (ubuntu-18.04) R (4.0.0 x86_64-pc-linux-gnu x86_64 linux-gnu) on GitHub Actions" }
          - {os: ubuntu-18.04,   r: 'release', rspm: "https://packagemanager.rstudio.com/cran/__linux__/bionic/latest"}
          - {os: ubuntu-18.04,   r: 'oldrel',  rspm: "https://packagemanager.rstudio.com/cran/__linux__/bionic/latest"}
          - {os: ubuntu-20.04,   r: 'devel',     rspm: "https://packagemanager.rstudio.com/cran/__linux__/xenial/latest"}
          - {os: ubuntu-20.04,   r: 'release',     rspm: "https://packagemanager.rstudio.com/cran/__linux__/xenial/latest"}
          - {os: ubuntu-20.04,   r: 'oldrel',     rspm: "https://packagemanager.rstudio.com/cran/__linux__/xenial/latest"}

[I have changed the windows '3.6' to 'release' and the last three ubuntu calls to 20.04 and the R versions to 'devel', 'release'
 and 'oldrel' and the repositories to point at xenial not bionic.  I have only done this here in the Email, not on disc!]

It would be nice to check all that occasionally but I am sure that at the frequency I'm pushing up new functions and tweaks,
so my colleagues can check things, I am putting far too much work on github with all those.  Could I just check windows
and macOS release and devel by commenting out lines there.  Or is it better to have different yaml files and just to put
the one you want in .github/workflows for each particular push?  No great difference?

OK.  I have pushed these changes with the commit comment "More work to get continuous integration testing ..." and the
actions seem to be running and I seem to have succeeded in pruning things down to that one action.

Wonderful if you can find the time to have a look at these questions and the run ... now I'm off to the dentist.
Hm: dentist ... bad day with R ... dentist ... OK, I'll take bad day with R any day!

Thanks (and to all),

Chris

  
    
#
Brilliant and so obviously correct.  As per response a minute ago to Dirk's 
help, I agree completely and have started moves that look to be in line with
that Dirk.

Apologies for my newbie/luser missteps and huge thanks to you both for your 
inputs.  I'm awed to have had this.

Chris

----- Original Message -----

  
    
#
Hi everybody,

I would like to take a minute to address the language used in the initial
e-mail, notably the "newbie/luser" terminology.
It was given in both the subject (" Two (newbie/luser) questions") as well
as the body ("...  robust R package that might be useful (mainly to R
newbies/lusers of even lower R/programming skills than mine).").

Personally I find the term "luser" (as of a "loser user") very
condescending, whether it is about yourself (because you are not an
experienced programmer) or others (because you perceive them as
inexperienced programmers).
To put it short, using such a condescending term inhibits yourself and,
especially, others. Its use is simply unwarranted. Using the term about the
users of your package, regardless of their experience, is derogatory.

If you are writing to this mailing list with issues developing an
R-package, you are neither a newbie nor a loser, and we collectively will
do our best to help you. So no need in derograting yourself.

Kind regards,
Stefan

Den tir. 16. mar. 2021 kl. 14.05 skrev Chris Evans <chrishold at psyctc.org>:

  
  
#
> Error: Process completed with exit code 1.

The error seems to be in building the documentation. There used to be a buglet in older versions of R, 
that caused some errors in processing to cause this kind of uninformative message from low level code. 
I don't remember the R version but the time frame seems consistent. 
It may well be that some \Sexpr code is spread over more than one line, which sometimes threw errors, sometimes silently ignored the lines after the first.

Maybe it's easiest just to require R >= "more recent version.

Georgi Boshnakov


-----Original Message-----
From: R-package-devel <r-package-devel-bounces at r-project.org> On Behalf Of Dirk Eddelbuettel
Sent: 16 March 2021 12:46
To: Duncan Murdoch <murdoch.duncan at gmail.com>
Cc: r-package-devel <r-package-devel at r-project.org>
Subject: Re: [R-pkg-devel] Two (newbie/luser) questions about adding CI to github hosted package
On 16 March 2021 at 06:48, Duncan Murdoch wrote:
| On 16/03/2021 5:58 a.m., Chris Evans wrote:
| You have 5 workflows, and their current content doesn't appear to 
| match the results on your actions page.  Pick one, run it, and I'll 
| see if I can spot the reason for a failure.

More granularly, one of these five workflow spawns nine builds of which the ones with older R version (3.3, 3.4, 3.5) fail as seen eg for run #3:

https://github.com/cpsyctc/CECPfuns/actions/runs/657501549

Maybe you want to comment out runs for R versions that old, or debug why they fail. The error shown in R 3.5 (a version soon three years out of date you are not obliged to care for: CRAN uses only current, previous and next
release) is not something I have seen before:

  ?? R CMD build ?????????????????????????????????????????????????????????????????
  * checking for file ?.../DESCRIPTION? ... OK
  * preparing ?CECPfuns?:
  * checking DESCRIPTION meta-information ... OK
  Error in parse_Rd("/tmp/RtmpydYbB0/Rbuild4d3e1cce5e37/CECPfuns/man/getCSC.Rd",  : 
    Value of SET_STRING_ELT() must be a 'CHARSXP' not a 'NULL'
  Execution halted
  Error: Error in proc$get_built_file() : Build process failed
  Calls: <Anonymous> ... build_package -> with_envvar -> force -> <Anonymous>
  Execution halted
  Error: Process completed with exit code 1.

Here proce$get_built_file points to `pkgbuild` which is part of a non-CRAN toolchain.  So this may not be caused by your package -- I can't tell. My preference is to keep it simpler: I usually get by with one workflow and generally at most two builds.

Dirk

--
https://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

______________________________________________
R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
#
Thanks. That's very kind and at some levels I know it's correct and I should not go there. I have been on the receiving end of some (mostly one) incredibly brutal public battering 
in relation to R so perhaps I am lightly scarred by that!

        

  
    
#
On 16/03/2021 9:04 a.m., Chris Evans wrote:
...
If you have something like this in your yaml file:

# Controls when the action will run.
on:

   pull_request:
     branches: [ master ]

   workflow_dispatch:


then it says the workflow will run when there is a pull request to the 
master branch, and also on request (that's what "workflow_displatch:" 
means).  You can leave out the "pull_request:" lines and only run on 
request.

To actually make the request, you go onto your Github site, go to the 
Actions page, then pick one of the workflows from the list on the left.
If it has "workflow_dispatch" in the file, you'll see a "Run workflow" 
button appear on the right.  It's actually a drop down box; you can 
choose to use the workflow from a different branch by choosing it from 
the list.

Something I'm not sure about is whether choosing a workflow from a 
different branch also selects your package code from that branch, or 
whether you have to choose that on the main Code page.  Experiment, it's 
free!
I think so.
This really depends on your audience.  It looks like the script you have 
came from the tidyverse people; they try to support 4 releases before 
the current one.  That's pretty extreme, you don't really need all 
those.  But I'd leave oldrel in place.  There are definitely lots of 
people out there who haven't updated to 4.0.x yet, for various reasons, 
e.g. not being admin on their computer.
I don't know.  This depends on all those paths in the rspm field being 
available.  If RStudio is making them available, that is probably fine.
My feeling is you want lots of versions, and lots of platforms, but not 
necessarily the product of those two things.  If something is going to 
fail because of a platform issue, it'll likely fail in the "release" 
version.  If it is going to fail because of R changes, it'll likely fail 
on all platforms.

As you've probably seen, when one of those jobs fails, you get email:  I 
don't like repetitive emails, so I limit the jobs I run.

Duncan Murdoch