Skip to content

Trying to install r-stan without using rocker

6 messages · Jesse McMullen-Crummey, Dirk Eddelbuettel, Elizabeth Tighe

#
Hi All,

   I found your mailing list via a stack overflow mention.  I?m currently on my second full day of trying to build a docker container to use as an AWS lambda fcn and am feeling pretty stuck - I was hoping someone here could help.  It needs to support python+R+rstan.


I currently construct the docker image from a python/debian base image.  I can get R working fine, but when I try to run an r-stan script within the lambda function I get a ?there is no package called Stan?, despite a good 5 minutes of the image build being dedicated to the download of rstan.  Am looking for advice here on how to proceed.  My current dockerfile follows?

################
FROM python:3.8-slim-bullseye

ENV DEBIAN_FRONTEND=noninteractive

ENV LC_ALL=C.UTF-8
ENV LANG=en_US.UTF-8
ENV TZ=:/etc/localtime
ENV PATH=/var/lang/bin:/usr/local/bin:/usr/bin/:/bin:/opt/bin
ENV LD_LIBRARY_PATH=/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib
ENV LAMBDA_TASK_ROOT=/var/task
ENV LAMBDA_RUNTIME_DIR=/var/runtime

# Define custom function directory
ARG FUNCTION_DIR="/var/task"
WORKDIR ${FUNCTION_DIR}

# Install aws-lambda-cpp build dependencies + r
RUN apt-get update && \
  apt-get install -y \
  g++ \
  make \
  cmake \
  unzip \
  libcurl4-openssl-dev \
  r-base 

COPY requirements.txt  .
COPY irt_dim_red.R .
COPY . .

#install the python requirements
RUN pip install -r requirements.txt

COPY scripts/install_stan.R install_stan.R

RUN Rscript "install_stan.R"
RUN R -e "install.packages('stringr')"
RUN R -e "install.packages('gtools')"
RUN R -e "install.packages('tidyr')"

# Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)
ENTRYPOINT [ "/usr/local/bin/python", "-m", "awslambdaric" ]
CMD [ "main.handler"]
Where the script install_rstan.R is:

# Following instructions from
# https://github.com/stan-dev/rstan/wiki/Installing-RStan-on-Linux

print("Running install_stan.R")

## Creating /home/rstudio/.R/Makevars

dotR <- file.path(Sys.getenv("HOME"), ".R")
if (!file.exists(dotR)) dir.create(dotR)
M <- file.path(dotR, "Makevars")
if (!file.exists(M)) file.create(M)
cat("\nCXX14FLAGS=-O3 -march=native -mtune=native -fPIC",
    "CXX14=clang++",
    file = M, sep = "\n", append = TRUE)

## Installing rstan

install.packages("rstan", dependencies=TRUE, type = "source")

Any help would be massively appreciated.

Jesse


?


Jesse McMullen-Crummey, PhD (he/him)
Data Engineer
#
Jesse,
On 13 June 2022 at 12:19, Jesse McMullen-Crummey wrote:
| Hi All,
| 
|    I found your mailing list via a stack overflow mention.  I?m currently on my second full day of trying to build a docker container to use as an AWS lambda fcn and am feeling pretty stuck - I was hoping someone here could help.  It needs to support python+R+rstan.

If something does not work, I usually backtrack.

You start from python:3.8-slim-bullseye. That is your right. But this is a
Debian-and-R mailing list, so we may not have started from there.

I maintain and curate several sets of Rocker containers building either on
 - plain Debian as our rocker/r-base which is also the official r-base
 - Ubuntu because I can then use Ubuntu binaries as from c2d4u or now also r2u

If I were, I'd the same. With either base set you should get to 'apt install r-cran-rstan'
easily, and adding your other package is easy. (If you look at work I worked
on recently via its site https://eddelbuettel.github.io/r2u/ you will see
that staring from eg eddelbuettel/r2u:22.04 you can run a single
   Rscript -e 'install.packages(c("rstan", "stringr", "gtools", "tidyr"))' 
without issues giving your the R side to which you can then add your Python
requirements. I usually try that first interactively in the same Docker
container.

If all that fails, maybe you want to emphasize Python first and use Anaconda
via mamba. Some folks had good luck with that (but don't mix and match, only
tears result from that).  We cannot help with *conda here.

Dirk


| I currently construct the docker image from a python/debian base image.  I can get R working fine, but when I try to run an r-stan script within the lambda function I get a ?there is no package called Stan?, despite a good 5 minutes of the image build being dedicated to the download of rstan.  Am looking for advice here on how to proceed.  My current dockerfile follows?
| 
| ################
| FROM python:3.8-slim-bullseye
| 
| ENV DEBIAN_FRONTEND=noninteractive
| 
| ENV LC_ALL=C.UTF-8
| ENV LANG=en_US.UTF-8
| ENV TZ=:/etc/localtime
| ENV PATH=/var/lang/bin:/usr/local/bin:/usr/bin/:/bin:/opt/bin
| ENV LD_LIBRARY_PATH=/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib
| ENV LAMBDA_TASK_ROOT=/var/task
| ENV LAMBDA_RUNTIME_DIR=/var/runtime
| 
| # Define custom function directory
| ARG FUNCTION_DIR="/var/task"
| WORKDIR ${FUNCTION_DIR}
| 
| # Install aws-lambda-cpp build dependencies + r
| RUN apt-get update && \
|   apt-get install -y \
|   g++ \
|   make \
|   cmake \
|   unzip \
|   libcurl4-openssl-dev \
|   r-base 
| 
| COPY requirements.txt  .
| COPY irt_dim_red.R .
| COPY . .
| 
| #install the python requirements
| RUN pip install -r requirements.txt
| 
| COPY scripts/install_stan.R install_stan.R
| 
| RUN Rscript "install_stan.R"
| RUN R -e "install.packages('stringr')"
| RUN R -e "install.packages('gtools')"
| RUN R -e "install.packages('tidyr')"
| 
| # Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)
| ENTRYPOINT [ "/usr/local/bin/python", "-m", "awslambdaric" ]
| CMD [ "main.handler"]
| Where the script install_rstan.R is:
| 
| # Following instructions from
| # https://github.com/stan-dev/rstan/wiki/Installing-RStan-on-Linux
| 
| print("Running install_stan.R")
| 
| ## Creating /home/rstudio/.R/Makevars
| 
| dotR <- file.path(Sys.getenv("HOME"), ".R")
| if (!file.exists(dotR)) dir.create(dotR)
| M <- file.path(dotR, "Makevars")
| if (!file.exists(M)) file.create(M)
| cat("\nCXX14FLAGS=-O3 -march=native -mtune=native -fPIC",
|     "CXX14=clang++",
|     file = M, sep = "\n", append = TRUE)
| 
| ## Installing rstan
| 
| install.packages("rstan", dependencies=TRUE, type = "source")
| 
| Any help would be massively appreciated.
| 
| Jesse
| 
| 
| ?
| 
| 
| Jesse McMullen-Crummey, PhD (he/him)
| Data Engineer
| 
| 
| 	[[alternative HTML version deleted]]
| 
| _______________________________________________
| R-SIG-Debian mailing list
| R-SIG-Debian at r-project.org
| https://stat.ethz.ch/mailman/listinfo/r-sig-debian
#
Hi All,

I?ve followed Dirk?s advice and am now starting from:

FROM eddelbuettel/r2u:22.04
While installing rstan using
RUN Rscript -e 'install.packages(c("rstan", "stringr", "gtools", "tidyr"))'

I can now get my lambda function running, but it throws an error when opening my Stan model, namely: 
"C++ compiler not found on system.?

I would have thought that it would be part of this base image.

Any thoughts would be appreciated.

Jesse
un 13, 2022, at 12:36 PM, Dirk Eddelbuettel <edd at debian.org> wrote:

  
  
#
On 13 June 2022 at 14:25, Jesse McMullen-Crummey wrote:
| Hi All,
| 
| I?ve followed Dirk?s advice and am now starting from:
| 
| FROM eddelbuettel/r2u:22.04
| While installing rstan using
| RUN Rscript -e 'install.packages(c("rstan", "stringr", "gtools", "tidyr"))'

(BTW I would actually use

   RUN install.r rstan stringr gtools tidyr

which is more idiomatic _for me_ using littler, but equivalent <wink> )

| I can now get my lambda function running,

Nice!

| but it throws an error when opening my Stan model, namely: 
| "C++ compiler not found on system.?
| 
| I would have thought that it would be part of this base image.

Indeed. That r2u image has g++-11.

So (r)stan may be configured differently / may have seen / expected a
different compiler?  I would check its docs. You should be able to square
that. 

Dirk
#
I'm not an expert, but I do use rstan on Ubuntu.

The install instructions here:

https://github.com/stan-dev/rstan/wiki/RStan-Getting-Started

include info on configuring the C++ Toolchain with a link for 
configuring on linux here:

 ?https://github.com/stan-dev/rstan/wiki/Configuring-C-Toolchain-for-Linux

in case this helps ...

Best,
Liz Tighe
On 6/13/2022 2:36 PM, Dirk Eddelbuettel wrote:
#
Thanks Liz.

I had that toolchain running as an install_stan.R script.  I ended up reverting to 20.04 and installing build-essential as well.  One of those seems to have done the trick!

Thanks!

Jesse