I build essential containers for statistical analysis tools, mainly for R, RStudio Server, Shiny Server and nginx load balancer for Shiny with PAM authentication module (since Shiny Server does not support authentication).
Add the steps involved:
1. First, ensure that iptables are running and create couple of rules for nginx
systemctl enable firewalld systemctl start firewalld firewall-cmd --permanent --direct --add-rule ipv4 filter IN_public_allow 0 -p tcp --dport 80 -m conntrack --ctstate NEW -j ACCEPT firewall-cmd --permanent --direct --add-rule ipv4 filter IN_public_allow 0 -p tcp --dport 443 -m conntrack --ctstate NEW -j ACCEPT firewall-cmd --reload
2. Install the latest docker, enable it, start and create a docker network for docker DNS resolution, so that container names work rather than IPs
yum -y update
yum install -y yum-utils yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo yum install -y docker-ce docker-ce-cli containerd.io systemctl enable docker systemctl start docker
docker network create -o com.docker.network.bridge.enable_ip_masquerade=true -o com.docker.network.bridge.enable_icc=true rstudio-net
3. Build R container
cd ~ mkdir R && cd R
vi Renviron.site
ORACLE_HOME=/oracle TNS_ADMIN=/oracle/network/admin OCI_LIB=/oracle/instantclient_12_1 ODBCINI=/exasol/.odbc.ini EXADRIVER = /exasol/EXASOL_ODBC-6.0.14/lib/linux/x86_64/libexaodbc-uo2214lv2.so EXAHOST = <exasol_ip_range>:8563
vi Dockerfile
FROM centos
LABEL "description"="A language for data analysis and graphics - http://www.r-project.org"
LABEL "version"="3.5.2-2.el7"
RUN rm -rf /etc/yum.repos.d/*
COPY system.repo /etc/yum.repos.d/
RUN yum -y install R cronie unixODBC-devel libcurl-devel libgit2-devel openssl-devel libssh2-devel libaio
ENV ORACLE_HOME=/oracle TNS_ADMIN=/oracle/network/admin OCI_LIB=/oracle/instantclient_12_1 LD_LIBRARY_PATH=/oracle/instantclient_12_1:/exasol/EXASOL_ODBC-6.0.14/lib/linux/x86_64 ODBCINI=/exasol/.odbc.ini
RUN R -e "install.packages(c('devtools','DBI','RODBC'), repos='<repo-url>'); devtools::install_github('exasol/r-exasol')"
COPY Renviron.site /usr/lib64/R/etc/
VOLUME /oracle /exasol /home /prod /stage
CMD /usr/sbin/crond -ndocker build -t=r:3.5.2-2.el7 .
!See Notes for installing ROracle package before building other containers that are based on this one!
4. Build RStudio Server container (need find a way of running rstudio-server under its own account rather than root. USER directive may not work because we also start crond, which should be run as root)
cd ~ mkdir rstudio && cd rstudio cp /etc/yum.repos.d/system.repo . wget https://s3.amazonaws.com/rstudio-ide-build/server/centos6/x86_64/rstudio-server-rhel-1.2.1335-x86_64.rpm
vi rserver.sh #!/bin/sh /usr/sbin/crond /usr/lib/rstudio-server/bin/rserver --server-daemonize 0
chmod 755 rserver.sh
vi sh.local #Add any required envvar overrides to this file, it is sourced from /etc/profile LD_LIBRARY_PATH=/oracle/instantclient_12_1:/exasol/EXASOL_ODBC-6.0.14/lib/linux/x86_64 OCI_LIB=/oracle/instantclient_12_1 TNS_ADMIN=/oracle/network/admin ORACLE_HOME=/oracle ODBCINI=/exasol/.odbc.ini PATH=/oracle/instantclient_12_1:$PATH export LD_LIBRARY_PATH OCI_LIB TNS_ADMIN ORACLE_HOME ODBCINI
vi Dockerfile FROM r:3.5.2-2.el7 LABEL "version"="1.2.1335" LABEL "description"="RStudio Server" RUN rm -rf /etc/yum.repos.d/* COPY system.repo /etc/yum.repos.d/ COPY rstudio-server-rhel-1.2.1335-x86_64.rpm /tmp RUN yum -y install /tmp/rstudio-server-rhel-1.2.1335-x86_64.rpm wget git RUN rm -rf /tmp/rstudio-server-rhel-1.2.1335-x86_64.rpm COPY sh.local /etc/profile.d/ COPY rserver.sh /usr/sbin EXPOSE 8787 VOLUME /oracle /exasol /home /prod /stage ENV ORACLE_HOME=/oracle TNS_ADMIN=/oracle/network/admin OCI_LIB=/oracle/instantclient_12_1 LD_LIBRARY_PATH=/oracle/instantclient_12_1:/exasol/EXASOL_ODBC-6.0.14/lib/linux/x86_64 ODBCINI=/exasol/.odbc.ini CMD /usr/sbin/rserver.sh
docker build -t=rstudio-server:1.2.1335 .
5. Configure PAM service for RStudio Server (for STAGE and DEV only)
vi /etc/pam.d/rstudio auth [user_unknown=ignore success=ok ignore=ignore default=bad] pam_securetty.so auth substack system-auth auth include postlogin auth required /usr/lib64/security/pam_listfile.so onerr=fail item=group sense=allow file=/etc/pam.d/rstudio-server_allowed_groups account required pam_nologin.so account include system-auth password include system-auth # pam_selinux.so close should be the first session rule session required pam_selinux.so close session required pam_loginuid.so session optional pam_console.so # pam_selinux.so open should only be followed by sessions to be executed in the user context session required pam_selinux.so open session required pam_namespace.so session optional pam_keyinit.so force revoke session include system-auth session include postlogin -session optional pam_ck_connector.so
6. Add AD user groups in /etc/pam.d/rstudio-server_allowed_groups
vi /etc/pam.d/rstudio-server_allowed_groups <add AD groups here>
7. Create a config file for RStudio Server
mkdir /etc/rstudio touch /etc/rstudio/rserver.conf
8. Build Shiny Server container (for PROD and STAGE only)
cd ~ mkdir shiny-server cd shiny-server
vi Dockerfile
FROM r:3.5.2-2.el7
LABEL "version"="1.5.9.923"
LABEL "description"="Shiny server"
RUN mkdir -p /usr/share/doc/R-3.5.2/html
RUN R -e "install.packages(c('shiny','rmarkdown'), repos='https://repo-r.example.com/repository/r-proxy-cran')"
COPY shiny-server-1.5.9.923-x86_64.rpm /tmp
RUN yum -y install /tmp/shiny-server-1.5.9.923-x86_64.rpm
RUN rm /tmp/shiny-server-1.5.9.923-x86_64.rpm
VOLUME /oracle /exasol /home /prod /stage
ENV ORACLE_HOME=/oracle TNS_ADMIN=/oracle/network/admin OCI_LIB=/oracle/instantclient_12_1 LD_LIBRARY_PATH=/oracle/instantclient_12_1:/exasol/EXASOL_ODBC-6.0.14/lib/linux/x86_64 ODBCINI=/exasol/.odbc.ini
CMD /opt/shiny-server/ext/node/bin/shiny-server /opt/shiny-server/lib/main.jsdocker build -t="shiny-server:1.5.9.923" .
8. Configure PAM service for Shiny Server
vi /etc/pam.d/shiny-server auth [user_unknown=ignore success=ok ignore=ignore default=bad] pam_securetty.so auth substack system-auth auth include postlogin auth required /usr/lib64/security/pam_listfile.so onerr=fail item=group sense=allow file=/etc/pam.d/shiny-server_allowed_groups account required pam_nologin.so account include system-auth password include system-auth # pam_selinux.so close should be the first session rule session required pam_selinux.so close session required pam_loginuid.so session optional pam_console.so # pam_selinux.so open should only be followed by sessions to be executed in the user context session required pam_selinux.so open session required pam_namespace.so session optional pam_keyinit.so force revoke session include system-auth session include postlogin -session optional pam_ck_connector.so
9. Add AD user groups in /etc/pam.d/shiny-server_allowed_groups
vi /etc/pam.d/shiny-server_allowed_groups <add AD groups here>
10. Build nginx container with PAM module (for PROD, STAGE and DEV)
cd ~ wget https://nginx.org/download/nginx-1.16.0.tar.gz gunzip nginx-1.16.0.tar.gz tar -xf nginx-1.16.0.tar cd nginx-1.16.0
wget https://github.com/sto/ngx_http_auth_pam_module/archive/master.zip mv master.zip ngx_http_auth_pam_module-master.zip unzip ngx_http_auth_pam_module-master.zip
yum groupinstall "Development Tools" yum install openssl-devel pam-devel pcre-devel
./configure --prefix=/opt/nginx-1.16.0 --with-http_ssl_module --add-module=ngx_http_auth_pam_module-master make install
cd ~ mkdir -p nginx/nginx-1.16.0 cd nginx/nginx-1.16.0 cp -r /opt/nginx-1.16.0/* . cd ..
vi Dockerfile
FROM centos
LABEL "version"="1.16.0"
LABEL "description"="nginx with PAM authentication module"
COPY nginx-1.16.0 /opt/nginx-1.16.0/
CMD ln -sf /dev/stdout /opt/nginx-1.16.0/logs/access.log \
&& ln -sf /dev/stderr /opt/nginx-1.16.0/logs/error.log
EXPOSE 80 443
ENTRYPOINT /opt/nginx-1.16.0/sbin/nginx -g "daemon off;"docker build -t="nginx:1.16.0" .
Generate/acquire SSL certificate.
vi /opt/nginx-1.16.0/conf/nginx.conf
events {
}
http {
server {
listen 80;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl;
server_name rstudio.example.com;
ssl_certificate /opt/nginx-1.16.0/conf/rstudio.crt;
ssl_certificate_key /opt/nginx-1.16.0/conf/rstudio.key;
ssl_protocols TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
ssl_prefer_server_ciphers on; location / {
proxy_pass http://rstudio-server:8787;
proxy_redirect http://rstudio-server:8787/ $scheme://$host/;
}
location /shiny/ {
rewrite ^/shiny/(.*)$ /$1 break;
proxy_pass http://shiny-server:3838;
proxy_redirect http://shiny-server:3838/ $scheme://$host/shiny/;
auth_pam "Authentication Required";
auth_pam_service_name "shiny-server";
}
}}Once containers are built, they can be saved into a tar file for later retrieval. You may need to have swift module for Openstack as a docker repo.
docker image save -o /docker-containers/r-3.5.2-2.el7.tar r:3.5.2-2.el7 docker image save -o /docker-containers/rstudio-server-1.2.1335.tar rstudio-server:1.2.1335 docker image save -o /docker-containers/shiny-server-1.5.9.923.tar shiny-server:1.5.9.923 docker image save -o /docker-containers/nginx-1.16.0.tar nginx:1.16.0
docker image load -i /docker-containers/r-3.5.2-2.el7.tar docker image load -i /docker-containers/rstudio-server-1.2.1335.tar docker image load -i /docker-containers/shiny-server-1.5.9.923.tar docker image load -i /docker-containers/nginx-1.16.0.tar
11. Run the containers
docker run -d --network rstudio-net --name rstudio-server \
--mount 'type=volume,dst=/home,volume-opt=type=nfs,volume-opt=device=:/cluster/home,"volume-opt=o=addr=<netapp-ip>,nfsvers=3,hard,retrans=2,nointr,nolock,timeo=600,tcp,fg,rw"' \
--mount 'type=volume,dst=/oracle,volume-opt=type=nfs,volume-opt=device=:/cluster/oracle,"volume-opt=o=addr=<netapp-ip>,nfsvers=3,hard,retrans=2,nointr,nolock,timeo=600,tcp,fg,rw"' \
--mount 'type=volume,dst=/exasol,volume-opt=type=nfs,volume-opt=device=:/cluster/exasol,"volume-opt=o=addr=<netapp-ip>,nfsvers=3,hard,retrans=2,nointr,nolock,timeo=600,tcp,fg,rw"' \
-v /etc/rstudio:/etc/rstudio:ro \
-v /etc/nsswitch.conf:/etc/nsswitch.conf:ro \
-v /etc/pam.d:/etc/pam.d:ro \
-v /usr/lib64/security/pam_vas3.so:/usr/lib64/security/pam_vas3.so:ro \
-v /usr/lib64/security/pam_listfile.so:/usr/lib64/security/pam_listfile.so:ro \
-v /etc/opt/quest:/etc/opt/quest/:ro \
-v /opt/quest:/opt/quest:ro \
-v /opt/quest/lib64/nss/libnss_vas4.so.2:/usr/lib64/tls/x86_64/libnss_vas4.so.2:ro \
-v /var/opt/quest/vas/vasd/.vasd40_ipc_sock:/var/opt/quest/vas/vasd/.vasd40_ipc_sock \
rstudio-server:1.2.1335(for shiny-server in PROD remove --mount options for /home and /stage)
docker run -d --network rstudio-net --name shiny-server \
--mount 'type=volume,dst=/home,volume-opt=type=nfs,volume-opt=device=:/cluster/home,"volume-opt=o=addr=<netapp-ip>,nfsvers=3,hard,retrans=2,nointr,nolock,timeo=600,tcp,fg,rw"' \
--mount 'type=volume,dst=/oracle,volume-opt=type=nfs,volume-opt=device=:/cluster/oracle,"volume-opt=o=addr=<netapp-ip>,nfsvers=3,hard,retrans=2,nointr,nolock,timeo=600,tcp,fg,rw"' \
--mount 'type=volume,dst=/exasol,volume-opt=type=nfs,volume-opt=device=:/cluster/exasol,"volume-opt=o=addr=<netapp-ip>,nfsvers=3,hard,retrans=2,nointr,nolock,timeo=600,tcp,fg,rw"' \
--mount 'type=volume,dst=/prod,volume-opt=type=nfs,volume-opt=device=:/cluster/prod,"volume-opt=o=addr=<netapp-ip>,nfsvers=3,hard,retrans=2,nointr,nolock,timeo=600,tcp,fg,rw"' \
--mount 'type=volume,dst=/stage,volume-opt=type=nfs,volume-opt=device=:/cluster/stage,"volume-opt=o=addr=<netapp-ip>,nfsvers=3,hard,retrans=2,nointr,nolock,timeo=600,tcp,fg,rw"' \
shiny-server:1.5.9.923 docker run -d -p 80:80 -p 443:443 --network rstudio-net --name nginx \
-v /opt/nginx-1.16.0/conf:/opt/nginx-1.16.0/conf:ro \
-v /etc/opt/quest:/etc/opt/quest/:ro \
-v /etc/nsswitch.conf:/etc/nsswitch.conf:ro \
-v /etc/pam.d:/etc/pam.d:ro \
-v /opt/quest:/opt/quest:ro \
-v /opt/quest/lib64/nss/libnss_vas4.so.2:/usr/lib64/tls/x86_64/libnss_vas4.so.2:ro \
-v /usr/lib64/security/pam_vas3.so:/usr/lib64/security/pam_vas3.so:ro \
-v /usr/lib64/security/pam_listfile.so:/usr/lib64/security/pam_listfile.so:ro \
-v /var/opt/quest/vas/vasd/.vasd40_ipc_sock:/var/opt/quest/vas/vasd/.vasd40_ipc_sock \
nginx:1.16.0It is possible to run them via systemd:
docker run -d --name r-container \
--mount 'type=volume,dst=/oracle,volume-opt=type=nfs,volume-opt=device=:/cluster/oracle,"volume-opt=o=addr=<netapp-ip>,nfsvers=3,hard,retrans=2,nointr,nolock,timeo=600,tcp,fg,rw"' \
--mount 'type=volume,dst=/exasol,volume-opt=type=nfs,volume-opt=device=:/cluster/exasol,"volume-opt=o=addr=<netapp-ip>,nfsvers=3,hard,retrans=2,nointr,nolock,timeo=600,tcp,fg,rw"' \
--mount 'type=volume,dst=/prod,volume-opt=type=nfs,volume-opt=device=:/cluster/prod,"volume-opt=o=addr=<netapp-ip>,nfsvers=3,hard,retrans=2,nointr,nolock,timeo=600,tcp,fg,rw"' \
r:3.5.2-2.el7docker stop r-container
vi /etc/systemd/system/docker-r-container.service
[Unit] Description=R container Requires=docker.service After=docker.service
[Service] Restart=on-failure ExecStart=/usr/bin/docker start -a r-container ExecStop=/usr/bin/docker stop r-container
[Install] WantedBy=default.target
systemctl enable docker-r-container systemctl start docker-r-container
Note that /oracle and /exasol NFS mounted directories are where the respective database drivers installed. RTWD uses instantclient_12_1 (instantclient-basic-linux.x64-12.1.0.2.0.zip downloaded from Oracle https://www.oracle.com/technetwork/topics/linuxx86-64soft-092277.html) which is simply unzipped in /oracle and networkadmin directory is added. Exasol drivers are downloaded from https:/www.exasol.com/support/secure/attachment/74441/EXASOL_ODBC-6.0.14.tar.gz and extracted into /exasol
/exasol/EXASOL_ODBC-6.0.14/config_odbc --host=<exasol-ip-range> --mode=config --odbcini /exasol/.odbc.ini --force
was run to produce exasol.odbc.ini file.
tnsnames.ora file should be placed in /oracle/network/admin directory:
RTDW-PROD =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = <oracle-server>)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = <service-name>)
)
)In addition we need to install ROracle package into running R container because the package requires Oracle client libraries that are installed into NFS mount /oracle/instantclient_12_1. ROracle package requires at least basic instantclient (instantclient-basic-linux.x64-12.1.0.2.0.zip) and SDK (instantclient-sdk-linux.x64-12.1.0.2.0.zip). Some configuration steps are required after the files are unzipped in /oracle folder namely:
cd /oracle mkdir rdbms cd rdbms ln -s ../instantclient_12_1/sdk/include public cd /oracle/instantclient_12_1 ln -s libclntsh.so.12.1 libclntsh.so
Then we start R container by running:
docker run --rm -d --network rstudio-net --name r-container \
--mount 'type=volume,dst=/oracle,volume-opt=type=nfs,volume-opt=device=:/cluster/oracle,"volume-opt=o=addr=<netapp-ip>,nfsvers=3,hard,retrans=2,nointr,nolock,timeo=600,tcp,fg,rw"' \
--mount 'type=volume,dst=/exasol,volume-opt=type=nfs,volume-opt=device=:/cluster/exasol,"volume-opt=o=addr=<netapp-ip>,nfsvers=3,hard,retrans=2,nointr,nolock,timeo=600,tcp,fg,rw"' \
--mount 'type=volume,dst=/prod,volume-opt=type=nfs,volume-opt=device=:/cluster/prod,"volume-opt=o=addr=<netapp-ip>,nfsvers=3,hard,retrans=2,nointr,nolock,timeo=600,tcp,fg,rw"' \
r:3.5.2-2.el7Then install ROracle package:
docker ps
docker exec -ti <r-container-ID> bash
R -e "install.packages('ROracle', repos='https://repo-r.example.com/repository/r-proxy-cran')"After that we should commit our change to the image:
docker ps docker commit <R-container-ID> r:3.5.2-2.el7