Integrating RStudio Workbench with Slurm#
Overview#
These steps describe how to integrate RStudio Workbench, formerly RStudio Server Pro1, with Launcher and Slurm.
In this configuration, the RStudio Workbench and Launcher services will be installed to one node in the Slurm cluster, and the RStudio Workbench Session Components will be installed on all other Slurm nodes.
Info
Launcher is a new feature of RStudio Workbench 1.22 that is only available under named user licensing. RStudio Workbench 1.2 without Launcher is available under existing server-based licensing. For questions about using Launcher with RStudio Workbench, please contact sales@rstudio.com.
Prerequisites#
- An existing Slurm cluster
- The minimum supported version of Slurm, 20.02.x
Pre-Flight Configuration Checks#
Verifying active Slurm compute nodes
-
On a machine with Slurm configured, ensure that you have one or more worker nodes that are ready to accept jobs as part of the Slurm cluster by running the following command:
Terminal$ sinfo
Verifying functionality with a test job
-
On a machine with Slurm configured, ensure that you are able to deploy a sample jobs to your Slurm cluster by running a test command:
Terminal$ srun date
-
You can also create a sample script called submit.sh:
File: submit.sh#!/bin/bash # #SBATCH --job-name=test #SBATCH --output=res.txt # #SBATCH --ntasks=1 #SBATCH --time=10:00 srun hostname srun sleep 60
-
Then submit the job by running the following command:
Terminal$ sbatch submit.sh
-
Verify that it runs successfully on the Slurm cluster.
Step 1. Install RStudio Workbench on one Slurm node#
You can install RStudio Workbench on any Slurm node that
has access to the Slurm tooling such as sbatch
, srun
, sinfo
, etc. This can
be a Slurm login/submission node, controller node, or compute node.
Note
You only need to install RStudio Workbench on one node. In later steps, you will install R and the RStudio Workbench Session Components on other Slurm nodes.
Step 2. Configure RStudio Workbench with Launcher#
-
On the node where RStudio Workbench is installed, add the following lines to the RStudio Workbench configuration file:
File: /etc/rstudio/rserver.conf# Launcher Config launcher-address=127.0.0.1 launcher-port=5559 launcher-sessions-enabled=1 launcher-default-cluster=Slurm launcher-sessions-callback-address=http://<SERVER-ADDRESS>:8787
-
We recommend that you do the following:
- In the
launcher-sessions-callback-address
setting, you should replace<SERVER-ADDRESS>
with the hostname or IP address of RStudio Workbench. -
You should also change the protocol and port if you are using HTTPS or a different port.
Note
If HTTPS is being used, ensure you have
launcher-use-ssl=1
configured in therserver.conf
file.
- In the
-
Ensure that the
launcher-sessions-callback-address
is reachable from the Slurm compute nodes.
Step 3. Configure Launcher settings and plugins#
-
On the node where RStudio Workbench is installed, add the following lines to the Launcher configuration file:
File: /etc/rstudio/launcher.conf[server] address=127.0.0.1 port=5559 server-user=rstudio-server admin-group=rstudio-server authorization-enabled=1 thread-pool-size=4 enable-debug-logging=1 [cluster] name=Slurm type=Slurm
Step 4. Configure profile for Launcher Slurm plugin#
-
On the node where RStudio Workbench is installed, add the following lines to the Launcher profiles configuration file:
File: /etc/rstudio/launcher.slurm.profiles.conf[*] default-cpus=1 default-mem-mb=512 max-cpus=2 max-mem-mb=1024
Step 5. Configure Launcher with Slurm#
-
On the node where RStudio Workbench is installed, add the following lines to the Launcher Slurm configuration file:
File: /etc/rstudio/launcher.slurm.confslurm-service-user=slurm enable-debug-logging=1
Note
If the Slurm CLI is installed in a non-default location, then it must be specified in the Slurm configuration file. For example:
File: /etc/rstudio/launcher.slurm.confslurm-service-user=slurm slurm-bin-path=/usr/local/bin
If you fail to add the non-default location in the configuration file, then the following error may return:
Error03 Aug 2021 17:42:42 [rstudio-slurm-launcher] ERROR slurm error 7 (Slurm command exited due to an unknown error: /bin/sh: scontrol: command not found
The slurm-service-user
should match the Slurm service user in your cluster.
Step 6. Verify Slurm configuration and cluster environment#
-
Verify that the following requirements are satisfied for RStudio Workbench and Launcher to work with your Slurm cluster:
- All Slurm nodes should have user's home directories mounted via shared file storage with matching user and group IDs across all nodes.
- The
root
user should have read access to all users' home directories. -
In your Slurm configuration file (
slurm.conf
), theMinJobAge
setting should be equal to or greater than thejob-expiry-time
setting in/etc/rstudio/launcher.conf
, which is 24 h by default. For both of them to be 24 h, you would need to setMinJobAge=86400
in yourslurm.conf
.Note
The
MinJobAge
setting inslurm.conf
is configured in seconds, rather than hours.
Step 7. Ensure that R is available on Slurm compute nodes#
- On each Slurm compute node in the cluster (where you did not install RStudio Workbench), you will need to install one or more versions of R and associated R packages to be able to start R sessions via Slurm.
- We recommend installing R to a shared file server or network drive location so that any installed packages are also available across all compute nodes. You can also make use of existing versions of R and environment modules that are available on the cluster.
-
When using multiple versions of R, the shared file
/var/lib/rstudio-server/r-versions
must be reachable by all Slurm nodes. Note that this file is generated by RStudio Workbench, and that its location may be changed by settingr-versions-path=<shared directory>/r-versions
inrserver.conf
.Info
For more information on using Launcher and Slurm with multiple versions of R and module loading, refer to the Multiple Versions of R and Module Loading section of the RStudio Workbench Launcher Administration Guide and the R Versions section of the RStudio Workbench Administration Guide.
- Alternatively, you can manually install R and R packages by following the steps to Install R on each Slurm compute node.
Step 8. Install RStudio Workbench session components on Slurm compute nodes#
On each Slurm compute node in the cluster (where you did not install RStudio
Workbench), you will need to install the RStudio Workbench session components and create an rstudio-server
user
to be able to start R sessions via Slurm.
-
Use the following commands to install the RStudio Workbench session components on each Slurm compute node:
sudo yum install libcurl-devel libuser-devel openssl-devel rrdtool postgresql-libs curl -O https://s3.amazonaws.com/rstudio-ide-build/session/rhel8/rsp-session-rhel8-2022.02.2-485.pro2.tar.gz sudo mkdir -p /usr/lib/rstudio-server sudo tar -zxvf ./rsp-session-rhel8-2022.02.2-485.pro2.tar.gz -C /usr/lib/rstudio-server/ sudo mv /usr/lib/rstudio-server/rsp-session*/* /usr/lib/rstudio-server/ sudo rm -rf /usr/lib/rstudio-server/rsp-session* rm -f ./rsp-session-rhel8-2022.02.2-485.pro2.tar.gz
sudo yum install libcurl-devel libuser-devel openssl-devel rrdtool postgresql-libs curl -O https://s3.amazonaws.com/rstudio-ide-build/session/centos7/rsp-session-centos7-2022.02.2-485.pro2.tar.gz sudo mkdir -p /usr/lib/rstudio-server sudo tar -zxvf ./rsp-session-centos7-2022.02.2-485.pro2.tar.gz -C /usr/lib/rstudio-server/ sudo mv /usr/lib/rstudio-server/rsp-session*/* /usr/lib/rstudio-server/ sudo rm -rf /usr/lib/rstudio-server/rsp-session* rm -f ./rsp-session-centos7-2022.02.2-485.pro2.tar.gz
sudo apt-get install curl libcurl4-gnutls-dev libssl1.0.0 libssl-dev libuser libuser1-dev rrdtool libpq5 curl -O https://s3.amazonaws.com/rstudio-ide-build/session/bionic/rsp-session-bionic-2022.02.2-485.pro2.tar.gz sudo mkdir -p /usr/lib/rstudio-server sudo tar -zxvf ./rsp-session-bionic-2022.02.2-485.pro2.tar.gz -C /usr/lib/rstudio-server/ sudo mv /usr/lib/rstudio-server/rsp-session*/* /usr/lib/rstudio-server/ sudo rm -rf /usr/lib/rstudio-server/rsp-session* rm -f ./rsp-session-bionic-2022.02.2-485.pro2.tar.gz
sudo apt-get install rrdtool libuser libuser1-dev libpq5 curl -O https://s3.amazonaws.com/rstudio-ide-build/session/debian9/rsp-session-debian9-2022.02.2-485.pro2.tar.gz sudo mkdir -p /usr/lib/rstudio-server sudo tar -zxvf ./rsp-session-debian9-2022.02.2-485.pro2.tar.gz -C /usr/lib/rstudio-server/ sudo mv /usr/lib/rstudio-server/rsp-session*/* /usr/lib/rstudio-server/ sudo rm -rf /usr/lib/rstudio-server/rsw-session* rm -f ./rsp-session-debian9-2022.02.2-485.pro2.tar.gz
sudo zypper install libuser-devel curl -O https://s3.amazonaws.com/rstudio-ide-build/session/opensuse/rsp-session-opensuse-2022.02.2-485.pro2.tar.gz sudo mkdir -p /usr/lib/rstudio-server sudo tar -zxvf ./rsp-session-opensuse-2022.02.2-485.pro2.tar.gz -C /usr/lib/rstudio-server/ sudo mv /usr/lib/rstudio-server/rsp-session*/* /usr/lib/rstudio-server/ sudo rm -rf /usr/lib/rstudio-server/rsp-session* rm -f ./rsp-session-opensuse-2022.02.2-485.pro2.tar.gz
Required dependencies
RHEL/CentOS:
- You must install
postgresql-libs
for this distribution because it is a requirement.
Debian/Ubuntu:
- You must install
libpq5
for this distribution because it is a requirement.
- You must install
-
To create
rstudio-server
user on each Slurm compute node, first check theuid
andgid
of therstudio-server
user by runningid rstudio-server
on the RStudio Workbench host machine. Then, on each compute node run the following commands:Terminal$ sudo groupadd --system --gid <gid of rstudio-server> rstudio-server $ sudo useradd --system --gid rstudio-server --uid <uid of rstudio-server> rstudio-server
Step 9. Restart RStudio Workbench and Launcher Services#
-
Run the following to restart services:
Terminal$ sudo rstudio-server restart $ sudo rstudio-launcher restart
Step 10. Test RStudio Workbench with Launcher and Slurm#
- In your browser, navigate to the RStudio Workbench interface and log in.
- Select New Session, then click the Start Session button.
- You can then use the RStudio Session, which will be running as a Slurm job.
Additional Documentation#
For more information on RStudio Workbench and Launcher, refer to the following reference documentation:
Troubleshooting RStudio Workbench and Slurm#
Refer to the support article on Troubleshooting Launcher and Slurm in RStudio Workbench for additional information on troubleshooting RStudio Workbench with Launcher and Slurm.
-
We have renamed RStudio Server Pro to RStudio Workbench. This change reflects the product’s growing support for a wide range of different development environments. Please see our official Announcement and review our FAQ regarding the name change from RStudio Server Pro to RStudio Workbench. ↩
-
We will continue to use the RStudio Server Pro name for references to versions prior to 1.4. ↩