Skip to content

Integrating RStudio Server Pro with Slurm#

Info

Launcher is a new feature of RStudio Server Pro 1.2 that is only available under named user licensing. RStudio Server Pro 1.2 without Launcher is available under existing server-based licensing. For questions about using Launcher with RStudio Server Pro, please contact sales@rstudio.com.

Overview#

These steps describe how to integrate RStudio Server Pro with Launcher and Slurm.

In this configuration, the RStudio Server Pro and Launcher services will be installed to one node in the Slurm cluster, and the RStudio Server Pro Session Components will be installed on all other Slurm nodes.

Prerequisites#

  • An existing Slurm cluster

Pre-Flight Configuration Checks#

Verifying active Slurm compute nodes

  • On a machine with Slurm configured, ensure that you have one or more worker nodes that are ready to accept jobs as part of the Slurm cluster by running the following command:

    Terminal
    $ sinfo
    

Verifying functionality with a test job

  • On a machine with Slurm configured, ensure that you are able to deploy a sample jobs to your Slurm cluster by running a test command:

    Terminal
    $ srun date
    

  • You can also create a sample script called submit.sh:

    File: submit.sh
    #!/bin/bash
    #
    #SBATCH --job-name=test
    #SBATCH --output=res.txt
    #
    #SBATCH --ntasks=1
    #SBATCH --time=10:00
    
    srun hostname
    srun sleep 60
    

  • Then submit the job by running the following command:

    Terminal
    $ sbatch submit.sh
    

  • Verify that it runs successfully on the Slurm cluster.

Step 1. Install RStudio Server Pro on one Slurm node#

You can install RStudio Server Pro on any Slurm node that has access to the Slurm tooling such as sbatch, srun, sinfo, etc. This can be a Slurm login/submission node, controller node, or compute node.

Note

You only need to install RStudio Server Pro on one node. In later steps, you will install R and the RStudio Server Pro Session Components on other Slurm nodes.

Step 2. Configure RStudio Server Pro with Launcher#

  • On the node where RStudio Server Pro is installed, add the following lines to the RStudio Server Pro configuration file:

    File: /etc/rstudio/rserver.conf
    # Launcher Config
    launcher-address=127.0.0.1
    launcher-port=5559
    launcher-sessions-enabled=1
    launcher-default-cluster=Slurm
    launcher-sessions-callback-address=http://<SERVER-ADDRESS>:8787
    

  • We recommend that you do the following:

    • In the launcher-sessions-callback-address setting, you should replace <SERVER-ADDRESS> with the hostname or IP address of RStudio Server Pro.
    • You should also change the protocol and port if you are using HTTPS or a different port.
  • Ensure that the launcher-sessions-callback-address is reachable from the Slurm compute nodes.

Step 3. Configure Launcher settings and plugins#

  • On the node where RStudio Server Pro is installed, add the following lines to the Launcher configuration file:

    File: /etc/rstudio/launcher.conf
    [server]
    address=127.0.0.1
    port=5559
    server-user=rstudio-server
    admin-group=rstudio-server
    authorization-enabled=1
    thread-pool-size=4
    enable-debug-logging=1
    
    [cluster]
    name=Slurm
    type=Slurm
    

Step 4. Configure profile for Launcher Slurm plugin#

  • On the node where RStudio Server Pro is installed, add the following lines to the Launcher profiles configuration file:

    File: /etc/rstudio/launcher.slurm.profiles.conf
    [*]
    default-cpus=1
    default-mem-mb=512
    max-cpus=2
    max-mem-mb=1024
    

Step 5. Configure Launcher with Slurm#

  • On the node where RStudio Server Pro is installed, add the following lines to the Launcher Slurm configuration file:

    File: /etc/rstudio/launcher.slurm.conf
    slurm-service-user=slurm
    enable-debug-logging=1
    

The slurm-service-user should match the Slurm service user in your cluster.

Step 6. Verify Slurm configuration and cluster environment#

  • Verify that the following requirements are satisfied for RStudio Server Pro and Launcher to work with your Slurm cluster:

    • All Slurm nodes should have user's home directories mounted via shared file storage with matching user and group IDs across all nodes.
    • The root user should have read access to all users' home directories.
    • In your Slurm configuration file (slurm.conf), the MinJobAge setting should be equal to or greater than the job-expiry-time setting in /etc/rstudio/launcher.conf, which is 24 h by default. For both of them to be 24 h, you would need to set MinJobAge=86400 in your slurm.conf.

      Note

      The MinJobAge setting in slurm.conf is configured in seconds, rather than hours.

Step 7. Ensure that R is available on Slurm compute nodes#

  • On each Slurm compute node in the cluster (where you did not install RStudio Server Pro), you will need to install one or more versions of R and associated R packages to be able to start R sessions via Slurm.
  • We recommend installing R to a shared file server or network drive location so that any installed packages are also available across all compute nodes. You can also make use of existing versions of R and environment modules that are available on the cluster.

    Info

    For more information on using Launcher and Slurm with multiple versions of R and module loading, refer to the Multiple Versions of R and Module Loading section of the RStudio Server Pro Launcher Administration Guide and the R Versions section of the RStudio Server Pro Administration Guide.

  • Alternatively, you can manually install R and R packages by following the steps to Install R on each Slurm compute node.

Step 8. Install RStudio Server Pro session components on Slurm compute nodes#

On each Slurm compute node in the cluster (where you did not install RStudio Server Pro), you will need to install the RStudio Server Pro session components to be able to start R sessions via Slurm.

  • Use the following commands to install the RStudio Server Pro session components on each Slurm compute node:

    sudo yum install libcurl-devel libuser-devel openssl-devel rrdtool
    
    curl -O https://s3.amazonaws.com/rstudio-ide-build/session/centos6/rsp-session-centos6-1.3.1073-1.tar.gz
    sudo mkdir -p /usr/lib/rstudio-server
    sudo tar -zxvf ./rsp-session-centos6-1.3.1073-1.tar.gz -C /usr/lib/rstudio-server/
    sudo mv /usr/lib/rstudio-server/rsp-session*/* /usr/lib/rstudio-server/
    sudo rm -rf /usr/lib/rstudio-server/rsp-session*
    rm -f ./rsp-session-centos6-1.3.1073-1.tar.gz
    
    sudo apt-get install curl libcurl4-gnutls-dev libssl1.0.0 libssl-dev libuser libuser1-dev rrdtool
    
    curl -O https://s3.amazonaws.com/rstudio-ide-build/session/xenial/rsp-session-xenial-1.3.1073-1.tar.gz
    sudo mkdir -p /usr/lib/rstudio-server
    sudo tar -zxvf ./rsp-session-xenial-1.3.1073-1.tar.gz -C /usr/lib/rstudio-server/
    sudo mv /usr/lib/rstudio-server/rsp-session*/* /usr/lib/rstudio-server/
    sudo rm -rf /usr/lib/rstudio-server/rsp-session*
    rm -f ./rsp-session-xenial-1.3.1073-1.tar.gz
    
    sudo apt-get install rrdtool libuser libuser1-dev
    
    curl -O https://s3.amazonaws.com/rstudio-ide-build/session/debian9/rsp-session-debian9-1.3.1073-1.tar.gz
    sudo mkdir -p /usr/lib/rstudio-server
    sudo tar -zxvf ./rsp-session-debian9-1.3.1073-1.tar.gz -C /usr/lib/rstudio-server/
    sudo mv /usr/lib/rstudio-server/rsp-session*/* /usr/lib/rstudio-server/
    sudo rm -rf /usr/lib/rstudio-server/rsp-session*
    rm -f ./rsp-session-debian9-1.3.1073-1.tar.gz
    
    sudo zypper install libuser-devel
    
    curl -O https://s3.amazonaws.com/rstudio-ide-build/session/opensuse/rsp-session-opensuse-1.3.1073-1.tar.gz
    sudo mkdir -p /usr/lib/rstudio-server
    sudo tar -zxvf ./rsp-session-opensuse-1.3.1073-1.tar.gz -C /usr/lib/rstudio-server/
    sudo mv /usr/lib/rstudio-server/rsp-session*/* /usr/lib/rstudio-server/
    sudo rm -rf /usr/lib/rstudio-server/rsp-session*
    rm -f ./rsp-session-opensuse-1.3.1073-1.tar.gz
    

Step 9. Restart RStudio Server Pro and Launcher Services#

  • Run the following to restart services:

    Terminal
    $ sudo rstudio-server restart
    $ sudo rstudio-launcher restart
    

Step 10. Test RStudio Server Pro with Launcher and Slurm#

  • In your browser, navigate to the RStudio Server Pro interface and log in.
  • Select New Session, then click the Start Session button.
  • You can then use the RStudio Session, which will be running as a Slurm job.

Additional Documentation#

For more information on RStudio Server Pro and Launcher, refer to the following reference documentation:

Troubleshooting RStudio Server Pro and Slurm#

Refer to the support article on Troubleshooting Launcher and Slurm in RStudio Server Pro for additional information on troubleshooting RStudio Server Pro with Launcher and Slurm.