7 Smoke Test Utility

The Smoke Test utility is provided with the RStudio Launcher Plugin SDK to provide the Plugin developer a way to exercise the basic functionality of the Plugin, particularly in a way that allows the developer to easily attach a debugger to the Plugin. The Smoke Test utility is not intended to replace integration testing with the RStudio Launcher and the RStudio product which will use the launcher.

7.1 Plugin Options

The Smoke Test utility will start the Plugin with the following options, which should not be changed:

--heartbeat-interval-seconds=0

--enable-debug-logging=1

--unprivileged=1 if the Smoke Test utility is launched without root privileges

Other options may be changed via the Plugin’s configuration file, which is usually located at /etc/rstudio/launcher.<plugin-name>.conf.

When running the Smoke Test utility in unprivileged mode (see Starting the Tester for more details) it is advisable to change the scratch-path option to a location that is readable and writable by the run-as user.

7.2 Starting the Tester

The Smoke Test utility takes two required arguments: the full or relative path to the Plugin to test, and the user to test as. It can be run in two modes: privileged and unprivileged. To run the Smoke Test utility in privileged mode, start it with root privileges. For example:

sudo ./rlps-smoke-test ../plugins/MyPlugin/rstudio-myplugin-launcher someUser.

To run the Smoke test utility in unprivileged mode, start it as the user specified in the second argument.

Privileged mode is the most common running environment for a Plugin; however, there are some situations where the Plugin cannot be run in privileged mode, such as within a Docker container, so it is advisable to test both scenarios.

When the Smoke Test utility starts, it will send the Plugin a Bootstrap Request and print the response. The Bootstrap response must be successful, or future requests will fail. Next the utility will enter a loop of printing the menu, waiting for input, and performing the requested action, until the user explicitly exits the utility. At any time, the user can provide q or Q as input to exit the utility.

7.3 Using the Tester

Each time the Smoke Test utility completes a requested action, it will print a menu of available actions. After an action is chosen, the utility will perform the action and print any responses from the Plugin. These responses must be manually verified, as their expected values may vary between Plugin implementations.

This section documents what each menu item will do. For an example of how a Plugin might respond to each of these tests, the Smoke Test utility may be run using the provided sample Local Launcher Plugin.

7.3.1 1. Get cluster info

This menu item will send a Cluster Info request to the Plugin and print the response. It should include all of the maximum and default resource limits, any available placement constraints, the container information, and the available queues. If any of these values are not configured or are not supported by the Plugin, they will not be included in the response.

7.3.2 2. Get all jobs

This menu item will send a Job State request to the plugin with the jobId field set to '*' and print the response. It should show all RStudio jobs in the underlying Job Scheduling System, regardless of state.

7.3.3 3. Get filtered jobs

This menu item will send a Job State request to the plugin with the tags field set to [ "filter job" ] and print the response. It should show any jobs that were launched with menu item 7 or menu item 10.

7.3.4 4. Get running jobs

This menu item will send a Job State request to the plugin with the statuses field set to [ "Running" ] and print the response. It should show any RStudio jobs in the underlying Job Scheduling System that are currently running. It may be useful to run menu item 10 immediately prior to running this menu item in order to have a Job that will still be running.

7.3.5 5. Get finished jobs

This menu item will send a Job State request to the Plugin with the statuses field set to [ "Finished" ] and print the response. It should show any RStudio jobs in the underlying Job Scheduling System that have finished running without a failure. Note that jobs which were killed (e.g. via menu item 17) will not be displayed in this response, but jobs which were stopped (e.g. via menu item 18) should be.

7.3.6 6. Get job statuses

This menu item will send a Job Status Stream request to the Plugin and print the responses. It will attempt to wait for at least one response for each job that was submitted to the Plugin via the Smoke Test utility since it last started, before canceling the stream request. All RStudio Jobs may be included in this response, but it is possible some may not be displayed as the request could be canceled early if there were many RStudio Jobs in the system before the Smoke Test utility was last started.

7.3.7 7. Submit quick job (matches filter)

This menu item submits a basic job to the Job Scheduling System via the Submit Job request and prints the response. This job will match the filter provided to the Job State request sent in menu item 3. The Job name is Quick Job 1.

If this menu item is followed by menu item 11 or menu item 12 the stdout output should be This is an environment variable! and the stderr output should be empty.

7.3.8 8. Submit quick job 2 (doesn’t match filter)

This menu item submits another basic job to the Job Scheduling System via the Submit Job request and prints the response. This job will not match the filter provided to the Job State request sent in menu item 3. The Job name is Quick Job 2.

If this menu item is followed by menu item 11 or menu item 12 the stdout output should be This is a shell command. and the stderr output should be empty.

7.3.9 9. Submit stderr job (doesn’t match filter)

This menu item submits a job to the Job Scheduling System which will emit stderr output via the Submit Job request and prints the response. This job will not match the filter provided to the Job State request sent in menu item 3. The Job name is Stderr job.

If this menu item is followed by menu item 11 or menu item 13 the stdout output should be empty and the stderr output should include an error about the incorrect usage of grep.

7.3.10 10. Submit long job (matches filter)

This menu item submits a longer running job to the Job Scheduling System via the Submit Job request and prints the response. This job will match the filter provided to the Job State request sent in menu item 3. The Job name is Slow job.

This job will iterate from 1 to 11, echoing ${i}... and then sleeping for ${i} seconds at each iteration of the loop. It may be useful to use menu item 11 or menu item 12 immediately after this menu item in order to validate that output streaming works correctly when the output is still being written after the stream has been opened.

7.3.11 11. Stream last job’s output (stdout and stderr)

This menu item sends a Job Output Stream request for both stdout and stderr output to the Plugin for the last Job that was submitted to the Plugin via the Smoke Test utility. If no Job has been submitted to the Plugin since the Smoke Test utility was started, an error will be emitted.

Even if there is no stdout or stderr output for the last job, there should still be a response from the Plugin indicating that the output is empty.

7.3.12 12. Stream last job’s output (stdout)

This menu item sends a Job Output Stream request for only stdout output to the Plugin for the last Job that was submitted to the Plugin via the Smoke Test utility. If no Job has been submitted to the Plugin since the Smoke Test utility was started, an error will be emitted.

Even if there is no stdout output for the last job, there should still be a response from the Plugin indicating that the output is empty.

7.3.13 13. Stream last job’s output (stderr)

This menu item sends a Job Output Stream request for only stderr output to the Plugin for the last Job that was submitted to the Plugin via the Smoke Test utility. If no Job has been submitted to the Plugin since the Smoke Test utility was started, an error will be emitted.

Even if there is no stderr output for the last job, there should still be a response from the Plugin indicating that the output is empty.

7.3.14 14. Stream last jobs’ resource utilization (must be running)

This menu item sends a Job Resource Utilization Stream request for the last Job that was submitted to the Plugin via the Smoke Test utility. If no Job has been submitted to the Plugin since the Smoke Test utility was started, an error will be emitted.

The Job must currently be running in order for resource utilization data to be returned. If the Job has already completed, there should be still be a response from the Plugin indicating that the stream is complete.

Running menu item 10 immediately before this menu item may provide the best possibility of observing resource utilization stream responses from the Plugin.

7.3.15 15. Get last job’s network information

This menu item sends a Job Network request to the Plugin for the last Job that was submitted to the Plugin via the Smoke Test utility. If no Job has been submitted to the Plugin since the Smoke Test utility was started, an error will be emitted.

The network information displayed in the response should match the network information of the machine which ran the Job, according to the underlying Job Scheduling system.

7.3.16 16. Submit a slow job and then cancel it

This menu item submits a slow job to the Job Scheduling System via the Submit Job request and then attempts to immediately cancel it via the Control Job request with the operation set to 4 (see the JobOperation section). Depending on the speed of the Job Scheduling system, it is possible that the Job will enter the Running state before the Cancel request is received. Because of this, it is not unexpected that an Error Response with an error code of InvalidJobState may be returned by the Plugin.

If the cancel operation completed successfully, invoking menu item 11, menu item 12, or menu item 13 should show that the Job emitted no output.

7.3.17 17. Submit a slow job and then kill it

This menu item submits a slow job to the Job Scheduling System via the Submit Job request, waits for one second, and then attempts to kill it via the Control Job request with the operation set to 3 (see the JobOperation section). Depending on the speed of the Job Scheduling system, it is possible that the Job will not have entered the Running state before the Kill request is received. Because of this, it is not unexpected that an Error Response with an error code of InvalidJobState may be returned by the Plugin.

If the kill operation completed successfully, invoking menu item 11, menu item 12, or menu item 13 should show that the Job emitted no output.

7.3.18 18. Submit a slow job and then stop it

This menu item submits a slow job to the Job Scheduling System via the Submit Job request, waits for one second, and then attempts to stop it via the Control Job request with the operation set to 2 (see the JobOperation section). Depending on the speed of the Job Scheduling system, it is possible that the Job will not have entered the Running state before the Kill request is received. Because of this, it is not unexpected that an Error Response with an error code of InvalidJobState may be returned by the Plugin.

If the kill operation completed successfully, invoking menu item 11, menu item 12, or menu item 13 should show that the Job emitted no output.

7.3.19 19. Submit a slow job, suspend it, and then resume it

This menu item submits a slow job to the Job Scheduling System via the Submit Job request, waits for one second, and then attempts to suspend it via the Control Job request with the operation set to 1 (see the JobOperation section). Next the utility waits for one second and then attempts to resume the job via the Control Job request with the operation set to 0 (see the JobOperation section).

Depending on the speed of the Job Scheduling system, it is possible that the Job will not have entered the Running state before the Suspend request is received. Because of this, it is not unexpected that an Error Response with an error code of InvalidJobState may be returned by the Plugin.

If the kill operation completed successfully, invoking menu item 11 or menu item 12 should show that the Job emitted "Done." to standard output after the Job finishes.