Skip to content

Troubleshooting Launcher and Kubernetes:
Verify Kubernetes Certificate Configuration#

Symptoms#

  • Unable to start new remote sessions in Kubernetes

Error messages#

When inspecting the log files for RStudio Server Pro, Launcher, and Kubernetes, you might see errors similar to the following:

RStudio Server Pro Home Page

Job information unavailable. Sessions are read-only.
Contact your administrator for help.

Error occurred while executing method (100)
Connection refused

void rstudio::core::http::TcpIpAsyncConnector::handleConnect(const rstudio_boost::system::error_code&, rstudio_boost::asio::ip::basic_resolver<rstudio_boost::asio::ip::tcp>::iterator) /var/lib/jenkins/workspace/IDE/pro-pipeline/v1.2-patch/src/cpp/core/include/core/http/TcpIpAsyncConnector.hpp:197

File: /var/lib/rstudio-server/monitor/log/rstudio-server.log

04 May 2020 04:22:35 [rserver] ERROR system error 111 (Connection refused); OCCURRED AT: void rstudio::core::http::TcpIpAsyncConnector::handleConnect(const rstudio_boost::system::error_code&, rstudio_boost::asio::ip::basic_resolver<rstudio_boost::asio::ip::tcp>::iterator) /var/lib/jenkins/workspace/IDE/pro-pipeline/v1.2-patch/src/cpp/core/include/core/http/TcpIpAsyncConnector.hpp:197; LOGGED FROM: rstudio::server::job_launcher::{anonymous}::ensureServerUserIsLauncherAdmin()::__lambda7::__lambda10 /var/lib/jenkins/workspace/IDE/pro-pipeline/v1.2-patch/src/cpp/server/ServerJobLauncher.cpp:678

File: /var/lib/rstudio-launcher/rstudio-launcher.log

04 May 2020 04:22:17 [rstudio-launcher] Bootstrapping plugin Kubernetes
04 May 2020 04:22:17 [rstudio-launcher] Sending message to plugin Kubernetes: {"messageType":1,"requestId":0}
04 May 2020 04:22:17 [rstudio-launcher] Plugin Kubernetes exited with code 1
04 May 2020 04:22:17 [rstudio-launcher] ERROR Could not start plugin Kubernetes; LOGGED FROM: rstudio::core::Error rstudio::job_launcher::plugins::PluginManager::start() /var/lib/jenkins/workspace/IDE/pro-pipeline/v1.2-patch/src/cpp/job_launcher/plugins/PluginManager.cpp:148
04 May 2020 04:22:17 [rstudio-launcher] ERROR system error 71 (Protocol error) [description=Failed to properly bootstrap plugin Kubernetes]; OCCURRED AT: rstudio::core::Error rstudio::job_launcher::plugins::Plugin::bootstrap() /var/lib/jenkins/workspace/IDE/pro-pipeline/v1.2-patch/src/cpp/job_launcher/plugins/Plugin.cpp:189; LOGGED FROM: int main(int, char* const*) /var/lib/jenkins/workspace/IDE/pro-pipeline/v1.2-patch/src/cpp/job_launcher/LauncherMain.cpp:240

File: /var/lib/rstudio-launcher/Kubernetes/rstudio-kubernetes-launcher.log

04 May 2020 04:22:17 [rstudio-kubernetes-launcher] ERROR asio.netdb error 1 (Host not found (authoritative)); OCCURRED AT: void rstudio::core::http::TcpIpAsyncConnector::handleResolve(const rstudio_boost::system::error_code&, rstudio_boost::asio::ip::basic_resolver<rstudio_boost::asio::ip::tcp>::iterator) /var/lib/jenkins/workspace/IDE/pro-pipeline/v1.2-patch/src/cpp/core/include/core/http/TcpIpAsyncConnector.hpp:150; LOGGED FROM: int rstudio::job_launcher::impls::entry_point::run(rstudio_boost::shared_ptr<rstudio::job_launcher::impls::ApiBase>, rstudio_boost::shared_ptr<rstudio::job_launcher::impls::FrameworkCommunicator>) /var/lib/jenkins/workspace/IDE/pro-pipeline/v1.2-patch/src/cpp/job_launcher/impls/EntryPoint.cpp:134

File: /var/lib/rstudio-launcher/Kubernetes/rstudio-kubernetes-launcher.log

04 May 2020 04:25:06 [rstudio-kubernetes-launcher] ERROR system error 22 (Invalid argument) [description=Invalid Kubernetes URL]; OCCURRED AT: rstudio::core::Error rstudio::job_launcher::impls::kubernetes::KubernetesApi::parseUrl() /var/lib/jenkins/workspace/IDE/pro-pipeline/v1.2-patch/src/cpp/job_launcher/impls/kubernetes/KubernetesApi.cpp:55; LOGGED FROM: int rstudio::job_launcher::impls::entry_point::run(rstudio_boost::shared_ptr<rstudio::job_launcher::impls::ApiBase>, rstudio_boost::shared_ptr<rstudio::job_launcher::impls::FrameworkCommunicator>) /var/lib/jenkins/workspace/IDE/pro-pipeline/v1.2-patch/src/cpp/job_launcher/impls/EntryPoint.cpp:134

File: /var/lib/rstudio-launcher/Kubernetes/rstudio-kubernetes-launcher.log

04 May 2020 04:28:47 [rstudio-kubernetes-launcher] ERROR system error 110 (Connection timed out); OCCURRED AT: void rstudio::core::http::TcpIpAsyncConnector::handleConnect(const rstudio_boost::system::error_code&, rstudio_boost::asio::ip::basic_resolver<rstudio_boost::asio::ip::tcp>::iterator) /var/lib/jenkins/workspace/IDE/pro-pipeline/v1.2-patch/src/cpp/core/include/core/http/TcpIpAsyncConnector.hpp:197; LOGGED FROM: int rstudio::job_launcher::impls::entry_point::run(rstudio_boost::shared_ptr<rstudio::job_launcher::impls::ApiBase>, rstudio_boost::shared_ptr<rstudio::job_launcher::impls::FrameworkCommunicator>) /var/lib/jenkins/workspace/IDE/pro-pipeline/v1.2-patch/src/cpp/job_launcher/impls/EntryPoint.cpp:134

Possible cause#

For RStudio Server Pro, Launcher, and Kubernetes to function properly, Launcher needs to be able to communicate to the Kubernetes API.

The following troubleshooting steps will help you verify that you have correctly configured the Kubernetes API URL in the Kubernetes plugin configuration file.

Troubleshooting steps#

Verify that you are using a valid Kubernetes API URL in the Kubernetes plugin configuration:

File: /etc/rstudio/launcher.kubernetes.conf

api-url=<KUBERNETES-API-ENDPOINT>

where <KUBERNETES-API-ENDPOINT> is the URL for the Kubernetes API such as https://kubernetes.yourcompany.com.

Ensure that the protocol is included at the beginning of the Kubernetes API URL, such as https://.

You should also include the port at the end of the Kubernetes API URL if you are using a different port, such as https://kubernetes.yourcompany.com:6443.

Verify that your Kubernetes API URL appears similar to the above example, that it was copied to the configuration file in its entirety, and that it is associated with the correct Kubernetes cluster.

Restart services and test#

After you've verified that you have configured the correct Kubernetes api-url in the Kubernetes plugin configuration file, restart the RStudio Server Pro and Launcher services:

Terminal

$ sudo rstudio-server restart
$ sudo rstudio-launcher restart

Verify that the services are running and try to start a new session from the RStudio Server Pro home page.

If the services are still not starting or you are still experiencing errors when starting a new session, then proceed to
Step 9 - Verify Kubernetes Token Configuration.