I recently setup a bastion server to use with my Kubernetes Rundeck deployment. I’ve done this before and the process is pretty straightforward. You add the node details to Rundeck, add the SSH Public key to the authorized_keys file on the Server, and then try connect from Rundeck.

This should just work, it’s simple SSH connection. If it doesn’t work it’s usually due to badly configured security group rules or network routes.

This time it failed with an authentication error.

Authentication failure connecting to node: "server1". Could not authenticate.
Failed dispatching to node server1: [jsch-scp] Failed copying the file: Authentication failure connecting to node: "server1". Could not authenticate.
Execution failed: 79679 in project Demo: [Workflow result: , step failures: {1=Dispatch failed on 1 nodes: [server1: AuthenticationFailure: [jsch-scp] Failed copying the file: Authentication failure connecting to node: "server1". Could not authenticate.]}, Node failures: {server1=[AuthenticationFailure: [jsch-scp] Failed copying the file: Authentication failure connecting to node: "server1". Could not authenticate.]}, status: failed]

I checked all the SSH keys to make sure everything was 100% correct. I then ran kubectl exec to get in to my Rundeck pod and tried to manually ssh from the pod to the server. That works 😕

I compared the entire setup to my known working configuration and the only difference was that the working servers were all using Ubuntu 18.04 and the failing server is running Ubuntu 22.04. After a bit of googling to try figure out my next move, I inspected the /var/log/auth.log file

Dec 12 11:17:01 ip-aa-bb-cc-dd CRON[1184]: pam_unix(cron:session): session closed for user root
Dec 12 11:21:50 ip-aa-bb-cc-dd sshd[1495]: userauth_pubkey: key type ssh-rsa not in PubkeyAcceptedAlgorithms [preauth]
Dec 12 11:21:51 ip-aa-bb-cc-dd sshd[1495]: error: Received disconnect from ww.xx.yy.zz port 19887:3: com.jcraft.jsch.JSchException: Auth fail [preauth]
Dec 12 11:21:51 ip-aa-bb-cc-dd sshd[1495]: Disconnected from authenticating user ubuntu ww.xx.yy.zz port 19887 [preauth]

Aha - key type ssh-rsa not in PubkeyAcceptedAlgorithms. It turns out that the RSA SHA-1 algorithm is being deprecated and in Ubuntu 22.04 (and perhaps earlier) it is disabled by default. It is easy enough to reenable.

Create the file /etc/ssh/sshd_config.d/10-rundeck-login.conf as below:

PubkeyAuthentication yes
PubkeyAcceptedKeyTypes=+ssh-rsa

Then restart/reload the sshd service

sudo systemctl reload sshd

The connection will work now. This is not a great long term solution because the ssh-rsa algorithm was deprecated due to security risks. However, until Rundeck updates the underlying jcraft package or stops using the ssh-rsa algorithm this is the only way to connect to a server running a new version of OpenSSH.