IBM Support

IBM AIX: send_pubkey_test: no mutual signature algorithm

Troubleshooting


Problem

Starting from OpenSSH 8.1.102.2103, RSA keys generated by ssh-rsa host key algorithm is starting to deprecate. As a result, SSH clients running OpenSSH 7.4 or below may get the mentioned error in the verbose ssh output when they try to establish ssh sessions with the latest OpenSSH version (version 8.1.102.2103 at the time this document has been released).
The reason why ssh-rsa host key algorithm is deprecating is because it is using SHA-1 hash algorithm, which lately became weak and easy to attack at low cost, and key length of 1024 bits which is nowadays considered a short key length which is easy to break.

In future releases of OpenSSH, ssh-rsa host key algorithm will be turned off by default, taking out RSA keys out of the scene.

Symptom

Verbose SSH output on client side shows:
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /home/obl/.ssh/id_rsa RSA SHA256:REEGRD03GiHfD6SQwhYQSZeQnI/56QaTfgjq3fgspjw
debug1: send_pubkey_test: no mutual signature algorithm

Cause

The ssh-rsa signature algorithm became weak for two reasons:
  1. They depend on weak host key algorithm SHA-1.
  2. Relatively short key length of 1024 bits.

Resolving The Problem

Enable the more secure RSA SHA-2 signature algorithms on the client side by one of two ways:
1. Adding this line to ssh_config file:
HostKeyAlgorithms +rsa-sha2-256,rsa-sha2-512
2.Using both as options with ssh command, as follows:
# ssh -oHostKeyAlgorithms=+rsa-sha2-256,rsa-sha2-512 user@host
Alternatively, we can enable UpdateHostKeys directive on ssh_config file on the client side as follows: 
UpdateHostKeys yes
This directive with allow the server to send replacement public keys to the clients if specific criteria are met.
In general, the following signature algorithms are considered safe and can replace ssh-rsa:
1. RSA SHA-2 signature algorithms “rsa-sha2-256” and /or “rsa-sha2-512”:
They have the advantage of using the same RSA keys as "ssh-rsa" but with more secure hash algorithms (sha2-256 or sha2-512). They have been supported since OpenSSH 7.5, and they will be used by default if relevant OpenSSH versions on both client and server are in place and the configuration files are using the defaults of host key algorithms. These signature algorithms use minimum key length of 3072 bits.
More on both ssh-sha2-256 and ssh-sha2-512 host key algorithms can be found at RFC8332 page.
2. SSH-ED25519 signature algorithm:
This has been supported since OpenSSH 6.5, but but being used by default. It needs to be listed explicitly in the configuration files to be used.
Find more in RFC8709 webpage.
3.ECDSA-SHA2-NISTP signature algorithms such as:
ecdsa-sha2-nistp256
ecdsa-sha2-nistp384
ecdsa-sha2-nistp521
They have been supported since OpenSSH 5.7. More at RFC5656.
Enable as follows:
1. If both sshd_config and ssh_config on both server and client are not left at their defaults, then we need to add the following:
  • sshd_config file on SSH target:
HostKeyAlgorithms rsa-sha2-256,rsa-sha2-512,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
PubkeyAcceptedKeyTypes rsa-sha2-256,rsa-sha2-512,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
  • ssh_config file on SSH source:
PubkeyAcceptedKeyTypes rsa-sha2-256,rsa-sha2-512,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
SUPPORT

If the instructions in this document do not lead to resolution of the problem, follow these instructions to open a case.  The product must be under warranty or have an active and valid support contract.

a.  Document or take screen captures of all symptoms, errors, or messages.

b.  Capture any logs or data relevant to the issue.

c.  Contact IBM to open a case.

   -For electronic support, visit the IBM Support Community:
     https://www.ibm.com/mysupport
   -If you require telephone support, visit this web page:
      https://www.ibm.com/planetwide/

d.  Provide a detailed description of the issue and reference this technote.

e.  Upload all of the details and data to the case.

   -You can attach files to the case in the IBM Support Community, or
   -Upload data to IBM test case server analysis at this URL:

    http://www.ibm.com/support/docview.wss?uid=ibm10733581

f.  Click here to submit feedback for this document.

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB08","label":"Cognitive Systems"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SWG10","label":"AIX"},"ARM Category":[{"code":"a8m0z000000cwO7AAI","label":"Communication Applications->SSH"}],"ARM Case Number":"TS006575108","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]

Document Information

Modified date:
29 August 2021

UID

ibm16484905

Page Feedback