IBM Support

IBM AIX: Various ssh problems after upgrading to OpenSSH 7.x

Question & Answer


Question

TARGET AUDIENCE:

Clients who upgraded to OpenSSH 7.x from OpenSSH 6.x or earlier

OBJECTIVE:

Provide instructions to resolve various problems after upgrading OpenSSH

Cause

OpenSSH versions 7.1 and higher have more stringent security settings than previous versions of OpenSSH. These settings include:

  • Logging in as the root user with a password is no longer allowed by default
  • Various ciphers, key exchange algorithms, key types, and MACs are not enabled by default
     

Users cannot log in if their session request includes any of the now disabled-by-default features.

In addition, there are a number of bugs in OpenSSH 7.1.102.1100 and 7.5.102.1100 that are fixed in OpenSSH 7.5.102.1500 and above.

IV82042: SSHD HANGS DUE TO A RACE CONDITION INVOLVING PTYS
IV87826: X11 FORWARDING USING OPENSSH FAILS IF $DISPLAY IS UNSET
IV89236: SSHD LEAVES PAG VALUE SET EVEN IF KERBEROS AUTHENTICATION FAILS
IV93325: SSH PRINTS "ERROR" AND SFTP FAILS WITH OPENSSH 7.1 SERVER
IV94291: SCP/SFTP FILE COPY PERFORMANCE DROPS IN OPENSSH 7.1
IV96200: ADDING VALUES WITH "+" TO HOSTKEYALGORITHMS DOESN`T WORK
IV97965: THE DEFAULT CIPHER LIST OF SSHD IS INCORRECT
IJ01208: MAN PAGES BROKEN IN OPENSSH 7.5
IJ02099: ALLOWPKCS12KEYSTOREAUTOOPEN FAILS IN OPENSSH 7 SERVER
IJ03122: OPENSSH 7.5 DOES NOT CREATE USER`S HOME DIRECTORY
IJ03680: COMPRESSION BROKEN IN OPENSSH 7.5
IJ05383: SSHD CRASH WITH KERBEROS

There is also an undocumented change starting in OpenSSH 7.5.102.1100 that reverses an errant change in 7.1.102.1100 that prevented sshd from exiting if UseLogin was set to false (the default) and the user changed their password when prompted.  The intent of the open source community is that sshd exits after a user changes their password during the authentication process (for example, due to the password being expired).  OpenSSH 7.5.102.1100 restores the functionality intended by the open source community.

Although not specific to OpenSSH, there are known bugs in AIX that might cause logins to hang.  APARs IJ04641 and IJ06228 apply to only multi-threaded products such as Tectia.

IV96177: LOGINS HANG TRYING TO WRITE TO UTMP
IJ04641: _ATFORK_PREPARE HANG
IJ06228: _ATFORK_PREPARE HANG WITHOUT LDAP

Answer

Short Answers:

To re-enable DSA keys, add the following lines to /etc/ssh/sshd_config on the server side and just the line PubkeyAcceptedKeyTypes to /etc/ssh/ssh_config on the client side.

HostKeyAlgorithms +ssh-dss
PubkeyAcceptedKeyTypes +ssh-dss

Note:  Because these settings add ssh-dss to the end of the respective options, this change might not resolve the problem on the ssh client side if there are multiple key types in users' known_hosts file for the server.  In this case, set HostKeyAlgorithms in /etc/ssh/ssh_config on the client to the full list of host key types with ssh-dss at the beginning.

HostKeyAlgorithms ssh-dss,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ssh-ed25519,ssh-rsa

To re-enable the old Diffie-Hellman KEX (key exchange) algorithm, add the following line to /etc/ssh/sshd_config and /etc/ssh/ssh_config.

KexAlgorithms +diffie-hellman-group1-sha1

To enable the same ciphers as in OpenSSH 6.x (plus the new ciphers available in OpenSSH 7.x), add the following line to /etc/ssh/sshd_config and ssh_config.

Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com

To re-enable root logins with a password, change the PermitRootLogin option in /etc/ssh/sshd_config to yes.

PermitRootLogin yes

The following error means that the other side does not support Diffie-Hellman keys based on moduli of at least 2048 bits. To resolve the problem, continue to use OpenSSH 6.x on AIX or upgrade the software on the other side to a level that supports moduli of at least 2048 bits.

ssh_dispatch_run_fatal: Connection to X.X.X.X port 22: DH GEX group out of range

Long Answers:

Root Login

By default in OpenSSH 7.1 and higher, the root user may log in only with non-interactive methods, such as with a public/private key pair. To allow root logins with a password, change the PermitRootLogin setting in /etc/ssh/sshd_config to yes:

PermitRootLogin yes

Ciphers

In the versions of OpenSSH on AIX before 7.1, the default cipher list was the same as the list of allowed ciphers:

aes128-ctr
aes192-ctr
aes256-ctr
arcfour256
arcfour128
aes128-cbc
3des-cbc
blowfish-cbc
cast128-cbc
aes192-cbc
aes256-cbc
arcfour
rijndael-cbc@lysator.liu.se

However, due to a bug in OpenSSH 7.1.102.1100, the default allowed cipher list contains only these values:

aes128-ctr
aes192-ctr
aes256-ctr
chacha20-poly1305@openssh.com
aes128-gcm@openssh.com
aes256-gcm@openssh.com

The other ciphers are still present in ssh, but they are not allowed by default. If a client tries to connect with one of the ciphers not on the allowed cipher list and the client doesn't accept any of the ciphers on the allowed cipher list, the server rejects the connection attempt with the error message "no matching cipher found."

To allow specific or additional ciphers in the sshd server, use the Ciphers option in /etc/ssh/sshd_config. You can specify a list of allowed ciphers or add individual ciphers with the "+" option. Examples:

Ciphers blowfish-cbc,aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,chacha20-poly1305@openssh.com,aes128-gcm@openssh.com,aes256-gcm@openssh.com
Ciphers +blowfish-cbc,arcfour256,arcfour128

To specify or add ciphers on the ssh client, use the same Ciphers option but instead set it in either the global /etc/ssh/ssh_config file or each user's individual $HOME/.ssh/config file.

This is the complete list of ciphers supported by OpenSSH as of version 7.5.102.2000.

3des-cbc
blowfish-cbc
cast128-cbc
arcfour
arcfour128
arcfour256
aes128-cbc
aes192-cbc
aes256-cbc
rijndael-cbc@lysator.liu.se
aes128-ctr
aes192-ctr
aes256-ctr
aes128-gcm@openssh.com
aes256-gcm@openssh.com
chacha20-poly1305@openssh.com

Notes:

  • The Ciphers option is a single line.
  • The ciphers in the Ciphers option are separated by commas without spaces.
  • The order of ciphers in the list determines their preference, with the highest preference at the beginning.
  • There is a bug in OpenSSH 7.1 (7.1.102.1100) that prevents the "+" option from working; it is fixed in OpenSSH 7.5 (7.5.102.1100).

Key Exchange Algorithms

In OpenSSH 7.1 and higher, the Diffie-Hellman Group 1 SHA1 algorithm is no longer allowed by default.  If the client does not support other key exchange algorithms, the connection will fail with the message "no matching key exchange method found."

To allow specific key exchange algorithms in the sshd server, use the KexAlgorithms option in /etc/ssh/sshd_config. You can specify a list of allowed key exchange algorithms or add individual algorithms with the "+" option. Examples:

KexAlgorithms diffie-hellman-group1-sha1,diffie-hellman-group14-sha1,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group-exchange-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,curve25519-sha256,curve25519-sha256@libssh.org,gss-group1-sha1-,gss-group14-sha1-,gss-gex-sha1-
KexAlgorithms +diffie-hellman-group1-sha1

To allow specific key exchange algorithms in the ssh client, use the same KexAlgorithms option but instead set it in either the global /etc/ssh/ssh_config file or each user's individual $HOME/.ssh/config file.

This is the complete list of key exchange algorithms supported by OpenSSH as of version 7.5.102.2000.

diffie-hellman-group1-sha1
diffie-hellman-group14-sha1
diffie-hellman-group14-sha256
diffie-hellman-group16-sha512
diffie-hellman-group18-sha512
diffie-hellman-group-exchange-sha1
diffie-hellman-group-exchange-sha256
ecdh-sha2-nistp256
ecdh-sha2-nistp384
ecdh-sha2-nistp521
curve25519-sha256
curve25519-sha256@libssh.org
gss-gex-sha1-
gss-group1-sha1-
gss-group14-sha1-

Notes:

  • The "KexAlgorithms" option is a single line.
  • The algorithms in the "KexAlgorithms" option are separated by commas without spaces.
  • The order of key exchange algorithms in the list determines their preference, with the highest preference at the beginning.

DSA Keys

In OpenSSH 7.1 and higher, DSA key types are no longer allowed by default.  There are two separate settings that control the use of key types.

HostKeyAlgorithms specifies the key type algorithms offered by the server or accepted by the client. Even if the server has a DSA key file and that key file is referenced by the HostKey option, it will not be used unless the ssh-dss algorithm is present in the HostKeyAlgorithms list.

To allow specific key type algorithms in the sshd server, use the HostKeyAlgorithms option in /etc/ssh/sshd_config. You can specify a list of allowed key type algorithms or add individual algorithms with the "+" option. Examples:

HostKeyAlgorithms ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ssh-ed25519,ssh-rsa,ssh-dss
HostKeyAlgorithms +ssh-dss

To have the ssh client accept specific key exchange algorithms, use the same HostKeyAlgorithms option but instead set it in either the global /etc/ssh/ssh_config file or each user's individual $HOME/.ssh/config file.

PubkeyAcceptedKeyTypes specifies which public key types are accepted by the server or offered by the client.

To allow specific public key types from an ssh client, use the PubkeyAcceptedKeyTypes option in /etc/ssh/sshd_config on the sshd server. You can specify a list of allowed key types or add individual types with the "+" option. Examples:

PubkeyAcceptedKeyTypes ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,ssh-rsa,ssh-dss
PubkeyAcceptedKeyTypes +ssh-dss

To have the ssh client offer specific key types, use the same HostKeyAlgorithms option but instead set it in either the global /etc/ssh/ssh_config file or each user's individual $HOME/.ssh/config file.

This is the complete list of key types supported by OpenSSH as of version 7.5.102.2000.

ssh-ed25519
ssh-ed25519-cert-v01@openssh.com
ssh-rsa
ssh-dss
ecdsa-sha2-nistp256
ecdsa-sha2-nistp384
ecdsa-sha2-nistp521
ssh-rsa-cert-v01@openssh.com
ssh-dss-cert-v01@openssh.com
ecdsa-sha2-nistp256-cert-v01@openssh.com
ecdsa-sha2-nistp384-cert-v01@openssh.com
ecdsa-sha2-nistp521-cert-v01@openssh.com

Notes:

  • The HostKeyAlgorithms and PubkeyAcceptedKeyTypes options are single lines.
  • The values in both options are separated by commas and without spaces.
  • The order of key algorithms in the list determines their preference, with the highest preference at the beginning.

MAC Algorithms

Certain MAC algorithms, most notably hmac-md5 and hmac-md5-96, are no longer allowed by default. If the client does not support newer MAC algorithms, the connection may fail with the message "no matching MAC found."

To allow specific or additional MAC algorithms in the sshd server, use the Macs option in /etc/ssh/sshd_config. You can specify a list of allowed MAC algorithms or add individual algorithms with the "+" option. Examples:

Macs umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-md5,hmac-md5-96
Macs +hmac-md5,hmac-md5-96

To have the ssh client use additional MAC algorithms, use the same Macs option but instead set it in either the global /etc/ssh/ssh_config file or each user's individual $HOME/.ssh/config file.

This is the complete list of MAC algorithms supported by OpenSSH as of version 7.5.102.2000.

hmac-sha1
hmac-sha1-96
hmac-sha2-256
hmac-sha2-512
hmac-md5
hmac-md5-96
hmac-ripemd160
hmac-ripemd160@openssh.com
umac-64@openssh.com
umac-128@openssh.com
hmac-sha1-etm@openssh.com
hmac-sha1-96-etm@openssh.com
hmac-sha2-256-etm@openssh.com
hmac-sha2-512-etm@openssh.com
hmac-md5-etm@openssh.com
hmac-md5-96-etm@openssh.com
hmac-ripemd160-etm@openssh.com
umac-64-etm@openssh.com
umac-128-etm@openssh.com

PrintLastLog Option

Between the 6.x and 7.x versions of OpenSSH, the open source community addressed a bug where compiling OpenSSH with DISABLE_LASTLOG did not mark the PrintLastLog option as being unsupported. This bug is fixed and so if the option PrintLastLog appears in the sshd_config file, sshd may display an error message stating that PrintLastLog is an unsupported option. This error is not fatal.

AuthorizedKeysFile Option

While the internal default setting for the AuthorizedKeysFile option has not changed, the entry for it in /etc/ssh/sshd_config is now uncommented by default. This overrides the internal setting in sshd.

When this option is commented out or not present, sshd will search the user's $HOME/.ssh/authorized_keys and $HOME/.ssh/authorized_keys2 files. With the new sshd_config file where AuthorizedKeysFile is uncommented, sshd will only search the $HOME/.ssh/authorized_keys file.

Note that when upgrading from a previous version of OpenSSH, the sshd_config file is not overwritten. Therefore, upgrading to OpenSSH 7.x where the previous version had the AuthorizedKeysFile option commented out will not cause a behavior difference in searching for matching keys. Only new installations or those where OpenSSH was removed entirely before installing a 7.x version will see this behavior difference.

To restore the original behavior, comment out the AuthorizedKeysFile entry in /etc/ssh/sshd_config.

UseDNS Option

Between the 6.x and 7.x versions of OpenSSH, the default value for the UseDNS option changed from "yes" to "no".  With this change, sshd no longer converts a client's IP address back into a host name.

This change causes host-based authentication to fail.  It also prevents sshd from putting the client system's host name in /etc/utmp, which is used by commands like "who" to display a list of logged-in users.  Instead, it writes the client's IP address to /etc/utmp.

To restore the 6.x behavior, add the following option to /etc/ssh/sshd_config:

UseDNS yes

"DH GEX group out of range" Error

Ephemeral keys used by the Diffie-Hellman algorithm are based on moduli (pre-generated prime numbers) of various sizes.  As of OpenSSH 7.x, the minimum bit size of a modulus has increased to 2048.  Old SSH implementations might not support moduli this large.  In this case, OpenSSH 7.x displays an error similar to this.

ssh_dispatch_run_fatal: Connection to X.X.X.X port 22: DH GEX group out of range

The minimum and maximum moduli sizes are hardcoded in OpenSSH 7.x and cannot be changed through a configuration option.  If the SSH software on the opposite end does not support moduli of at least 2048 bits, there are only two options to resolve the problem:  1) Continue to use OpenSSH 6.x on AIX, or 2) upgrade the software on the other side to a version that supports moduli of at least 2048 bits.

SUPPORT:

If the instructions in this document do not lead to resolution of the problem, follow these instructions to open a case.  The product must be under warranty or have an active and valid support contract.

a.  Document or take screen captures of all symptoms, errors, or messages.

b.  Capture any logs or data relevant to the issue.

c.  Contact IBM to open a case.

   -For electronic support, visit the IBM Support Community:
     https://www.ibm.com/mysupport
   -If you require telephone support, visit this web page:
      https://www.ibm.com/planetwide/

d.  Provide a detailed description of the issue and reference this technote.

e.  Upload all of the details and data to the case.

   -You can attach files to the case in the IBM Support Community, or
   -Upload data to IBM test case server analysis at this URL:

    http://www.ibm.com/support/docview.wss?uid=ibm10733581

f.  Click here to submit feedback for this document.

[{"Product":{"code":"SWG10","label":"AIX"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Component":"--","Platform":[{"code":"PF002","label":"AIX"}],"Version":"Version Independent","Edition":"","Line of Business":{"code":"LOB08","label":"Cognitive Systems"}}]

Document Information

Modified date:
15 September 2021

UID

isg3T1025737

Page Feedback