A Tricky AS2 Issue and its Lessons

Body

A few months ago, I had a tricky AS2 issue that was rather difficult to diagnose due to the junction of different settings. A customer had just installed 5020603 and was testing prior to going into production. They were planning to migrate off of a non-IBM AS2 product, but were using it to act as the client to send AS2 data to SB2BI for their testing prior to the migration. Every attempt to send AS2 data to SB2BI failed with an error.

There were a few of things that together caused the problem. I am going to explain those and mention some best practices related to the conditions that might have helped avoid it. Those were as follows:

1. The AS2 partner in SB2BI had been set up to use TLS 1.2 for the testing. However, up until then, only TLS 1.0 had ever been used in production with the non-IBM AS2 software.

The best practice to use here is to limit changes to the minimum number possible at one time. It would have been better to first try to duplicate on the new software what was already being done with the old software, i.e. a connection at TLS 1.0. The time to look at adding a second change, the change from TLS 1.0 to TLS 1.2, would be after getting the original setup to work on the new system. The fewer changes that you introduce at one time the better, because it reduces the scope of possible causes for

any issue that comes up.

2. The non-IBM AS2 client being used was not sending the "signature_algorithms" extension in the Client Hello. That is something that can only be seen by doing a TCP capture during an attempt to connect. This mattered because the RFC that covers TLS 1.0 has different content than the RFC that covers TLS 1.2. At TLS 1.2, the client is not required to send the "signature_algorithms" extension in the Client Hello, but if it does not, and an RSA cipher is being used, which was the case here, then the server is required to assume a signature algorithm of RSA SHA1 from the client.

This illustrates the importance of keeping your software up to date. Newer software would likely have either (a) automatically sent the "signature_algorithms" extension when TLS 1.2 was selected or (b) if it was not programmed for that, automatically blocked the selection of RSA SHA256 as the signing algorithm once TLS 1.2 had been selected. If it had done either, the issue might have been avoided.

3. The partner setup on the AS2 client software was configured to use RSA SHA256 as the signing algorithm. That would not have mattered had it not been for the fact that conditions #1 and #2 were also in place.

Just as with #2, this illustrates the importance of keeping your software up to date, and for the same reason. 

4. The client was only sending two items in the cipher suites portion of the Client Hello, TLS_EMPTY_RENEGOTIATION_INFO_SCSV and SSL_RSA_PSK_WITH_RC4_128_SHA. The former is not really a cipher suite, even though it is sent with the cipher suites. It has other uses. So, in effect, the client was only offering one possible cipher suite in its negotiation with the server.

While the sending of only one cipher suite was not the cause of the problem in this case, and is not prohibited, the best practice is to not limit yourself to so few options in your cipher suite selection. That is because there is no guarantee that a trading partner's server will have that one option enabled on their side, and if it does not, the session will fail due to the lack of a mutually acceptable cipher suite.

The set of conditions in place meant that a connection failure was inevitable. The Client Hello by the non_IBM AS2 software only allowed for SSL_RSA_PSK_WITH_RC4_128_SHA at TLS 1.2, but did not send the "signature_algorithms" extension. [Note: SB2BI operates using SSL or TLS based on which is used in the request from the client. Therefore, it does not care if the names of the cipher suites presented by the client contain "TLS" or "SSL". It will use the requested

one. ]  The SB2BI AS2 server was therefore required to expect a signature algorithm of RSA SHA1, but was receiving a signature algorithm of RSA SHA256, so it dropped the connection.
 
This was a situation in which the non-IBM AS2 client was at fault as it should not have been possible for it to both (a) not send the "signature_algorithms" extension and (b) use an RSA cipher suite with RSA SHA256 as the signature algorithm during a TLS 1.2 transfer attempt. By doing so, it violated the TLS 1.2 RFC. However, the average AS2 software user probably does not read the RFC to determine if their AS2 software is compliant with it. The non-IBM AS2 software that the customer had been using had worked for them, albeit at a different TLS version which operated under a different RTC. Due to that, it looked to the customer like the issue was with SB2BI until shown differently.