APAR status
Closed as program error.
Error description
A file is transferred from a Linux system to an FTP server running on Microsoft Windows using the IBM MQ Managed File Transfer via a Protocol Bridge Agent. The filename contains a mix of ASCII and Japanese characters. After the transfer has been completed, the filename on the Windows system is corrupt, containing multiple non-ASCII characters in place of each Japanese character. In addition, it is observed that when attempting to transfer files from the FTP server back to the Linux system, where the files contain characters outside of the ASCII range, IBM MQ Managed File Transfer fails the transfer reporting that it cannot locate the files.
Local fix
Problem summary
**************************************************************** USERS AFFECTED: This issue affects users of the Protocol Bridge Agent who are transferring files to/from an FTP/FTPS server which have a name (which includes the location of where the file is, or is going to on the FTP server) which contains characters which are outside of the 7-bit ASCII range. Platforms affected: MultiPlatform **************************************************************** PROBLEM DESCRIPTION: When the Protocol Bridge Agent is used to transfer a file to or from an FTP server, data sent over the control channel which is established between the Protocol Bridge Agent and the FTP server is encoded by default using the UTF-8 encoding sequence, as strongly recommended by RFC 2640 in section 2: https://tools.ietf.org/html/rfc2640 which extended the original FTP specification to add support for a level of internationalisation. The Protocol Bridge Agent is capable of using other encoding formats, and is configured through the use of the "controlEncoding" property, as described in the Knowledge Center on the following URI: https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_8.0.0/com. ibm.wmqfte.doc/protocol_bridge_properties_format.htm However it should be noted that the FTP specification does not provide a mechanism for negotiation of the character encoding used on the control channel. This means that changing the value of "controlEncoding" may not have desirable effects unless the user also configures the FTP(S) server to match the defined character encoding as specified on the "controlEncoding" property. This requires the FTP server to provide a mechanism to configure the encoding. Unfortunately not all FTP servers have adopted the above RFC 2640 recommendation of using UTF-8 for the default encoding of the control channel. For example the Microsoft Windows server 2012 IIS FTP server running in the EN locale appears to interpret the bytes sent over the control channel using the Windows-1252 encoding. The result of this is that is you are transferring a file to the Windows FTP server, where the filename is: ABAB?ABAB which has the UTF-8 encoded byte values: 0x41, 0x42, 0x41, 0x42, 0xe5 0x8c 0xbb, 0x41, 0x42, 0x41, 0x42 When this is transferred to the Windows FTP server, three characters are seen in place of the 3-byte UTF-8 encoded character: ABABå?»ABAB It was also observed that other FTP clients ('lftp' and 'FileZilla') can successfully transfer the same filename to the Windows FTP server, while preserving the character representation.
Problem conclusion
An expired RFC draft publication provides a hint for what command the Microsoft Windows IIS FTP server is expecting to receive to negotiate the character encoding used on the control channel, which is the RFC: https://tools.ietf.org/html/draft-ietf-ftpext-utf-8-option-00 This states: "The user issues the OPTS UTF-8 command to indicate its willingness to send and receive UTF-8 encoded pathnames over the control connection. Prior to sending this command, the user should not transmit UTF-8 encoded pathnames." After testing, it was found that the Windows FTP server has a "UTF8" feature, which makes its receptive to the FTP enabling command: OPTS UTF8 ON This results in the Windows FTP server interpreting the bytes received as being encoded in UTF-8. As a consequence, support to make use of this options command has been added to the Protocol Bridge Agent. In order to configure the Protocol Bridge Agent to request this option, the following attribute and value needs to be added to the "tns:ftpServer" element of the Protocol Bridge Agent's "ProtocolBridgeProperties.xml" file: requestUTF8ControlEncoding="true" For example, for a default location MQ installation your "ProtocolBridgeProperties.xml" file will be located on the filesystem at the location: /var/mqm/mqft/config/<MFT_QMGR>/agents/<PBA_AGENT_NAME>/Protocol BridgeProperties.xml replacing <MFT_QMGR> with your MFT queue manager name, and <PBA_AGENT_NAME> with your protocol bridge agent's name. Within this file you will find an element with the name "tns:ftpServer". Add the above attribute and value this element, for example as follows: <tns:ftpServer name="FTP_server_host" host="FTP_server_address" platform="WINDOWS" timeZone="Europe/London" locale="en_GB" fileEncoding="UTF-8" controlEncoding="UTF-8" requestUTF8ControlEncoding="true" listFormat="unix" limitedWrite="false" /> By default this option is not used, in keeping with the recommendations of the FTP specification. --------------------------------------------------------------- The fix is targeted for delivery in the following PTFs: Version Maintenance Level v8.0 8.0.0.8 v9.0 CD 9.0.4 v9.0 LTS 9.0.0.2 The latest available FTE maintenance can be obtained from 'Fix List for WebSphere MQ File Transfer Edition 7.0' http://www-01.ibm.com/support/docview.wss?uid=swg27015313 The latest available MQ maintenance can be obtained from 'WebSphere MQ Recommended Fixes' http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037 If the maintenance level is not yet available information on its planned availability can be found in 'WebSphere MQ Planned Maintenance Release Dates' http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309 ---------------------------------------------------------------
Temporary fix
Comments
APAR Information
APAR number
IT19455
Reported component name
WMQ MFT V8.0
Reported component ID
5724H7252
Reported release
800
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2017-02-27
Closed date
2017-06-29
Last modified date
2017-06-29
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
WMQ MFT V8.0
Fixed component ID
5724H7252
Applicable component levels
R800 PSY
UP
[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSYHRD","label":"IBM MQ"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"8.0","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]
Document Information
Modified date:
29 June 2017