APAR status
Closed as program error.
Error description
Information Server 8.1 on AIX In DataStage, the customer is able to insert columnize UTF-8 simplified Chinese data into a VARCHAR column in a UTF-8 DB2 data base using an ODBC Enterprise stage. Using the ODBC Enterprise stage attempt to read the data back into DataStage but are observing data corruption. The customer has updated Data Direct drivers. Tried reading the data in as a VARCHAR and using a string_to_ustring conversion in a transfortmer without success.
Local fix
As a workaround the customer can read the data in as vargraphic e.g select ADMIN_SYS_TP_CD, CAST(ADMIN_CLIENT_ID as vargraphic(20)) ADMIN_CLIENT_ID from IS_ADMINCLIENT In this case the ODBC stage should have ADMIN_CLIENT_ID as nvarchar . This fix is included in 8.0.1 fix pack 3
Problem summary
**************************************************************** USERS AFFECTED: All platforms , jobs using ODBC EE to read UTF-8 characters **************************************************************** PROBLEM DESCRIPTION: Using the ODBC Enterprise stage to read columnize UTF-8 simplified Chinese data from DB2 back into DataStage but are observing data truncation. **************************************************************** RECOMMENDATION: Install patch for JR33566 The fix has been included in the IS 8.1 FP1. Apply the FixPack1 for IS 8.1. ****************************************************************
Problem conclusion
When UTF-8 data is stored in char/varchar columns, ODBC EE is unable to read the data correctly as it fetches the data as SQL_C_CHAR type. Implemented option Use Ustrings, to read all char/varchar columns as unicode strings. When this option is selected , char/varchar columns are retrieved as SQL_C_WCHAR type. Note: Since we are attempting to read varchar/char columns as nvarchar/nchar , hence the output will be inflated nvarchar/nchar columns. e.g if the user is writing nvarchar(20) to a column of varchar(80), then it will be read back as nvarchar(80) . This is because the application cannot make a guess about the number of utf8 characters in the column. It could be from 20 charactersto 80 characters. If the user changes the column definition in the GUI to nvarchar(20), they will see a warning message like "Implicit conversion from source type "ustring[max=80]" to result type "ustring[max=20]": Possible truncation of variable length string." To get the correct column definition, the customer should load the imported table definition and check the box that says "Ensure all char columns use unicode", whenever the option Use UStrings is set to true.
Temporary fix
Comments
APAR Information
APAR number
JR33566
Reported component name
WIS DATASTAGE
Reported component ID
5724Q36DS
Reported release
801
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2009-07-06
Closed date
2009-07-14
Last modified date
2010-12-09
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Modules/Macros
ODBCEE
Fix information
Fixed component name
WIS DATASTAGE
Fixed component ID
5724Q36DS
Applicable component levels
R801 PSN
UP
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSVSEF","label":"IBM InfoSphere DataStage"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"8.0.1","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]
Document Information
Modified date:
09 December 2010