APAR status
Closed as program error.
Error description
The filter option should really be made mutually exclusive with multiple readers as in most situations it will not function as desired. The problem will occur with any filter that causes the number of bytes read to be different (in this case less) than the number of bytes in the record. The reason for this is the way that the multiple readers functionality works. In the case of a delimited file it is split up based on its total byte length. Taking a simple case where the file length is 256 and we have 2 readers, reader 1 will be set to read from byte 0 to 128 and reader 2 from 128 to 256. Now it is very likely byte 128 falls in the middle of a record. In this case reader 1 will read from 0 to 128 plus the number of bytes to the next record delim. Reader 2 will start reading at byte 128 until it finds the first record delim and ignore this data, starting actual reading at the next record. This guarantees there is no overlap and the correct number of records is read. Sequential file stage using multiple readers and filter option: example: -readers 2 -filter 'cut -c1-4' The problem occurs when the first reader is calculating its current offset. Since the filter is passing fewer bytes than are actually in the file, the first reader will calculate its offset to be earlier in the file than it actually is meaning it will read a bunch of records past where it is supposed to stop. Unfortunately the solution to this would mean preprocessing the file which would negate any performance improvements of running multiple readers on the file. The answer to this is this combination of arguments is not supported and should be flagged as such.
Local fix
do not use filter and multiple node option together
Problem summary
**************************************************************** USERS AFFECTED: All versions of IS **************************************************************** PROBLEM DESCRIPTION: When the filter and readers options are used in conjunction the results are only correct under certain situations. The user is not made aware of that. **************************************************************** RECOMMENDATION: Install patch JR30649 This change is included in 8.1 Fix Pack 1. ****************************************************************
Problem conclusion
A warning was added to alert the user to the possibility of improper function in 8.0.1 and beyond.
Temporary fix
Comments
APAR Information
APAR number
JR30649
Reported component name
WIS DATASTAGE
Reported component ID
5724Q36DS
Reported release
752
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2008-10-02
Closed date
2009-01-20
Last modified date
2009-12-17
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Modules/Macros
SERVER
Fix information
Fixed component name
WIS DATASTAGE
Fixed component ID
5724Q36DS
Applicable component levels
R752 PSN
UP
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSVSEF","label":"IBM InfoSphere DataStage"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"7.5.2","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]
Document Information
Modified date:
17 December 2009