Incident validation
When an incident is reported, it must be validated by gathering documentation immediately. This
documentation should include the following information:
- The time and place that the incident occurred.
- The source, which is the logical unit (LU) names of the primary and secondary LUs. If the session is a TELNET session, the source also includes the IP address of the secondary LU.
- The name and type of application that was being used; and if possible, the transaction that was being executed.
- The name of the PU, LINE, and major node of the secondary LU, if applicable.
- Additional trace data needs to be collected to determine whether a pattern of data exists to this incident.
Example
IST2424I 3270 DATA STREAM ERROR - NETA.TSO0002 NETA.TCPM0001
IST2425I PLU SUBAREA = X'0001' INDEX = X'0000' ELEMENT = X'0058'
IST2425I SLU SUBAREA = X'0001' INDEX = X'0001' ELEMENT = X'0009'
IST2441I JOBNAME = JHACKER SID = EAABEEC331E8DB02
IST2426I IPADDR = 192.168.98.254..61691
IST2427I DATE = 2016/01/25 TIME = 15:47:56 ID = 1
IST2428I ROW = 9 COLUMN = 16
IST2429I OUTBOUND - SEQ = X'0001' OFF = 598 LEN = 39
IST2431I 40404040 40404040 D1C1C3D2 E2D6D540 * JACKSON *
IST2430I INBOUND - SEQ = X'0001' OFF = 284 LEN = 39
IST2431I 40404040 40404040 F1F2F3F4 F5F6F7F8 * 12345678*
IST314I END- The date and time of this incident is identified in message IST2427I and in the formatted trace data. The ID shows a unique identifier for this incident and this is the first one since VTAM® was started.
IST2427I DATE = 2016/01/25 TIME = 15:47:56 ID = 1 - The secondary LU is identified in message IST2424I as NETA.TCPM0001. The following information
displays this LU. Message IST271I shows that this LU is an application that the job named TELNET
opens. Messages IST1727I and IST1669I identify the domain service name and IP address of the user.
Note: TCPM0001 is an application that acts as a secondary LU, which is not supported for 3270 IDS monitoring.
IST2424I 3270 DATA STREAM ERROR - NETA.TSO0002 NETA.TCPM0001 d net,id=NETA.TCPM0001 IST097I DISPLAY ACCEPTED IST075I NAME = NETA.TCPM0001, TYPE = DYNAMIC APPL 456 … IST231I APPL MAJOR NODE = TCPAPPLS IST271I JOBNAME = TELNET, STEPNAME = TELNET, DSPNAME = IST19405 … IST1727I DNS NAME: JOEHACKER.FARFARAWAY.EXAMPLE.COM IST1669I IPADDR..PORT 192.168.98.254..61691 IST171I ACTIVE SESSIONS = 0000000001, SESSION REQUESTS = 0000000000 IST206I SESSIONS: IST634I NAME STATUS SID SEND RECV VR TP NETID IST635I TSO10002 ACTIV-P EAABEEC331E8DB02 0004 0009 NETA IST314I END - The name of the PLU application is TSO0002. This user is logged onto TSO. The following
information displays the application information. Message IST271I shows the TSO user ID. Messages
IST2433I and IST2434I show the application 3270 IDS parameter values. Message IST2435I confirms that
an 3270 IDS data steam error
occurred.
IST2424I 3270 DATA STREAM ERROR - NETA.TSO0002 NETA.TCPM0001 D NET,ID=TSO0002,E IST097I DISPLAY ACCEPTED IST075I NAME = TSO0002, TYPE = APPL 479 IST486I STATUS= ACT/S, DESIRED STATE= ACTIV … IST231I APPL MAJOR NODE = TSO1A IST213I ACBNAME FOR ID = TSO10002 … IST271I JOBNAME = JHACKER, STEPNAME = OS390R5, DSPNAME = IST71E8A … IST2433I DSMONITR = YES, DSCOUNT = 15, DSACTION = (CONSOLE,NONE) IST2434I DSTRUST = LOCALLU IST2435I SESSIONS MONITORED = 1, ERRORS DETECTED = 1 IST171I ACTIVE SESSIONS = 0000000001, SESSION REQUESTS = 0000000000 IST206I SESSIONS: IST634I NAME STATUS SID SEND RECV VR TP NETID IST635I TCPM0001 ACTIV/E-S EAABEEC331E8DB02 0009 0004 NETA IST314I END D NET,TSOUSER,ID=JHACKER IST097I DISPLAY ACCEPTED IST075I NAME = JHACKER, TYPE = TSO USERID 623 IST486I STATUS= ACTIV, DESIRED STATE= N/A IST576I TSO TRACE = OFF IST262I ACBNAME = TSO0002, STATUS = ACT/S IST262I LUNAME = TCPM0001, STATUS = ACT/S IST1727I DNS NAME: JOEHACKER.FARFARAWAY.EXAMPLE.COM IST1669I IPADDR..PORT 192.168.98.254..61691 IST2203I CHARACTER SET 02B9 - CODE PAGE 0417 IST314I END D A,JHACKER IEE115I 15.58.22 2016.025 ACTIVITY 638 JOBS M/S TS USERS SYSAS INITS ACTIVE/MAX VTAM OAS 00000 00011 00002 00033 00003 00002/00300 00004 JHACKER OWT A=0025 PER=NO SMC=000 PGN=N/A DMN=N/A AFF=NONE CT=000.032S ET=01.04.21 WUID=TSU00029 WKL=TSO SCL=TSO P=1 RGP=N/A SRVR=NO QSC=NO ADDR SPACE ASTE=1EFD6940 - The information of a secondary LU might identify the PU, LINE, and major node. In this example,
the information of the PU, LINE, and major node is not available. However, you can use the TCPIP
commands NSLOOKUP and TRACERTE to confirm the ID and location
of the secondary LU. Information about
router206indicates the approximate location.For more information about TCPIP commands, see z/OS Communications Server: IP System Administrator's Commands.
nslookup 192.168.98.254 EZB3170I Server: dns.example.com EZB3172I Address: 192.168.100.4 EZB3170I Name: joehacker.farfaraway.example.com EZB3172I Address: 192.168.98.254 READY tracerte 192.168.98.254 CS V2R1: Traceroute to 192.168.98.254 (192.168.98.254) 1router65.faraway.example.com (192.168.105.65) 2 ms 0 ms 0 ms 2router1.faraway.example.com (10.6.0.1) 1 ms 0 ms 0 ms 3router41a.faraway.example.com (192.168.120.41) 0 ms 0 ms 0 ms 4routeredge201.faraway.example.com (192.168.106.201) 0 ms 0 ms 5router1a.farfaraway.example.com (192.168.184.1) 15 ms 18 ms 21 ms 6router8.faraaway.example.com (192.168.34.8) 12 ms 7router208.faraaway.example.com (192.168.106.208) 2 ms 9 ms 11 ms 8router12.faraaway.example.com (192.168.96.120) 7 ms 12 ms 10 ms 9 joehacker.farfaraway.example.com (192.168.98.254) 2 ms 2 ms 1 ms READY - You can use the TCPIP Netstat command to show the time when the connection
started.
For more information about TCPIP commands, see z/OS Communications Server: IP System Administrator's Commands.
Tip: Information about the IP session is recorded in type 119 SMF records. Subtypes 1 and 2 contain information about the TCP connection. Subtypes 21 and 22 contain information about the TELNET connection. For TSO sessions, type 30 records contain information about the TSO user.netstat all (port 55516 MVS TCP/IP NETSTAT CS V2R1 TCPIP Name: TCPCS 20:57:34 Client Name: TELNET Client Id: 00000024 Local Socket: ::ffff:192.168.105.112..23 Foreign Socket: ::ffff:192.168.98.254..61691 BytesIn: 00000000000000002422 BytesOut: 00000000000000009580 SegmentsIn: 00000000000000000247 SegmentsOut: 00000000000000000320 StartDate: 01/25/2016 StartTime: 17:33:56 Last Touched: 20:47:56 State: Establsh … Application Data: EZBTNSRV TCPM0001 TSO10002 ET B ---- READY - The following information of messages from IST2428I to IST2431I indicates the overlay in the
3270 data steam. Near row 9 and column 16 in the 3270 display buffer, a field that contains the
string
JACKSONis replaced by the string12345678. Messages IST2429I and IST2430I show the respective PIUs where the fields can be found.IST2428I ROW = 9 COLUMN = 16 IST2429I OUTBOUND - SEQ = X'0001' OFF = 598 LEN = 39 IST2431I 40404040 40404040 D1C1C3D2 E2D6D540 * JACKSON * IST2430I INBOUND - SEQ = X'0001' OFF = 284 LEN = 39 IST2431I 40404040 40404040 F1F2F3F4 F5F6F7F8 * 12345678*Tip: Message IST2431I shows part of the raw 3270 data stream, which might include different 3270 orders. The presence of the Start Field order (x'1D') might indicate that a field attribute has been overlaid, which might cause the incident report. Another order is the Start Field Extended (x'29'). For more information about the 3270 data stream, see 3270 Data Stream Programmer's Reference. - The following generalized trace facility (GTF) trace data shows information about the buffers. Start additional traces of VTAM buffers to verify whether the sequence is repeated. The TCPIP packet trace data can also be collected. The TELNET option of the TCPIP packet trace formatter can be used to display the 3270 data stream orders.
For more information about the TCPIP packet trace, see z/OS Communications Server: IP Diagnosis Guide.
(11)VTAM TH=40000000 00000000 00010001 00000001 1800000B 00580001 051F RH=0380C0 (12) SEQ 0001-0001 F5C21140 402901C0 40F4F040 40E44040 40404040 *5B. ..{ 40 U * 404040C3 C8D9C9E2 E3C9C1D5 40404040 40404008 * CHRISTIAN .* … 114DC829 01C0E9C5 F94040D7 40C8E240 40D44040 *.(H..{ZE9 P HS M * 40D4C1E2 D6D54040 40404040 40404011 4DF02901 * MASON .(0..* C06CF6C3 4040D740 4040C940 40404040 D1C1C3D2 *{%6C P I JACK* E2D6D540 40404040 40404040 114ED829 01C06DF6 *SON .+Q..{_6* … 40404040 40404040 40C8C5E7 E2E3D9C9 D5C74DF0 * HEXSTRING(0* F05D4011 5D7E1D60 *0) .)=.- * (11)VTAM TH=40000000 00000000 00000001 00010001 1C000058 000B0001 0298 RH=0393A0 (12) SEQ 0001-0001 7D4AD811 40E9C3F1 4040E440 40404040 D4404040 *'¢Q. ZC1 U M * C1D3C5E7 E8E24040 40404040 40404040 11C1F9C3 *ALEXYS .A9C* F54040E4 4040E240 40D44040 40D4C1E2 D6D54040 *5 U S M MASON * 40404040 40404040 4011C3C9 C3F94040 E440C8E2 * .CIC9 U HS* … 40E4D540 40C940D4 404040D4 C1E2D6D5 40404040 * UN I M MASON * 40404040 40404011 4AC1F6F0 4040D740 40404040 * .¢A60 P * 40404040 F1F2F3F4 F5F6F7F8 F9404040 40404040 * 123456789 * 114AE9F6 F14040D7 40404040 40D44040 40D4C1C4 *.¢Z61 P M MAD* C9E2D6D5 40404040 40404040 40114BF9 C5F54040 *ISON ..9E5 * … C8C540E5 C1D3E4C5 40E3D67A 40404040 40404040 *HE VALUE TO: * 40404040 404040C8 C5E7E2E3 D9C9D5C7 4DF0F05D * HEXSTRING(00)* 40 * *