Streaming logs as JSON Lines over TCP to an analytics platform
You can run Transaction Analysis Workbench batch jobs that stream log data in JSON Lines format over a network to a TCP socket on an analytics platform, such as Elastic or Splunk. The TCP socket can be secure or unsecure.
Before you begin
You must configure the analytics platform to listen on a TCP port for JSON Lines. You need to know the analytics platform host name (or IP address) and port number.
For secure TCP, you need to know the corresponding security details that Transaction Analysis Workbench will require: for example, the location on z/OS® of the security certificates, such as a SAF key ring.
You need to know the location on z/OS of the logs that you want to forward.
About this task
Streaming gets data off z/OS without the storage and processing costs of writing data to temporary staging files on z/OS, and then transferring those files off z/OS.
You can configure some analytics platforms, such as Elastic and Splunk, to listen on a TCP port and ingest data as it arrives.
You can configure some log forwarding tools, such as Logstash (from the Elastic Stack), to listen on a TCP port and forward data to a variety of destinations. The destinations can include analytics platforms that do not natively ingest data via TCP. For example, you can configure Logstash to receive JSON Lines input over TCP, and then forward the data in a different, platform-specific output format over HTTP, to Elasticsearch or other destinations. Such log forwarding tools expand the range of destinations for data from Transaction Analysis Workbench.
Many analytics platforms ingest data in JSON Lines format. In some cases, a key-value (KV)
pair
data format such as JSON Lines is the preferred format.
To stream logs in JSON Lines format to a TCP socket, use the JSON and STREAM commands of the Transaction Analysis Workbench report and extract utility.
You can either write JCL yourself to run these commands, or you can use Transaction Analysis Workbench ISPF dialog option 1 to create the JCL for you.
The dialog option offers only a small subset of the data sources and record types supported by Transaction Analysis Workbench. You can tailor the JCL created by the dialog, or write JCL yourself, to forward data from any of the data sources and record types supported by Transaction Analysis Workbench.
The following procedure uses the dialog option to create JCL. If you prefer to write JCL yourself, skip to the example JCL following the procedure.
Procedure
Example
The following JCL extracts selected fields from CICS monitoring facility performance class
records in a dumped SMF data set, converts them to JSON Lines in ASCII, and then forwards them over
TCP in a single stream to a listening analytics platform, such as Elastic (specifically, Logstash)
or Splunk. The analytics platform is installed on a computer with host name analytics
,
listening on unsecure TCP port number 6789.
//UIDFUW JOB NOTIFY=&SYSUID 1
//FUWBATCH EXEC PGM=FUWBATCH
//STEPLIB DD DISP=SHR,DSN=<FUW HLQ>.SFUWLINK 2
//SMFIN DD DISP=SHR,DSN=HLQ.SMF.DAILY
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
OUTZONE(Z) 3
STREAM NAME(STREAM1) HOST(analytics) PORT(6789) +
TIMEFORMAT(ISO8601) 4
JSON CODE(CMF) STREAM(STREAM1)
FIELDS( 5
TRAN
USRCPUT
/* More fields...
)
/*
Notes on the JCL:
Here is a line from the output stream:
{"time":"2015-11-30T08:00:00.000001Z","type":"cmf-6e13","tran":"TRNA","usrcput":0.003456,…}
For a secure TCP port, add the following parameters to the STREAM command in the previous JCL listing:
SECURITY(TLS*) FIPS KEYRING(my/fuw.stream)
where my/fuw.stream
identifies the RACF®
key ring that contains the CA certificate used by Splunk or Logstash.
You can forward multiple record types in the same stream. For example, to add SMF type 30 job termination records to the stream, append the following lines to the SYSIN data set:
JSON CODE(SMF:30.) STREAM(STREAM1)
FIELDS(
SMF30JBN
SMF30CPT
/* More fields...
)
CODE(SMF:30.)
COND SMF30STP EQ 5 /* Filter to select job termination records
Here is a corresponding output line:
{"time":"2015-11-30T08:00:01.000002Z","type":"smf-30","smf30jbn":"PRDJOB1","smf30cpt":0.123006,…}
The ISPF dialog creates JCL that forwards logs from a single data source, such as a single SMF data set. You can tailor that JCL, or write JCL yourself, to forward logs from multiple data sources. For example, to add IMS transaction index records to the stream in the previous example JCL, add a DD statement that refers to an IMS system log data set (SLDS):
//LOGIN DD DISP=SHR,DSN=IMSP.SLDS
then append the following lines to SYSIN:
IMSVRM=154
IMSINDEX
JSON CODE(IMS:CA01) STREAM(STREAM1)
FIELDS(
TRANCODE
PROGRAM
USERID
IMSID
/* More fields...
)
where 154
identifies the release level of the IMS system.
Each CSV or JSON command can only write to a single stream. However, you can specify multiple STREAM commands in a SYSIN data set; CSV and JSON commands in the same SYSIN data set can write to different streams.