Integrated Accelerator for zEnterprise Data Compression (zEDC) Everything you need to get started quickly. Get started - Play overview Transcript

Welcome to the Integrated Accelerator for zEDC content solution, your homepage for technical resources.

The Integrated Accelerator for zEDC, available beginning with IBM® z15™ and LinuxONE III, reduces the cost of storing, transporting and processing data. It replaces the zEDC Express adapter with on-chip compression, providing increased throughput and capacity: for the z15, up to 8 times faster application elapsed time with no additional CPU time compared to a z14 with zEDC Express for compression and decompression *

If you enabled compression selectively with zEDC Express, you might now enable compression more pervasively, including:

  • Storage (SMF logstreams, BSAM and QSAM data sets, DFSMShsm, DFSMSdss processing and zFS data)
  • In-application compression using standard Java packages or zlib APIs
  • Databases (Db2® LOBs, Db2 archive logs, and ContentManager OnDemand)
  • Network transmission (Sterling Direct:Connect™)
  • Linux
Big picture 1. Assess data and applications for compression. 2. Define what to compress. Consider encryption: compress first for efficiency. 3. Run the workload. 4. Test and assess compression. 5. Put compression into production. How to get started
Overview

Use z/OS® data compression for SMF logstreams, BSAM and QSAM data sets, and DFSMShsm and DFSMSdss processing. This requires the priced zEDC software feature, which you enable with an IFAPRDxx parmlib member. (For compression of zFS data, see the z/OS zFS data tab.)

Enabling the zEDC feature in IFAPRDxx
Planning

If you have not been using zEDC, plan for the following.

  • Compression of data at the data set level or the system level
  • Compression during DFSMShsm and DFSMSdss processing of migration and backup, or of dump data

If you have been using zEDC, review your definitions for restrictions that you might remove, to apply compression more broadly. For example:

  • Compression of sequential data sets that is selectively enabled by application.
  • For migration to tape, balancing zEDC against data set compression. Consider removing any restrictions on what you compress.

To allocate a zEDC compressed format data set, ensure the allocation amount meets a minimum space requirement.

Space requirement
Setup
  1. Define or review definitions.

    If you have not been using zEDC, request compression:

    • Of data before writing it to SMF logstreams, with the PERMFIX and COMPRESS keywords in the SMFPRMxx parmlib member. This can be done globally or by log stream. Learn more.
    • Of QSAM or BSAM data sets:

      At the data set level, by defining a data class with the COMPACTION options ZR (zEDC Required) or ZP (zEDC Preferred). Learn more.

      At the system level, by defining the storage management subsystem (SMS) with the COMPRESS options ZEDC_R (zEDC Required) or ZEDC_P (zEDC Preferred) in the IGDSMSxx parmlib member. The data class COMPACTION value must be set to Y. Learn more.

    • When creating PDSEs, by defining a discrete FACILITY class profile, STGADMIN.SMS.ALLOW.PDSE.COMPACT. Only SMS-managed PDSEs are eligible to be compressed. You can specify compression at the data set or system level, as described for QSAM and BSAM data sets. Learn more.
    • During hsm migration and backup and processing of dump data, with the ZCOMPRESS parameter of the SETSYS command and the ZCOMPRESS parameter of the DEFINE DUMPCLASS command. Learn more

    If you have been using zEDC, review SMS storage classes and definitions for applications, and use of zEDC in DFSMShsm processing as described above.

  2. Run the applications.
  3. Assess the compression. For performance metrics use:
  4. Put your new definitions into production.
Overview

Compress data in the z/OS Distributed File Service (zFS) to save disk space. This requires the priced zEDC software feature, which you enable with an IFAPRDxx parmlib member.

You can define a new file system that is always compressed, and compress existing zFS data.

Restrictions The compression process Enabling the zEDC feature in IFAPRDxx
Planning

Specify the compression of zFS data as follows.

New data:

  • Set a global default used by all formatting methods when formatting a new file system, with an option in the IOEFSPRM configuration file, format_compression=on. The default for this option is to set compression off.
  • Explicitly specify compression on a formatting method even if the global default is that compression is off, with the -compress keyword.

Existing data:

Use the zfsadm compress command. This is a long-running administrative command. Operator messages are issued during the operation. Monitor progress of the operation with the FSINFO command. During this process, background tasks on the zFS owning system process every object in the file system. Application access is fully allowed to the file system during the operation.

Setup
  1. Define or review definitions to ensure that the zFS data is compressed.

    If you have not been using zEDC:

    Enable compression as the default for all formatting methods when formatting a new file system with an option in the IOEFSPRM configuration file, format_compression=on. The default for this option is to set compression off.

    Alternatively, if you leave the global default for compression off, you can explicitly request compression on a formatting method with the -compress keyword.

    Compress existing data with the zfsadm compress command.

    If you have been using zEDC:

    Ensure that the setting for the format_compression option in the IOEFSPRM configuration file is format_compression=on.

    Compress existing data with the zfsadm compress command.

  2. Assess the compression by comparing the size of the file system with compression to the size before compression. For performance metrics, use:
  3. Make changes as needed.
  4. Put the compression for zFS into production.
Overview

Support for in-application compression is provided through zlib, which is an open source data compression library. The zlib compression library provides in-memory compression and decompression functions, including integrity checks of the uncompressed data.

zEDC uses a modified version of the zlib compression library. The IBM-provided zlib-compatible C library provides a set of wrapper functions that use zEDC compression when appropriate and software-based compression services when zEDC is not appropriate.

zlib
Planning

If you are not already using zEDC, define applications to use compression as follows:

  • In a Java application, use the java.util.zip compression classes.
  • In a C application, use the supported zlib APIs. Link or re-link applications to use the IBM-provided zlib, which is an archive file in the z/OS UNIX System Services file system and can be statically linked into your applications.
  • In a COBOL program, see Using zlib compression from a COBOL program.

If you are already using zEDC, review your applications to consider applying compression more broadly.

java.util.zip zlib functions zlib home page
Setup

If you are not already using zEDC

Enable zEDC for an application.

Ensure that z/OS input buffers specified in your Java application are large enough to qualify for hardware compression. To see the minimum size for compression and decompression, issue the D IQP command and check the values for DEFMINREQSIZE and INFMINREQSIZE in the response. (These values are static.) Use the environment variables _HZC_DEFLATE_THRESHOLD and _HZC_INFLATE_THRESHOLD in your application to control the minimum buffer sizes that must be met for hardware compression. For the best experience, use larger buffers for compression, such as 32KB - 256KB, rather than small buffers such as 256 bytes. Small buffers work very well, but large buffers are better.

For C applications, link or re-link the application to use the IBM-provided zlib.

Path for the zlib archive file: /usr/lpp/hzc/lib/libzz.a

Path for the zlib header files: /usr/lpp/hzc/include/

Re-link an application when a new service is provided in zlib in order to use the updated zlib and take advantage of the new function. zlib will always be compatible -- Integrated Accelerator for zEDC will never break zlib’s ability to call into the operating system to get accelerated compression.

If you are already using zEDC

Ensure that the z/OS input buffers for the Java application are sufficient for what is listed for DEFMINREQSIZE and INFMINREQSIZE displayed with the D IQP command.

You do not need to re-link the application to use the IBM-provided zlib.

Assess and complete:

  1. Run the application.
  2. Assess the compression. For performance metrics, use:
    • SMF record type 113 for synchronous processing. (zlib is always synchronous on a z15.)
    • SMF record type 30 to see the change in elapsed time and CPU time for the job.
  3. If necessary, adjust your application and run it again.
  4. Put the application into production.
Overview

Use Integrated Accelerator with zEDC to compress Db2 large object (LOB) table spaces, archive logs, and IBM Content Manager OnDemand archives.

Compressing LOB data with zEDC can reduce the size of data in a LOB table space and ensure that the LOB table space and its associated base table space remain available for inserting data. Compress LOB data and Content Manager On Demand archives to achieve faster query response times, reduced disk use, and improved archive performance.

Compressing LOB data for Db2 V12 and higher requires the priced zEDC software feature, which you enable with an IFAPRDxx parmlib member.

LOB table spaces Content Manager OnDemand Enabling the zEDC feature in IFAPRDxx
Planning

If you have not been using zEDC

Plan to enable compression for:

  • Individual table spaces
  • Db2 directory.

If you have been using zEDC

Review your current definitions and consider applying compression more broadly. Review:

  • Definitions for compression in the CREATE TABLESPACE or ALTER TABLESPACE statement and the COMPRESS_DIRLOB subsystem parameter.
  • For Content Manager OnDemand, ARS.CFG parameters that might be limiting compression.
CREATE TABLESPACE COMPRESS_DIRLOB Content Manager OnDemand ARS.CFG
Setup
  1. Define or review definitions to ensure that the data is available to be compressed.

    If you have not been using zEDC

    Enable compression for individual table spaces by specifying the COMPRESS YES option on the CREATE TABLESPACE or ALTER TABLESPACE statement.

    Enable compression for the Db2 directory by setting the COMPRESS_DIRLOB subsystem parameter to YES.

    If you have been using zEDC

    Review the definitions for compression in the CREATE TABLESPACE or ALTER TABLESPACE statement and the COMPRESS_DIRLOB subsystem parameter.

    For Content Manager OnDemand, review ARS.CFG parameters that might be limiting the use of compression.

  2. Test the Db2 application that uses the LOB data.
  3. Assess the compression. Check:
    • The size of the table space in Db2 once compression is used
    • SMF record type 30.
  4. Make changes as needed.
  5. Put compression for Db2 into production.
Overview

Use IBM Sterling Direct:Connect for z/OS to transmit compressed data from z/OS to z/OS or z/OS to other platforms. Sterling Direct:Connect for z/OS is a separately orderable product.

Sterling Direct:Connect for z/OS
Planning

If you have not been using zEDC, plan for transmitting compressed data. See the initialization parameter ZEDC of Sterling Direct:Connect.

If you have been using zEDC, review your current definitions and consider applying compression more broadly. Transmission of compressed data might be selective based on capacity. Consider removing any restrictions on what you compress.

ZEDC parameter of Sterling Direct:Connect
Setup
  1. Define or review definitions.

    If you have not been using zEDC, request compression when transmitting data with SterlingDirect:Connect using the ZEDC initialization parameter.

    If you have been using zEDC, review parameters for Sterling Direct:Connect as described above.

  2. Transmit data.
  3. Assess the compression. For performance metrics, use:
    • Synchronous execution: SMF record type 113
    • Asynchronous execution:
      • SMF record type 30
      • RMF EADM reporting (RMF 74.10).
  4. Make changes as needed.
  5. Put your new definitions into production.
ZEDC parameter of Sterling Direct:Connect
Overview

Compression with the Integrated Accelerator for zEDC is available to applications that use zlib or gzip in user space as well as to the kernel zlib.

zlib gzip
Planning

The prerequisites for using the Integrated Accelerator for zEDC depend on your virtualization environment, for example Linux in LPAR mode or Linux as a guest of KVM or z/VM. For details, see the topic on data compression with the Integrated Accelerator for zEDC for the appropriate Linux distribution in Linux on IBM Z and LinuxONE: Device Drivers, Features, and Commands.

In addition, accelerated compression must be supported by the zlib or gzip in the Linux distribution. To determine if your zlib or gzip supports Integrated Accelerator for zEDC, search the zlib and gzip binaries for this string: DFLTCC. If it is found, the support is available.

By default, decompression is always accelerated and compression is accelerated if software requests compression level 1. Consider whether you want to change the compression level that is requested by software, or override the default so that Integrated Accelerator for zEDC is used when software requests a compression level other than 1. The possible compression levels, 1 – 9, indicate compression quality, with 9 being the highest quality. Higher quality might come at the expense of performance. Integrated Accelerator for zEDC provides compression that is approximately equivalent to level 1 (best peformance).

For additional information, as well as hints and tips on zlib acceleration, see Linux on the Mainframe.

Setup

For the kernel, compile the kernel zlib with support for the Integrated Accelerator for zEDC by selecting the kernel configuration option CONFIG_ZLIB_DFLTCC.

For applications in user space, perform the following to get on-chip accelerated compression.

  1. Ensure that environment variable DFLTCC is set to 1, which enables on-chip accelerated compression.
  2. Check the compression level requested by your software. If it is not level 1 but can be configured, set it to level 1, or use environment variable DFLTCC_LEVEL_MASK to ensure that on-chip accelerated compression is used for the requested compression level, as described in the next step.
  3. If you want on-chip accelerated compression when software requests a compression level other than 1, use environment variable DFLTCC_LEVEL_MASK. For example, set the value to 0x01ff to request on-chip acceleration for compression levels 0 - 8 (all but the highest quality commpression).
  4. Run the application.
  5. Assess the compression and make changes if necessary.
Documentation Comprehensive content collection (c3)

Find information about Integrated Accelerator for zEDC

Explore the collection
Technical resources

This summary of Integrated Accelerator for zEnterprise Data Compression provides use cases and proof points.

Read the summary

Reduce Storage Occupancy and Increase Operations Efficiency with IBM zEnterprise Data Compression.

Read the IBM Redbook

IBM Z Batch Network Analyzer (zBNA) Tool

Download the tool

This white paper describes the benefits of combining pervasive encryption and compression with the z15.

Read the white paper

This blog provides hints and tips for using compression with Linux.

New Read the blog
Overview of Integrated Accelerator for zEDC

Get an overview of Integrated Accelerator for zEDC.

Watch the video
Accelerated data compression with Linux on IBM z15: Managing data growth

Learn how accelerated data compression reduces storage requirements and more.

New Watch the video
Accelerated data compression for Linux databases on IBM Z15

Learn how you can boost the performance of your Linux databases.

New Watch the video
Accelerated data compression with Linux on LinuxONE: Managing data growth

Learn how accelerated data compression reduces storage requirements and more.

New Watch the video
Accelerated data compression for Linux databases on LinuxONE

Learn how accelerated data compression reduces storage requirements and more.

New Watch the video
Notes

* Measurements were collected in a controlled environment running an IBM developed workload under z/OS comprised of an equal mix of compression and decompression. Individual results may vary. Results are workload dependent.

What's new

The z/OS data, z/OS zFS data, and z/OS databases tabs in the How to get started section were updated to include the requirement for the priced zEDC software feature. The z/OS applications tab was updated with information about using zlib compression from a COBOL program.

Minor edits were made to the Linux tab of the How to get started section.

Links to Db2 documentation were updated to use the Db2 13 for z/OS library. Other minor edits were made for clarity and consistency.

Links to z/OS documentation were updated to use the z/OS 2.5 library.

Transcript added for Integrated Accelerator for zEDC overview video.

The Big Picture section has been modified for accessibility.

The page was updated to describe the support for Linux, including a new Linux tab in the How to get started section and new resources in the Technical resources section.

A white paper was added to the Other Resources tab of the Technical resources section.

The overview video has been replaced with an extended video that provides additional detail.

A c3 was added to the Content Solution tab of the Technical resources section.
Some links from the How to get started section now go to the c3.