Influence of striping of FCTC devices numbers over FICON channels

Three shared FICON channels with three "straight" FCTC connections each: FCTC connection I/O rates

Figure 1 shows the I/O rates per FCTC connection achieved on the sending side (LPAR2) during the relocation process for the case that three shared FICON channels are used with three "straight" FCTC connections each. The FCTC device numbers are assigned in a sequential manner in this case.

In Figure 1, the FCTC connection I/O rates are measured in SSCH/s (Start Subchannel per second), using the left scale. For each FICON channel a different color was used, as follows:

• Orange was used for the data rates of the FCTC connections defined on the FICON channel with CHPID 3C
• Green was used for the data rates of the FCTC connections defined on the FICON channel with CHPID 3D
• Blue was used for the data rates of the FCTC connections defined on the FICON channel with CHPID 3E

In addition, Figure 1 shows the link data rate (LDR) for the complete ISFC logical link as a dashed black line. The link data rate is measured in MB/s, using the right scale.

Observation

The FCTC connections with the lowest device numbers – D000 and D001, both on the FICON channel with CHPID 3C – are designated by z/VM® for unidirectional use by receiving LPAR1. For these FCTC connections only very low I/O rates are observed (coinciding bottom red and yellow graphs). While not shown in Figure 1, the device busy percentages in these cases stay at 0%.

The highest I/O rate with 214 SSCH/s is observed for FCTC connection with device number D002 (topmost orange graph); while exhibiting that high value, the device busy percentage (not shown) for this FCTC connection stays at about 100%. Because the FCTC connection with device number D002 is the only connection on the FICON channel with CHPID 3C, the channel I/O rate for that FICON channel is also 214 SSCH/s here.

For the FCTC connections on the FICON channels with CHPIDs 3D and 3E, in all cases an I/O rate of about 90 SSCH/s is observed (coinciding middle green and blue graphs). For each of these FCTC connections a device busy percentage of about 100% (not shown). For the FICON channels with CHPIDs 3D and 3E the channel I/O rate is 270 SSCH/s each (3 * 90 SSCH/s).

The added I/O rate for the ISFC link is 754 SSCH/s (214 SSCH/s + 2 * 270 SSCH/s). In addition, Figure 1 shows that during the relocation process the write data rate for the ISFC logical link (black dashed graph) almost constantly stays at about 565 MB/s.

Three shared FICON channels with three FCTC connections each and "striped" FCTC device numbers

Figure 2 also shows the I/O rates per FCTC connection achieved on the sending side (LPAR2) during the relocation process for the case that three FICON channels are used with 3 FCTC connections each; however, here the device numbers were assigned in the IOCDS in a “striped” manner.

In Figure 2, the same color assignment as in Figure 1 is used (3C: orange; 3D: green; 3E: blue). However, because here the device numbers were assigned in a “striped” manner, the device numbers corresponding to one FICON channel are not arranged in contiguous ranges; instead, every third FCTC device corresponds to the same FICON channel.

Observation

Again the FCTC connections with the lowest device numbers – D000 and D001 – are designated for unidirectional use by receiving LPAR1. But because of the “striped” device number assignment in the IOCDS now on different FICON channels (with CHPIDs 3C and 3D). For these FCTC connections only very low I/O rates are observed (bottom coinciding orange and green graphs), along with a (not shown) device busy rate of 0%. But as a result of the “striped” device number assignment, the designated FCTC connections now are on two different FICON channels.

For the remaining FCTC connections on these FICON channels (D003 and D006 on the FICON channel with CHPID 3C, D004 and D007 on the FICON channel with CHPID 3D, topmost coinciding orange and green graphs) an I/O rate of about 128 SSCH/s (along with a 100% device busy rate) is observed. In sum, the channel I/O rate for the FICON channels with CHPIDs 3C and 3D is 256 SSH/s here.

For the three FCTC connections on the FICON channel with CHPID 3E (device numbers D002, D005 and D008, middle coinciding blue graphs) an I/O rate of about 90 SSCH/s is observed (along with a 100% busy rate). In sum, the channel I/O rate for the FICON channel with CHPID 3E is 270 SSCH/s here.

The added I/O rate for the ISFC link is 782 SSCH/s (2 * 256 SSCH/s + 270 SSCH/s), which is about 3.7% higher than for the non-striped case shown in Figure 1. In addition, Figure 2 shows that the write data rate for the ISFC logical link (black dashed graph) almost constantly stays at about 590 MB/s during the relocation process, which is about 4.4% higher than for the non-striped case.

One shared FICON channel with two FCTC connections: FCTC connection I/O rates

For comparison, Figure 3 shows the I/O rates per FCTC connection achieved during the relocation process for the case that one FICON channel is used with two FCTC connections.

Observation

The FCTC connection with the lower device number exhibits a very low device I/O rate in this case. In addition, Figure 3 shows that while I/O is performed the write data rate for the ISFC logical link (black dashed graph) almost constantly stays at about 165 MB/s.

Conclusions from influence of striping of FCTC devices numbers over FICON channels

Figure 1 shows the case of an ISFC logical link configuration using shared FICON channels and "straight" FCTC connections with sequentially assigned device numbers. The two FCTC connections with the lowest device numbers (D000 and D001, lower orange graphs) are not used for relocation data transfer because of the way z/VM designates certain FCTC connections for unidirectional use (see Avoidance of write collisions). Due to the sequential device number assignment, both of these FCTC connections are from the same FICON channel with CHPID 3C.

The remaining only FCTC connection (device number D002) on the FICON channel with CHPID 3C is loaded with a significantly higher I/O rate than the FCTC connections on the other FICON channels (CHPIDs 3D and 3E). In fact, this single FCTC connection I/O rate is similar to that shown in Figure 3. There, one FCTC connection (D051) of a two FCTC connection / single FICON channel configuration is used for the relocation data transfer, while the other FCTC connection (device number D050) is designated for use by the communication partner by z/VM. In other words, only one FCTC connection on that FICON channel is used for relocation data transfer. Figure 29 also conveys that the data rate of the ISFC logical link in this situation is about 165 MB/s. This is about 20% less than the maximal possible data transfer rate per FICON channel of about 206 MB/s that is achievable with one FICON channel if sufficient FCTC connections are used (see Figure 1). Apparently, the use of only one FCTC connection from a FICON channel for data transfer prevents exploiting all of the usable bandwidth from that FICON channel.

Figure 2 shows the case of an ISFC logical link configuration also using shared FICON channels with "straight" FCTC connections, but with device numbers assigned in a striped manner. Again the two FCTC connections with the lowest device numbers (D000 and D001, lower orange and lower green graph) are not used for relocation data transfer. However, with a “striped” device number assignment in the IOCDS, now the two “unidirectional” connections are from two different FICON channels. With two FCTC connections from each of these FICON channels now being used for data transfer, hitting the “one connection bandwidth limit” as observed in the non-striped case is attenuated. For the ISFC logical link overall, a higher throughput results than in the non-striped case.

In summary, for an ISFC logical link configuration with shared FICON channels and "straight" FCTC connections where three FCTC connections with contiguous device numbers are configured per FICON channel, the first two FCTC connections are designated for unidirectional use by z/VM, causing the first FICON channels data transfer capacity not being fully used for data transfer. It was shown that a better ISFC logical link throughput can be achieved by distributing the “unidirectional” FCTC connections on two different FICON channels, because then the remaining FCTC connections on these FICON channels are used for data transfer.

Seven FICON channels with sixteen "straight" FCTC connections: FCTC connection I/O rates

Figure 4 shows the I/O rates per FCTC connection achieved during the relocation process for the case that ISFC logical link configuration with sixteen "straight" FCTC connections on seven FICON channels is used. Four FCTC connections are configured on the FICON channel with CHPID 3C; two FCTC connections are configured on each of the other six FICON channels. The orange and red graphs show the I/O rates for the four FCTC connections on the FICON channel with CHPID 3C; two of them are the lowest graphs, and the other two are the highest graphs. The I/O rates of all other FCTC connections are shown in blue.

Observation

The I/O rates for the FCTC devices corresponding to the FICON channel with CHPID 3C exhibit a bi-fold behavior: The FCTC connections with device numbers D000 and D007 are almost not used (bottom orange and yellow graph); the FCTC connections with device numbers D015 and D01C (topmost red graphs) are used with a similar I/O rate than the FCTC connections corresponding to the other FICON channels.

Conclusion

The first two FCTC connections with device numbers D000 and D007 (lower orange graphs, lowest device numbers) are not used for relocation data transfer from BOER3740 to BOER3739 because of the way z/VM dedicates certain FCTC connections for unidirectional use (see Avoidance of write collisions).

For the ISFC logical link configuration used for this example, the FCTC connections with the two highest and the two lowest device numbers were selected from the same FICON channel (CHPID 3C), resulting in a symmetrical ISFC logical link configuration; the remaining FCTC connections were selected from the other six FICON channels, and with device numbers within the range in between.

As a consequence, the four FCTC connections on the FICON channel with CHPID 3C are designated by z/VM for unidirectional use (two in either direction). For the relocation process this ISFC logical link configuration results in an almost uniform distribution of I/O operations over fourteen FCTC connections and seven FICON channels, with each FICON channel being used with two FCTC connections.

This ISFC logical link configuration yielded the most optimal relocation process achieved in our project. This is caused by uniform distribution of the I/O load over all available FICON channels. including the FICON channel on which the FCTC connections designated for unidirectional use are assigned.