Introduction

Large File Support

httpd is now built with support for files larger than 2GB on modern 32-bit Unix systems. Support for handling >2GB request bodies has also been added.

IBM HTTP Server on IBM i 7.1 is based on Apache HTTP Server Version 2.2.11, but it didn't have large file support enabled when it was initially released two years ago. Nowadays, large files like DVDs, HD movies and large sized program install image etc have become more and more common and popular. The requirements to provide the support to allow customers to download/upload large files through IBM HTTP Server for i has become more important and necessary. This new feature is only added to IBM i 7.1 and higher starting from HTTP Server Group SF99368 level 16 and will not be ported back to any earlier releases. This is an update that we would normally have delivered on a release boundary due to the potential impact. Due to the importance to allow large files to be handled by the IBM i HTTP Server we felt this change could not wait for the next new IBM i release.

How to check if your HTTP Server is large file support enabled

There are three ways to simply verify if your HTTP Server is large file support enabled or not, you can follow any of the below three options to do that.

1. Check HTTP Server large file support PTFs/HTTP Server Group.

(1) HTTP Server Group PTF: SF99368 level 16

(2) HTTP Server Large file support PTFs: SI47649, SI47662, SI47663,SI47664, SI42003

Note: These five PTFs are Corequisite PTFs, so they must be applied or removed together. These five PTFs and other large file support related PTFs are all included in the HTTP Server Group SF99368 level 16 . It's suggested to install the HTTP Server Group instead of applying individual PTFs.

2. Check HTTP Server error log

First, please make sure error log is already enabled for your instance. You can use IBM Web Administration for i tool to select your HTTP Server instance and click the Logging link from the left navigation menu. See figure 1 for example:

 

Figure 1 Enable HTTP Server error log

 

By default, the error log is located in /www/[instance_name]/logs/error_log.Q*********.

Note: The "*" represents the timestamp when the error log is generated, please make sure you check the error log with correct date and time.

After HTTP server is fully started, open the error log and check. See figure 2 for example, you should find "with large file support " message from error log if your instance is large file support.

Figure 2 Check error log for large file support

 

3. Specify '-V' parameter to STRTCPSVR command

Specify '-V' (uppercase) parameter to STRTCPSVR command to show the server compile settings, for example:

STRTCPSVR SERVER(*HTTP) HTTPSVR(WEBSVR '-V')

See figure 3 for example: the phase "with large file support" indicates your HTTP Server has large file support.

Figure 3 Check HTTP Server compile settings for large file support

Note: Specify '-V' parameter only displays all the compile settings, the HTTP Server won't be actually started.

HTTP Server Plug-in module consideration
HTTP Server for i code has been recompiled against macro _LARGE_FILES in order to support large file. All IBM and third-party developed HTTP Server Plug-in modules will not work correctly with the large file support HTTP Server anymore if they are not recompiled in the same way. Below section describe the reason and how to get old Plug-in modules work with large file support HTTP Server.

Background:

There is a structure request_rec defined in HTTP Server's header file "httpd.h" which is used in all HTTP Server plug-in modules code. This structure has many members are defined as apr_off_t data type, for example:

struct request_rec {
... ...
apr_off_t sent_bodyct; /* byte count in stream is for body */
apr_off_t bytes_sent; /* body byte count, for easy access */
... ...
apr_off_t clength; /* The "real" content length */
apr_off_t remaining; /* Remaining bytes left to read from the request body */
apr_off_t read_length; /* Number of bytes that have been read from the request body */
apr_finfo_t finfo;
... ...
};

struct apr_finfo_t {
... ...
apr_off_t size; /** The size of the file */
apr_off_t csize; /** The storage size consumed by the file */
... ...
};

In order to support large file(>2G), HTTP Server code has been recompiled with the macro _LARGE_FILES so the data type apr_off_t will be automatically mapped to 8 bytes version integer which is large enough to hold a large number(i.e. file size large than 2G). For all HTTP Server plug-in modules, they all needed to be recompiled with the same macro in order to get the module work correctly with the large file support HTTP Server. Below section describes all IBM provided HTTP Server Plug-in modules changes and details steps on how to get third-party HTTP Server Plug-in modules to work.

IBM WebSphere Application Server Plug-in modules

If you have IBM i HTTP server associated to WebSphere Application Server, you must install the corresponding WebSphere Application Server fix packs that include the large file support Plug-in(new *SRVPGM name is QSVTAP22LF) first.

The minimum required version of fix pack that include the large file support Plug-in are:

WebSphere Application Server V6.1 - Fix pack 6.1.0.45

WebSphere Application Server V7.0 - Fix pack 7.0.0.25

WebSphere Application Server V8.0 - Fix pack 8.0.0.5

WebSphere Application Server V8.5 - Fix pack 8.5.0.1.

All fix pack can be downloaded from Fix Central: http://www-933.ibm.com/support/fixcentral/

We take WebSphere Application Server V7.0 for example, the below table shows what the value of directive LoadModule should be before and after large file support.

Large File Support	LoadModule directive value in httpd.conf
N	LoadModule was_ap20_module /QSYS.LIB/QWAS7x.LIB/QSVTAP22.SRVPGM.
Y	LoadModule was_ap20_module /QSYS.LIB/QWAS7x.LIB/QSVTAP22LF.SRVPGM

Table 1 LoadModule directive comparison before and after large file support

Note: The letter "x" represents the WebSphere Application Server installation library, it can be A, B, C... depends on how many installations you have on your system.

By alternatively, If the large file support HTTP Server Group is installed but you are unable to install the required fix pack in short time due to some reasons, IBM also delivers PTF SI48802 to help users restore correct large file support WebSphere Application Server Plug-in modules to all corresponding WebSphere Application Server libraries(i.e. QWAS7A, QWAS85B etc) on your system. With that PTF applied, it allows you continue to run your existing business correctly on WebSphere Application Server before you have a time slot to install the required fix pack. If you need to do the restore work again in future, besides remove and apply the PTF again, you can also directly run this CL command:

CALL PGM(QHTTPSVR/QZHBLFPLG) This command does the same work as applying the PTF. PTF SI48802 is also included in HTTP Group SF99368 level 16.

Note:

• (1) Plug-in module will not be restored if the QSVTAP22LF service program already exists in the library. If you want to replace it, you need to manually rename/move/delete the existing QSVTAP22LF service program and then call the QHTTPSVR/QZHBLFPLG program or removing PTF SI48802 and applying it again.
• (2) If a new WebSphere Application Server installation is installed after the PTF applied and the fix pack level is older than the required, you need to call the QHTTPSVR/QZHBLFPLG program or removing PTF SI48802 and applying it again to get the correct Plug-in modules restored to the new created WebSphere Application Server libraries.
• (3) If the PTF SI48802 is applied first and then you install fix pack which is older than the required, the Plug-in runtime service program QSVTCMMNH will be replaced by the older version from the fix pack which is not compatible with the QSVTAP22LF service program. In this case, you need to manually rename/move/delete the existing QSVTAP22LF service program and then call the QHTTPSVR/QZHBLFPLG program or removing PTF SI48802 and applying it again to get the correct version Plug-in service programs restored.
• (4) The Plug-in runtime service program QSVTCMMNH shipped by PTF SI48802 is compatible with both large file Plug-in QSVTAP22LF and non large file Plug-in QSVTAP22 service programs.
• (5) IBM provides PTF SI44812 which automatically updates httpd.conf to use the large file support WAS Plug-in service program QSVTAP22LF on IBM i 7.1 when the HTTP Server is first started. This PTF requires that the previous plugin-cfg.xml which is specified by the WebSpherePluginConfig directive and the WAS plug-in service program QSVTAP22/QSVTAP20/QSVT2070/QSVT2270 which is specified by the LoadModule directive to be in the standard location, otherwise users will need to update the httpd.conf manually. 

The standard location for the WebSpherePluginConfig  directive has the following pattern:  

WebSpherePluginConfig/QIBM/UserData/WebSphere/AppServer/<version>/<edition>/profiles/<WAS_profile>/config/IHS_<HTTP_server_name>/plugin-cfg.xml

The standard location for the LoadModule directive, without large file support, has the following pattern:

LoadModule was_ap20_module /QSYS.LIB/<WAS_product_library>.LIB/QSVTAP22.SRVPGM

Here is an example, If you are upgrading from IBM i 6.1 to i 7.1:

LoadModule was_ap20_module /QSYS.LIB/QHTTPSVR.LIB/QSVTAP20.SRVPGM
LoadModule was_ap20_module /QSYS.LIB/QHTTPSVR.LIB/QSVTAP22.SRVPGM
LoadModule was_ap20_module /QSYS.LIB/QHTTPSVR.LIB/QSVT2070.SRVPGM
LoadModule was_ap20_module /QSYS.LIB/QHTTPSVR.LIB/QSVT2270.SRVPGM

 

If the correct large file support WebSphere Application Server Plug-in can't be found or correctly loaded by HTTP Server, the HTTP Server will fail to start and below error message will be displayed on green screen:

Error message: HTTP Server could not find the large file support WebSphere Application Server Plug-in <Plug-in path>.

See figure4 for example:

Figure 4 Error message of large file support WebSphere Application Server Plug-in not found

 

Hit F1 to see the detail HTP8908 diagnostic message:

• Message - HTTP Server could not find the large file support WebSphere Application Server Plug-in <Plug-in path>.
• Cause - The HTTP Server attempted to load the large file support WebSphere Application Server
• Plug-in &1 but was unsuccessful.
• Recovery - The large file support WebSphere Application Server Plug-in &1 may be misspelled or does
• not exist. Correct the Plug-in name or install the latest WebSphere Application Server
• fix pack and start the server again.

See figure 5 for example:

Figure 5 HTP8908 message

 

If you start HTTP Server from the Web Administration for i GUI, a similar error message will be displayed too. Below section illustrates the Web Administration for i GUI changes for this.

IBM Web Administration for i provides a Web-based GUI interface to manage IBM HTTP server and application servers. When you start/restart a HTTP server or associate a HTTP server with a WebSphere Application Server by IBM Web Administration for i, the GUI will check if the WebSphere Application Server large file Plug-in can be found or not.

If the correct Plug-in can not be found on the system while starting/restarting HTTP server, or associating HTTP server with an existing WebSphere Application Server, the following error message is thrown:

• Error message: HTTP server <instance name>; can not be started. HTTP Server is enabled with large file support but the required plugin is not found from the associated WebSphere Application Server. Update the WebSphere Application Server to version <minimum required version>; or above.

In following example, start HTTP server while the WebSphere Application Server large file Plug-in can not be found.

Figure 6 Start HTTP server while the large file Plug-in can not be found

 

In following example, associate the HTTP server with a WebSphere Application Server which does not have the large file Plug-in.

Figure 7 Associate HTTP server with WebSphere Application Server without large file Plug-in

 

In Create Application Server (only WebSphere Application Server) and Create WebSphere Portal wizard, the GUI also provides option to associate HTTP server with the application server after the profile is created. In the wizard, if the WebSphere Application Server installation is not at the required minimum level, the following warning message is shown to indicate that the WebSphere Application Server needs to be updated to ensure the entire Web environment work correctly,

Warning message: HTTP Server is enabled with large file support on the system but the WebSphere Application Server installation is not at the required level. HTTP Server can not be started after the profile is created. Update the WebSphere Application Server to version <minimum required version> or above before or after the profile is created to ensure the entire Web environment work well.

Following gives an example in creating WebSphere Application Server V8.0.0.3.

Figure 8 Create WebSphere Application Server

 

For more detail about WebSphere Application Server large file support Plug-in module, please refer to related section in WebSphere Application Server infomation center.

• WebSphere Application Server V8.5 -
• https://www.ibm.com/support/knowledgecenter/SSEQTP_8.5.5/com.ibm.websphere.base.iseries.doc/ae/welcome_basei.html
• WebSphere Application Server V9.0 -
  https://www.ibm.com/support/knowledgecenter/SSEQTP_9.0.5/as_ditamaps/was9_welcome_base.html

Other IBM provided Plug-in modules

Except IBM WebSpehre Application Server Plug-in module, all other IBM provided Plug-in module have been correctly recompiled, tested and ship by PTF together with HTTP server large file support PTFs in the same HTTP Group SF99368 level 16. It's strongly suggested install the HTTP Server Group as a whole to ensure all your Web solutions run correctly.

i) IBM Integrated Web Application Server for i / IBM Integrated Web Services Server for i

The HTTP Server Plug-in of IBM Integrated Web Application Server and IBM Web Services Server are the same:

LoadModule mod_ibm_lwi /QSYS.LIB/QHTTPSVR.LIB/QLWIIHSMOD.SRVPGM

It has been recompiled to support large file. If you have IBM i HTTP Server associated to IBM Integrated Web Application Server for i or IBM Integrated Web Services Serve for i, please apply PTF SI42003 or install the HTTP Group.

ii) Domino

The HTTP Server Plug-in of Domino Server is:

LoadModule ibm_dom_server_http_module /QSYS.LIB/QHTTPSVR.LIB/QZSRVAP20A.SRVPGM

It has been recompiled to support large file. If you have IBM i HTTP Server to connect to domino server, please apply PTF SI42277 or install the HTTP Group.

iii) Tomcat

The HTTP Server Plug-in of Tomcat Server is:

LoadModule jk_module /QSYS.LIB/QHTTPSVR.LIB/QZTCJK.SRVPGM

It has been recompiled to support large file. If you have IBM i HTTP Server to connect to own installed tomcat server on IBM i 7.1, please apply PTF SI47667 or install the HTTP Group..

 

Third Party plug-in modules:

For all ISVs' and customers' own developed plug-in modules, all of them MUST be recompiled before installing the large file support HTTP Server PTFs/HTTP Group level 16. Please follow below 4 steps to ensure your own developed plug-in module can still correctly work with large file support HTTP server.

Step 1 Review your plug-in source code to check if any primary integer data types are directly used for the apr_off_t members.

As mentioned in above section, there are some members defined as apr_off_t data type in the structure request_rec, if your plug-in code is using any of them and assign them to primary integer data types(i.e. int), please correct them to use the same data type definition in the httpd.h.

For example:

If your plug-in code is using the member bytes_sent and the below code snippet sample is not correct.

Wrong:

void functionXXX(request_rec* r){

int size = r->bytes_sent;

}

Correct:
void functionXXX(request_rec* r){

apr_off_t size = r->bytes_sent;

}

Please check and update your code to use the same data type defined in the httpd.h rather than using primary data type directly.

Step 2 Recompile your plug-in code with macro _LARGE_FILES

After step 1, your plug-in needs to be recompiled with the marco _LARGE_FILES to ensure your modules is still working correctly with the large file support HTTP server.

Below is an example of how to recompile your plug-in module.

Suppose our example plug-in module source code is /home/plugin/testMod.c and the plug-in module service program library/name is PLUGIN/TESTMOD. There are two ways to recompile your code.

i) Add _LARGE_FILES to the DEFINE parameter when using the CRTCMOD CL command to compile your code

CRTCMOD MODULE(PLUGIN/TESTMOD) SRCSTMF('/home/plugin/testMod.c') INCDIR('/qibm/proddata/httpa/include') DEFINE(AS400 AS400_UTF8 '_LARGE_FILES') LOCALETYPE(*LOCALEUTF) TERASPACE(*YES)

See Fig.9 for example:

Figure 9 Re-compile Plug-in against macro _LARGE_FILES

Then run CRTSRVPGM command to create the plug-in service program.

ii) Add #define _LARGE_FILES to the beginning of your module's source files.

See Fig.10 for example:

Source code: /home/plugin/testMod.c

Figure 10 Add macro _LARGE_FILES to source code

 

Then create the plug-in modules and service program.

Note: If your module has more than one source files, add the macro definition to all of them.

Step 3 Ensure the new plug-in has the correct authority as before

See Fig.11 for example:

Plug-in module: /QSYS.LIB/PLUGIN.LIB/TESTMOD.SRVPGM:

Figure 11 Change Plug-in *SRVPGM authority

 

Note: The authority of your plug-in may be different from the example, use the same authority as the old one.

Step 4 Restart HTTP server and test

Restart your HTTP Server instance to get the new compiled plug-in module loaded and run your tests to see if everything works well as before.

Summary

The large file support new feature has been added to HTTP Server for i on IBM i 7.1 to provide the ability to allow users to transfer large size files(>2G) through HTTP Server. User can install HTTP Group SF99368 level 16 to enable your HTTP instance with large file support on your IBM i 7.1 system.

In order to ensure your Web solutions contiue to work correctly with large file support HTTP Server, the following two actions must be performed.

(1) If WebSpere Application Server is using to connect with HTTP Server, the mimium requried version fix pack(6.1.0.45, 7.0.0.25, 8.0.0.5 and 8.5.0.1) must be installed.

(2) If third-party HTTP Server Plug-in module is used, it must be re-compiled against marco _LARGE_FILES

Resources

• [1] Apache HTTP Server 2.2 new features - http://httpd.apache.org/docs/2.2/new_features_2_2.html
• [2] WebSphere Application Server 8.5 Knowledge Certer -
• https://www.ibm.com/support/knowledgecenter/SSEQTP_8.5.5/as_ditamaps/was855_welcome_base_dist_iseries.html
• [3] WebSphere Application Server 9.0 Knowledge Certer -
• https://www.ibm.com/support/knowledgecenter/SSEQTP_9.0.5/as_ditamaps/was9_welcome_base.html