Water cooling requirements

Determining minimum required system flow and pressure

• Allowable system inlet water temperature range is 6 - 16°C (43 - 61°F), using standard building chilled water (BCW). A special water system for model 9125-F2A is typically not required.
• Required flow rate to the cabinet is 3.7 - 79.4 lpm (1 -21 gpm), depending on inlet water temperature and the number of nodes populated in the rack. Colder inlet water temperatures require less flow then warmer water temperatures. Racks that are partially populated with nodes require less flow than maximum populated racks.
• Minimum water pressure required across the hose ends is 0.34 - 2.32 bar (5 - 33.7 psi), depending on the minimum flow required.

Determining minimum required system flow and pressure for a normal operating system

1. Determine the inlet water temperature. This is the temperature of the chilled water that is present in the room where the system is installed.
2. Determine the minimum design flow rate for your rack configuration using tables Required building chilled water flow, pressure drop and Required building chilled water flow, pressure drop and outlet temperature (English units). Under normal operating conditions the flow into the frame will be double that of the numbers shown in the tables. This is because both Water Conditioning Units (WCU) in the frame operate simultaneously.
   Note: If there is a possibility that you will add nodes to a partially populated system at a later date, you should plan for it during the initial installation.
3. Determine the minimum pressure required across the ends of the hoses to provide the flow rate determined in the step above. Maximum continuous operating pressure must not exceed 2.32 bars (33.7 psi).

Determining additional required system flow for a fault condition

Define the maximum facility pressure drop that can be present across the ends of the model 9125-F2A hoses, and locate the equivalent pressure drop in tables Required building chilled water flow, pressure drop and Required building chilled water flow, pressure drop and outlet temperature (English units). The corresponding flow rate multiplied by two represents the maximum flow a single rack could experience under a worst-case fault condition. The facility must be able to accommodate this condition for at least one 9125-F2A racks in the installation.

Principal of operation

• Two pairs of hoses and two WCU's are in parallel at the water cooling input of the model 9125-F2A. Normally both of the WCU's operate in parallel to cool the rack, with half of the water flow circulating through each of the water cooling units. The control valve in the WCU regulates the flow of water that goes through it in order to maintain the proper water temperature of the system-side water in the 9125-F2A. If the facility water temperature gets cooler, the valve opening will shrink. If the facility water temperature gets hotter, the valve will become more open. There must be enough pressure across the ends of the hoses to force the minimum required amount of water through the WCUs. This is information is available in Determining minimum required system flow and pressure.
• The 9125-F2A has two pairs of hoses and two WCUs. This allows the system to continue to function even if one of the WCUs should experience some type of failure. If a WCU fails in a way that does not enable it to cool the load, the other WCU will pick up the entire rack load. In this case, the valve in the functioning WCU will open wider to increase the water flow through it to maintain system temperatures. In addition, water will likely be shut off to the rear door heat exchanger of the rack to shed the water cooling load, so the single WCU can cool the processors in the rack. This causes the heat that is exhausted to the room to rise. By removing the WCU from the front side, the faulty WCU can be replaced concurrently, without shutting the system down to restore the rack water cooling system to its fully redundant state. It is possible that a faulty WCU could have its input valve stuck in the full opened position. If this happens, the rack will require two times the amount of available water flow until the faulty WCU is repaired. The facility must be capable of providing this additional flow for this fault case. This is information is available in Determining minimum required system flow and pressure.
• The system-side water is completely isolated from facility water by the water-to-water heat exchangers in the WCUs. Heat is transferred from the system-side water to the facility water by thermal conduction through the water-to-water heat exchangers. System-side water is maintained by IBM® service personnel using special water treated with a corrosion inhibitor.
• Provided that the room dew point is within class 1 specification, condensation will not form on the system side of the water cooling system. This is because the system-side water is regulated above the temperature at which condensation occurs. Condensation will not form on the facility-side water-cooling system because the components, including the quick disconnects and hoses, are insulated. Protection against condensation at the point where the hoses connect to facility plumbing and condensation protection of the facility side plumbing itself is a customer responsibility.

Calculating building chilled water return temperature (SI units)

T^return (°C) = T^supply (°C)+ 14.4 (Q (^MWU (kW)/ (MWU BCW Flow (lpm))

Return water temperature can be calculated under normal operating conditions using the previous table, Required Building Chilled Water Flow per MWU (lpm).

Calculating building chilled water return temperature (English units)

T^return (°F) = T^supply (F) + 2.0 (Q^MWU (kBTU/hr) / (MWU BCW Flow (gpm))

Return water temperature can be calculated under normal operating conditions using the Required Building Chilled Water Flow per MWU (gpm) table.

Facility water momentary flow interruption and out of specification temperature

The system is designed to tolerate a momentary interruption of facility water flow or temperature increase that might occur as a result of a failure in the facility water distribution system.

Momentary interruption of facility water flow

The system can tolerate complete loss of facilities water flow for 60 seconds. There is a high probability that nodes will be powered off by thermal protection circuits internal to the 9125-F2A if loss of flow exceeds this time.

Over-temperature of facility water

The system can tolerate a 14°C (26°F temperature rise above the maximum facilities operating temperature at a given flow rate. It should be assumed that the system will be powered off by the thermal protection circuits internal to the 9125-F2A for facility water temperatures greater than this maximum temperature rise. Continuous operation in an over-temperature condition of 1 - 14°C (2 - 25°F) cannot be sustained, even though the system will run without powering down. When the maximum allowable facility water temperature of 16°C (60.8°F) is exceeded, or when the system-side water temperature cannot be regulated, an error is sent to the customer, and corrective action is required.

Under-temperature facility water

If facility water is below the minimum allowable temperature of 4.4 °C (40 °F) an error will be surfaced to the customer and corrective action will be required. The only problem that can result from under-temperature facility water is condensation on the facility side of the system. The temperature at which condensation might occur is dependant on the severity of the facility water under-temperature and the air temperature and humidity in the room.

Facility water quality and fowling

In general the 9125-F2A requires standard building chilled water temperatures without any special requirements. The quality of the facilities water is as follows.

• Total hardness must not exceed 200 mg/L as calcium carbonate.
• The pH must be 7 - 9.
• Turbidity must be less than 10 Nephelometric Turbidity Unit (NTU).
• Bacteria must be less than 1000 10 colony forming unit (CFU)/ml.
• Water should be as free of particulate matter as feasible.

Deionized water with benzotriazole solution installation and maintenance

IBM is responsible for supplying and maintaining the internal frame secondary loop side water. This protects the system processor books and distribution plumbing from damage that could result from the use of contaminated water.

IBM supplies a water solution that is mixed with benzotriazole (BTA), a corrosion inhibitor, for the internal secondary cooling loop of the frame when it is installed and when any repairs are performed that require water to be added.

IBM uses certified suppliers for the water solution that satisfy all pertinent environmental control requirements.

Deionized water with benzotriazole disposal

The customer must dispose of the water solution in accordance with applicable laws and regulations and product characteristics at the time of disposal.

Internal frame water solutions

IBM will supply the system-side water.

Water and benzotriazole solution

Benzotriazole (BTA) is mixed with the deionized (DI) water to a concentration of 1000 parts per million by weight.

Deionized water

The deionized water used in IBM water cooling systems conforms to type II, grade A specifications in ASTM D1193-06 entitled, standard specifications for reagent water. Specifications are as follows:

• Electrical resistivity at 25 °C > 0.5 MΩ•cm
• Total organic carbon < 50 µg/L
• Sodium < 5 µg/L
• Chloride < 5 µg/L
• Total Silica < 3 µg/L
• Total organic carbon < 50 µg/L
• Heterotrophic bacteria count (HBC) less than 10 colony forming unit (cfu) / 1000 mL as measured per ASTM F1094 or IBM approved equivalent

Benzotriazole

Benzotriazole (BTA) is purchased from Sigma-Aldrich or an alternative IBM approved supplier, and is defined as follows:

• Product Name: Benzotriazole, 99%
• Product Number: B11400
• Brand: Aldrich Chemical
• Substance Name: 1H-Benzotriazole
• Chemical abstracts service number: 95-14-7
• Formula: C⁶H⁵N³
• Molecular weight: 119.12

Connection of facility water to the system

Two pairs of insulated 25.4 mm (1 in.) inside diameter (38.4 mm outside diameter / 1.51 in.) hoses, which are specified by IBM, connect facility water to the system. The hoses are available in 1.83 m (6 ft) and 4.27 m (14 ft) lengths, and can be purchased from IBM or purchased directly from the hose assembly manufacturer using a part number. The hoses must be attached to the facility water source, and the system ends must be properly positioned above the raised floor prior to the server arriving so that the rack can be rolled into positions, up to the hoses, without delay.

Installation of IBM system connection hoses in facility

The following figure shows the four hoses (two supply hoses and two return hoses) connected to the building water manifolds under the raised floor with the ends that connect to the system properly positioned 188 mm (7.4 in.) above the floor.

The facility end of the provided hoses is a cut hose without a fitting. It is the customer's responsibility to determine the connection technique on this side.

It is suggested, but not required, that a shutoff valve be provided in front of the hose assembly as shown. This shutoff valve is not required for maintenance of the equipment, but can be useful if hose removal is ever desired by the customer.

The 1.83 m (6 ft) and 4.27 m (14 ft) hose length is the customer's responsibility and depends on the distance from the facility manifolds to the rack. The facility end of the hose can be cut to the desired length by the hose installer.

Route the hoses through the floor cutout as shown, being sure to avoid sharp edges of metal and to leave some slack in the hose. Cable management is required for signal cables that exit the rear of the rack and must be considered when routing the hoses. In the proceeding figure the cable management tray is provided as part of the facility at the rear of the rack.

IBM hose assembly ordering information

Four hoses (two supply and two return) and one hose positioning fixture is required for each rack that will be installed. The hose positioning fixture is removed after the hoses are connected to the system.

Ordered hose kits from IBM arrive with the system. However, it is recommended that you order the hose kits before the system, so that they can be attached to the facilities plumbing in advance. The hose kits must be ordered directly from the hose kit manufacturer.

Hose assembly ordering information

Two supply hoses, two return hoses, and one hose positioning fixture are required for each rack to be installed.

In order for the hose kits to arrive before the system, the hose kits must be ordered from IBM by ordering the 9125-F2A Site Preparation/Install Support Model 9125-F2B. The 9125-F2A Site Preparation/Install Support Model (9125-F2B) enables the ability to place an order for the Coolant Supply/Return Hoses (#6876 or #6877) prior to delivery of the system. This allows the site chilled water plumbing to be completed ahead of the system arrival, so the hose kits can be attached to facilities plumbing in advance, reducing overall installation time. Order either of the following if hoses are required before the systems.

• 9125-F2B 6876 coolant supply and return hoses, 1.83 m (6 ft)
• 9125-F2B 6877 coolant supply and return hoses, 4.27 m (14 ft)

If it is not necessary to receive the hose kits prior to installation, then you can order the following hose kits:

• Feature code 6876 and part number 45D2215.
  • Two 1.83 m (6 ft) hose assembly (black supply) –P/N 45D0907
  • Two 1.83 m (6 ft) hose assembly (white return) – P/N 45D0908
  • One hose positioning fixture – P/N 45D2245
• Feature code 6877 and part number 45D2214.
  • Two 4.27 m (14 ft) hose assembly (black supply) – P/N 45D1952
  • Two 4.27 m (14 ft) hose assembly (white return) – P/N 45D1951
  • One hose positioning fixture – P/N 45D2245

Hose materials

Only IBM hoses can be used to connect to the IBM 9125-F2A server. This protects against leaks and condensation that can result from using insufficient hose assemblies and incompatible quick disconnect couplings.

The hose contains the following components:

• Gates terminator hose: 25.4 mm (1 in.) inner diameter (Gates part number 308504)
• Armacell and Armaflex Hose insulation: Armacell part number APT15838 919-304-3846
• Bentley Harris Expando flame retardant plus protective mesh covering (Bentley Harris part number BSBHFRP-175)
• Aeroquip System-side quick disconnect couplings (IBM part number 45D0909 or 45D0915)
• Aluminum-bronze elbow and stainless steal clamp

Quick disconnect couplings insulator information

The quick disconnect coupling insulators shown in the following figure, are shipped with the system.

The quick disconnect couplings insulators must be attached as shown in Quick disconnect couplings without insulator (left) and with insulator (right) or condensation might occur on the quick disconnect.

Purging air from facilities-side hoses

It is recommended that you purge the air from the facilities connection hoses when the contractor installs the hoses.

The hoses can contain a maximum of 0.51 liters per meter (0.041 gallons per foot) of air.

A hose purge operation is not done when the system is connected to the hoses because it is assumed that purging the hoses is completed by the hose installation contractor if it is thought to be necessary in the facility. The suggested procedure to purge air from the hoses during installation is as follows.

1. After final connection of the hoses to the facility supply and return is made, connect the supply quick disconnect coupling to the return quick disconnect coupling as shown in the preceding figure.
2. After the connectors are twisted together, open the valve handle on one of the two connectors to lock the quick disconnects together.
3. Slowly open the valve on the other connector, allowing the air trapped in the hoses to be moved slowly into the return side of the facility. Slowly removing the air prevents a large amount of air from entering the return all at once and allows the air to find its way to the facility's high point vent.
4. After purging is complete, disconnect the quick disconnects from one another and place them in their proper locations in the positioning fixture above the raised floor, in preparation for rack installation.