IBM Support

AIX Live Update Across Frames using PowerVC incase of Insufficient System Resource Availability

Technical Blog Post


Abstract

AIX Live Update Across Frames using PowerVC incase of Insufficient System Resource Availability

Body

Contents

  1. Introduction

  2. Available Live Update features till AIX 7.2 TL2

  3. Environment details for Live Update Across Frames

  4. Live Update Across Frames functionality details

  5. Virtual Machine Migration Requirements for Live Update

  6. How to perform Live update across frames

  7. Conclusion

1. Introduction

At Several occasions there might be need for AIX Operating System to get upgraded by installing interim fixes (Ifixes) in form of Kernel/Kernel Extensions or by moving to a newer Service Pack/Technology level. This affects business critical workloads as the system may need a reboot for the newer version to take effect. This issue can be solved by using the Live Update (LU) operation which allows the application of Ifixes / Service Packs/Technology levels without requiring a system restart.

One major obstacle for the adoption of live update is the availability of 2X processor and memory resources on the Server which hosts the original Virtual Machine (VM). On packed systems there might not be enough free processor and/or memory to allow the creation of the surrogate VM, or not enough to run several live updates concurrently

To overcome these constraints, AIX 7.2 TL3 provides a solution by performing Live Update using an alternate Server having sufficient processor, memory to accommodate both original and surrogate. Live Update Across Frames functionality is available only for the PowerVC Based Live Update.

This blog will provide insights into how users can leverage this new functionality and detailed steps to be followed.

2. Available Live Update features till AIX 7.2 TL2

Basic knowledge of the available features is pre-cursor for new Live Update users to appreciate the new feature (Live Update across Frames) better. The evolution of Live Update features in AIX 7.2 is depicted in the figure below.

image

Figure 1: Evolution of Live Update

To learn more about each of these features, the following links can be referred.

3. Environment details for Live Update Across Frames

Prerequisites for the Live Update Across Frames feature

1. System Firmware

  • Ax770_063
  • Ax773_056
  • Ax780_056
  • FW840
  • FW860
  • FW910
  • FW920

2. IBM Power Virtualization Center (PowerVC)

  • 1.3.3.1 and later

3. IBM PowerVM Novalink

  • 1.0.0.8 and later

4. Hardware Management Console (HMC)

  • 860 SP2
  • 870
  • 910
  • 920 and later

5. Virtual I/O Server (VIOS)

  • 2.2.6.20 and later

4. Live Update Across Frames functionality details

When there is a shortage of free system resources on the original server to accommodate both the original and the surrogate VMs during a Live Update operation, administrators can either choose a different destination server or let Live Update pick one based on certain pre-defined parameters in order to complete the operation. Next section elaborates those factors in details.

The whole process works as follows:

  1. VM migration to a destination Server (Step 1)
  2. Live Update at the destination Server (Step 2)
  3. VM migration back to Original Server (Step 3)

PowerVC provides the flexibility to choose the destination Server irrespective of whether it is HMC managed or Novalink managed. For example, Source Server can be a PowerVC-HMC managed Server and destination can be a PowerVC-NOVA managed server or vice-versa.

We will discuss the procedure with the help of an example. In the flowchart in Figure 2, VM1 is the node which needs to undergo LU. VM1 is hosted by Server1. LU always requires 2X system resources to accommodate both the original and surrogate VMs. If the administrators observe that Server1 does not have 2X free resources and Server2 does, they can opt for Live Update Across Frames feature.

  • In Step1, VM1 migrates to Server2, using Live Migration feature of PowerVC (also known as Live Partition Mobility or LPM).
  • In Step2, Live Update is performed on Server2. Post end of the Live Update, VM1 runs with updated kernel. All resources used by the original VM are freed up.
  • In Step3, VM1 is migrated back to Server1 using PowerVC Live Migration again. After the Live Migration, VM1 does not consume any resources on Server2 anymore.

image

image

imageFigure 2: Live Update Across Frames flowchart.

The requirements for VM migration are explained in next section. if user is familiar with VM migration with PowerVC environment can skip next section and go to section 6 "how to perform Live Update Across Frames".
 

5.  VM Migration Requirements for Live Update

Live Update Across Frames operation utilizes existing PowerVC VM Migration functionality. Following requirements must be fulfilled for making the setup migration capable.

1. Host Requirements

  • Enough Processor and memory resource as to be on the destination Host.
  • Migration can be performed on VM whose HOST is either managed by HMC, Nova link
  • Logical memory block size as to be same on the source and destination Host.
  • Host should be in active state.
  • VM processor compatibility mode should be supported on Destination Host

Host and Host group
A Host group can be used to group logical Hosts regardless of the underlying architecture (system, network, storage configurations) that may or may not be common. Only requirement of adding a host to host group is host should be managed by the same PowerVC. Any newly added host will part of ‘default’ host group. Host can part of only one Host group; existing host can be added to any other Host group.

Every host group has an attribute called placement policy.VM can be migrated to Host within Host Group only. Destination Host for Migration will be selected based on the placement policy set for the Host Group.

For more information on the Host Group and Placement policy refer below links,

https://www.ibm.com/support/knowledgecenter/en/SSXK2N_1.3.3/com.ibm.powervc.standard.help.doc/powervc_host_group_overview_hmc.html
https://www.ibm.com/support/knowledgecenter/SSXK2N_1.3.3/com.ibm.powervc.standard.help.doc/powervc_placement_policy_hmc.html

image             Screenshot 4: List of the Host connected to PowerVC.

image            Screenshot 5: List of Host Groups in PowerVC.

image            Screenshot 6:  Host Group host_gp1 with two Hosts and placement policy set to Striping.

 

2. Storage Requirements

  • Original and Destination Host must have same Storage Connectivity Group(SCG) configuration.
    Number of VIOS, Fiber Channel (FC) ports configuration in the Fabric must be same in destination host as in original host.

image

Figure 3. Representation of SCG with Fabric connection.

Figure 3 shows an example of Fiber Channel (FC) port and Storage Connectivity Group (SCG) configuration. In this figure, HOST1 and HOST2 have 2 VIOS each and FC adapters are connected to different fabric. We have two SCG named TEST and DEV. TEST will use only VIOS A1 and VIOS B2 FC ports connected to Fabric A. SCG DEV will use all VIOS’s and FC ports connected to Fabric A and B.

  • FC port channel configuration and port tagging
    All Fiber channel ports connected to the VIOS in the Hosts are listed in the PowerVC configuration à Fibre channel port configuration page. Depending on the Storage connectivity with fabric requirement, FC ports are tagged.
    In Screenshot 7, a single FC port of VIOS in both the Hosts has been tagged with label “ZZ1-ZZ5” which are connected to the same fabric. More information about FC port tagging can be found in PowerVC Config FC ports HMC
    https://www.ibm.com/support/knowledgecenter/en/SSXK2N_1.3.3/com.ibm.powervc.standard.help.doc/powervc_config_FC_ports_hmc.html

    imageScreenshot 7:  Fiber Channel Port Tagging
     

  • Storage Connectivity Group with FC Port Tag

Storage connectivity group will be created with sets of VIOS, Fiber Channel (FC) fabrics and ports for deployment and migration of VM.

In Screenshot 8, “VSCSI-NPIV” SCG is created with VSCSI mapping as boot volume and NPIV as Data volume. Disks attachment to the VM is restricted with adding the VIOS’ and FC port labeled “ZZ1-ZZ5”. 
In Screenshot 9,Addition of VIOS from different Hosts to group will allow the Migration of VM to another Host in SCG. You can read more about Storage connectivity Group in powervc_storage_connectivity_groups

https://www.ibm.com/support/knowledgecenter/SSXK2N_1.3.3/com.ibm.powervc.standard.help.doc/powervc_storage_connectivity_groups.html?view=embed#powervc_storage_connectivity_groups__details

image

                                Screenshot 8: SCG creation with Port Tag ZZ1-ZZ5

image

Screenshot 9:  Different Host’s VIOS addition to SCG

3. Network Requirements

  • Both the Source and Destination Host should be compatible from Network perspective. This means the Virtual Networks which are part of the VM undergoing LU, should be available in the destination server as well. Also, VLAN IDs of Virtual Networks should be bridged to Shared Ethernet Adapter (SEA) in both the source and destination VIOS of Hosts.
  • Source and destination Hosts should be connected to the Same Virtual Switch/Sub network.
  • SR-IOV Networks should be supported in both source and destination Host.
  • Source and destination Host SR-IOV adapter’s Physical port must have same port label, Same type of Physical adapter, Number of logical ports and bandwidth allocated to the Virtual machine.

4.  Remote Management and Control (RMC) Connection

Virtual machine and VIOS’s state and RMC Status should be active.


The users who are planning for VM migration should also be knowing the restrictions below.

  • VM migration is not supported to the Host which in not part of Original Host’s Host group.
  • Collocation rule with affinity or anti affinity will restrict the virtual machine migration to destination Host.

More information can be found about this topic in  powervc_relocation_reqs_hmc.
https://www.ibm.com/support/knowledgecenter/en/SSXK2N_1.3.3/com.ibm.powervc.standard.help.doc/powervc_relocation_reqs_hmc.html

6.  How to perform Live Update Across Frames

Changes outlined below have been added to the existing LU functionality in order to support the Across Frames feature. VM will be migrated to the user-specified destination Host or a Host selected from source Host’s Host Group.

A. Modifications of the lvupdate.data file in /var/adm/ras/liveupdate location
To support live update across frames two new attributes have been added to pvc stanza in the lvupdate.data file.

1. The destination attribute
The destination attribute provides the destination Host information it can be set to either,  

  • Destination host name as in PowerVC host page
  • ANY

If destination Host name is provided and all migration pre-requisites are satisfied, then VM will be migrated to the destination host before starting LU.

If “ANY” option is provided in the destination attribute and all migration pre-requisites are satisfied, then VM will be migrated to a destination host selected by the Host Group Placement policy.

 

2. The force_migration attribute
The force_migration attribute forces Live Update to happen on another host even if sufficient resources are available on the current host.
The force_migration attribute can be either “yes”, “no”, or missing/empty. The missing/empty value is treated as “no”. Any other value is invalid.

 

In Screenshot 10, we can see how lvupdate.data file modified when destination Host is specified as ANY

 

image



 

 



 

 

Screenshot 10: liveupdate.data file with ANY option

In Screenshot 11, the lvupdate.data file is set with Host name for destination attribute

image



 

 

 

 

Screenshot 11:  The lvupdate.data with destination attribute set with Host name

 

B. Live Update Across Frames operation from the VM telnet Console
Following screenshots explain the LU flow that is observed from the telnet console of VM.

In Screenshot 12 and 13, PowerVC token created and LU is triggered. Preview check messages are displayed.
imageScreenshot 12: PVC token generation

image

Screenshot 13: LU authentication, geninstall command and LU preview checks

imageScreenshot 14: live update VM migration preview checks


In screenshot 15, VM migrated to destination Host , LU operation performed and VM migrated back to source Host.

imageScreenshot 15: Actual VM migration and LU completion

 

C. Live update Operation view from PowerVC Graphical User Interface (GUI)
Following screenshots explain the VM migration process as observed from the PowerVC_GUI

Stage 1: VM Migrating to the Destination Host fvt-zz1-vnova from fvt-zz5-nova Host
image

Screenshot 16: VM is migrating to destination host

Stage 2:  VM migrated to the destination Host fvt-zz1-nova and IP address will be same
image

Screenshot 17: VM has migrated to the destination host


Stage 3: Live update operation started with Surrogate VM (partition) creation with temporary IP address and  after original VM  deletion, original IP assigned to the surrogate VM
image

Screenshot 18: Surrogate created with temporary IP address and the LU is in progress.

Stage 4: VM Migrating back to Source host

image

Screenshot 19: VM is migrating back to source host with Original IP address.


Stage 5: LU operation Completed

image

Screenshot 20: Post LU completion, the VM will be back at the original host


In screenshot 21, Shows the PowerVC GUI Message’s requested by the live update.

image

Screenshot 21: PowerVC GUI messages

 

7. Conclusion

This article has provided an insight into performing Live Update of AIX Operating System without system reboot even when there is no availability of sufficient free processor and memory resources on the original Server (i.e. Host). Live Update Across Frames allows administrators to choose a different destination Server with adequate resources to perform Live Update. Live Update Across Frames is only supported on the PowerVC managed Hosts and uses PowerVC’s VM Migration feature.

Authors: Sharath kumar N R, Sanchita Sinha, Madusudanan Vasudevan, Kasinadh P Divvela
 

[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SWG10","label":"AIX"},"Component":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"","Edition":"","Line of Business":{"code":"LOB08","label":"Cognitive Systems"}}]

UID

ibm16156195

Page Feedback